Study finds a ‘Moore’s Law’ for AI agents

Via Superhuman

“Study finds a ‘Moore’s Law’ for AI agents

Source: METR

A new study shows that frontier models might be improving at a much faster rate than we previously thought. The research group METR found that the length of tasks LLMs can tackle seems to be doubling every seven months. That’s even faster than Moore’s Law, which claims that chip capacity doubles roughly every two years.

Putting that into perspective: Claude Sonnet 3.7 can now tackle tasks that take humans about an hour. Compare that to 2023’s GPT-4, which was only capable of five-minute tasks, like searching the web for a specific fact — or 2022’s GPT-3.5, which could only do things that take us less than a minute, like recalling something it already knows.

What’s next? If the trend holds steady, it means four-hour tasks might only be a year or two away. And just wait until 2029, when METR predicts AI will be able to do things that take humans an entire month, like starting a business or making a new scientific discovery. Still, since LLMs are so new, we only have four or five years of data to go off, meaning we should probably take these findings with a grain of salt.”