That was actually the case for Claude Opus 4.5, the most recent model of Anthropic’s strongest mannequin, which was launched in late November. In December, METR introduced that Opus 4.5 seemed to be able to independently finishing a activity that may have taken a human about 5 hours—an enormous enchancment over what even the exponential development would have predicted. One Anthropic security researcher tweeted that he would change the route of his analysis in mild of these outcomes; one other worker on the firm merely wrote, “mother come choose me up i’m scared.”
However the fact is extra sophisticated than these dramatic responses would recommend. For one factor, METR’s estimates of the talents of particular fashions include substantial error bars. As METR explicitly said on X, Opus 4.5 would possibly be capable of commonly full solely duties that take people about two hours, or it’d succeed on duties that take people so long as 20 hours. Given the uncertainties intrinsic to the strategy, it was unattainable to know for certain.
“There are a bunch of ways in which individuals are studying an excessive amount of into the graph,” says Sydney Von Arx, a member of METR’s technical workers.
Extra essentially, the METR plot doesn’t measure AI talents writ massive, nor does it declare to. To be able to construct the graph, METR exams the fashions totally on coding duties, evaluating the problem of every by measuring or estimating how lengthy it takes people to finish it—a metric that not everybody accepts. Claude Opus 4.5 would possibly be capable of full sure duties that take people 5 hours, however that doesn’t imply it’s anyplace near changing a human employee.
METR was based to evaluate the dangers posed by frontier AI techniques. Although it’s best identified for the exponential development plot, it has additionally labored with AI firms to judge their techniques in higher element and printed a number of different unbiased analysis initiatives, together with a widely covered July 2025 study suggesting that AI coding assistants would possibly really be slowing software program engineers down.
However the exponential plot has made METR’s fame, and the group seems to have an advanced relationship with that graph’s usually breathless reception. In January, Thomas Kwa, one of many lead authors on the paper that launched it, wrote a blog post responding to some criticisms and making clear its limitations, and METR is at the moment engaged on a extra intensive FAQ doc. However Kwa isn’t optimistic that these efforts will meaningfully shift the discourse. “I feel the hype machine will principally, no matter we do, simply strip out all of the caveats,” he says.
Nonetheless, the METR staff does suppose that the plot has one thing significant to say in regards to the trajectory of AI progress. “It’s best to completely not tie your life to this graph,” says Von Arx. “But additionally,” she provides, “I wager that this development is gonna maintain.”
