Anthropic's AI models are improving roughly twice as quickly as they were a year ago, according to an 11-week investigation by the AI safety research organization METR. The report, published Thursday, suggests that the company has made approximately two years of historical progress over the last year. METR researchers stated they are confident this acceleration is not merely the result of increased computing power, but is instead driven by the AI itself, with its contribution to progress growing over time.
Internal productivity gains at the company are significant, with employees using Anthropic’s most capable internal model reporting they are, on average, 12 times as productive as they would be without AI assistance. This figure represents a doubling of productivity compared to what staff reported just a few months earlier while using Mythos 5. While much of the current acceleration stems from AI assisting human researchers, METR warned that "there are some signs that fully autonomous contributions will play a larger role soon."
The investigation revealed that Anthropic has trained three advanced models that have not yet been released to the public. These models, which researchers identified by the codenames Model 2, Model 3, and Model 4, completed training in July, August-September, and September-October, respectively. METR also identified an unreleased model from 2022 referred to as Unreleased 2022.
METR researchers noted they found no clear signs of diminishing returns or observable bottlenecks that could prevent even faster acceleration. The findings arrive as Anthropic prepares to go public and as CEO Dario Amodei has publicly called for AI labs to slow the pace of development. However, the report's full scope remains unclear because Anthropic was permitted to redact portions of the final document as part of the agreement with METR.
