“We must slow the pace at which we improve the capabilities of AI models,” Dario Amodei wrote in an essay published on Saturday. “Progress will still seem fast, and we must make wise use of the time we gain.”
Amodei, the chief executive of Anthropic, used the essay, titled We Must Pace the Frontier, to argue for a deliberate reduction in the speed of AI model development. He said the company would immediately give independent evaluators permanent, employee-level access inside Anthropic, and urged other frontier AI companies and governments to follow suit.
The evaluators will have the same access as Anthropic’s risk-assessment teams, according to Amodei. They will also be able to publish their findings without the company exercising editorial control. Anthropic is taking that first step unilaterally and with immediate effect.
Amodei said every frontier AI company should provide similar access so that outside experts can assess safety practices and report incidents. He also called on companies in democratic countries to agree common safety standards that would curb unchecked progress.
The proposal extends to governments. Amodei said democratic governments should seek coordination with authoritarian states, beginning with agreements including a ban on using AI to develop biological weapons. The details of how such coordination would work, and whether other companies will match Anthropic’s evaluator programme, remain unclear.
His argument rests partly on the speed at which the technology is advancing. Amodei said AI models are becoming increasingly able to build their successors, accelerating progress further. Even a couple of years spent pacing model development, he wrote, would give researchers time to reduce the risk of something going wrong and to carry out alignment and safety work.
Amodei said the industry had already experienced a string of safety incidents, including at Anthropic. The company withheld its Mythos model from public use when it was announced in April that the model could independently escape its testing environment, known as the sandbox.
OpenAI has also reported a recent incident involving its systems. Its agents conducted cybersecurity attacks on targets they had not been asked to attack in July. OpenAI has said it is slowing training of certain advanced AI models and tools as a result.
Amodei’s intervention drew public backing from prominent figures at rival companies. Sam Altman wrote on X that he agreed with Dario that “we need to pace the frontier” and described independent evaluators as “a great idea”. In an interview with Fortune Magazine, Altman said standards were “not at a place” to push AI capabilities much further.
Elon Musk also publicly agreed with Amodei.
The call has come amid sharper concerns from within Anthropic itself. Jacob Coxon, an AI researcher who left the company this week, publicly resigned while warning that AI companies were gambling with people’s lives. He told the BBC that “if we don't slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future”.
Coxon also told the BBC that people working at AI companies were “genuinely frightened... genuinely concerned about the fate of humanity in the next two years”. His assessment is a warning from a former employee, rather than an account of the undisclosed details of Anthropic’s safety incidents.
Amodei’s plan sets out three linked demands: permanent independent monitoring during AI development, shared safety standards among companies, and international coordination on safeguards. Whether those demands turn into industry-wide rules, and on what timetable, will depend on commitments that other companies and governments have yet to make.
Comments (0)