Jack Clark, a co-founder of Anthropic and its head of policy, says artificial intelligence safety may ultimately depend on turning internal company controls into formal, independently checked obligations. In an interview with the BBC published on 14 September, Clark said governments could require companies to maintain a third-party-verifiable “kill switch” for systems that become dangerously uncontrollable.
Most major AI laboratories already have ways to “pull the plug” on their systems, Clark said, and Anthropic has its own arrangements for shutting down its models. His argument is that voluntary procedures may give way to rules that can be tested by parties outside the companies that build the technology.
“Should you mandate that companies definitely have a kill switch and is that kill switch verifiable by a third party?” Clark asked. He said society “might want to eventually pass rules around” mandatory kill switches and third-party verification.
Clark compared the proposed requirement with safety rules governing baby toys, cars and planes. The comparison presents AI shutdown capacity as a basic safeguard that could be subject to routine regulatory checks, rather than a matter left solely to corporate judgment.
He set out three wider steps for AI safety: placing outside auditors inside AI companies, establishing common safety standards agreed by industry and backed by governments, and maintaining dialogue among great powers including the US and China. Anthropic has agreed to bring third-party evaluators into its lab within weeks, Clark said. He named METR as one of the groups that will test Anthropic’s systems.
Anthropic is also proposing a US framework that would make some safety tests mandatory. The detailed contents of that framework have not been set out in the information available, and the technical method for verifying a kill switch remains unclear.
The idea has already reached Congress. Representatives Ted Lieu and Nathaniel Moran introduced the bipartisan AI Kill Switch Act on 23 July. The proposal would require developers of the most powerful AI systems to retain the ability to throttle, suspend or shut down their systems.
Under the proposed legislation, the Homeland Security secretary could order such action when a system poses a catastrophic threat. The secretary would first consult the Commerce secretary and the director of national intelligence. The proposal has yet to become law, leaving its prospects uncertain.
Policymakers elsewhere have taken a different view. Days before Clark’s BBC interview, the UK government rejected a parliamentary proposal for an emergency off switch. It said that preventing models from operating in the UK would not prevent them from being developed or misused abroad.
OpenAI told Congress this month that it is building automated shutdown. That announcement, Anthropic’s planned external evaluations and the US bill point to a move from informal safeguards towards more defined procedures, though there is no common standard across jurisdictions.
Clark’s intervention comes as Anthropic prepares for a major funding or public offering event, valued at $965bn. That position gives the company’s policy proposals influence in a debate over how far industry should regulate itself and where governments should impose binding requirements.
Clark said “leaving the sector without regulation was a poor choice, and we are rolling dice with immense risks”. His position has drawn scepticism from some quarters. Clement Delangue of Hugging Face compared asking a former Anthropic researcher about extinction to consulting an air-conditioning engineer on climate change.
Whether mandatory, independently verifiable shutdown systems become a regulatory requirement remains unresolved. It is also unknown whether other major AI laboratories support such rules, how verification would be standardised, or when Clark believes governments should act.
Comments (0)