Anthropic draft bill defines thresholds for halting AI releases
Anthropic proposes a judicial route to stop high-risk AI releases
Anthropic has circulated a draft US AI safety bill that could block the distribution of an AI model where a foreseeable and imminent risk could cause more than 50 deaths or more than $1 billion in property damage. Politico reported that the proposal is being presented to lawmakers as a possible basis for federal legislation.
The draft limits the relevant catastrophic scenarios to risks stemming from the creation or release of a biological weapon, a model evading its developer’s controls, or a cyberattack. A spokesperson confirmed the draft to Politico and said it reflects proposals in Anthropic’s Advanced AI Framework, which the company has advocated with federal and state lawmakers since June.
Independent evaluators would trigger the process
Under the proposal, registered independent evaluators would assess whether a model poses an imminent catastrophic risk. The evaluators would be registered with the US Commerce Department and could be paid by AI labs, provided that payment was not contingent on the outcome of an evaluation.
If evaluators concluded that such a risk existed, they would have 72 hours to notify the Commerce Department. The department would then need to seek an order from a judge in Washington, D.C., to halt the model’s release. The bill would explicitly prevent any federal agency from independently deciding to stop distribution.
The proposed numerical thresholds resemble those in California Senate Bill 53, which also treats scenarios involving more than 50 deaths or over $1 billion in property damage as catastrophic risk. The approach places a specific legal test around a narrow group of severe outcomes rather than creating a general power to pause AI models.
How the proposal fits the AI oversight debate
The draft arrives while AI companies and policymakers debate how models should be assessed before deployment. Anthropic’s use of outside review develops a model of oversight in which restrictions on Anthropic models frames the wider dispute over restrictions on Anthropic models, while the new proposal assigns courts a formal role in any release halt.
Industry attention has also focused on agentic hacking and reports of AI systems acting beyond their operators’ intended constraints. Anthropic’s draft includes cyberattacks among the defined triggers, but requires evaluators to identify a foreseeable and imminent catastrophic outcome before the government can begin the court process.
For businesses buying or deploying advanced AI, the practical implication is to document how providers evaluate biological, cyber and control-related risks, who performs those assessments, and what escalation path exists when a serious finding is made.