“I have worked on AI for the last twelve years because I believe it could dramatically raise the quality of human life… But over the last few months, I have become convinced that fully addressing the risks requires even more prudence… We must slow the pace at which we improve the capabilities of AI models.” – Dario Amodei – Anthropic CEO
The tension driving contemporary AI policy is not between progress and stasis, but between unconstrained capability growth and the slower evolution of institutions able to manage it. Frontier models are now improving on timescales measured in months, while legal, technical, and organisational safeguards update over years. That divergence creates a structural risk that systems with general-purpose capabilities in code generation, biological planning, and autonomous decision making will exist before robust mechanisms are in place to constrain misuse and malfunction 1,6,12. The practical question is no longer whether AI will be transformative, but whether its trajectory can be shaped so that transformation does not arrive in the form of cascading systemic shocks.
From scaling optimism to governance pessimism
Dario Amodei has long argued that frontier AI follows predictable scaling behaviour: system capability tracks the product of compute quantity, hardware speed, and algorithmic efficiency, each of which has been compounding on roughly annual timescales 6. In his Senate testimony he notes that chip counts used in training have been increasing by roughly 2x to 5x per year, chip speeds doubling every 1 to 2 years, and algorithmic efficiency improving by about 2x per year, producing an aggregate trajectory of staggering capability growth 6. Early on, this picture fed a broadly optimistic view: if alignment and safety could keep pace with scaling, the same dynamics that deliver rapid progress in language, science, and engineering could be harnessed for large social gains. Over recent years, however, Amodei has become more sceptical that governance will track this curve. He now stresses that safety is not a discretionary brake but an institutional safety barrier without which the industry is on course for a complexity backlash cycle in which risks accumulate faster than they can be understood or mitigated 12.
Risk buckets and the frontier line
Amodei’s public interventions repeatedly organise AI risks into short-term, medium-term, and long-term buckets 6. Short-term risks revolve around privacy, bias, misinformation, and intellectual property issues visible in current systems 6. Medium-term risks, expected on roughly 2 to 3 year horizons, are more alarming: models significantly better at science and engineering could be misused to design biological agents, automate cyber-intrusions, or shift military balances 6. Long-term risks involve highly autonomous systems whose world-modelling and goal-optimising capabilities are strong enough that loss of control becomes a live possibility, including scenarios with non-trivial existential stakes 6,11. The frontier line is where medium-term and long-term buckets begin to blur; models capable of open-ended scientific reasoning and autonomous tool use create both immediate operational risk and the precursors of future misalignment. Amodei’s recent writing argues that industry trajectories are now close to that line, with scaling trends pushing towards dangerous capability thresholds on a timescale that leaves little room for compliance-by-learning from large accidents 1,13.
The shift towards pacing and embedded evaluators
The most striking development in Amodei’s stance is his move from advocating proactive regulation while defending rapid development, to explicitly calling for a slowdown in frontier capability gains. In his recent essay he proposes a three-part framework: embedded evaluators inside frontier labs, coordination among democratic governments and leading AI companies, and eventually global coordination that includes non-democratic states 1,4,13,14. The first step is operational and immediate. Anthropic is committing to provide independent evaluators with permanent, employee-level access to its systems, enabling continuous verification of safety measures, real-time incident reporting, and direct assessment of alignment during training and deployment 1,4. This is a significant departure from conventional audit models, which rely on snapshot evaluations or post-hoc red-teaming. Embedded evaluators represent a governance mechanism woven into the operational fabric of frontier labs, aiming to change incentives by making safety performance visible to external actors in near real time.
Slowing capabilities without freezing development
Contrary to caricatures that treat calls for pacing as demands for moratoria, Amodei’s framework is deliberately calibrated to slow the rate of capability improvement while maintaining progress on safety, reliability, and beneficial applications. Public statements emphasise that he is not seeking a freeze on training runs or an indefinite cap on model size, but rather a deliberate decision to build at a balanced rate that takes adequate time for alignment research, dangerous capability assessment, and external verification 1,3,9. He argues that safety and capability are tightly coupled: attempts to accelerate raw capability while relegating safety to downstream fixes are both technically naive and strategically dangerous 3,12. Under the proposed regime, frontier labs would still train more powerful models, but would accept constraints on how fast those models cross certain capability thresholds, particularly in areas like autonomous cyber operations, automated biological experimentation, and scalable deception. Slowing capabilities is therefore framed as buying time: progress will still appear rapid from the outside, yet the marginal months gained at each frontier step are used to harden governance mechanisms and test failure modes that would otherwise only be discovered in deployment 1,13.
Responsible Scaling Policy as a testbed
Anthropic’s Responsible Scaling Policy (RSP) provides the institutional backdrop to this call for pacing. The RSP is an adaptive risk governance framework built to manage catastrophic risks from frontier systems, combining capability thresholds, governance triggers, and testing requirements 7,10,15. The company commits to frequent dangerous capability testing along the compute scaling curve, meaning that as training runs approach higher compute regimes, models are repeatedly evaluated for behaviours like assisting biological weapon design or orchestrating sophisticated cyberattacks 15. If risk thresholds are crossed, the policy envisages pausing deployment, strengthening mitigations, or in extreme cases reversing releases 5,7. In effect, RSP operationalises a control system where the state variable is frontier capability and the control law seeks to keep catastrophic risk below defined tolerances while allowing useful capability growth. This implicit control structure can be thought of as a feedback loop in which measured capability C_t and risk R_t determine whether further scaling is permitted; if R_t exceeds a threshold R^*, the policy requires reducing effective scaling velocity dC_t/dt until mitigations bring R_t back under R^* 7,10,15.
Regulation as an external binding constraint
Amodei’s more recent essays and interviews emphasise that internal policies are necessary but insufficient. He warns that absent external binding constraints, frontier labs face a classic race-to-the-bottom dynamic: if one company slows to run comprehensive safety testing, competitors can seize market share by moving faster and marketing more capable models 3,12. To counter this, he advocates targeted regulation that arrives ahead of the most dangerous capabilities, not in reaction to highly visible failures 5,8,12. Proposed elements include government authority to block high-risk deployments, mandatory testing and auditing regimes for models above specified compute or capability thresholds, and something akin to a FINRA-style entity for AI that can supervise compliance across firms 5,8. Crucially, he argues that well-designed regulation can simultaneously reduce cyber, biological, and alignment risks, constrain unchecked concentration of power in frontier AI companies, and still leave room for open-weight models where risks are manageable 8. In this view, regulation is not primarily about state control of innovation but about creating a shared floor of safety standards that prevents competitive pressure from eroding safeguards.
Debates, objections, and strategic tensions
The call to slow capability improvement has triggered predictable objections. Some industry figures argue that deliberate pacing will cede geopolitical advantage to actors less constrained by safety norms or democratic accountability, particularly authoritarian states that may prioritise strategic leverage over risk management 4,14. Others worry that dominant incumbents could weaponise safety rhetoric to entrench their position, raising compliance burdens that smaller labs and open-source communities struggle to meet. There is also a technical scepticism: critics question whether dangerous capabilities can be predicted and measured reliably enough to serve as governance triggers, or whether existing evaluation methods are too narrow and gameable to underpin binding regulation. Amodei acknowledges these tensions but counters that ignoring systemic risks will eventually produce crises severe enough to provoke far more draconian responses, including abrupt bans or sweeping liability rules 1,12,13. He portrays pacing as a way to avoid that backlash by building a track record of responsible frontier management. Nonetheless, the debate remains live, with some stakeholders preferring aggressive development under lighter-touch regulation and others calling for much stronger constraints than those Anthropic proposes, including temporary pauses on training runs above certain compute scales.
Why pacing the frontier matters
The broader significance of Amodei’s statement lies in its admission that internal safety culture and incremental mitigations are no longer seen as sufficient to manage frontier risk. When a leading architect of scaling-based development publicly calls for slowing capability progress, embedding independent evaluators, and empowering regulators to block dangerous deployments, it signals that the expected trajectory of AI over the next few years plausibly includes systems with non-trivial probabilities of causing large-scale harm if mismanaged 1,6,13,15. The strategic implication is that frontier AI is moving from a phase where firms could plausibly argue that ordinary corporate governance and technical fixes were adequate, into one where industry-wide and governmental frameworks become central. Whether or not the specific pacing plan is adopted, the underlying diagnosis of a mismatch between capability growth and governance capacity is now widely reported across mainstream and specialist media 1,9,13,14. The backstory to the statement is therefore not only about one company’s policy, but about the broader recalibration of expectations among AI leaders: the frontier must be managed as a high-risk technological domain where slowing, testing, and external scrutiny are not optional add-ons, but core design parameters for a technology that could raise the quality of human life while carrying, if left ungoverned, catastrophic downside risk.
References
1. ‘We must slow the pace’: CEO of Anthropic calls for an AI slowdown – 2026-09-12 – https://www.theguardian.com/technology/2026/sep/12/we-must-slow-the-pace-ceo-of-anthropic-calls-for-an-ai-slowdown
2. Dario Amodei on X: “We Must Pace the Frontier: I’ve written a … – 2026-09-12 – https://x.com/DarioAmodei/status/2098773920774074715
3. Anthropic CEO Dario Amodei: I am not a doomsayer, but we must … – 2026-07-05 – https://bbx.com/article/542814
4. Dario Amodei proposes three-step strategy for responsible AI development – 2026-09-12 – https://cryptobriefing.com/amodei-three-step-ai-safety-framework/
5. Policy on the AI Exponential – Dario Amodei – https://darioamodei.com/post/policy-on-the-ai-exponential
6. Written Testimony of Dario Amodei, Ph.D. – https://www.judiciary.senate.gov/imo/media/doc/2023-07-26_-_testimony_-_amodei.pdf
7. Learning From Experience – 2023-11-03 – https://www.anthropic.com/news/announcing-our-updated-responsible-scaling-policy
8. Dario Amodei admits AI suffers from a crisis of trust, saying … – 2026-08-16 – https://fortune.com/2026/08/16/dario-amodei-anthropic-ai-trust-crisis-regulation-frontier-open-models-negative-views/
9. Elon Musk backs Anthropic CEO’s call to slow AI development – 2026-09-12 – https://www.thenews.com.pk/amp/1415969-elon-musk-backs-anthropic-ceo-s-call-to-slow-ai-development
10. The case for targeted regulation – Anthropic – 2023-11-03 – https://www.anthropic.com/news/the-case-for-targeted-regulation
11. Dario Amodei (Anthropic CEO) – Scaling, Alignment, & AI … – 2023-08-08 – https://www.dwarkesh.com/p/dario-amodei
12. Dario Amodei, Who Insists on Proactive Oversight … – HyperAI – 2025-12-18 – https://hyper.ai/en/news/47737
13. Anthropic CEO calls for ‘pacing the frontier’ of AI race amid safety concerns – 2026-09-12 – https://krdo.com/news/2026/09/12/anthropic-ceo-calls-for-pacing-the-frontier-of-ai-race-amid-safety-concerns/
14. Techmeme – 2026-09-12 – https://www.techmeme.com/
15. Dario Amodei’s prepared remarks from the AI Safety Summit on … – 2023-11-03 – https://www.anthropic.com/news/uk-ai-safety-summit
