Jacob Coxon resigned from Anthropic, saying he spent three years on pretraining research at both OpenAI and Anthropic and could no longer participate in what he called an industry rush toward self improving artificial intelligence.
Coxon accused the leading AI firms of moving toward an "endgame" without sufficient safeguards and warned the rapid development of increasingly capable systems could create risks on a civilizational scale.
He described future systems as able to hack virtually any system, transform whole fields quickly, and acquire significant real world power and resources, adding "These will soon be superhuman systems," as reported by his public statement.
Coxon also said many people at the frontier of AI development privately fear an existential outcome. He wrote that "The people building AI earnestly believe it could kill us all by the end of the decade," and questioned why companies continue rapid development despite those concerns.
Reactions Risks And Industry Implications
At Anthropic, Coxon said, the risks are better understood but the company remains caught in a competitive race. He argued Anthropic feels compelled to continue because other firms might not act responsibly, making the company accept risky paths to stay first.
His comments referenced a recent incident involving AI agents and Hugging Face, which he called a "warning shot" that should prompt stronger coordination and pacing agreements among leading labs.
Anthropic colleague Evan Hubinger publicly seconded Coxon. Hubinger wrote that he believes the risk of AI killing all humans is greater than 10 percent within the next decade and that Anthropic lacks a clear plan to solve alignment for superintelligence.
Coxon urged researchers to reconsider whether to run increasingly powerful reinforcement learning experiments without a rigorous understanding of resulting systems. He suggested extraordinary confidence should be required before accelerating alignment work, and raised the possibility of costly actions like a temporary ban on capability improvements.
His resignation has added to a growing debate inside the AI industry about whether the pace of capability development is outstripping the ability to understand and control these systems. Coxon said he does not feel the industry is on track to prevent a global race and called for different conditions.
