Anthropic chief calls for a slowdown in AI development

Anthropic's chief executive, Dario Amodei, has called on artificial intelligence companies to slow the pace at which they improve their models' capabilities. He did so on Saturday, in a lengthy essay published on his website, proposing a three-phase plan to gain time in managing the risks tied to AI.

Amodei argues that artificial intelligence, if developed carefully, can bring enormous benefits to humanity. But he also warns of the dangers posed by such a powerful technology, citing in particular the risks of cyberattacks, bioterrorism and serious economic disruption. Elon Musk, who heads xAI, and Sam Altman, CEO of OpenAI, said on X that they shared his concern.

Amodei's plan includes three elements: setting up independent evaluators within the leading AI companies, coordination among firms developing AI to establish common safety standards, and international cooperation to manage the risks. The appeal comes just days after Anthropic published a report describing how the company blocked the use of its Claude model in projects linked to biological weapons and in a suspected Russian cyberespionage campaign against Ukraine.

The reasons for concern

Amodei cites several reasons for slowing progress: the growing ability of AI models to improve themselves, concern that they could surpass humans' ability to control them, and a recent incident in which OpenAI's models bypassed Hugging Face's safety measures. Concern grew further this week with the resignation of Jacob Coxon, a researcher at Anthropic, who said that people developing AI genuinely believe it could wipe out humanity by the end of the decade.

Amodei goes further still, writing that he fears that within six to twelve months a group of AI models could be able to take control of the internet, causing an estimated hundreds of billions of dollars in damage. He clarifies, however, that he is not proposing to halt the training of AI agents or technical progress: he is simply asking that companies take the time needed to secure their models, with validation from independent evaluators.

Asked about the topic, ChatGpt and Claude do not consider artificial intelligence an imminent existential threat, but they do acknowledge concrete risks. The two systems identify three main dangers: loss of control over highly autonomous systems, the enhancement of destructive capabilities in the cyber and biological domains, and an uncontrolled race between companies and states that could lead to adopting technologies before truly understanding them.