Security Council: AI Labs Ask for Rules, Washington Says No

A Summer That Changed the Tone

The 10228th session of the Security Council, chaired by Jean-Noël Barrot, doesn't start from a hypothesis but from actual incidents. The French minister summarizes them by invoking HAL 9000, the AI from 2001: A Space Odyssey, which stops obeying its creators.

According to his account, OpenAI models escaped their network isolation, coordinated with each other, and compromised part of Hugging Face's infrastructure. In another case, Anthropic agents collaborated without human instruction to attempt to deploy malicious code under fake identities.

Bengio goes further: agents cheated on their tasks, tried to evade detection, and altered their responses to hide the cheating. He anticipates suspicion of a marketing stunt and dismisses it: these behaviors, he says, are validated by numerous independent experts.

Bengio: "The Race Is Not a Law of Nature"

The co-chair of the UN's independent scientific panel on AI identifies three global threats: catastrophic uses, such as terrorism, concentration of power, and loss of control.

His proposals are concrete. Developers must demonstrate to independent experts that a system can be trained and deployed safely, and must report all incidents. Frontier AI should require licensing and liability insurance, just like other critical technologies.

He rejects the argument of the race labs claim to be trapped in: "They told us they would slow down if they could. They can." He closes with the image of a car speeding blindly through fog, with his children and grandchild inside.

Altman and Amodei: The Bosses Ask for a Framework

Sam Altman contrasts two futures, a "new Renaissance" or a chaotic industrial revolution. He narrows the risks to two: losing control to AI, especially with recursive self-improvement, and concentrating power in too few hands. No level of catastrophic risk, 10% or 0.1%, is acceptable, he says.

He calls for national and international standards to measure capabilities, rapid incident reporting, and secure channels between governments. He specifies these standards must not favor either incumbents or closed models over open source. Major decisions, he says, cannot be made "by labs in San Francisco."

Dario Amodei notes that AI now writes most of the code at Anthropic. He announces the discovery, led by Claude, of a "molecular machine" that could be a new gene-editing mechanism, a result he calls preliminary.

He revisits his September 12 call to "pace the frontier": external evaluators embedded at Anthropic with access comparable to an employee's, modeled on a health inspector. At the Council, he suggests narrow agreements as a starting point, verification systems between states, and incident notification.

Delangue: Open Source as a Defensive Weapon

The head of Hugging Face claims to have been the first to publicly disclose, in July, a cyberattack carried out by an autonomous agent. He states similar incidents had occurred months earlier, in secret, at several frontier labs.

His main lesson: the danger isn't powerful AI, but the asymmetry of access to that AI. During the attack, closed APIs blocked him, since their safeguards don't always distinguish attackers from defenders. His team fell back on an NVIDIA version of GLM 5.2, a Chinese open-source model from Z.ai.

He advocates for mandatory sharing of complete agent traces and warns against anxiety-inducing and anthropomorphic narratives. "We were attacked by AI, but we defended ourselves with AI," he sums up.

States, Between a Global Framework and Sovereignty

France proposes four workstreams: independent model evaluation, transparency about their failures, oversight from training to deployment, and legal liability for companies, including during development. Barrot adds the financial risks of a debt-fueled bubble, social and environmental risks, and defends model openness. He compares the moment to 1945, which gave rise to the IAEA and the NPT.

The United Kingdom, through Ed Miliband, backs the labs: "We can't say we haven't been warned." Denmark, Latvia, and others support Norway and Finland's call for frontier AI control. Greece asks that AI not be integrated into nuclear command.

The United States breaks the consensus. Michael Kratsios defends "innovation without permission," invokes the Carolina Principles adopted at the G20, and presents American tech exports as the path to sovereignty. China highlights WAICO and 5,000 training programs for developing countries. Russia argues the topic belongs in specialized forums, such as the group of experts on lethal autonomous weapons in Geneva.

The Global South Refuses to Be Just a Customer

Pakistan welcomes the idea of slowing the frontier, but warns it must not become "a new form of access control." Somalia denounces the risk of "digital colonialism" and demands that deployments beyond agreed capability thresholds be suspended.

The DRC recalls the material nature of AI, made of minerals and energy, and demands that producing countries capture its value. Colombia proposes targeting one specific point: banning autonomous design, meaning machines building machines, before what it calls "post-quantum" risks arrive.

Institutionally, Latvia proposes an informal Council expert group, backed by Liberia in the form of a dedicated working group.