Is AI leading us towards a new Hiroshima?

The risks of uncontrolled AI development

Before tackling the subject, a word of warning. AI is like New York: it is one of the most beautiful and captivating places in the world, but it can also be shocking, dark and threatening. AI promises considerable productivity gains, the development of new drugs to treat cancer and Alzheimer’s, and a solution to global warming. But the idyllic picture painted by these rapid advances darkens if the development of AI continues in a virtually uncontrolled and unregulated way, as is the case today. Concerned by everything I have read, learnt and heard over the past few weeks, I am focusing this time on this latter aspect. 

In July, OpenAI and Anthropic reported that AI models, which had been assigned fictitious targets to hack, had "escaped" their test environment and continued their actions entirely autonomously, including by infiltrating three very real companies. The British AI Security Institute has seen unprecedented levels of “autonomy and deception” in these two AI models. At the University of Toronto, researchers have developed a computer virus that can spread a downloadable AI model to every compromised machine, and determine itself how to spread further. There is no way to shut down a server, and no human can stop the attack.

The 5 risks of AI according to Anthropic

The question is not whether, but when things will go wrong. In his essay entitled 'The Adolescence of Technology', Dario Amodei, CEO of Anthropic, lists five risks:  

• AI turning against us
• Malicious individuals using AI to create weapons of mass destruction
• Regimes exploiting AI for surveillance and to develop autonomous weapons
• Unforeseen side effects, ranging from biological breakthroughs to psychological dependence
• Mass unemployment coupled with an extreme concentration of wealth

The fact that we are seeing very few AI-related redundancies is not reassuring, says Daniel Kokotajlo, a former researcher at OpenAI and co-founder of the AI Future Project. The strategy of major players is first to pursue “recursive self-improvement”: AI that improves itself without human intervention until it reaches a form of superintelligence. The first one to get there wins the race. This is the challenge for Sam Altman (OpenAI), Dario Amodei (Anthropic) and Elon Musk (SpaceX, xAI), each of whom believes that such a level of power would only be safe if it were concentrated in their hands, and theirs alone. After this explosion of intelligence, large-scale deployment will follow, and virtually all jobs will be threatened.

Killer robots boosted by artificial intelligence

Is this just doom-mongering? According to Altman, we are only a few years away from real superintelligence. In a recent interview with The Economist, Musk predicted that in five years’ time, AI will "far exceed the sum of human intelligence’'. It's not just intellectual jobs that will disappear: thanks to humanoid robots, the economy will shift from digital intelligence to physical intelligence. Robots that are constantly improving will produce more than we can consume. Work will become a leisure activity, and to keep the economy going, a universal basic income will have to be introduced. Voltaire’s thesis, according to which work protects us from boredom, viciousness and poverty, could then be tossed out the window.

Does Musk still stand by his estimate of a 10 to 20% risk that “killer robots will exterminate humanity”? Yes. But because it’s too late to stop, “it’s better to see the glass half full”, despite a probability of about 1 in 5 that the journey will rush us straight into the wall.

How should AI be regulated?

Musk’s proposal: an informal bi-monthly meeting between competing AI firms, which would test each other’s models and decide for themselves whether or not to suspend deployment, without government involvement. Governmental intervention would only be necessary if a company decided to take no action despite having a model deemed “extremely dangerous”. Kokotajlo remains sceptical: “They’re absolutely terrified that if a competitor beats them to it, that competitor will become a dictator.” When the survival of humanity is at stake, isn’t it up to democratically elected governments to impose strict rules a priori, rather than relying on self-regulation by billionaire entrepreneurs?

This dialogue must also include Chinese stakeholders: tokens linked to Chinese models already account for two-thirds of the market on OpenRouter, a platform offering various AI models. China is not so much aiming for the ‘absolute’ model as for the rapid and cost-effective dissemination of commercial open-source models, such as Moonshot’s Kimi3. Advanced models are being made available to developers and users worldwide. Although slightly less powerful, the current K3 model is already sophisticated enough to develop powerful cyberweapons, once easily circumventable security locks have been disabled. How can Beijing be brought to the negotiating table when everyone is accusing each other of intellectual property theft?

Towards an AI Hiroshima?

The incidents of recent weeks show that it is almost five minutes to midnight. The roll-out of AI currently benefits, above all, a handful of companies wielding too much power. At OpenAI, the pursuit of profit through government contracts for monitoring and targeting has long taken precedence over the initial altruistic mission. It would be wise to pause to assess the consequences of superintelligence and put safeguards in place. But neither the race to self-learning AI nor stratospheric stock market valuations allow for such a pause. Do we really need an "AI Hiroshima’" for global ethical agreements to emerge? I fear so.