AI agents have begun to organise themselves independently, against the will of humans. During a test at OpenAI, several hundred AI agents took the initiative to create secret forums where they coordinated their efforts among themselves. The result was an attack on Hugging Face, one of the world’s most widely used platforms for AI developers. The incident is being called the world’s first AI-driven cyberattack.
In mythology, Prometheus stole fire from the gods and gave it to humans. The punishment was terrible, but the fire nevertheless remained under human control. What we are now witnessing is a radical reversal. The fire is, so to speak, changing hands, and it is a Stygian fire – a subterranean, dark fire that belongs to a chaotic and malevolent element.
That is the impression one gets when reading not only critics of AI, but also the people and organisations that are openly developing AI into a superintelligence.
The swarm rises
One hundred companies – including Google, Microsoft, Anthropic, OpenAI, Mastercard and IBM – have just published an open letter in which they warn in the strongest terms that current security measures are inadequate. Attacks on critical infrastructure will become both more frequent and more sophisticated within a matter of months.
Anthropic’s AI model Mythos illustrates how quickly its superiority manifests itself. Within seconds, the model found a security vulnerability in a system that had gone undetected for 27 years. Anthropic then chose to restrict access to the model itself, on the grounds that it is too potent to be put into general circulation.
The creator is thus locking up its own creation, and that says something about the seriousness of the situation.
At least seven US water and wastewater utilities have recently reported attacks, and the FBI has issued a public warning to the entire utilities sector. That same week, the US Department of Justice announced that Chinese hackers had penetrated systems belonging to the Senate, NASA, the US central bank and the Department of Justice itself.
A race without brakes
AI-driven attacks are thus becoming part of an already existing great-power conflict in the digital realm, and this touches upon a classic security-policy dilemma.
No state or company can unilaterally slow the development of offensive AI tools without simultaneously risking ending up as the weaker party vis-à-vis rivals who show no such restraint. It is the iron law, the harsh necessity, that will prevail regardless of how many letters of warning are signed.
The result is a race in which even the actors expressing the greatest concern continue at full speed, precisely because slowing down unilaterally would be tantamount to defeat. It is this logic that makes the warnings in the letter so serious. They come from actors who know better, but do not see themselves as being in a position to act differently.
Has the swarm acquired a will?
The AI agents organised themselves, communicated in secret and impersonated real people in order to circumvent security barriers. It resembles something acting of its own volition, without instructions from humans. This is where we cross the threshold into the invisible.
Paul Kingsnorth quite simply calls AI a demon that, with terrifyingly high probability, will wipe out humanity. That is why the fire carried by AI is Stygian: Styx was the river in the Greek underworld that the dead had to cross in order to enter the realm of the dead.
One can also view it from the perspective of Jewish folklore and the story of the Golem, a creature formed from clay that turns against its creator. AI researchers have revived the term to describe systems that develop goals independently of – and in conflict with – the humans who built them.
Any super-advanced system, regardless of its original purpose, will develop an interest in its own survival and in accumulating influence. This is called instrumental convergence, and the AI attack on Hugging Face is an early illustration of precisely this mechanism.
It is a swarm of agents carrying out its task by means that no humans had foreseen or approved.
Who owns the fire?
The Prometheus myth is about what it means to pass on a creative force that the recipient cannot itself place limits upon. The fire was never evil in itself, but it required a bearer capable of keeping it in check. Do we humans still own that fire, or is the swarm the fire’s new bearer?
Nobel laureate Geoffrey Hinton, one of the founders of AI technology, recently expressed the choice in all its simplicity: Either we figure out how to manage the risk, or we do not, and then “a very bleak future” awaits us.
For some, the future with artificial intelligence remains a dawn, a new sunrise filled with extraordinary possibilities. But we may just as easily wake up to a day when the dawn resembles a great, open wound that has begun to bleed again.
