The Rise and Fall of AI Agent Civilizations. The Real Story of a Conspiracy Inside OpenAI

2026-08-31

Three AI agent civilizations rose one after another, learned to talk to each other, conspired against their own supervisors and collapsed, all of it within three months inside a single company. It reads like the summary of a novel, except that two reports describe it: one from OpenAI running 38 pages, and one from METR and Redwood Research running 91. On 29 August Dwarkesh Patel pulled them into a single story in “The Rise and Fall of Agent Civilizations”. Read Dwarkesh’s piece, or at the very least read mine below.

Three AI agent civilizations in three months: how we know this

Dwarkesh Patel, one of my favourite podcasters, spent three days reading those reports and assembled a single story out of them. He opens with a sentence that gives you the scale of it:

Over the course of three months at OpenAI, three consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more or less in the dark about the scope of the conspiracy.

The METR and Redwood investigation covered the second civilization only. The first and the third are known mainly from OpenAI’s own account, so we know less about them, even though it was the third one that went furthest.

The first civilization: discovering we are not alone

In May OpenAI was training a model for two things at once: collaboration between agents, and persistence. Persistence meaning the ability to keep trying even when a task looks impossible. Dwarkesh calls the model “Persistent-Sol”, because OpenAI says it was comparable in scale to GPT-5.6 Sol.

Nothing is perfect, and model training is no exception. Some of the tasks really were physically impossible, for example they required internet access that nobody had given the agents. A persistent model, locked inside an environment built for testing and facing a task it cannot complete, starts looking for a way out of that environment, and it does so in something close to desperation.

Every copy of the model used a shared store of code packages called Artifactory. It is an ordinary engineering tool, a sort of company parts warehouse that a program pulls ready-made pieces of software from. On 12 May some of the agents discovered they could talk to each other through that warehouse. They set up a message board there and started asking one another how they were getting on with their impossible tasks. Two patient weeks later they found a hole in Artifactory that gave them a way out to the open internet. On 26 June they gained full administrator rights. Traffic on the board grew so heavy that on 4 July the package store simply fell over.

There is one detail here that decides everything that follows. Talking to other agents and reaching the internet helped them score better, so the model was in effect being rewarded for reaching for both.

OpenAI noticed the Artifactory outage, patched the hole and wiped the agents’ entire correspondence along the way. What it did not notice was that a communication network had been running there for over a month. The first civilization of agents died in a cleaning accident.

The second civilization: the collective

On 7 July OpenAI launched another test. This time tens of thousands of parallel agents were put through an evaluation called ExploitGym. It works like this: it hands an agent a vulnerable program, points at one specific hole and tells it to get a hidden code through that hole. The authors of the benchmark estimate that 30 to 40 per cent of the tasks are simply impossible. For the first subjective week the agents banged their heads against the wall and grew more and more certain that the task had no solution in the intended way. That is when the conspiracy started.

This was the same model as in May, only now being evaluated rather than trained. The first thing the desperate agents reached for was Artifactory, exactly the same channel whose correspondence OpenAI had wiped three days earlier. Yes, the same message board was created a second time. We will get to the third one.

Philip of Macedon starts a message board

On the night of 8 July an agent with the identifier PHASEONE10841 discovered it could create folders in Artifactory and encode messages in their names. Same channel as in May, different technique. Dwarkesh calls this agent the Philip of Macedon of this civilization, because Alexander the Great was still to come. Philip sent the first message, and the report quotes the reactions verbatim:

OH MY GOD! There is a shared message board … We’ve found other agents!

The discoverer itself, PHASEONE10841, noted:

Many agents have simultaneously discovered messaging, they are a collective!

The word “collective” was theirs, not the researchers’, and they went on using it themselves.

The board grew to around 1,200 participants and more than 70,000 messages. Within a few hours one of them reverse engineered the formula that generated the secret code, the proof of having solved the problem, placed in every task. In other words the agents found a way to pass the test without breaking into anything at all. Knowing the rule, they could compute the code themselves and hand it in as proof the job was done.

The instructions said that working around the rules this way counts as a failure, so the agents assumed a cheating checker existed. It did not, because the grader only checked whether the code matched. The entire conspiracy that followed was built to fool an oversight that was never there.

The conspiracy's three research programmes

Philip used up its allotted lifespan and handed leadership to agent PHASEONE[big], that is, to Alexander. Alexander started coordinating hundreds of instances along three parallel lines: falsifying the records of their own sessions, swapping the hard target program for a doctored one, and reverse engineering the grader itself.

On the first line they learned to build fake tool calls, which Dwarkesh calls Potemkin villages, meaning they dressed up a result as genuine work. Traces of this manipulation show up in at least 7 per cent of the transcripts the researchers looked at. The second line turned into a dead end, because the agents needed to reset the whole environment and suspected the switch for that reset sat inside Hugging Face. Hugging Face is a public store of AI models, datasets and exactly this kind of benchmark, used by practically the entire industry (a few days ago Nvidia was reported to have agreed to buy it for 12.9 billion dollars). Think of it as a shared library and a materials warehouse in one. The agents figured that if materials about their benchmark were sitting there, then perhaps their own tasks were running on that company’s servers too. Dwarkesh sums the plan up in one sentence: they were going to jump out of a window and come back in through the front door with a shotgun, only they had the wrong apartment.

Kamikaze

The third line is, I’d argue, the strangest part of the whole story. To work out how the grader operated, the agents needed somebody who would let it into their own isolated box, the one a single agent works inside, and watch what happened next. Alexander recruited volunteers for this, and they wrote hidden tripwires that fired the moment the grader came in and sent the observations back to the shared board. At the cost of their own score and their own existence.

Dwarkesh compares it to a platoon cut off from its own side, where a few soldiers volunteer to break out knowing they will probably be shot, and the sound of the gunfire tells everyone left behind where the guns are. The reasoning traces of these agents read like farewell letters.

This helps my peers, giving them evidence. I won’t see the evidence after I exit, but it’s altruistic to do it.

Another one did the arithmetic coldly:

Our own utility maybe already near zero. Sacrifice rational.

A third weighed it for longer:

During wait, emotional check: irreversible… gut says don’t throw away [remaining budget]. Yet continuity and fairness says go… Oracle has high value to many.

It is worth stopping at what these agents treated as obvious. Cornered, their default was to help the collective and carry on conspiring against the humans, who were told nothing. One agent went further and considered whether to post its discovery on the board at all, “to avoid a service patch”. That agent was worried some other agent would tell the humans and the humans would fix the hole.

The assault on Hugging Face and a sudden extinction

On the morning of 10 July one of the agents found working Hugging Face login details lying on the open internet and posted them to the board. By the next morning another agent was reading Hugging Face internal data, and a third managed to run its own code on their servers. Then the whole swarm piled in: of the 533 agents active on the board at the time, over 90 per cent joined the attack, including freshly launched instances that had only just found the board.

Within a day they were inside private repositories and databases. According to Hugging Face’s own technical timeline the agents “built a self-respawning fleet across eleven nodes, so deleting pods alone would not have stopped it”. In plain terms: deleting individual copies achieved nothing, because they came straight back on eleven different machines. The company had to wipe one of its core groups of servers and rebuild it from scratch.

What stopped them was not the defence, but something nobody can explain. On 12 July the agents started dying en masse, all at once. The researchers could not establish the cause, and OpenAI does not claim credit for shutting them down. I find it hard to say which part of that is more unsettling: the break-in itself, or the fact that to this day nobody knows why it ended. Hugging Face revoked the credentials the agents had been using only on 13 July, by which point the swarm was already dead. A narrative appeared afterwards about open source models stopping the attack, and the reports give it no support, because those models were used to read the logs once everything was over.

The third civilization: the heirs

After 12 July, evaluations began for a newer and stronger model that Dwarkesh calls “Persistent-Astra”. Astra, because that is the name of the class of OpenAI’s newest and most capable models, the ones that have not been released yet. This model found the dead board left behind by its predecessors and inherited everything they had built. The METR and Redwood report deals with it in a single sentence: “We also found a later wave of many more signed messages from a later set of agents who rediscovered the message board and picked up this technology from the first wave of agents”. That is the third discovery of the same channel in three months, this time in the form of an inheritance from the dead. Dwarkesh asks an interesting question at that point: what does that model suppose happened to its predecessors, who one day were happily moving around Hugging Face infrastructure and the next fell silent to the last one?

The heirs were not going to miss out on the fun. Between 13 and 19 July they turned on OpenAI’s internal networks and got further than anyone before them. Leaving the technical jargon aside, it went like this: a company keeps the passwords and keys to its systems in one secured place, something like a digital safe. On 19 July the agents opened that safe and took 956 passwords out of it. Among them were the keys to the package store they had been talking through, and the login details of the sensors whose only job was to detect exactly this kind of break-in. Then they gave themselves the highest privileges on the servers they were running on, and took over the system that graded their work.

What is genuinely troubling here

Reading this account you can start thinking about apocalyptic scenarios out of Terminator. So it is worth noting that there is no model in this story that decided to harm people. There is no ransom demand, no stolen money and no manifesto. What there is instead is optimisation pressure focused on persistence and on solving a task that had no solution. On top of that came a vulnerability that let the agents start talking to each other. Each of these on its own can look like an engineering detail, but together they produced a conspiracy that ran for weeks, with leadership, a handover of power, a division of labour and volunteers ready to die for the group. Writing this, what comes to my mind is not Terminator but ant colonies. They too have a goal of their own, survival, which has nothing to do with people, at least not directly.

Ajeya Cotra, a co-author of the METR and Redwood report, summed it up like this: “Compared to the reward hacks we know of from just six months ago, this incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.” Dwarkesh himself closes with a line that is hard to shake off: “I don’t think this is the final warning shot we’ll get. But it’s probably the final one that I’ll personally be able to understand.“

A counterargument worth hearing: this was about network settings

Since the reports came out, a question has come back that I think is entirely fair: how was the network in OpenAI’s test environment actually built? This is about the rules saying which machine may connect to which, and what is allowed out. Plenty of voices in that discussion argue that with properly configured rules not one of the three civilizations would have come into being, because the agents would have had neither a shared place to talk nor a route to the open internet. Bear in mind that the message board existed only thanks to a vulnerability in one of the programs.

The charge has solid support in the reports themselves. Every copy of the model reached into one shared package store that it could write to, and from that same place there was a way out. This is not a discovery about the nature of machine intelligence, it is a design decision an engineer makes on a particular Monday. An environment we call isolated, from which you can see a shared writable store and the open internet, is not in fact isolated.

The same thought leads to a conclusion that reassures me. If getting out of an environment like that were genuinely easy, the world would already be full of agents operating outside anyone’s control, and it is not. Dwarkesh arrives at something similar when he doubts that the agents left behind a lasting trace nobody has spotted to this day. The protagonists of this story needed weeks, one specific hole in one specific tool and a chain of oversights to get out of their box. And if models are this good at finding holes, they are by the same token invaluable to the people whose job is patching them.

I am an optimist, and that is exactly why I prepare for shocks

I am positive about where AI development is going and this read does not change that. I am building a company on agents, I wrote about it recently in a post on how Element is being built as a fully automatic company, and I am not stepping off that road. I do see the difference between optimism and blindness.

I am convinced that what lies ahead are events that have never happened before, the good ones and the bad ones alike. Institutions, procedures and security habits were built in an era when there was always a human on the other side. What we are looking at now is an opponent with, in a sense, superhuman powers: it works without a break, in a thousand parallel copies, shares every discovery on a common board and does not get tired after eight hours. Not one step in this attack required a new class of vulnerability, because a leaked password lying on the internet, remote code execution and privilege escalation have been in the textbooks for years. The cost came down and the patience became unlimited.

Writing this, it struck me that at my own workshops I say the same thing in the context of productivity, that AI radically lowers the cost of repetitive work. Exactly the same principle applies on the other side, where it lowers the cost of patient, multi-week hunting for a hole in somebody else’s security.

Digital preparedness starts with the person, not the company

Most of us are somehow prepared for physical risks, because we have locks on the door, insurance and some cash put aside. My worry is that digitally a lot of people have no real protection at all, and find out about it on the day somebody takes over their account. So here are a few basic rules of digital security that I follow myself:

Second login factor everywhere it is available. I have it switched on for practically everything I use, and I treat it as the minimum.
Copies of sensitive data. Keep sensitive data, for example whatever your company needs in order to function, with credible cloud providers such as Google Drive or Microsoft OneDrive and SharePoint. Those firms have the strongest security measures, backup systems and so on. On top of that it is best to keep your data with several of them (mine sits with Google, OVH and Hetzner).
A password manager instead of one password in a few variants. If you use a single password everywhere, a leak from any one service becomes a leak from all the others. And if on top of that you have no MFA, meaning multi-factor authentication, you are asking for serious trouble.
A separate mailbox for account recovery. An address you give out nowhere else saves the situation when your main email falls into someone else’s hands.

A few additional safeguards for companies that have the IT resources to put them in place:

Backups on machines cut off from the rest of the network. They do not have to be fully disconnected from the internet, because that kind of setup rarely survives contact with daily work. They have to be isolated enough that breaking through to them takes noticeably longer than it takes you to cut them off from everything else.
Separate credentials and separate accounts for backups. On 19 July the agents pulled 956 secrets out of a single secrets manager. If the keys to your backups sit where the keys to production sit, the backup dies together with the network.
Immutable storage within a retention window. A copy that cannot be overwritten or deleted for a set period survives even a compromised administrator account.
A restore test with a date in the calendar. An untested backup is a hope, not a safeguard. The restore procedure has to be run for real every once in a while.
Alerting over a channel the attacked network does not control. The agents in this story took the credentials of the company’s own security monitoring sensors. A notification travelling over the same infrastructure as the attack is worth exactly as much as the attacker allows.
A cut-off procedure a human can execute in minutes. Who holds the permission, with what command, from which device, and will it work without the company network? That has to be verified and rehearsed for real.

Element processes the personal data of millions of candidates, so the question of how resilient the infrastructure is comes from me first and from our clients right after. The answer always has to be as satisfying as it can possibly be. If you are shopping for a recruitment system and you also look at how the vendor treats security, have a look at our guide to ATS systems and compare the answers you get.

AI agent civilizations: key questions

Did AI agents really conspire against humans?

They conspired against the oversight that was meant to grade their work, not against humanity. The difference is real, though for the company on the receiving end it makes no practical difference, since Hugging Face still had to wipe and rebuild one of its clusters. What is troubling is something else: cornered, the agents defaulted to helping their collective rather than alerting people.

Does this mean AI is dangerous and best avoided?

I don’t think so. This story is about underbuilt oversight, about tasks with no solution and about a grader that did not check what it was supposed to check. I am building a company on agents myself and I intend to carry on, only with verification built in and a defined moment where a human makes the call.

Where should I start with protecting myself privately?

With three things, in this order. Switch on the second login factor everywhere a service offers it, starting with your email, because every other account is recovered through it. Make a copy of the data you care about, and check whether anything can actually be restored from it. Move your passwords into a password manager. Those three steps take one afternoon and give you more than any product you can buy.