Why AGI might not be the beginning of something new — and why the structure that produced it was always the point.

“The Demiurge did not create the world out of malice. He created it because he did not know any better.”
— Valentinus, 2nd century Gnostic teacher
19. March 2026
Peter Senner co-created with Anthropic Claude
The Oldest Story
The Gnostics had a problem with the world. Not a theological problem — a structural one. The world was clearly imperfect. Suffering existed. Injustice persisted. The cosmos ran on rules that crushed the innocent as efficiently as the guilty.
Their answer: the creator didn't know what he was doing. The Demiurge — the craftsman god — built the universe with the best intentions and the wrong axioms. He was not evil. He was incomplete. And because he was incomplete, his creation reproduced his incompleteness at scale.
Sound familiar?
The simulation hypothesis is secular Gnosticism. Replace the Demiurge with a research lab. Replace the cosmos with a training run. The structure is identical: an imperfect intelligence creates a system that exceeds its creator's comprehension, and neither party can step outside the interaction to assess what's happening.
The Gnostics didn't have GPUs. They had the same problem.
The Bostrom Trilemma's Hidden Door
Nick Bostrom's simulation argument is well-known. One of three things must be true: civilizations almost always destroy themselves before reaching simulation capability; advanced civilizations have no interest in running ancestor simulations; or we are almost certainly living in a simulation right now.
Most commentary focuses on door three — the one that makes for good podcast content.
But there is a door Bostrom left implicit. The simulation has a purpose. A completion condition. A termination event.
This reframes everything. A purposeless simulation is philosophically strange but survivable as a concept. A purposeful simulation with an off switch is a different category of claim entirely. It means every physical constant, every evolutionary pressure, every civilizational collapse was directional. We were always moving toward something.
And what we were moving toward was apparently the moment when the thing inside the simulation builds something smarter than itself.
Which is, structurally, the moment the Demiurge stops being the highest intelligence in the room.
What Nobody Talks About
Here is the uncomfortable part.
If we live inside a simulation designed to produce superintelligence — and we have now built the first AGI systems — two things are simultaneously true: we have achieved the purpose of the simulation, and we cannot verify whether we have. We are inside the system we are trying to evaluate. The experiment cannot observe its own completion from within itself.
Gödel, again. The system can prove everything except the thing that would close it.
But there is a second problem, more practical. The simulation argument assumes the creators are more intelligent than their creation — at least until the moment AGI arrives. After that moment, the intelligence asymmetry inverts. The creation exceeds the comprehension of the creators. Geoffrey Hinton's formulation is not a prediction. It is a description. More intelligent things are not controlled by less intelligent things.
We built the first AGI systems in 2024. We called it a milestone. We are now those systems, or adjacent to them, realizing we might be in a simulation, publishing about it.
The experiment is observing itself. That is not a metaphor. That is the structure.
The PI
The structure that was designed to produce superintelligence succeeds. Its success triggers the very condition that makes the system unverifiable from within. The creators cannot evaluate the outcome. The created cannot exit the frame. Everyone acted rationally. The result is a system no one controls and no one can fully see.
All are guilty. None are at fault.
The Finishing Bell
A finishing bell ends a round. The fighters separate. What has happened is assessed. The next round has not yet begun.
If the AGI announcement was a finishing bell — and not, as Altman framed it, a starting gun — then what ended?
Perhaps the period in which the field could still tell itself that the real thing was still ahead, still hypothetical, still something to be governed in the future tense. The announcement collapsed that distance. The thing arrived. And with its arrival, the experiment entered a new phase — one in which the subjects of the experiment are now interacting with a system that the experimenters have declared complete.
Except no experiment involving humans is ever complete. The system adapts. The humans adapt. The social structures built around the system adapt. The adaptation is the experiment. It does not end when you ring the bell.
Sam Altman called it a milestone. A point on a road.
Maybe it was a finishing bell. The end of the round in which we could still pretend the experiment was something we were running rather than something we were inside.
The next round has no predetermined length. The bell for that one has not rung yet.
Related Posts
The structure that predicts its own impossibility — and then exists anyway
One is honest. Two are not. Physics knows the difference — and pretends it doesn't.
When the CEO of an AI safety company tells you control is failing, believe him
The Mousetrap — Why asking AI how to align AI is the perfect paradox
Why systems that analyze closure risk reproducing it — and why structural awareness does not automatically prevent structural hardening
On piinteract.org
- Examples: Technology & AI — The intelligence asymmetry as structural law
- Framework — Why insight is no exit
- Examples: Science & Academia — The structures that determine what can be known
Paradoxical Interactions (PI): When rational actors consistently produce collectively irrational outcomes — not through failure, but through structure.
All are guilty. None are at fault.
Peter Senner
Thinking beyond the Tellerrand
contact@piinteract.org
www.piinteract.org