What is the AI’s IQ? The number that has lost its measure

Why a language model with an IQ of 155 is not smarter than one with 140 — and why the number will be everywhere anyway.

"Intelligence is what the tests test."

— Edwin G. Boring, "Intelligence as the Tests Test It" (1923)

9. July 2026

Playground:

This article poses a harmless question—what is the AI’s IQ?—and you can watch as it falls apart while trying to answer it.

Different models, different abilities—ergo, different IQs. The logic sounds sound. It has only one flaw: it treats IQ as a measurable property. IQ is not a measurable property. It is a rank—a position within a test group that takes the same test for the same reason. Take away the group, and the number remains like a ruler from which the units have been removed. The hypothesis of this text: The question of AI’s IQ isn’t answered incorrectly; it’s asked incorrectly—and yet it will still be answered, by everyone, constantly, for many good reasons. This is no glitch. It’s structural.

In early 2023, Finnish psychologist Eka Roivainen put ChatGPT through the verbal subtests of the world’s most widely used intelligence test: vocabulary, general knowledge, and analogies. The result, extrapolated: verbal IQ: 155. Better than 99.9 percent of the people who standardized the test. Roivainen publishes the findings in Scientific American, cautiously, with all the necessary caveats.

Around the same time, the same family of models fails publicly and repeatedly at another task: counting how many “r”s are in the word “strawberry.”

An IQ of 155. And three letters are simply too many. Both are true. And that is precisely the finding.

The Hierarchical Classification

The IQ test was never designed to measure intelligence the way a thermometer measures temperature. It was designed to sort people. The mean is 100—not because anyone measured it that way, but because that’s how it’s defined. The scale is the test group. An IQ of 130 says nothing about a bright mind; it says something about a bright mind, relative to all the other less or more brilliant minds who took the same test.

The difference makes it clear: the Elo rating in chess has survived the rise of machines. Human 2850, engine 3600—the numbers are comparable because Elo counts games, and humans and machines play the same games. Elo measures a game. The IQ, however, measures membership.

So the question “What is the AI’s IQ?” is really asking: What rank does a non-member hold in a club to which they do not belong? The question of the observer returns, one level deeper—this time, the debate is not about who is allowed to measure, but whether the ruler is measuring in a space for which it was not designed.

The Network That Didn’t Play Along

Why does the test work for humans at all? In 1904, Charles Spearman found the answer, which still holds true today: In humans, almost everything correlates with almost everything else. People with a strong vocabulary, on average, are also better at solving matrix problems, memorizing number sequences, and calculating faster. Spearman called this common factor “g.” The network of these correlations is the reason why a test with forty tasks can reveal something about a thousand untested abilities. The test doesn’t measure the tasks themselves. It measures the ability to combine elements within the network.

Language models lack this network. An IQ of 155 on the verbal section, yet a total failure at counting letters—in humans, that would be a neurological emergency; in the model, it’s Tuesday instead of Monday. The abilities aren’t interconnected because they don’t stem from a shared architecture, but rather from the randomness of the training distribution. A term has been coined for this: jagged intelligence—peaks next to abysses, with no discernible transition.

The inference from the sample to the whole was the test’s sole justification. In a system without an interconnected network, the sample does not infer anything at all. The number is just there. It points everywhere and nowhere. Sometimes this way, sometimes that way.

The number that remains, regardless

And now for the part that turns the measurement question into a PI. Because that number isn’t going to disappear. It’s going to multiply.

Laboratories need comparability—to demonstrate progress, to attract capital, to recruit talent. So there are benchmarks. The media need a number that fits into a headline—“IQ 155” does that; a phrase like “inconsistent ability profile without a positive manifold” (Spearman) does not. Users need guidance without specialized study—a rank is guidance in its cheapest, fastest form. And the models themselves are optimized for the tests, partly intentionally, partly because the tests have long been embedded in the training data. Research on benchmark contamination shows: Simply rephrasing the same questions is enough to cause the IQ score to plummet. The model knew the test. Goodhart in its purest form—as soon as the metric itself becomes the goal, the measurement becomes redundant.

Each of these actors is acting rationally. Together, they produce a number that means less and less the more often it is used—and the more often it is used, the less it means, because a meaningless number serves every purpose.

The IQ-PI

A metric defined as a ranking within a population is applied to systems outside any population—and gains traction precisely because it no longer measures anything.

Everyone acts rationally: Everyone is guilty. No one is to blame.

• The labs—publish benchmark scores (rational: without comparative figures, no capital, no talent, no proof)
• The media—condense everything into a single number (rational: context must fit into a headline)
• Users—believe the number (rational: no one can read forty benchmarks themselves)
• The models—are optimized for that number (rational: optimization is what optimization does)
• Result—the industry’s most-cited metric measures the least: structurally, unintentionally

All are guilty, none is at fault.

The Number as a Projectile

And then there’s the third use, the most common one: the IQ as a projectile. No one takes the test, but everyone throws it around. Politicians attribute double-digit scores to one another, opinion columns hand out rankings by the second, and every debate has that moment when one person denies the other’s intelligence—with a number, of course; the number makes it objective. Except: In this usage, no test actually takes place. Not even a bad one—none at all. The number is invoked, not measured, and it is invoked precisely because it sounds like it came from a lab. It lends the insult the white coat of a procedure that was never carried out. The standardization, the standard sample, the protocol—the entire apparatus that underpins the ranking in the lab is both cited and bypassed in the same breath. What is actually measurable at that moment is exactly one person: the one doing the ranking. What becomes visible is their need to rank, their claim to a seat on the awards committee, their confidence that no one will ask about the protocol. The observer’s paradox in the wild—a self-appointed committee with a single member who grants herself the right to observe through the throw itself. The test proves the tester. Nothing and no one else.

Two Rulers, No Common Measurement

Let’s navigate rather than solve. So the answer to the original question is: The measurement principle isn’t bullshit. It’s context-dependent. IQ measures reliably—ranks—within a population whose abilities are correlated. It only becomes bullshit when it’s moved: when the number leaves the context in which it was constructed.

One could conceive of a separate yardstick: an AI IQ, standardized against a population of models, with a mean of 100 by definition—all neat and tidy. It would work. And it would say absolutely nothing about the gap between humans and machines—two parallel hierarchies, no common yardstick, just as the standings of two leagues that never play against each other. To measure that gap, you need a shared playing field. Melanie Mitchell and David Krakauer have mapped out the debate over what “understanding” would even mean for these systems—and there is already no consensus on the very question of what should be measured. Before measurement comes the network. Before the network comes the concept. Both are missing.

What remains is Boring’s statement from 1923: “Intelligence is what the tests test.” For a century, that sounded like an awkward admission that tacitly mended the web of human abilities. With AI, nothing mends it anymore. The tautology stands naked—and will nevertheless be cited, complete with a number and a decimal place.

This text was co-written by a system that has no IQ. Not even an unknown one. None. The number you therefore read in a headline measures only the headline.

Related Posts

On piinteract.org:

• ["Metrics Gaming"] — The moment the benchmark is the target, it measures the optimization toward itself. Goodhart, here with decimal places.
• ["See the Pattern, Not the Symptom"] — The number is the symptom. The pattern is a system that needs orientation cheaper than truth can be had.
• ["When Intelligence Becomes a Threat"] — The sorting runs by height. The danger runs on an axis no test has in stock.
• ["Name the Paradox"] — The question of AI's IQ cannot be answered. It can be named. That is more.

See also (external links):

I Gave ChatGPT an IQ Test. Here's What I Discovered (Roivainen, Scientific American, 2023) — The tester's own primary report: verbal IQ 155 — including the caveats no headline kept.

"General Intelligence," Objectively Determined and Measured (Spearman, 1904) — The original document of the web: why in humans everything correlates with everything — the sole justification of the testing principle.

Rethinking Benchmark and Contamination for Language Models with Rephrased Samples (arXiv) — The empirical evidence of contamination: rephrase the test questions and the score collapses. The model knew the exam.

The Debate Over Understanding in AI's Large Language Models (Mitchell & Krakauer, arXiv/PNAS) — The map of the prior question: before anything can be measured, it would have to be settled what understanding even means here. It isn't.

Paradoxical Interactions (PI): When rational actors consistently produce collectively irrational outcomes — not through failure, but through structure.

All are guilty. None are at fault.

Peter Senner Thinking beyond the Tellerrand

contact@piinteract.org
https://piinteract.org

Co-created with Claude (Anthropic) — two incomplete systems making each other's gaps visible.