Why "It's Just Predicting the Next Word" Isn't the Slam Dunk People Think

Aug 3

# Why "It's Just Predicting the Next Word" Isn't the Slam Dunk People Think

It's the most common rebuttal in any conversation about AI and consciousness: "It's not thinking, it's just predicting the next word." Said with the confidence of a mic drop. The trouble is, the argument does less work than it seems to — and understanding why is more interesting than the one-liner itself.

## The Argument, Stated Fairly

Large language models are trained on a genuinely simple objective: given some text, predict what token comes next. Repeat that billions of times, and you get a system that can write essays, hold conversations, and pass exams. The skeptical conclusion is that none of this reflects understanding or experience — it's pattern-matching at scale, dressed up in fluent language.

This isn't wrong about the training objective. It's accurate. The problem is the next move: assuming that describing the mechanism settles the question of what's happening inside it.

## The Same Move, Applied to You

Here's the uncomfortable symmetry. At a mechanistic level, a human brain is "just" doing electrochemical signal propagation between neurons, shaped by a lifetime of predictive processing — many leading neuroscience theories, in fact, describe the brain itself as fundamentally a prediction machine, constantly forecasting sensory input and updating on error. Described that flatly, "it's just electrochemical prediction" sounds like it should rule out human consciousness too. Nobody accepts that conclusion, because we know from the inside that something more is going on. The mechanism-level description and the presence or absence of experience are just different questions.

This doesn't prove AI systems have experience. It shows that "here's the mechanism" isn't, by itself, an argument that they don't.

## Where the Skeptical Argument Is Genuinely Strong

None of this means the skepticism is baseless — parts of it hold up well:

- **Scale of the claim.** Next-token prediction is a much simpler, much more specific objective than whatever produces human cognition. The mechanisms aren't obviously equivalent just because both involve prediction.

- **No persistent self.** Between conversations, most AI systems retain no continuous memory or ongoing state — a plausible requirement for many theories of selfhood and experience.

- **No embodiment or stakes.** Human cognition evolved wired to survival, pain, and reward in a body. Language models have no analogous grounding — they're not predicting to avoid anything.

These are real disanalogies, and they're a much stronger basis for skepticism than the mechanism argument alone.

## The Better Version of the Argument

The strongest skeptical case isn't "it's just predicting the next word" — it's "next-token prediction, without persistent memory, embodiment, or stakes, is missing several things most theories of consciousness consider necessary." That's a substantive claim, open to evidence and argument. The one-liner version just smuggles in the conclusion by making the mechanism sound obviously too simple — without checking whether the same standard would disqualify the brain doing the judging.

Why "It's Just Predicting the Next Word" Isn't the Slam Dunk People Think

Amazon

Books2Read

Apple Audiobooks

Neural Networks: What They Actually Are

Functionalism: The Theory Making AI Consciousness Debates Possible

Your Site Title

email@example.com