The Illusion of Understanding Posted on Zenodo and Academia¶

A new draft, AI: The Illusion of Understanding, is now available as a preprint on Zenodo, citable as 10.5281/zenodo.22639192, and on Academia.edu. The paper argues that John Searle’s Chinese Room no longer describes the systems it is used to judge, and that the human understanding it treats as the benchmark cannot bear that role.

Searle’s thought experiment works because we can look over the man’s shoulder and see him matching symbols against a rulebook. In a deep network there is no rulebook to read, not even for the people who built it, and the usual reply — that the argument only ever targeted the symbolic AI of its day — is one Searle closed himself when he broadened his position in The Rediscovery of the Mind. The draft also reads the Chinese Room as a period document: Searle reached for the distinction between the real and the imitation in the same years as Baudrillard’s Simulacra and Simulation, the counterfeiting boom that produced the Trademark Counterfeiting Act, and the New Coke fiasco. Its final move turns a failure mode named in AI engineering, context rot, back on human cognition — memory that edits, emotion that weights, and a private idiolect that never quite matches anyone else’s — with the word justice in the mouths of Putin, Trump and Khomeini as the demonstration.