The Seduction of the Machine
Imagine the first time you asked a machine a question and it answered as though it understood you. The response arrived in fluent, confident prose, structured and contextualized, with a tone that felt almost human. It did not hesitate. It did not hedge. It did not say it was uncertain. And that is the uncomfortable truth at the center of this article: the machine speaks with an authority it does not, in any scientifically demonstrable sense, possess. In the space of a single exchange, it seemed to know you, to follow your reasoning, to anticipate what you needed next, an illusion of understanding so complete that it has become easy to mistake fluency for thought.
The assumption is seductive because the evidence for it is everywhere and immediate. Large language models (LLMs), the architecture that powers ChatGPT, Claude, Gemini, and their peers, generate text of extraordinary coherence. They pass professional examinations. They draft legal briefs. They write code that compiles. They explain quantum mechanics in plain language and, in the same session, compose a sonnet. This fluency has led even technically sophisticated observers to reach for words like ‘understanding’, ‘reasoning’, and ‘knowledge’. Yet beneath the performance, a different picture is taking shape, assembled not by critics on the margins but by the very researchers, mathematicians, and architects who built this technology. Their argument, emerging with increasing clarity from peer-reviewed journals, landmark reports, and billion-dollar funding decisions, is that language prediction and genuine intelligence are not the same thing. What follows is the evidence for that distinction and why it should change how you plan.
Figure 1: The Automation Chess Player (1770)
The player convinced audiences that a machine could think. It could not. Two centuries later, the question remains whether convincing performance should be mistaken for genuine understanding.
Source: https://en.wikipedia.org/wiki/Mechanical_Turk.
Section 1: a library that has never left the building
LLMs do not understand language; instead, they manipulate symbols that have never been grounded in experience. This is not an abstract debate but a formal scientific claim, and it has a name: the ‘symbol grounding problem’. First articulated by cognitive scientist Stevan Harnad and re-examined in a 2023 paper in the Philosophical Transactions of the Royal Society, the problem asks how abstract symbols, such as the words and tokens a machine manipulates, acquire genuine meaning.[2], [3] In a traditional symbolic system, every word is defined by other words, in a closed loop of mutual reference. A dictionary defines ‘red’ as ‘a color at the long-wavelength end of the visible spectrum’, but unless the reader has seen red, the definition is merely one symbol explaining another. Harnad’s argument is that the true meaning requires grounding in sensorimotor experience: the body, the senses, the encounter with a physical world. LLMs, by design, have none of these. A 2025 evaluation study tested thirteen of the field’s leading LLMs, including models from OpenAI, Google, and Meta, giving each one grounding-related tasks cold, with no prior examples or coaching.[4] The result held across every system tested. Not one model could connect a symbol to its real-world referent, or reason about cause and effect the way a person who has actually experienced the world can. Every model could describe the concept. None could demonstrate that it understood what the concept referred to. The study confirmed that no model achieved the contextual, causally grounded understanding that the frame and symbol grounding problems demand. The limitation is not a software bug. It is an architectural constraint.
The scale of this constraint becomes vivid when examined through human comparison. In a public lecture delivered at MIT in early 2024, Yann LeCun, a Turing Award laureate and one of the foundational architects of modern neural networks, stated that a typical LLM processes approximately 20 trillion tokens of text yet cannot match the physical-world reasoning of a four-year-old child.[5] The child, LeCun argued, has a body. She falls. She picks things up. She learns that objects persist when she looks away, that water is cold, that fire hurts, that gravity is not optional. Each lesson arrives not as a token but as a consequence of a direct, sensorimotor encounter with cause and effect. An LLM has read every text ever written about swimming. It can describe buoyancy, technique, and the physiology of hypothermia with forensic accuracy. It has never felt cold. More data will not fix this. The problem is not that the model has read too little; it is that reading is not the same kind of knowledge as feeling cold. The information missing from the LLM’s training corpus is not text that has not yet been digitized. It is experience that has never been, and cannot be, written down.
When the machine cannot help but lie
Hallucination in LLMs is not a flaw waiting to be fixed; it is built into how the system works. In January 2024, a peer-reviewed paper by Xu, Jain, and Kankanhalli formally proved this claim using results from learning theory.[6] The authors demonstrated that LLMs cannot learn all computable functions and will therefore inevitably produce outputs that are inconsistent with ground truth. This is not a claim about occasional errors in rare, unusual situations. It is a theorem, a mathematical proof that holds every time, for every sufficiently capable LLM. Mathematics establishes that for any sufficiently general LLM, there will always exist queries for which the model generates fluent, confident, and factually incorrect responses. No amount of fine-tuning, retrieval augmentation, or architectural refinement can eliminate this property. Hallucination is not a failure mode. It is a defining feature of a system that generates language by predicting the next most plausible token rather than by consulting a verified model of the world.
The practical manifestations of this structural property are now well-documented, and they are accelerating in unexpected directions. According to reporting by TechCrunch, OpenAI acknowledged that hallucination in its most advanced reasoning models is not only present but increasing.[7] On OpenAI’s own PersonQA benchmark, an internal evaluation designed to measure factual accuracy, the model o3 hallucinated in 33% of responses. Its companion model, o4-mini, performed worse still: a hallucination rate of 48%, meaning the model generated incorrect or fabricated information nearly half the time. These are not legacy systems. They are the most capable LLMs ever deployed. Imagine hiring a consultant who, when faced with a question they cannot answer, never says ‘I don’t know’, but always produces a confident, fluent, plausible-sounding response. Now imagine a consultant advising a national security council, or certifying a medical diagnosis, or drafting a financial regulation. The danger is not that the system is occasionally wrong. The danger is that it is structurally incapable of knowing when it is wrong and structurally incapable of saying so.
Running out of words to learn from
The dominant paradigm of AI development, which is basically to train larger models on more data, is approaching a physical ceiling. In a landmark 2024 analysis titled ‘Will We Run Out of Data?’, Epoch AI projected that all high-quality, publicly available human-generated text will be consumed by frontier LLM training at some point between 2026 and 2032.[8] This is not a projection about compute costs or energy infrastructure. It is about the finite stock of a specific resource: the accumulated written record of human civilization. Every book, academic paper, news article, forum thread, and digitized manuscript that has ever been made publicly accessible constitutes a corpus of knowable and limited size. LLMs train on this corpus. The largest models have already ingested the greatest portion of it. Under current training trajectories, the library will close. LLMs are a civilization that learned everything it knows from every library ever built, and the library is now running out of new shelves.
The consequences of data exhaustion extend beyond a technical bottleneck into macroeconomic territory. Bruegel, the Brussels-based economic research institute, noted in a 2025 policy brief that by mid-2024, LLMs were already encountering diminishing returns on additional training data and compute.[9] Gary Marcus, a cognitive scientist and AI researcher, reached a similar conclusion in a widely circulated analysis in late 2024.[10] The performance gains that accompanied each new generation of scaled models have begun to plateau. Much of the industry has already responded by shifting toward smaller, task-specific models and specialized agentic systems rather than simply building bigger general-purpose ones, a trend this insight returns to later. Yet national AI strategies and sovereign investment plans have been slower to adapt. Many were drafted when the assumption of continuous, near-limitless scaling still held, and few have been revised to account for a paradigm that is already changing beneath them. The Stanford HAI AI Index (2025) records US$252.3 billion in corporate AI investment in a single year.[11] The risk is not that this capital is wasted outright. The risk is that a meaningful share of it remains anchored to planning assumptions written for yesterday’s paradigm, even as the technology itself moves on.
When the architects walk out of their own building
The strongest signal that LLMs are not the endpoint of AI comes from one of the people who helped build the field. In late 2025, Yann LeCun, Turing Award laureate and former chief scientist at Meta, argued publicly that LLMs merely simulate understanding rather than genuinely comprehending the world and that they are a dead-end for achieving grounded intelligence.[12] In 2026, he left Meta to co-found AMI Labs and led a seed round of roughly US$1.03 billion to pursue world models rather than further scaling language models, a move analysts interpreted as a bet on a different architecture rather than a larger version of the current one. LeCun’s claim rests on a simple observation: a four-year-old child, he notes, has seen far more varied sensory information through vision, motion, and touch than any text-trained model can ever read and therefore builds a causal picture of the world that no amount of token prediction can reproduce. Where a language model predicts the next word, a child predicts what will happen when a glass tips off a table. This is a difference in kind rather than in degree. LeCun’s world model agenda aims to give machines that same ability to represent objects, dynamics, and physical consequences, which would move them from talking about the world to planning actions within it. When an engineer who helped design the current engine begins investing his own capital in a different engine, he is not declaring the existing machine useless. He is saying that on its own it cannot take us all the way to genuine understanding of the physical world, and that a new kind of system will be needed.
A complementary signal comes from Ilya Sutskever, whose concern lies less with understanding and more with control once such systems exist. In 2024, Sutskever, co-designer of the GPT architecture and former chief scientist at OpenAI, co-founded Safe Superintelligence Inc. with the stated mission of developing superintelligent AI that surpasses human capability while keeping safety paramount and then left OpenAI to pursue that mission full-time.[13] Within months, Safe Superintelligence raised around US$1 billion from Andreessen Horowitz, Sequoia Capital and other leading investors at a multibillion-dollar valuation, with its materials emphasizing that safety and capabilities must be advanced together as linked technical problems rather than treated as separate stages.
Sutskever does not claim that language models can never become more capable. He insists that if systems, including future world models, do achieve grounded intelligence and begin acting on rich internal representations of reality, then questions of alignment and safety must be solved at the same pace as questions of capability. In effect, LeCun is arguing that token prediction alone cannot deliver genuine understanding, and so new world modelling architectures are necessary, while Sutskever is arguing that whatever architectures come next will be dangerously incomplete if safety is not built into them from the outset. Taken together, their departures suggest that the architects of the current paradigm now doubt its sufficiency on two fronts at once. One doubts its ability to understand the world, the other doubts its ability to be safely scaled once it does, and the next wave of capital is increasingly being aimed at architectures that must answer both concerns rather than merely making the existing models larger.
From predicting language to understanding the world
The frontier beyond large language models is not speculative but already reshaping how researchers define machine intelligence. World models are one clear expression of this shift. They are systems designed to build internal representations of how physical environments behave so that a model can predict, plan, and reason about causes and consequences rather than merely choose the next word in a sentence. LeCun’s JEPA architecture, described in analyses of his world models work, is the most prominent example.[14] It learns structured states of the world and the relations between them in a way that encodes basic physics object permanence and causal links that humans acquire through direct experience. A parallel line of research, neurosymbolic AI, aims to combine the pattern recognition strength of neural networks with the rigor of symbolic logic so that systems can both recognize patterns and reason about them in a verifiable way. Together, these paradigms are a direct response to the limitations set out earlier in the insight. They are funded and institutionally serious attempts to build architectures that move beyond token prediction toward something closer to grounded understanding.
A second development reinforces the same conclusion from the perspective of practice. Small language models embedded in agentic systems are increasingly used in place of single general models in settings where precision matters more than fluency. A 2025 preprint and subsequent analysis from NVIDIA’s developer community both document the advantages of narrowly scoped models that operate in defined domains with clear constraints rather than answering every question in every domain.[15] In these architectures, clusters of specialized models handle different tasks, and their outputs are checked against tools and rules that can be audited. The LLM becomes one component among many rather than the place where all reasoning happens. The implications for the thesis of this insight are significant. In the emerging systems that matter most, language prediction is increasingly treated as a surface that wraps decisions, not as the mechanism that produces them, and that is precisely what would be expected if the machine that speaks is not the machine that thinks.
A US$96 billion question for the Arab world
The stakes of architectural succession are nowhere higher than in the Gulf states, where national competitiveness strategies have been explicitly built on the transformative promise of AI. PwC’s Global Artificial Intelligence Study projects that AI will contribute US$96 billion to the UAE’s GDP by 2030, roughly 13.6% of total output, a figure independently corroborated by Emirates NBD Research on the UAE’s AI economy.[16], [17] The UAE’s National AI Strategy, Saudi Vision 2030, and Qatar’s National AI Strategy are all, to varying degrees, premised on the assumption that the current LLM paradigm will remain durable and will continue to improve over the coming decade. Investment in data center infrastructure, high-end graphics processing units (GPUs), and AI-native government services has accelerated at a pace. These are rational decisions made in good faith on the basis of the evidence available when those strategies were drafted. The question that now emerges from the preceding sections is whether strategic planning horizons of five to ten years can safely rest on a technology that the field’s own researchers increasingly describe as structurally unable to understand the world it is helping to govern. Peer-reviewed studies and the funding decisions of the architecture’s founders together suggest that language prediction is an extraordinary surface but a finite foundation. For a region wagering tens of billions on that foundation, the real risk is not that the investment fails. It is that it succeeds on terms that the next paradigm will no longer share.
The geopolitical dimension compounds the urgency. The competition between the United States and China in AI is increasingly understood as a race not merely for current capability but for first-mover advantage in the next architectural paradigm. The World Models Race (Introl, 2026) documents the intensifying contest over who first achieves grounded, world-modeling intelligence, the capacity that LLMs demonstrably lack.[18] For a region that has invested billions in positioning itself as a global AI hub, the ability to anticipate architectural succession, not merely adopt whatever paradigm the major powers export, may be the defining strategic question of the decade. Investing in LLM infrastructure without accounting for what follows is structurally similar to building oil refinery capacity as the world moves toward renewables—not wrong today but potentially misaligned with tomorrow. The Gulf has already shown it can change course at scale through Vision era diversification. The question now is whether it will use that same capacity to pivot its AI strategy before the architecture beneath it moves on.
The machine that speaks is not the machine that thinks
Return, for a moment, to the opening encounter: the fluent, confident machine that answered as though it understood you. That experience was real. The fluency was genuine. The utility, in the right contexts, with appropriate constraints, is not in question. The LLM is an extraordinary instrument. It has compressed decades of human knowledge into a searchable, generative surface, democratizing access to expertise that once required years of professional training and significant financial resources. The Stanford HAI AI Index (2025) documents a civilizational shift in how knowledge is accessed, generated, and applied.[19] These gains are real. They are consequential. They are also insufficient, on their own, as the basis for the full architecture of trust that governments, corporations, and individuals are now being asked to extend to these systems. The evidence reviewed in this insight from the Royal Society to Epoch AI, from Bruegel to the funding decisions of the field’s most credentialed architects, converges on a single inference: that language prediction, however sophisticated, is a necessary but insufficient condition for genuine machine intelligence.
The machine that speaks is not the machine that thinks. The distance between those two sentences, measured in unsolved mathematics, ungrounded symbols, structural hallucinations, and the irreducible gap between token and experience, is the most important frontier in technology today. The world has invested US$252.3 billion in a single year to close it. The architects who built the current paradigm have walked out of their own buildings to fund something different. The data is running out. The ceiling is in view. What comes next will not be an incremental improvement on what exists. It will be a different kind of machine altogether, one that does not merely predict what we might say, but begins, at last, to understand what we mean.
[1] Njenga Kariuki, Artificial Intelligence Index Report 2025, Stanford University – Human Centered Artificial Intelligence, 1-77, 2025, Retrieved from https://hai.stanford.edu/ai-index/2025-ai-index-report/economy.
[2] Stevan Harnad, “The symbol grounding problem,” Physica D: Nonlinear Phenomena 42, no. 1-3 (1990): 335–346, https://dl.acm.org/doi/10.1016/0167-2789(90)90087-6.
[3] Ellie Pavlick, “Symbols and grounding in large language models,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 381, no. 2251 (2023), https://doi.org/10.1098/rsta.2022.0041.
[4] Shoko Oka, “Evaluating large language models on the frame and symbol grounding problems: A zero-shot benchmark,” arXiv, 2025, https://doi.org/10.48550/arXiv.2506.07896.
[5] Yann LeCun, LLMs and diffusion limitations: A 4‑year‑old child has seen 50x more information than the biggest LLMs, USA, March 10, 2024, Retrieved from https://www.youtube.com/watch?v=jmkTM2VSQoY.
[6] Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli, “Hallucination is inevitable: An innate limitation of large language models,” arXiv, 2024, https://doi.org/10.48550/arXiv.2401.11817.
[7] Vincent, J., “OpenAI’s new reasoning AI models hallucinate more,” TechCrunch, 2025, https://techcrunch.com/2025/04/18/openais-new-reasoning-ai-models-hallucinate-more/.
[8] Pablo Villalobos, Tamay Besiroglu et al., “Will we run out of data to train large language models? Limits of LLM scaling based on human‑generated data,” Epoch AI, 2024, https://epoch.ai/publications/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data.
[9] Bertin Martens, How DeepSeek has changed artificial intelligence and what it means for Europe, Bruegel, March 20, 2025, https://www.bruegel.org/policy-brief/how-deepseek-has-changed-artificial-intelligence-and-what-it-means-europe.
[10] Gary Marcus, “Confirmed: LLMs have indeed reached a point of diminishing returns,” Marcus on AI (Substack), November 8, 2024, Retrieved from Substack, https://garymarcus.substack.com/p/confirmed-llms-have-indeed-reached.
[11] Kariuki, Artificial Intelligence Index Report 2025.
[12] “AI godfather warns language models are a ‘dead end,’ says even a house cat understands the world better,” Ynetnews, November 16, 2025, https://www.ynetnews.com/tech-and-digital/article/bk9yu1fgwg.
[13] Safe Superintelligence Inc., Safe Superintelligence Inc. USA, 2024, https://ssi.inc/.
[14] Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas, “Self‑supervised learning from images with a Joint‑Embedding Predictive Architecture,” arXiv, 2023, https://doi.org/10.48550/arXiv.2301.08243.
[15] Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, Pavlo Molchanov, “Small language models are the future of agentic AI,” arXiv, 2025, https://doi.org/10.48550/arXiv.2506.02153.
[16] “The potential impact of AI in the Middle East,” PwC Middle East, 2018, https://www.pwc.com/m1/en/publications/potential-impact-artificial-intelligence-middle-east.html.
[17] Emirates NBD Research, “The USD 277bn impact of Artificial Intelligence in the GCC,” May 30, 2018, https://www.emiratesnbdresearch.com/en/articles/the-usd-277bn-impact-of-artificial-intelligence-in-the-gcc.
[18] Blake Crosley, “World Models Race 2026: How LeCun, DeepMind, and World Labs Are Redefining the Path to AGI,” Introl, January 2, 2026, https://introl.com/blog/world-models-race-agi-2026.
[19] Kariuki, Artificial Intelligence Index Report 2025.
