“What I think we have to do going forward…is figure out ways to remove some of the knowledge and to keep what I call this cognitive core. It’s this intelligent entity that is stripped from knowledge but contains the algorithms and contains the magic of intelligence and problem-solving and the strategies of it and all this stuff.” – Andrej Karpathy – Ex-OpenAI, Ex-Tesla AI
Andrej Karpathy’s observation about the need to “strip away knowledge whilst retaining the cognitive core” represents one of the most penetrating insights into contemporary artificial intelligence development. Speaking on Dwarkesh Patel’s podcast in October 2025, Karpathy—formerly a leading figure at both OpenAI and Tesla’s Autopilot programme—articulated a fundamental tension at the heart of modern AI: the current generation of large language models have become prodigious memorisers, yet this very capability may be constraining their potential for genuine intelligence.
The Paradox of Pre-training
To comprehend Karpathy’s thesis, one must first understand the architecture of contemporary AI systems. Large language models are trained on vast corpora—often 15 trillion tokens or more—through a process called pre-training. During this phase, models learn to predict the next token in a sequence, effectively compressing the entire internet into their neural networks. Karpathy describes this compressed representation as only “0.07 bits per token” for a model like Llama 3 70B, highlighting the extraordinary degree of compression occurring.
This compression serves two distinct functions, which Karpathy carefully delineates. First, models accumulate factual knowledge—the content of Wikipedia articles, the specifics of historical events, the details of scientific papers. Second, and more crucially, they develop what Karpathy terms “algorithmic patterns”—the capacity for in-context learning, the ability to recognise and complete patterns, the fundamental mechanisms of reasoning itself.
The problem, as Karpathy sees it, is that contemporary models have become too adept at the former whilst the latter remains the true seat of intelligence. When a model can regurgitate passages verbatim or recite obscure facts, it demonstrates remarkable memory. But this same capability creates what he calls a “distraction”—the model becomes reliant on its hazy recollections of training data rather than developing robust reasoning algorithms that could operate independently of specific factual knowledge.
The Cognitive Core Concept
Karpathy’s proposed solution is to isolate and preserve what he terms the “cognitive core”—an intelligent entity stripped of encyclopaedic knowledge but retaining the fundamental algorithms of problem-solving, the strategies of thought, and what he describes as “the magic of intelligence.” This concept represents a profound shift in how we conceptualise artificial intelligence.
Consider the analogy to human cognition. Humans are remarkably poor memorisers compared to AI systems. Present a human with a random sequence of numbers, and they’ll struggle after seven or eight digits. Yet this apparent limitation forces humans to develop robust pattern-recognition capabilities and abstract reasoning skills. We’re compelled to “see the forest for the trees” precisely because we cannot memorise every individual tree.
Karpathy suggests that AI systems would benefit from similar constraints. A model with less memory but stronger reasoning capabilities would be forced to look up factual information whilst maintaining sophisticated algorithms for processing that information. Such a system would more closely resemble human intelligence—not in its limitations, but in the way those limitations drive the development of generalisable cognitive strategies.
The implications extend beyond mere technical architecture. Karpathy envisions cognitive cores as compact as one billion parameters—potentially even smaller—that could operate as genuine reasoning engines rather than glorified databases. These systems would “know that they don’t know” when confronted with factual questions, prompting them to retrieve information whilst applying sophisticated analysis. The result would be AI that thinks more than it remembers, that reasons rather than recites.
From Evolution to Engineering: The Path Not Taken
Karpathy’s perspective on AI development diverges sharply from what he calls the “Richard Sutton viewpoint”—the notion that we should build AI systems analogous to biological intelligence, learning from scratch through reinforcement learning in the manner of animals. Instead, Karpathy argues we’re building what he evocatively describes as “ghosts” or “spirit entities”—ethereal intelligences that emerge from imitating human-generated text rather than evolving through environmental interaction.
This distinction illuminates a crucial divergence in AI philosophy. Biological intelligence, as embodied in animals, emerges from evolution operating over millions of years, with vast amounts of capability “baked in” to neural circuitry. A zebra foal runs within minutes of birth not through reinforcement learning but through evolutionary encoding. Humans similarly arrive with substantial cognitive machinery pre-installed, with lifetime learning representing maturation and refinement rather than learning from first principles.
By contrast, contemporary AI systems learn through what Karpathy terms “crappy evolution”—pre-training on internet documents serves as a compressed, accelerated alternative to evolutionary optimisation. This process creates entities fundamentally different from biological intelligence, optimised for different tasks through different mechanisms. The current approach imitates the products of human intelligence (text, code, conversations) rather than replicating the developmental process that creates intelligence.
The Limits of Current Learning Paradigms
Karpathy’s critique extends to reinforcement learning, which he describes with characteristic bluntness as “terrible.” His concerns illuminate deep problems in how AI systems currently learn from experience. In reinforcement learning, a model generates hundreds of solution attempts, and those that arrive at correct answers have every intermediate step up-weighted, whilst failed attempts are down-weighted. Karpathy calls this “sucking supervision through a straw”—extracting minimal learning signal from vast amounts of computational work.
The fundamental issue is noise. When a solution works, not every step along the way was necessarily correct or optimal. The model may have taken wrong turns, pursued dead ends, or stumbled upon the answer despite flawed reasoning. Yet reinforcement learning broadcasts the final reward across the entire trajectory, reinforcing both good and bad reasoning indiscriminately. Karpathy notes that “you may have gone down the wrong alleys until you arrived at the right solution,” yet every mistaken step gets marked as something to do more of.
Humans, by contrast, engage in sophisticated post-hoc analysis. After solving a problem, we reflect on which approaches worked, which didn’t, and why. We don’t simply label an entire problem-solving session as “good” or “bad”—we dissect our reasoning, identify productive and unproductive strategies, and refine our approach. Current AI systems lack this reflective capacity entirely.
This limitation connects to broader questions about how AI systems might achieve continual learning—the ability to genuinely learn from ongoing experience rather than requiring massive retraining. Karpathy suggests that humans engage in a nightly “distillation phase” during sleep, processing the day’s experiences and integrating insights into long-term knowledge. AI systems have no equivalent mechanism. They simply restart from the same state each time, unable to evolve based on individual experiences.
Model Collapse and the Entropy Problem
A subtle but critical concern in Karpathy’s analysis is what he terms “model collapse”—the tendency of AI systems to produce outputs that occupy “a very tiny manifold of the possible space of thoughts.” Ask ChatGPT to tell a joke repeatedly, and you’ll receive the same three jokes. Request reflection on a topic multiple times, and you’ll observe striking similarity across responses. The models are “silently collapsed,” lacking the entropy and diversity that characterises human thought.
This phenomenon creates profound challenges for synthetic data generation, a technique labs use to create additional training material. If models generate training data for themselves or subsequent models, this collapsed distribution gradually dominates the training corpus. Training on one’s own outputs creates a dangerous feedback loop—each generation becomes less diverse, more stereotyped, more “collapsed” than the last. Karpathy suggests this may not even be a solvable problem, noting that humans similarly “collapse over time,” becoming more rigid and less creative as they age, revisiting the same thoughts and patterns with decreasing learning rates.
The contrast with children is illuminating. Young minds, not yet “overfitted” to the world, produce shocking, creative, unexpected responses precisely because they haven’t collapsed into standard patterns of thought. This freshness, this maintenance of high entropy in cognitive processes, may be essential to genuine intelligence. Yet our current training paradigms actively work against it, rewarding convergence towards common patterns and penalising deviation.
The Decade of Agents: Why Progress Takes Time
When Karpathy states this will be “the decade of agents” rather than “the year of agents,” he draws on hard-won experience from five years leading Tesla’s Autopilot programme. His insights into why artificial intelligence deployment takes far longer than demonstrations suggest carry particular weight given this background.
The central concept is what Karpathy calls “the march of nines.” Getting something to work 90% of the time—the level typically showcased in demonstrations—represents merely the first nine in “99.9%.” Each additional nine requires equivalent effort. During his tenure at Tesla, the team progressed through perhaps two or three nines over five years. More crucially, numerous nines remain before self-driving cars achieve true autonomy at scale.
This pattern isn’t unique to autonomous vehicles. Karpathy argues it applies across safety-critical domains, including software engineering. When code errors can leak millions of Social Security numbers or create critical security vulnerabilities, the cost of failure becomes prohibitively high. The demo-to-product gap widens dramatically. What works impressively in controlled conditions fails in countless edge cases when confronting reality’s full complexity.
Waymo’s experience illustrates this challenge. Despite providing “perfect drives” as early as 2014, the company still operates limited deployments requiring elaborate teleoperation infrastructure and supervision. Humans haven’t been removed; they’ve been rendered invisible, beaming in remotely to handle edge cases. The technology lives in a “pulled-back future”—functional but not yet economical, capable but not yet scalable.
Contemporary AI agents face analogous challenges. Whilst Claude and GPT-5 Pro demonstrate remarkable capabilities, they remain what Karpathy characterises as “elementary school students”—savants with perfect memory but lacking robust reasoning across all necessary dimensions. They’re “cognitively deficient” in ways users intuitively recognise even if they can’t articulate precisely what’s missing.
The Software Engineering Puzzle
Perhaps no domain better illustrates the puzzling contours of current AI capabilities than software engineering. Karpathy notes, somewhat ruefully, that whilst these systems were meant to enable “any economically valuable task,” API revenue remains “dominated by coding.” This supposedly general intelligence overwhelmingly excels at one specific domain.
This concentration isn’t accidental. Code enjoys unique properties that make it ideal for current AI architectures. Software development has always operated through text—terminals, editors, version control systems all manipulate textual representations. LLMs, trained on internet text, encounter code as a native format. Moreover, decades of infrastructure exist for handling code textually: diff tools for showing changes, IDEs for navigation, testing frameworks for verification.
Contrast this with domains lacking such infrastructure. Creating presentations involves spatial arrangement and visual design—there’s no “diff” for slides that elegantly shows modifications. Many knowledge work tasks involve physical documents, in-person interactions, or tacit knowledge that resists textual representation. These domains haven’t been pre-optimised for AI interaction in the way software development has.
Yet even in coding, Karpathy remains sceptical of current capabilities for cutting-edge work. When building nanoChat, a repository implementing a complete ChatGPT clone in simplified form, he found AI tools valuable for autocomplete and handling familiar patterns but inadequate for novel architectural decisions. The models kept trying to impose standard approaches when he deliberately chose non-standard implementations. They couldn’t comprehend his custom solutions, constantly suggesting deprecated APIs and bloating code with unnecessary defensive programming.
This points to a deeper truth: current models excel at reproducing common patterns from their training data but struggle with code “that has never been written before”—precisely the domain of frontier AI research itself. The recursive self-improvement that some forecast, where AI systems rapidly enhance their own capabilities, founders on this limitation. Models can accelerate work within established paradigms but cannot yet pioneer truly novel approaches.
The Trajectory of Intelligence Explosion
Karpathy’s perspective on potential intelligence explosions diverges sharply from both pessimistic and optimistic extremes. He sees AI not as a discrete, alien technology but as a continuation of computing’s evolution—part of an ongoing automation trend stretching back through compilers, high-level programming languages, and computer-aided design tools. From this view, the “intelligence explosion” has already been occurring for decades, visible in the exponential GDP growth curve that represents accumulated automation across countless domains.
This framing leads to counterintuitive predictions. Rather than expecting AI to suddenly accelerate economic growth from 2% annually to 20%, Karpathy suggests it will enable continued progress along the existing exponential trajectory. Just as computers, the internet, and mobile phones transformed society without producing visible discontinuities in aggregate growth statistics, AI will diffuse gradually across industries, maintaining rather than disrupting established growth patterns.
This gradualism doesn’t imply insignificance. The compounding effects of sustained exponential growth produce extraordinary transformation over time. But it does suggest that simple extrapolations from impressive demonstrations to imminent superintelligence misunderstand how technology integrates into society. There will be no discrete moment when “AGI” arrives and everything changes. Instead, we’ll experience continuous advancement in capabilities, continuous expansion of automation, and continuous adaptation of society to new technological possibilities.
The analogy to the Industrial Revolution proves instructive. That transformation didn’t occur through a single breakthrough but through cascading improvements across multiple technologies and practices, gradually shifting society from 0.2% annual growth to 2%. Similarly, AI’s impact will emerge through countless incremental deployments, each automating specific tasks, enabling new workflows, and creating feedback loops that accelerate subsequent progress.
The Human Element: Education in an AI Future
Karpathy’s work on Eureka, his educational initiative, reveals his deepest concerns about AI’s trajectory. He fears not that AI will fail but that “humanity gets disempowered by it,” relegated to the sidelines like the portly, passive citizens of WALL-E. His solution lies in radically reimagining education around the principle that “pre-AGI education is useful; post-AGI education is fun.”
The analogy to fitness culture illuminates this vision. Nobody needs physical strength to manipulate heavy objects—we have machines for that. Yet gyms proliferate because exercise serves intrinsic human needs: health, aesthetics, the satisfaction of physical mastery. Similarly, even in a world where AI handles most cognitive labour, humans will pursue learning for its inherent rewards: the pleasure of understanding, the status of expertise, the deep satisfaction of mental cultivation.
But achieving this vision requires solving a technical problem: making learning genuinely easy and rewarding. Currently, most people abandon learning because they encounter material that’s too difficult or too trivial, bouncing between frustration and boredom. Karpathy describes the experience of working with an expert language tutor who maintained a perfect calibration—always presenting challenges at the edge of current capability, never boring, never overwhelming. This created a state where “I was the only constraint to learning,” with knowledge delivery perfectly optimised.
Replicating this experience at scale represents what Karpathy sees as education’s great technical challenge. Current AI tutors, despite their sophistication, remain far from this standard. They can answer questions but cannot probe understanding, identify gaps, or sequence material to create optimal learning trajectories. The capability exists in exceptional human tutors; the challenge lies in encoding it algorithmically.
Yet Karpathy sees this challenge as tractable. Just as AI has transformed coding through autocomplete and code generation, it will eventually transform education through personalised, responsive tutoring. When learning becomes “trivial”—not in the sense of requiring no effort but in the sense of encountering no artificial obstacles—humans will pursue it enthusiastically. Not everyone will become an expert in everything, but the ceiling on human capability will rise dramatically as the floor on accessibility descends.
The Physics of Understanding: Karpathy’s Pedagogical Philosophy
Karpathy’s approach to teaching reveals principles applicable far beyond AI. His background in physics instilled what he describes as finding “first-order terms”—identifying the essential, dominant factors in any system whilst recognising that second and third-order effects exist but matter less. This habit of abstraction, of seeing spherical cows where others see only messy complexity, enables the creation of minimal, illustrative examples that capture phenomena’s essence.
MicroGrad exemplifies this approach perfectly. In 100 lines of Python, Karpathy implements backpropagation—the fundamental algorithm underlying all neural network training. Everything else in modern deep learning frameworks, he notes, is “just efficiency”—optimisations for speed, memory management, numerical stability. But the intellectual core, the actual mechanism by which networks learn, fits in 100 comprehensible lines. This distillation makes the previously arcane accessible.
The broader principle involves “untangling knowledge”—reorganising understanding so each concept depends only on what precedes it. This creates “ramps to knowledge” where learners never encounter gaps or leaps that would require them to take claims on faith. The famous transformer tutorial embodies this, beginning with a simple bigram model (literally a lookup table) and progressively adding components, each motivated by solving a specific limitation of what came before.
This approach contrasts sharply with the standard academic practice of presenting solutions before establishing problems, or introducing abstractions before concrete examples. Karpathy sees such approaches as, in his words, “a dick move”—they rob learners of the opportunity to grapple with challenges themselves, to develop intuition about what solutions might work, and to appreciate why particular approaches succeed where alternatives fail. The pedagogical crime isn’t challenging students; it’s presenting answers without first establishing questions.
Leading Theorists: The Intellectual Lineage
Richard Sutton and the Bitter Lesson
Richard Sutton, a pioneering reinforcement learning researcher, articulated what became known as “the bitter lesson”—the observation that simple, scalable methods leveraging computation consistently outperform approaches incorporating human knowledge or structural assumptions. His perspective suggests that the path to artificial general intelligence lies through learning algorithms powerful enough to discover structure from experience, much as evolution discovered biological intelligence.
Sutton’s famous assertion that “if you got to the squirrel, you’d be most of the way to AGI” reflects this view. Animal intelligence, in his framework, represents the core achievement—the fundamental learning algorithms that enable organisms to navigate environments, solve problems, and adapt to challenges. Human language and culture, whilst impressive, represent relatively minor additions to this foundation.
Karpathy respectfully dissents. His “we’re building ghosts, not animals” formulation captures the divergence: current AI systems don’t replicate the learning processes that create biological intelligence. They imitate the products of human intelligence (text, code, reasoning traces) rather than replicating its developmental origins. This distinction matters profoundly for predicting AI’s trajectory and understanding its capabilities and limitations.
Geoffrey Hinton and the Neural Network Renaissance
Geoffrey Hinton, often termed the “godfather of AI,” pioneered the neural network approaches that underpin contemporary systems. His persistence through decades when neural networks were unfashionable, his development of backpropagation techniques, and his later work on capsule networks and other architectures established the foundation for today’s large language models.
Karpathy studied directly under Hinton at the University of Toronto, experiencing firsthand the intellectual ferment as deep learning began its ascent to dominance. Hinton’s influence appears throughout Karpathy’s thinking—the emphasis on learning from data rather than hand-crafted rules, the focus on representation learning, the conviction that scale and simplicity often trump elaborate architectural innovations.
Yet Karpathy’s view extends beyond his mentor’s. Where Hinton focused primarily on perception (particularly computer vision), Karpathy grapples with the full scope of intelligence—reasoning, planning, continual learning, multi-agent interaction. His work synthesises Hinton’s foundational insights with broader questions about cognitive architecture and the nature of understanding itself.
Yann LeCun and Convolutional Networks
Yann LeCun’s development of convolutional neural networks in 1989 represented the first successful application of gradient descent to real-world pattern recognition. His work on handwritten digit recognition established core principles: the power of hierarchical feature learning, the importance of translation invariance, the value of specialised architectures for specific domains.
Karpathy’s reconstruction of LeCun’s 1989 network, time-travelling 33 years of algorithmic improvements, reveals his appreciation for this lineage. He found that pure algorithmic advances—modern optimisers, better architectures, regularisation techniques—could halve error rates. But achieving further gains required more data and more computation. This trinity—algorithms, data, compute—advances in lockstep, with no single factor dominating.
This lesson shapes Karpathy’s predictions about AI’s future. He expects continued progress across all three dimensions, with the next decade bringing better algorithms, vaster datasets, more powerful hardware, and more efficient software. But no breakthrough in any single dimension will produce discontinuous acceleration. Progress emerges from the intersection of many incremental improvements.
The Broader Intellectual Context
The debate Karpathy engages extends beyond specific individuals to fundamental questions about intelligence itself. Does intelligence arise primarily from general learning algorithms (the Sutton view) or from accumulated structure and innate mechanisms (the evolutionary perspective)? Can we build intelligence by imitating its products (the current LLM approach) or must we replicate its developmental processes? Will artificial intelligence remain fundamentally tool-like, augmenting human capability, or evolve into genuinely autonomous agents pursuing their own goals?
These questions connect to century-old debates in psychology and cognitive science between behaviourists emphasising learning and nativists emphasising innate structure. They echo discussions in evolutionary biology about the relative roles of genetic determination and developmental plasticity. They parallel arguments in philosophy of mind about whether intelligence requires embodiment or can exist as pure information processing.
Karpathy’s position threads between extremes. He acknowledges both the power of learning from data and the necessity of architectural structure. He recognises both the distinctiveness of AI systems and their illuminating analogies to biological intelligence. He balances optimism about AI’s potential with realism about current limitations and the difficulty of translating demonstrations into robust, deployed systems.
The Cognitive Core in Context: A New Paradigm for Intelligence
The concept of a cognitive core stripped of factual knowledge represents more than a technical proposal—it’s a reconceptualisation of what intelligence fundamentally is. Rather than viewing intelligence as encompassing both reasoning algorithms and accumulated knowledge, Karpathy proposes treating these as separate, with reasoning capability as the essence and factual knowledge as external resources to be accessed rather than internalised.
This separation mirrors certain aspects of human cognition whilst diverging in others. Humans do maintain a distinction between knowing how to think and knowing specific facts—we can reason about novel situations without direct experience, applying general problem-solving strategies learned in one domain to challenges in another. Yet our factual knowledge isn’t purely external; it shapes the very structure of our reasoning, creating rich semantic networks that enable rapid, intuitive judgement.
The proposal to strip AI systems down to cognitive cores involves accepting tradeoffs. Such systems would need to perform external lookups for factual information, introducing latency and dependency on knowledge bases. They would lack the pattern-matching capabilities that arise from vast memorisation, potentially missing connections between superficially unrelated domains. They might struggle with tasks requiring seamless integration of many small facts, where lookup costs would dominate processing time.
Yet the gains could prove transformative. A genuine cognitive core—compact, efficient, focused on algorithmic reasoning rather than fact retrieval—could operate in settings where current models fail. Edge deployment becomes feasible when models don’t require storing terabytes of parameters. Personalisation becomes practical when core reasoning engines can be fine-tuned or adapted without retraining on entire knowledge corpora. Interpretability improves when reasoning processes aren’t obscured by retrieval of memorised patterns.
Most profoundly, genuine cognitive cores might avoid the collapse and loss of entropy that plagues current models. Freed from the burden of maintaining consistency with vast memorised datasets, such systems could explore more diverse solution spaces, generate more varied outputs, and maintain the creative flexibility that characterises human cognition at its best.
Implications for the Decade Ahead
Karpathy’s decade-long timeline for agentic AI reflects hard-earned wisdom about technology deployment. His experience with autonomous vehicles taught him that impressive demonstrations represent merely the beginning of a long productisation journey. Each additional “nine” of reliability—moving from 90% to 99% to 99.9% accuracy—requires comparable effort. Safety-critical domains demand many nines before deployment becomes acceptable.
This reality shapes expectations for AI’s economic impact. Rather than sudden disruption, we’ll witness gradual diffusion across domains with varying characteristics. Tasks that are repetitive, well-defined, purely digital, and allowing high error rates will automate first. Call centre work exemplifies this profile—short interaction horizons, clear success criteria, limited context requirements, tolerance for occasional failures that human supervisors can catch.
More complex knowledge work will resist automation longer. Radiologists, consultants, accountants—professionals whose work involves lengthy timescales, subtle judgements, extensive context, and high costs of error—will see AI augmentation before replacement. The pattern will resemble Waymo’s current state: AI handling routine cases whilst humans supervise, intervene in edge cases, and maintain ultimate responsibility.
This graduated deployment creates an “autonomy slider”—a continuous spectrum from pure human operation through various degrees of AI assistance to eventual full automation. Most jobs won’t flip discretely from human to machine. Instead, they’ll slide along this spectrum as AI capabilities improve and organisations develop confidence in delegation. This process will unfold over years or decades, not months.
The economic implications differ from both optimistic and pessimistic extremes. We won’t see overnight mass unemployment—the gradual nature of deployment, the persistence of edge cases requiring human judgement, and society’s adaptation through creating new roles all mitigate disruption. But neither will we see disappointing underutilisation—the compound effect of many small automations across countless tasks will produce genuine transformation.
The Path Forward: Research Priorities
Karpathy’s analysis suggests several critical research directions for developing robust, capable AI systems. First, developing methods to isolate cognitive cores from memorised knowledge whilst maintaining reasoning capability. This might involve novel training objectives that penalise rote memorisation whilst rewarding generalisation, or architectural innovations that separate knowledge storage from reasoning mechanisms.
Second, creating effective continual learning systems that can distil experience into lasting improvements without catastrophic forgetting or model collapse. This requires moving beyond simple fine-tuning toward something more akin to the reflection and consolidation humans perform during sleep—identifying patterns in experience, extracting lessons, and integrating insights whilst maintaining diversity.
Third, advancing beyond current reinforcement learning to richer forms of learning from experience. Rather than broadcasting sparse reward signals across entire trajectories, systems need sophisticated credit assignment that identifies which reasoning steps contributed to success and which didn’t. This might involve explicit review processes where models analyse their own problem-solving attempts, or meta-learning approaches that learn how to learn from experience.
Fourth, developing multi-agent systems with genuine culture—shared knowledge bases that agents collectively maintain and evolve, self-play mechanisms that drive capability improvement through competition, and organisational structures that enable collaboration without centralized control. Current systems remain fundamentally solitary; genuine agent economies will require breakthroughs in coordination and communication.
Fifth, and perhaps most ambitiously, maintaining entropy in AI systems—preventing the collapse toward stereotyped outputs that currently plagues even frontier models. This might involve explicit diversity penalties, adversarial training to prevent convergence, or inspiration from biological systems that maintain variation through mechanisms like mutation and recombination.
Conclusion: Intelligence as Engineering Challenge
Andrej Karpathy’s vision of the cognitive core represents a mature perspective on artificial intelligence—neither breathlessly optimistic about imminent superintelligence nor dismissively pessimistic about current limitations. He sees AI as an engineering challenge rather than a mystical threshold, requiring patient work across multiple dimensions rather than awaiting a single breakthrough.
This perspective derives from direct experience with the messy reality of deploying AI systems at scale. Self-driving cars that work perfectly in demonstrations still require years of refinement before handling edge cases reliably. Coding agents that generate impressive solutions for common problems still struggle with novel architectural challenges. Educational AI that answers questions adequately still falls far short of expert human tutors’ adaptive responsiveness.
Yet within these limitations lies genuine progress. Models continue improving along multiple dimensions simultaneously. Infrastructure for deploying and managing AI systems grows more sophisticated. Understanding of these systems’ capabilities and constraints becomes more nuanced. The path forward is visible, even if it stretches further than optimists anticipated.
The concept of stripping knowledge to reveal the cognitive core captures this mature vision perfectly. Rather than pursuing ever-larger models memorising ever-more data, we might achieve more capable intelligence through subtraction—removing the crutch of memorisation to force development of robust reasoning algorithms. Like humans compelled to abstract and generalise because we cannot remember everything, AI systems might benefit from similar constraints.
This vision offers hope not for sudden transformation but for steady progress—the kind that compounds over decades into revolutionary change. It suggests that the hard technical problems of intelligence remain tractable whilst acknowledging their genuine difficulty. Most importantly, it positions humans not as passive observers of AI’s ascent but as active participants in shaping its development and ensuring its integration enhances rather than diminishes human flourishing.
The decade ahead will test these ideas. We’ll discover whether cognitive cores can be effectively isolated, whether continual learning mechanisms can be made robust, whether the demo-to-product gap can be bridged across diverse domains. The answers will shape not just the trajectory of AI technology but the future of human society in an increasingly automated world. Karpathy’s contribution lies in framing these questions with clarity, drawing on hard-won experience to guide expectations, and reminding us that the most profound challenges often yield to patient, disciplined engineering rather than waiting for miraculous breakthroughs.
