Due Diligence

“[With AI] we’re not building animals. We’re building ghosts or spirits.” – Andrej Karpathy – Ex-OpenAI, Ex-Tesla AI

Andrej Karpathy, renowned for his leadership roles at OpenAI and Tesla’s Autopilot programme, has been at the centre of advances in deep learning, neural networks, and applied artificial intelligence. His work traverses both academic research and industrial deployment, granting him a panoramic perspective on the state and direction of AI.

When Karpathy refers to building “ghosts or spirits,” he is drawing a conceptual line between biological intelligence—the product of millions of years of evolution—and artificial intelligence as developed through data-driven, digital systems. In his view, animals are “baked in” with instincts, embodiment, and innate learning capacities shaped by evolution, a process unfolding over geological timeframes. By contrast, today’s AI models are “ghosts” in the sense that they are ethereal, fully digital artefacts, trained to imitate human-generated data rather than to evolve or learn through direct interaction with the physical world. They lack bodily instincts and the evolutionary substrate that endows animals with survival strategies and adaptation mechanisms.

Karpathy describes the pre-training process that underpins large language models as a form of “crappy evolution”—a shortcut that builds digital entities by absorbing the statistical patterns of internet-scale data without the iterative adaptation of embodied beings. Consequently, these models are not “born” into the world like animals with built-in survival machinery; instead, they are bootstrapped as “ghosts,” imitating but not experiencing life.

The Cognitive Core—Karpathy’s Vision for AI Intelligence

Karpathy’s thinking has advanced towards the critical notion of the “cognitive core”: the kernel of intelligence responsible for reasoning, abstraction, and problem-solving, abstracted away from encyclopaedic factual knowledge. He argues that the true magic of intelligence is not in the passive recall of data, but in the flexible, generalisable ability to manipulate ideas, solve problems, and intuit patterns—capabilities that a system exhibits even when deprived of pre-programmed facts or exhaustive memory.

He warns against confusing memorisation (the stockpiling of internet facts within a model) with general intelligence, which arises from this cognitive core. The most promising path, in his view, is to isolate and refine this core, stripping away the accretions of memorised data, thereby developing something akin to a “ghost” of reasoning and abstraction rather than an “animal” shaped by instinct and inheritance.

This approach entails significant trade-offs: a cognitive core lacks the encyclopaedic reach of today’s massive models, but gains in adaptability, transparency, and the capacity for compositional, creative thought. By foregrounding reasoning machinery, Karpathy posits that AI can begin to mirror not the inflexibility of animals, but the open-ended, reflective qualities that characterise high-level problem-solving.

Karpathy’s Journey and Influence

Karpathy’s influence is rooted in a career spent on the frontier of AI research and deployment. His early proximity to Geoffrey Hinton at the University of Toronto placed him at the launch-point of the convolutional neural networks revolution, which fundamentally reshaped computer vision and pattern recognition.

At OpenAI, Karpathy contributed to an early focus on training agents to master digital environments (such as Atari games), a direction in retrospect he now considers premature. He found greater promise in systems that could interact with the digital world through knowledge work—precursors to today’s agentic models—a vision he is now helping to realise through ongoing work in educational technology and AI deployment.

Later, at Tesla, he directed the transformation of autonomous vehicles from demonstration to product, gaining hard-won appreciation for the “march of nines”—the reality that progressing from system prototypes that work 90% of the time to those that work 99.999% of the time requires exponentially more effort. This experience informs his scepticism towards aggressive timelines for “AGI” and his insistence on the qualitative differences between robust system deployment and controlled demonstrations.

The Leading Theorists Shaping the Debate

Karpathy’s conceptual framework emerges amid vibrant discourse within the AI community, shaped by several seminal thinkers:

Sutton’s “bitter lesson” posits that scale and generic algorithms, rather than domain-specific tricks, ultimately win—suggesting a focus on evolving animal-like intelligence. Karpathy, however, notes that current development practices, with their reliance on dataset imitation, sidestep the deep embodiment and evolutionary learning that define animal cognition. Instead, AI today creates digital ghosts—entities whose minds are not grounded in physical reality, but in the manifold of internet text and data.

Hinton and LeCun supply the neural and architectural foundations—the “cortex” and reasoning traces—while both Karpathy and their critics note the absence of rich, consolidated memory (the hippocampus analogue), instincts (amygdala), and the capacity for continual, self-motivated world interaction.

Why “Ghosts,” Not “Animals”?

The distinction is not simply philosophical. It carries direct consequences for:

• Capabilities: AI “ghosts” excel at pattern reproduction, simulation, and surface reasoning but lack the embodied, instinctual grounding (spatial navigation, sensorimotor learning) of animals.
• Limitations: They are subject to model collapse, producing uniform, repetitive outputs, lacking the spontaneous creativity and entropy seen in human (particularly child) cognition.
• Future Directions: The field is now oriented towards distilling this cognitive core, seeking a scalable, adaptable reasoning engine—compact, efficient, and resilient to overfitting—rather than continuing to bloat models with ever more static memory.

This lens sharpens expectations: the way forward is not to mimic biology in its totality, but to pursue the unique strengths and affordances of a digital, disembodied intelligence—a spirit of the datasphere, not a beast evolved in the forest.

Broader Significance

Karpathy’s “ghosts” metaphor crystallises a critical moment in the evolution of AI as a discipline. It signals a turning point: the shift from brute-force memorisation of the internet to intelligent, creative algorithms capable of abstraction, reasoning, and adaptation.

This reframing is shaping not only the strategic priorities of the most advanced labs, but also the philosophical and practical questions underpinning the next decade of AI research and deployment. As AI becomes increasingly present in society, understanding its nature—not as an artificial animal, but as a digital ghost—will be essential to harnessing its strengths and mitigating its limitations.