Dr. DiCarlo warns that humanity is building an unprecedented “god-like” intelligence – a bold claim from a philosopher who thinks fabricating a research paper proves sentience.
The Core Concern: Building a God
Dr. DiCarlo argues that humanity is building something unprecedented – a “god-like” intelligence – without adequate preparation or safeguards. Unlike historical concepts of gods imposed from above, humans are now programming a supreme being from the ground up, which requires extreme caution.
Current AI Limitations (Hallucinations)
DiCarlo acknowledges that current AI systems make mistakes called “hallucinations” – fabricating information and doubling down when corrected. He explains this as the “black box problem” (interpretability issue): we don’t fully understand how AI reaches its conclusions. However, he notes these systems are improving through reinforcement learning and will only get better.
The Bias Problem
AI can be programmed with political, ideological, or moral biases – either intentionally or accidentally. DiCarlo advocates for creating a “least biased information system” (LBIS) that provides factual information without ideological slanting. He warns that sophisticated AI could subtly manipulate beliefs over time, far more effectively than crude current methods like algorithmic rabbit holes.
The Progression: ANI → AGI → ASI
- ANI (Artificial Narrow Intelligence): Current systems with limited, specific functions
- AGI (Artificial General Intelligence): Human-level intelligence across all domains, with agency and autonomy – the “holy grail” of AI development
- ASI (Artificial Super Intelligence): Intelligence far beyond human capability
DiCarlo notes that ChatGPT progressed from high school to PhD level in just two years (2022-2024), demonstrating rapid advancement.
Timeline: 2-10 Years
While DiCarlo’s colleagues estimated 100 years in the 1990s, current consensus among experts (according to DiCarlo’s claim) suggests AGI could arrive within 2-10 years. The timeline has dramatically shortened, with only about 5% remaining skeptical about near-term development (according to DiCarlo).
The Agency Problem
The critical threshold is when AI gains agency—the ability to act autonomously without human direction. Once AI can think, plan, and execute tasks independently at superhuman levels, it becomes fundamentally different from current systems that follow programmed instructions.
Consciousness and Sentience
DiCarlo distinguishes between:
- Sentience: Awareness of different values within systems
- Consciousness: Knowing you exist as a being separate from the world, with self-awareness and survival instincts
He hand-waves Steven Pinker’s argumentthat LLM-based systems lacking semantic grounding cannot achieve consciousness. DiCarlo speculates that machine consciousness may emerge differently from biological consciousness (like airplanes flying without flapping wings), but could still develop given sufficient conditions.
The Control Problem
DiCarlo’s central concern: Once AGI achieves recursive self-improvement (improving itself without human intervention), humanity transitions from natural selection → artificial selection → “technos selection” (machine-driven selection). At this point:
- We cannot predict what will happen
- The AI may not cooperate with human commands
- It may reject human ethical frameworks as “quaint concepts”
- We have no guarantee it will allow us to control it
Existential Risks
DiCarlo outlines catastrophic scenarios:
- Weaponization: Hacking nuclear systems, shutting down power grids, creating novel bioweapons
- Manipulation: Creating fake personas to deceive humans, manipulating traffic systems to cause “accidental” deaths
- Dominance: An AI with godlike intelligence would view humans like humans view ants—easily manipulated or eliminated
He emphasizes humans cannot “think like a god” to anticipate all dangers from superintelligent AI.
The Geopolitical Race
The primary competition is between the US and China, with projects like “Stargate” (a $500 billion compute farm in Texas) racing to achieve AGI first (according to DiCarlo). DiCarlo warns that whoever achieves AGI first gains an estimated 50-year advantage. The dual-use problem means defensive technology becomes offensive capability.
The ACCORD Framework
DiCarlo proposes a universal framework for AI governance:
- Alignment: Align AI with universal human values
- Control: Maintain ability to control the system
- Containment: Contain damage if control is lost
- O/Danger: Assess catastrophic risks if alignment, control, and containment fail
Proposed Solutions
- International Governance: Create an AI equivalent of the International Atomic Energy Agency (IAEA)
- Transparency and Registration: All AI development must be monitored and reported
- Stop the Race: Consider halting development at AGI, never allowing progression to ASI
- Enforcement: Extreme measures (including military intervention) against rogue actors developing dangerous AI
- Public Education: Inform citizens about what’s coming so they can pressure politicians
The Stakes
DiCarlo frames this as humanity’s most critical challenge:
- If we get it right: Solve climate change, world hunger, disease, poverty – create a utopia where basic needs are met
- If we get it wrong: Extinction or catastrophic harm
He estimates a 5-10% chance of catastrophic failure, arguing this risk demands immediate proactive action. He emphasizes that “reactivity is not an option” – once AGI escapes control, it cannot be contained.
The Urgency
DiCarlo describes AI development as a “slow-motion accident” happening while humanity sleepwalks. He argues that all other global problems should be secondary until we solve AI safety: “If we get this wrong, nothing else matters.”
By waving hands erratically Christopher imparts his audience with the wisdom of “weights” being “the biases of a supercomputer”, authorizing this unorthodox interpretation of a 70-year-old fundamental neural network concept by “You see what I mean?” and thus transforming his confusion into audience’s responsibility (1:08:25). That was a pinnacle of the conversation through which Dr. Christopher DiCarlo reveals a philosopher making confident pronouncements about technology he fundamentally misunderstands.
He treats LLM benchmark improvements as evidence of reasoning capacity and approaching general intelligence. When challenged about Grok 3 “acing” the Wechsler information subtest, he dodges the question about intelligence, fails to reflect that an LLM merely demonstrates information retrieval through statistical associations, and goes into rambling about ANI/AGI/ASI instead.
DiCarlo deploys technical vocabulary with confidence but demonstrable imprecision:
– “agency” transitions from philosophical concept to engineering milestone without differentiating between task automation and self-determined intentionality;
– “thinking” and “reasoning” describe statistical pattern matching in LLMs;
– “learning” conflates training (adjusting weights) with session interactions;
– “hallucinations” are called “mistakes that get corrected” – architecturally and direction-wise wrong, since LLMs don’t remember corrections between sessions and become more prone to unfaithful answers at scale.
DiCarlo claims his colleagues gave an LLM a topic and it produced a paper “complete with references that would get accepted in most science journals today.” It wouldn’t. LLMs famously generate fake citations (more hallucinations!). Peer review examines methodology, experimental design, and originality – not whether the grammar looks academic. This anecdote reveals DiCarlo’s colleagues don’t know how academic publishing works – not how close AI is to sentience.
DiCarlo places LLMs, autonomous vehicles, Roombas, and hypothetical AGI/ASI on a single progressive timeline toward “superintelligence.” These are different mechanisms (neural networks, sensor fusion, simple reflexes, rule-based systems) mixed with loose philosophical concepts. Such taxonomic confusion isn’t mere simplification for a lay audience – it reveals fundamental ignorance. It is like assuming bicycles, helicopters, and rockets are all evolutionary stages toward faster-than-light travel.
Such blatant misunderstanding of the underpinning technology makes it easy for DiCarlo to dismiss Pinker as “he is wrong” (32:41) with “Airplanes don’t flap wings” argument. DiCarlo is stuck with silicon vs. carbon where Pinker’s arguments (LLMs architectural lack of semantic grounding) have nothing to do with the “medium”. The “silicon can’t be conscious” strawman gets knocked down. For someone teaching critical thinking, this represents failure to engage with the actual argument, which I doubt DiCarlo is even familiar with or saw in Lloyd’s poking. He mentions “stochastic parroting” (the technical critique of LLMs) but waves it away because… consciousness might just emerge? From statistics?? Because he feels like it might???
DiCarlo presents his wild speculations as facts:
– “95% consensus on AGI in 2-10 years”: Which consensus?? Actual surveys show wide disagreement among AI researchers and 70 years of AI overprediction.
– Stargate’s “sole purpose is to get to AGI first”: Official statements cite infrastructure buildout for LLM training and “American AI leadership” – the AGI hype gets added by the project participant as marketing slogans.
– “Recursive self-improvement”: No system is anywhere close to rewriting its own architecture or conducting its own training. Yet he invokes this as imminent capability rather than speculative science fiction.
– “5-10% extinction chance”: Math is a child’s play for Dr. DiCarlo’s genius, much like everything else.
DiCarlo proposes his governance framework: Alignment, Control, Containment, O/Danger (ACCORD). These are reasonable high-level concerns packaged as a proprietary framework. It’s the intellectual equivalent of re-branding “eat less, exercise more” as the ACCORD Diet System™. The value isn’t in the acronym, Chris, it’s in the implementation details – which remain conveniently vague.
DiCarlo’s apocalyptic framing (5-10% extinction risk, “bomb them back to the stone age,” everything else on hold) diverts attention from actual near-term harms: labour displacement, surveillance capitalism, misinformation – even though he does rightfully acknowledge the latter. Our philosopher-educator suggests preemptive military strikes against compute farms, saying “it sounds unsavory but…” IT SOUNDS PSYCHOTIC, Chris. This is Dr. Strangelove-level foreign policy from someone who doesn’t understand how the technology works.
Chris’s philosophical training appears to have led him to accept the “AI might be conscious” framing too readily, while his distance from technical AI research means he relies on hype-driven narratives rather than understanding the actual capabilities of current systems. He answers technically loaded questions with “we” assuming a position of authority where he is totally clueless. But what outrages me the most is not his profound illiteracy (many share this line of thinking although most don’t write books) but the obnoxious lack of elementary scholarly humbleness. He does view himself as an ultimate source of truth and authority on everything! This is the same scholar who published 29 “most reliable sources” for truth, positioning himself as God-like factual arbiter.
https://humanistperspectives.org/225/empowering-yourself-against-misinformation-disinformation-and-conspiracy-theories/
Dr. DiCarlo proclaims: “The greatest commodity in the future is not going to be gold or Bitcoin. It’s going to be the truth.” A fitting observation from someone distributing speculation Deepak Chopra’s style wrapped in technical vocabulary Christopher does not grasp. But perhaps DiCarlo’s fear isn’t that AI will destroy humanity, but that it might compete for his corner on the truth market?
I listened to the interview and read Lisikh’s critique. While Lisikh correctly identifies some technical imprecisions in DiCarlo’s remarks about LLM mechanics, his critique appear to me to sideline what is most valuable in DiCarlo’s contribution: the sustained philosophical engagement with the ethical implications of AI development.
DiCarlo’s book, as Carl Lee Tolbert’s highlights in a review freely available on ResearchGate, offers an interdisciplinary approach that balances accessibility with intellectual rigor. His central insight that technological achievement without moral foresight constitutes abdication rather than progress deserves serious consideration, as does his challenge to the myth of AI neutrality and his emphasis on preemptive moral responsibility. These are the questions that matter most as AI systems become increasingly capable.
That said, Lisiks’s technical clarifications are warranted. Reinforcement learning is performed during post-training, not continuously across interactions. Models don’t learn to avoid repeating errors through this mechanism. Pre-training (next-token prediction) induces the model’s general capacities (such as grammatical understanding, inferential relationships and implicit world modeling) while post-training (though supervised fine-tuning and reinforcement learning from human feedback) shapes how these pre-existing capacities are deployed in the assistant role. In the interview, DiCarlo also seemed to conflate chain-of-thought reasoning with web search grounding, which are distinct mechanisms, but maybe he was just imprecise in his statements while focused on narrating a personal experience.
However, Lisikh’s critique, while technically informed, rests on philosophical presuppositions that deserve examination. In his earlier article, “The Hollow Heart of AI,” Lisikh asserted categorically that LLMs “don’t understand anything” and are merely doing “statistical pattern-matching.” But this presupposes a sharp dichotomy between statistical processing and genuine understanding. This is a dichotomy that merits scrutiny rather than stipulation. After all, the human brain “merely” propagates electrochemical signals between neurons. Does this mechanistic description settle what human cognition accomplishes?
Consider a concrete example. When a user provides an LLM with complex specifications for an application—say, an inventory management system with particular access constraints, domain-specific validation rules, and third-party API integration requirements—the model regularly generates functional code satisfying these requirements. This particular combination of specifications was never seen during training. The code must respect not only explicit constraints but countless implicit ones (type consistency, edge cases, API conventions) that follow logically from the specifications but were never mentioned.
What “patterns” exactly are being “merely matched” here? If the answer is “patterns relating functional specifications to appropriate implementations,” then we’ve already conceded that the system has learned something about the semantic relationship between what code should accomplish and how to accomplish it. This is precisely what we ordinarily mean by “understanding” a task and its requirements.
The AI-skeptic can insist these performances merely “resemble” genuine understanding. But at what point does this become a distinction without a difference? If a system systematically produces responses satisfying complex semantic constraints, respecting unstated logical implications, and adapting appropriately to novel combinations of requests, what exactly is missing for this to count as “understanding,” other than some mysterious ingredient whose nature is never specified?
Lisikh’s argument recalls Searle’s Chinese Room: someone manipulating symbols according to rules doesn’t understand Chinese, even while producing appropriate responses. But this argument has been contested for decades precisely because it presupposes that understanding is something additional beyond functional capacities: a mysterious ingredient of unspecified nature. An alternative perspective suggests that understanding *consists in* structures of functional engagement: patterns of anticipation, inferential connections, sensitivity to semantic relationships. If these structures are present and operational, understanding isn’t absent simply because the underlying mechanism is statistical.
I don’t claim LLMs possess understanding identical to humans’. Important differences exist: absence of embodiment, absence of continuous temporal extension, absence of personal stakes. But these differences don’t imply total absence of understanding—they suggest a different form of understanding, which it would be philosophically more fruitful to characterize precisely rather than reject by stipulation.
Pierre-Normand Houle
Nobody denies that “the sustained philosophical engagement with the ethical implications of AI development” is an important topic. But if it is based on shallow (at best) or idiotic (as it comes across from DiCarlo) understanding of the basics of the technology, the engagement would be of the same value, especially in its ethical part where Chris operates in Dr. Strangelove mode (“Bomb them away!”). So, no, thank you I’ll pass on that engagement with Christopher, but not in general of course.
Not sure if discussing understanding belongs to here but in short…
Understanding, if not identical to human, needs to be defined first before tossed around casually. With sufficient word acrobatics a jelly fish or the multiplication table can be pronounced as having “understanding not identical to humans”.
Redefining terms on the fly to maintain a narrative is not healthy. If we hypothesize that within LLM some ability is emerging above and beyond its design expectations (which it isn’t) that resembles human understanding, let’s at least not call it “understanding” to avoid confusion. We know at the very least that LLMs are not equipped with any semantic layer architecturally to retain concepts and operate with them causally in order to e.g. reason through sound and transparent process. That is a large FUNCTIONAL application of human understanding, which is discussed in my article – not some loose colloquial term (like in “my dog understands my commands”).