Chapter 12: Mice in the Imaginarium

Chapter Overview:

• Main Focus: This chapter explores the unique cognitive abilities that emerged with the neocortex in early mammals. Bennett argues that the neocortex's primary function is not just pattern recognition, but the simulation of alternative scenarios and possibilities – a capacity he calls "vicarious trial and error." This "imaginarium" in the minds of mammals enabled them to plan ahead, learn from imagined mistakes, and adapt more flexibly to changing environments (Bennett, 2023, p. 189). He emphasizes how this ability may have been crucial for early mammals who were small and who had the first-mover advantage while hiding from dinosaurs and other large predators, and how this may have been a key factor in their evolutionary success (Bennett, 2023, p. 164, 200).
• Objectives:
• Explain the concept of vicarious trial and error and its significance in mammalian intelligence.
• Describe the three key abilities enabled by the neocortex: vicarious trial and error, counterfactual learning, and episodic memory.
• Connect these abilities to the structure and function of the neocortex, particularly the prefrontal cortex.
• Discuss the advantages of model-based reinforcement learning over model-free approaches.
• Highlight the role of internal models in planning and decision-making.
• Fit into Book's Structure: This chapter delves into the specific advantages conferred by the neocortex in mammals. It follows the discussion of the neocortex as a generative model (Chapter 11) and precedes the exploration of model-based reinforcement learning (Chapter 13). It represents a crucial step in Bennett's argument, illustrating how the capacity for simulation transformed the landscape of intelligence.

Key Terms and Concepts:

• Neocortex: The outermost layer of the cerebral cortex, responsible for higher-level cognitive functions. Relevance: The neocortex is the key innovation in mammals, enabling the "imaginarium."
• Vicarious Trial and Error: The ability to mentally simulate different actions and their potential outcomes before acting in the real world. Relevance: This is presented as a core function of the neocortex, enabling more efficient and flexible learning.
• Counterfactual Learning: Learning from imagined scenarios and "what-if" questions. Relevance: This ability allows mammals to learn from mistakes they didn't make, expanding the scope of learning beyond direct experience.
• Episodic Memory: Memory for specific past events and experiences. Relevance: Episodic memory provides a rich source of information for simulating future scenarios and making predictions.
• Prefrontal Cortex (PFC): The front part of the frontal lobe, involved in planning, decision-making, and working memory. Relevance: The PFC is crucial for controlling and directing the simulations generated by the neocortex.
• Model-Based Reinforcement Learning: A type of reinforcement learning that uses an internal model of the world to simulate future outcomes and guide decision-making. Relevance: This approach is contrasted with model-free reinforcement learning, which relies on direct experience and is less flexible.
• Model-Free Reinforcement Learning: A type of reinforcement learning that learns through direct experience and does not rely on simulating future outcomes. Relevance: This contrasts with the model-based approaches of mammals and other species with neocortices (Bennett, 2023, p. 209)
• Detour Task: An experimental task used to assess spatial reasoning and planning abilities. Relevance: Performance on detour tasks demonstrates the advantage of having an internal model of the world.

Key Figures:

• Edward Tolman: A psychologist who proposed the concept of cognitive maps. Relevance: Tolman's work provided early evidence for internal models in animals and is cited as an inspiration for later research on vicarious trial and error.
• David Redish and Adam Johnson: Neuroscientists who studied vicarious trial and error in rats. Relevance: Their research provided direct evidence for the neural basis of this ability in the hippocampus and is presented as one of the more surprising and recent discoveries in neuroscience (Bennett, 2023, p. 190).
• Tony Dickinson: A psychologist who studied goal-directed behavior and habits. Relevance: Dickinson's work is discussed in the context of model-based vs. model-free reinforcement learning and the interplay between goals and habits, by highlighting the tension between the frontal neocortex and the basal ganglia. The basal ganglia are associated with automatic ‘habitual’ behavior (model-free) whereas the aPFC and other frontal areas are associated with more deliberative ‘goal-directed’ behavior (model-based), though both work through similar temporal difference mechanisms (Bennett, 2023, p. 214).
• Karl Friston: A neuroscientist known for his work on the free energy principle and active inference. Relevance: Friston's concept of active inference provides a theoretical framework for understanding how generative models in the brain drive behavior. Bennett uses the term ‘active inference’ to highlight how the neocortex enables behavior that was not possible with the previous vertebrate brain, by creating a predictive generative model that could simulate future outcomes based on ‘intent’, and which could thereby adjust those actions in anticipation to more efficiently reach the intended outcome (Bennett, 2023, p. 194, 216).
• Antonio Damasio: A neuroscientist whose work focuses on the role of emotions and feelings in decision-making. Relevance: Damasio's work is relevant to the discussion of how internal states and emotions can influence simulated scenarios and thus behavior. His studies of patients with damage to specific areas in their neocortex, specifically his patient ‘L’ who suffered from a stroke, revealed that the prefrontal cortex evolved to control what a mammal simulates, which explains why his patient ‘L,’ whose frontal cortex was damaged, had no inner dialogue (Bennett, 2023, p. 205, 217).

Central Thesis and Supporting Arguments:

• Central Thesis: The neocortex, through its capacity for vicarious trial and error, counterfactual learning, and episodic memory, enables mammals to simulate alternative possibilities, learn from imagined mistakes, and plan ahead, conferring a significant advantage in adaptability and decision-making. Further, the author argues that the same microcircuitry of the neocortex which supports perception may also underlie these new more sophisticated simulating capabilities of the neocortex (Bennett, 2023, p. 195).
• Supporting Arguments:
• Behavioral flexibility: Mammals exhibit greater behavioral flexibility compared to other vertebrates, suggesting more advanced planning and decision-making abilities.
• Neural evidence: Recordings from the brains of rats navigating mazes show activity consistent with vicarious trial and error.
• Adaptive advantages: Simulating possible futures allows for more efficient learning, avoidance of costly mistakes, and better adaptation to dynamic environments.
• Model-based learning: The neocortex enables model-based reinforcement learning, a more flexible approach than model-free methods.

Observations and Insights:

• The importance of the "first move": Bennett argues that the capacity for simulation is particularly advantageous in situations where an animal has the first move, allowing it to plan and strategize before acting.
• The interplay of simulation, memory, and internal models: Vicarious trial and error relies on access to past experiences (episodic memory) and an accurate model of the world.
• The role of the prefrontal cortex in controlling simulations: The PFC acts as a "conductor," directing the simulations generated by the neocortex and integrating them with internal states, goals, and motivations.
• Evolutionary advantages and limitations: Model-based behavior tends to be more evolutionarily “successful” than model-free behavior (Bennett, 2023, p. 201, 207, 209) because it enables vicarious counterfactual reasoning and thus a more nuanced strategy for assigning “credit” where credit is due (Bennett, 2023, p. 206). But model-based planning comes at a cost; ‘simulating’ is more computationally expensive than the simpler model-free approaches, requiring larger brains with greater caloric needs and longer developmental periods in childhood. This suggests there was likely some environmental pressure that caused this “expensive” trade-off to be worthwhile for mammals (Bennett, 2023, p. 206, 210).

Unique Interpretations and Unconventional Ideas:

• The emphasis on vicarious trial and error as a primary function of the neocortex: This contrasts with the traditional focus on the neocortex's role in sensory processing and "higher" cognitive functions.
• The link between simulation and seemingly irrational behaviors: Bennett suggests that some seemingly irrational behaviors, like Dickinson's “habits” (Bennett, 2023, p. 215), are actually model-free behaviors which were previously useful, but which are now being applied inappropriately in a particular context. Bennett emphasizes how this is the source of much irrational behavior in humans.

Problems and Solutions:

Categorical Items:

Bennett distinguishes between model-free and model-based reinforcement learning, highlighting the increased flexibility and adaptability afforded by the latter. He also categorizes different types of memory (episodic, procedural) and relates them to the neocortex's simulation abilities.

Literature and References: (Refer to the book's bibliography for full citations)

• Works by Tolman, Redish, Johnson, Dickinson, Friston, Damasio, and others are cited.
• Studies on rat navigation, primate behavior, and the neural basis of planning and memory are referenced.

Areas for Further Research:

• The precise neural mechanisms underlying vicarious trial and error, counterfactual learning, and episodic memory need further investigation.
• The development and refinement of internal models in the brain across species is not fully understood.
• The interplay between model-based and model-free learning in different decision-making contexts warrants more research.

Critical Analysis:

• Strengths: The chapter provides a compelling and well-supported argument for the importance of simulation in mammalian intelligence. The integration of research from neuroscience, psychology, and AI strengthens the analysis.
• Weaknesses: The focus on model-based learning may downplay the role of other factors, such as emotions and social influences, in shaping behavior. Some of Bennett's interpretations, particularly regarding the supposed advantages provided by the “first-mover” advantage and subsequent neocortical upgrades (Bennett, 2023, p. 189, 199) are speculative.

Practical Applications:

• Understanding the role of simulation in learning and decision-making can be applied to improve educational methods, develop more effective training programs, and design better AI systems.

Connections to Other Chapters:

• Chapter 10 (Neural Dark Ages): This chapter directly follows Chapter 10 which describes the evolutionary transition from the limited cognitive abilities of earlier reptiles and fish to the sophisticated neocortical machinery of early mammals (Bennett, 2023, p. 165). It describes the evolution of the neocortex as a major restructuring in the brain not seen since early vertebrates. It sets the stage by highlighting that the neocortex made up only a very small fraction of the brain in early mammals, and would later explode to become 70% of human brains today (Bennett, 2023, p. 165).
• Chapter 11 (Generative Models): This chapter builds on the idea of the neocortex as a generative model by demonstrating its specific capabilities of vicarious trial and error, counterfactual learning and episodic memory. This ties together his theory of intelligence as consisting of systems which can model, and subsequently simulate, both the outside world (the neocortex, Ch. 11) and the internal world (the prefrontal cortex, Ch. 13) (Bennett, 2023, p. 209).
• Chapter 13 (Model-Based Reinforcement Learning): This chapter lays the groundwork for the discussion of model-based learning by establishing the importance of internal models and simulations in guiding decisions.
• Chapter 14 (Secret to Dishwashing Robots): This chapter foreshadows the next chapter’s explanation for how fine motor skills emerged in early mammals, including hand-eye coordination and the manipulation of small objects, and the author makes the provocative suggestion that this ability came from mentally rehearsing such skills in the “imaginarium” and thus training an internal model of the animal’s body which was a key ingredient in early placental mammals’ dominance of their respective ecosystems (Bennett, 2023, p. 209). This echoes his previous argument in Ch. 6 about the benefit of using ‘mental rehearsal’ to improve performance on certain tasks.
• Chapters 16, 17 and 18: These three chapters on social intelligence, theory of mind and imitation learning in primates, are also closely linked to the idea of simulating, since Bennett argues that mentalizing (Ch. 16) builds upon the prior breakthrough of simulating, and he goes so far as to suggest that mentalizing is merely simulating one’s own model of another’s internal states, and then applying that same model to one’s self. In other words, primate mentalizing builds on mammalian simulating (Ch. 3) (Bennett, 2023, p. 296). Similarly, primates’ ability to imitate the motor skills of others, a key part of how primates transmit knowledge and skills across generations via social learning, is closely tied to the idea of simulating, where primates simulate themselves performing the tasks that they see others perform (Bennett, 2023, p. 317). And a primate’s capacity to learn new motor skills and to anticipate future needs, both key features of primate intelligence, build upon this ability to simulate, plan, and engage in episodic and counterfactual learning (Bennett, 2023, p. 296).

Surprising, Interesting, and Novel Ideas:

• Vicarious trial and error as a core function of the neocortex: This challenges traditional views of the neocortex's role, highlighting the importance of simulation and imagination (Bennett, 2023, p. 189).
• Counterfactual learning as a driver of adaptation: Learning from mistakes we didn't make expands the scope of learning and enables more flexible behavior (Bennett, 2023, p. 192-196). He supports this with an example of a “restaurant row” experiment with rats who forgo quick access to food to try and get better food, and who, if they do not receive the better food, exhibit signs of ‘regret’ for their decision, by pausing and looking back in the direction of the other food (Bennett, 2023, p. 194). This suggests that such rats are simulating, or at least considering, alternative outcomes and therefore inferring that perhaps they made the wrong decision.
• The link between episodic memory and simulation: Bennett argues that episodic memory, the ability to recall specific past events, is crucial for constructing simulations of future possibilities (Bennett, 2023, p. 197). This suggests a strong link between the evolutionary emergence of episodic memory, simulation, and the mammalian neocortex.

Discussion Questions:

• How does vicarious trial and error enhance learning and decision-making compared to relying solely on direct experience?
• What are the evolutionary advantages of counterfactual thinking, and how might it contribute to human intelligence?
• How does the prefrontal cortex orchestrate and control the simulations generated by the neocortex?
• What are the limitations of model-based reinforcement learning, and when might model-free approaches be more effective?
• How might the concept of the "imaginarium" be applied to enhance creativity and problem-solving in humans?

Visual Representation:

[Neocortex] --> [Vicarious Trial and Error] + [Counterfactual Learning] + [Episodic Memory] --> [Flexible Behavior & Adaptive Decision-Making] ^ | | v [Prefrontal Cortex (Control)] [Model-Based Reinforcement Learning]

TL;DR: