Ruminating on replay during the awake state
Related Articles
Constructing future behavior in the hippocampal formation through composition and replay
The hippocampus is critical for memory, imagination and constructive reasoning. Recent models have suggested that its neuronal responses can be well explained by state spaces that model the transitions between experiences. Here we use simulations and hippocampal recordings to reconcile these views. We show that if state spaces are constructed compositionally from existing building blocks, or primitives, hippocampal responses can be interpreted as compositional memories, binding these primitives together. Critically, this enables agents to behave optimally in new environments with no new learning, inferring behavior directly from the composition. We predict a role for hippocampal replay in building and consolidating these compositional memories. We test these predictions in two datasets by showing that replay events from newly discovered landmarks induce and strengthen new remote firing fields. When the landmark is moved, replay builds a new firing field at the same vector to the new location. Together, these findings provide a framework for reasoning about compositional memories and demonstrate that such memories are formed in hippocampal replay.
Preserving and combining knowledge in robotic lifelong reinforcement learning
Humans can continually accumulate knowledge and develop increasingly complex behaviours and skills throughout their lives, which is a capability known as ‘lifelong learning’. Although this lifelong learning capability is considered an essential mechanism that makes up general intelligence, recent advancements in artificial intelligence predominantly excel in narrow, specialized domains and generally lack this lifelong learning capability. Here we introduce a robotic lifelong reinforcement learning framework that addresses this gap by developing a knowledge space inspired by the Bayesian non-parametric domain. In addition, we enhance the agent’s semantic understanding of tasks by integrating language embeddings into the framework. Our proposed embodied agent can consistently accumulate knowledge from a continuous stream of one-time feeding tasks. Furthermore, our agent can tackle challenging real-world long-horizon tasks by combining and reapplying its acquired knowledge from the original tasks stream. The proposed framework advances our understanding of the robotic lifelong learning process and may inspire the development of more broadly applicable intelligence.
Sleep-stage dependent patterning of slowly propagating brain activity
Endogenous brain activity plays a key role during sleep, facilitating important functions like learning and memory consolidation. Its precise mechanisms are not yet understood. Recent functional MRI (fMRI) research has revealed structured multi-second waves that travel across the cortex along a hierarchical functional gradient during wakefulness. However, how these waves are modulated across different sleep stages remains unclear. Here, we present compelling evidence of significant changes in both the frequency and spatiotemporal patterns of these waves across sleep stages. Notably, during rapid eye movement (REM) sleep, we observed a predominance of propagations from sensory/motor areas to higher-order networks. These patterns were distinct from other sleep stages, with key regions like the thalamus, pons, and visual cortex—linked to PGO waves—activating at the earliest phase of waves. Similar to PGO waves, these fMRI waves were coupled with REM activity, suggesting their potential role in supporting memory consolidation and learning processes.
Historical loss weakens competitive behavior by remodeling ventral hippocampal dynamics
Competitive interactions are pervasive within biological populations, where individuals engage in fierce disputes over vital resources for survival. Before the establishment of a social hierarchy within the population, this competition becomes even more intense. Historical experiences of competition significantly influence the competitive performance; individuals with a history of persistent loss are less likely to initiate attacks or win escalated contests. However, it remains unclear how historical loss directly affects the evolution of mental processes during competition and alters responses to ongoing competitive events. Here, we utilized a naturalistic food competition paradigm to track the competitive patterns of mutually unfamiliar competitors and found that a history of loss leads to reduced competitive performance. By tracking the activity of ventral hippocampal neuron ensembles, we identified clusters of neurons that responded differently to behavioral events during the competition, with their reactivity modulated by previous losses. Using a Recurrent Switch Linear Dynamical System (rSLDS), we revealed rotational dynamics in the ventral hippocampus (vHPC) during food competition, where different discrete internal states corresponded to different behavioral strategies. Moreover, historical loss modulates competitive behavior by remodeling the characteristic attributes of this rotational dynamic system. Finally, we found that an evolutionarily conserved glutamate receptor-associated protein, glutamate receptor-associated protein 1 (Grina), plays an important role in this process. By continuously monitoring the association between the attributes of the dynamic system and competitiveness, we found that restoring Grina expression effectively reversed the impact of historical loss on competitive performance. Together, our study reveals the rotational dynamics in the ventral hippocampus during competition and elucidates the underlying mechanisms through which historical loss shapes these processes.
Dopaminergic modulation and dosage effects on brain state dynamics and working memory component processes in Parkinson’s disease
Parkinson’s disease (PD) is primarily diagnosed through its characteristic motor deficits, yet it also encompasses progressive cognitive impairments that profoundly affect quality of life. While dopaminergic medications are routinely prescribed to manage motor symptoms in PD, their influence extends to cognitive functions as well. Here we investigate how dopaminergic medication influences aberrant brain circuit dynamics associated with encoding, maintenance and retrieval working memory (WM) task-phases processes. PD participants, both on and off dopaminergic medication, and healthy controls, performed a Sternberg WM task during fMRI scanning. We employ a Bayesian state-space computational model to delineate brain state dynamics related to different task phases. Importantly, a within-subject design allows us to examine individual differences in the effects of dopaminergic medication on brain circuit dynamics and task performance. We find that dopaminergic medication alters connectivity within prefrontal-basal ganglia-thalamic circuits, with changes correlating with enhanced task performance. Dopaminergic medication also restores engagement of task-phase-specific brain states, enhancing task performance. Critically, we identify an “inverted-U-shaped” relationship between medication dosage, brain state dynamics, and task performance. Our study provides valuable insights into the dynamic neural mechanisms underlying individual differences in dopamine treatment response in PD, paving the way for more personalized therapeutic strategies.
Responses