Self-supervised video processing with self-calibration on an analogue computing platform based on a selector-less memristor array
Related Articles
Deep Bayesian active learning using in-memory computing hardware
Labeling data is a time-consuming, labor-intensive and costly procedure for many artificial intelligence tasks. Deep Bayesian active learning (DBAL) boosts labeling efficiency exponentially, substantially reducing costs. However, DBAL demands high-bandwidth data transfer and probabilistic computing, posing great challenges for conventional deterministic hardware. Here we propose a memristor stochastic gradient Langevin dynamics in situ learning method that uses the stochastic of memristor modulation to learn efficiency, enabling DBAL within the computation-in-memory (CIM) framework. To prove the feasibility and effectiveness of the proposed method, we implemented in-memory DBAL on a memristor-based stochastic CIM system and successfully demonstrated a robot’s skill learning task. The inherent stochastic characteristics of memristors allow a four-layer memristor Bayesian deep neural network to efficiently identify and learn from uncertain samples. Compared with cutting-edge conventional complementary metal-oxide-semiconductor-based hardware implementation, the stochastic CIM system achieves a remarkable 44% boost in speed and could conserve 153 times more energy.
The logarithmic memristor-based Bayesian machine
The demand for explainable and energy-efficient artificial intelligence (AI) systems for edge computing has led to growing interest in electronic systems dedicated to Bayesian inference. Traditional designs of such systems often rely on stochastic computing, which offers high energy efficiency but suffers from latency issues and struggles with low-probability values. Here, we introduce the logarithmic memristor-based Bayesian machine, an innovative design that leverages the unique properties of memristors and logarithmic computing as an alternative to stochastic computing. We present a prototype machine fabricated in a hybrid CMOS/hafnium-oxide memristor process. We validate the versatility and robustness of our system through experimental validation and extensive simulations in two distinct applications: gesture recognition and sleep stage classification. The logarithmic approach simplifies the computational model by converting multiplications into additions and enhances the handling of low-probability events, which are crucial in time-dependent tasks. Our results demonstrate that the logarithmic Bayesian machine achieves superior performance in terms of accuracy and energy efficiency compared to its stochastic counterpart, particularly in scenarios involving complex probabilistic models. This approach enables the development of energy-efficient and reliable AI systems for edge devices.
A spatiotemporal style transfer algorithm for dynamic visual stimulus generation
Understanding how visual information is encoded in biological and artificial systems often requires the generation of appropriate stimuli to test specific hypotheses, but available methods for video generation are scarce. Here we introduce the spatiotemporal style transfer (STST) algorithm, a dynamic visual stimulus generation framework that allows the manipulation and synthesis of video stimuli for vision research. We show how stimuli can be generated that match the low-level spatiotemporal features of their natural counterparts, but lack their high-level semantic features, providing a useful tool to study object recognition. We used these stimuli to probe PredNet, a predictive coding deep network, and found that its next-frame predictions were not disrupted by the omission of high-level information, with human observers also confirming the preservation of low-level features and lack of high-level information in the generated stimuli. We also introduce a procedure for the independent spatiotemporal factorization of dynamic stimuli. Testing such factorized stimuli on humans and deep vision models suggests a spatial bias in how humans and deep vision models encode dynamic visual information. These results showcase potential applications of the STST algorithm as a versatile tool for dynamic stimulus generation in vision science.
Brain inspired iontronic fluidic memristive and memcapacitive device for self-powered electronics
Ionic fluidic devices are gaining interest due to their role in enabling self-powered neuromorphic computing systems. In this study, we present an approach that integrates an iontronic fluidic memristive (IFM) device with low input impedance and a triboelectric nanogenerator (TENG) based on ferrofluid (FF), which has high input impedance. By incorporating contact separation electromagnetic (EMG) signals with low input impedance into our FF TENG device, we enhance the FF TENG’s performance by increasing energy harvesting, thereby enabling the autonomous powering of IFM devices for self-powered computing. Further, replicating neuronal activities using artificial iontronic fluidic systems is key to advancing neuromorphic computing. These fluidic devices, composed of soft-matter materials, dynamically adjust their conductance by altering the solution interface. We developed voltage-controlled memristor and memcapacitor memory in polydimethylsiloxane (PDMS) structures, utilising a fluidic interface of FF and polyacrylic acid partial sodium salt (PAA Na+). The confined ion interactions in this system induce hysteresis in ion transport across various frequencies, resulting in significant ion memory effects. Our IFM successfully replicates diverse electric pulse patterns, making it highly suitable for neuromorphic computing. Furthermore, our system demonstrates synapse-like learning functions, storing and retrieving short-term (STM) and long-term memory (LTM). The fluidic memristor exhibits dynamic synapse-like features, making it a promising candidate for the hardware implementation of neural networks. FF TENG/EMG device adaptability and seamless integration with biological systems enable the development of advanced neuromorphic devices using iontronic fluidic materials, further enhanced by intricate chemical designs for self-powered electronics.
Psychological booster shots targeting memory increase long-term resistance against misinformation
An increasing number of real-world interventions aim to preemptively protect or inoculate people against misinformation. Inoculation research has demonstrated positive effects on misinformation resilience when measured immediately after treatment via messages, games, or videos. However, very little is currently known about their long-term effectiveness and the mechanisms by which such treatment effects decay over time. We start by proposing three possible models on the mechanisms driving resistance to misinformation. We then report five pre-registered longitudinal experiments (Ntotal = 11,759) that investigate the effectiveness of psychological inoculation interventions over time as well as their underlying mechanisms. We find that text-based and video-based inoculation interventions can remain effective for one month—whereas game-based interventions appear to decay more rapidly—and that memory-enhancing booster interventions can enhance the diminishing effects of counter-misinformation interventions. Finally, we propose an integrated memory-motivation model, concluding that misinformation researchers would benefit from integrating knowledge from the cognitive science of memory to design better psychological interventions that can counter misinformation durably over time and at-scale.
Responses