An Optimal Control Theory for Learning in the Brain

Join us on Tuesday, September 3rd from 1-3pm for a virtual talk by Alexander Meulemans, Research Scientist at Google, Paradigms of Intelligence Team. There will be a Q&A and discussion session after the talk.

Abstract: Understanding the human brain and developing artificial intelligence (AI) are among the most significant scientific endeavors of our time. Neuroscience and AI enjoy a symbiotic relationship: understanding the mechanisms of the human mind drives scientific discovery and inspires novel AI techniques, and conversely, AI offers valuable insights to conceptualize how the brain accomplishes specific tasks. In the work I will present, I focus on the concept of credit assignment, the process of assigning credit or blame to specific neural computations and synaptic connections for obtaining a behavioral outcome. Credit assignment is key for both rapid modulation of ongoing neural processes and long-term learning through synaptic change. Inspired by neuroscientific evidence of feedback signals guiding both functions, we develop the least-control principle, a control-centric credit assignment theory that unites rapid neural modulation with gradual synaptic learning. Our principle formulates learning as a least-control problem: error feedback optimally guides network dynamics towards states of minimal loss, while synaptic plasticity steadily reduces the necessary control effort to reach such states. The resulting learning rules are local in space and time, and compatible with feedforward and recurrent neural networks. In practice, our principle leads to strong performance matching that of leading gradient-based learning methods on feedforward and equilibrium networks. By showing a duality between our least-control principle and the expectation-maximization method for learning in latent-variable probabilistic models, we offer a fresh control-centric perspective on learning in the brain, complementary to previous theories such as the Bayesian Brain and Active Inference.