Leveraging Modern AI to Catalyze a New Era in Naturalistic Computational Cognitive Science
Kempner Research Fellow Wilka Carvalho and his collaborators have a plan to help cognitive scientists use AI to study human behavior in realistic settings
Wilka Carvalho (left) studies how AI can supplement traditional cognitive science theories and methodologies, providing a bridge between diverse approaches.
Photo credit: Anthony Tulliani
To better understand the mysteries of human cognition, scientists at the Kempner Institute are turning to a powerful new ally: artificial intelligence. Kempner Research Fellow Wilka Carvalho and his collaborators have developed an ambitious framework for using AI to broaden the scope of cognitive science, offering researchers new tools and methodologies to study how people think and make decisions in realistic contexts.
Carvalho and his collaborators recently published a series of three papers presenting this framework. The first lays out a broad theoretical case for what they call “naturalistic computational cognitive science.” The second introduces a software tool, NiceWebRL, that helps cognitive scientists design and analyze experiments in increasingly realistic online environments. The third uses NiceWebRL to test Carvalho’s new theory of how humans generalize to new goals in large worlds with many possible goals and objects.
The Case for Studying Cognition in “Naturalistic” Settings
One of Carvalho’s main research goals is to better understand how humans pursue multiple goals in naturalistic settings. The term “naturalistic” means that the settings, which may be virtual, reflect important aspects of the real world. Traditionally, computational cognitive scientists have developed models of sequential decision-making in small, simplified laboratory settings with few possible things that can be done.
But real life is more complicated. The world is often large with many possible things that can be done and people must juggle many goals at the same time in dynamic contexts that can involve multiple agents who might cooperate or compete.
“I want to study learning in more naturalistic settings,” Carvalho explains. Such studies, however, present significant technical challenges, which is why he is also focused on building tools that make it easier for researchers to add realism to their experiments.
In his first paper, “Naturalistic Computational Cognitive Science: Towards Generalizable Models and Theories That Capture the Full Range of Natural Behavior,” co-authored with Andrew Lampinen of Google DeepMind, Carvalho argues for expanding beyond simple experimental tasks. He proposes that AI can supplement traditional cognitive science without replacing earlier methods or theories, instead providing a bridge between diverse approaches.
A central opportunity, Carvalho notes, lies in creating models that can generalize, predicting behavior not only in controlled experiments but also across a wide range of naturalistic contexts.
For example, many cognitive science studies involve presenting humans with stimuli such as images. Carvalho points out that, traditionally, researchers have used simple images in such studies. One benefit of simple images is that they allow researchers to control every aspect of their appearance. Now, generative AI tools give researchers a similar level of control over complex, naturalistic images. This means that insights derived from studies that use simple images can be tested with realistic images without sacrificing precise experimental control.
Introducing NiceWebRL: An AI Tool for Human Experiments
Drawing on his computer science background, Carvalho sees opportunities for cognitive scientists to make better use of state-of-the-art AI and machine learning. His second paper, “NiceWebRL: a Python library for human subject experiments with reinforcement learning environments,” introduces NiceWebRL, a software package that streamlines the design and analysis of web-based human experiments that leverage machine learning tools and environments.
These experiments often resemble video games, where participants act as players pursuing goals in virtual environments. Web-based studies can recruit participants from diverse backgrounds far more easily than traditional lab-based research, but until now, applying AI tools to such studies has been difficult. AI software is typically written in the computer language Python, while web experiments usually rely on JavaScript.
NiceWebRL bridges this gap, allowing researchers to use Python-based tools to run experiments on the web, and to analyze the rich datasets collected from such experiments.
“There’s a growing ecosystem of interesting models and environments in machine learning,” Carvalho says. “I wanted to use those models for cognitive science.”
NiceWebRL lowers the technical barriers for cognitive scientists to leverage advanced AI tools, paving the way to new insights into how the human brain organizes information, learns, and adapts.
A New Theory of Human Generalization
Carvalho’s third paper, “Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machines,” builds on both the theoretical framework of naturalistic computational cognitive science and the methodological innovations of NiceWebRL. In it, he introduces a new algorithm called multitask preplay that is designed to model how humans pursue multiple goals.
The central idea is that people can learn about a future goal while actively working on a different one. Carvalho illustrates this with the example of exploring a new city: while walking to a shopping center, you might notice the entrance to a park. Later, when you decide to visit a park, your brain recalls that earlier observation: “you saw a park yesterday; go that way and take a right.”
Carvalho developed this theory in collaboration with Kempner associate faculty member Sam Gershman, Harvard postdoctoral researcher Sam Hall-McMaster, and University of Michigan professor Honglak Lee. Their preprint, “Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machines,” shows how to derive predictions from multitask preplay in simple settings and then test those predictions in more complex virtual environments using NiceWebRL.
Carvalho and his co-authors argue that if human brains are using a biological version of the preplay algorithm, then they will reuse paths that happened to be useful for prior goals, rather than attempting to take an untrodden path. The evidence from the paper supports this hypothesis. In fact, study participants were faster at reaching new goals in situations where they could reuse paths that they were already familiar with, even when the paths they took were not the fastest ones available.
Beyond serving as a testable theory of human behavior, multitask preplay also holds promise for artificial intelligence. Robots equipped with such algorithms could adapt more effectively to unfamiliar environments, drawing on prior experience to solve new problems.
“Robots are going to enter homes and factories,” Carvalho notes. “Each of those places is unique. A robot may bring prior knowledge, but it still has to learn about a new setting. Multitask preplay could make that process much more efficient, allowing robots to learn about many goals from every individual experience.”
***********
To learn more about the work of Carvalho and his collaborators, check out the three papers in the series:
- Carvalho, Wilka, and Andrew Lampinen. “Naturalistic Computational Cognitive Science: Towards generalizable models and theories that capture the full range of natural behavior.” arXiv preprint arXiv:2502.20349 (2025).
- Carvalho, Wilka, Vikram Goddla, Ishaan Sinha, Hoon Shin, and Kunal Jha. “NiceWebRL: a Python library for human subject experiments with reinforcement learning environments.” aXiv preprint arXiv:2508.1563 (2025).
- Carvalho, Wilka, Sam Hall-McMaster, Honglak Lee, and Samuel J. Gershman. “Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machines.” arXiv preprint arXiv:2507.05561 (2025).
About the Kempner Institute
The Kempner Institute seeks to understand the basis of intelligence in natural and artificial systems by recruiting and training future generations of researchers to study intelligence from biological, cognitive, engineering, and computational perspectives. Its bold premise is that the fields of natural and artificial intelligence are intimately interconnected; the next generation of artificial intelligence (AI) will require the same principles that our brains use for fast, flexible natural reasoning, and understanding how our brains compute and reason can be elucidated by theories developed for AI. Join the Kempner mailing list to learn more, and to receive updates and news.
PRESS CONTACT:
Deborah Apsel Lang | deborah_apsel_lang[at]harvard.edu
