Join us for a talk by Leyla Isik, Clare Boothe Luce Assistant Professor of Cognitive Science at John Hopkins University. This talk is part of the Kempner Seminar Series, a research-level seminar series that covers topics related to the basis of intelligence in natural and artificial systems.
Humans see the world in rich social detail. We effortlessly recognize not only objects and people in our environment, but also social interactions between people. The ability to perceive and understand others’ interactions is critical to function in our social world, yet the underlying neural computations remain poorly understood. In this talk, I will first argue that social interaction perception should be studied with the same computational vision tools that are now widely applied to other areas of vision, like scene and object recognition. I will then present our recent research using a large-scale, naturalistic video dataset and condition-rich fMRI experiment, demonstrating that social interaction information is extracted hierarchically by the visual system along the recently proposed lateral visual pathway.
In ongoing work with this same dataset, we find that, unlike static scene and object vision, current AI vision models do a poor job of matching human behavior and neural responses to dynamic, social scenes. To help close this gap, we developed a novel graph neural network model, SocialGNN, that instantiates insights from cognitive (neuro)science. SocialGNN reproduces human judgments of social interactions in both controlled and natural videos using only visual information, without any explicit model of agents’ minds or the physical world. Critically, the model’s relational, graph-based structure and processing are required for accurate social interaction recognition. Together, this research suggests that social interaction recognition is a core human ability that relies on structured visual representations, highlighting a path forward for better human-aligned AI.