Research Roundup

Papers are listed by publication/upload date, with the most recent first.

• Abreu, Natalie, Edwin Zhang, Eran Malach, and Naomi Saphra. “A Taxonomy of Transcendence.” arXiv preprint arXiv:2508.17669 (2025).
• Luo, Yan, Drake Du, Hao Huang, Yi Fang, and Mengyu Wang. “CurveFlow: Curvature-Guided Flow Matching for Image Generation.” arXiv preprint arXiv:2508.15093 (2025).
• Carvalho, Wilka, Vikram Goddla, Ishaan Sinha, Hoon Shin, and Kunal Jha. “NiceWebRL: a Python library for human subject experiments with reinforcement learning environments.” arXiv preprint arXiv:2508.15693 (2025).
• Hu, Jennifer, Felix Sosa, and Tomer Ullman. “Re-evaluating Theory of Mind evaluation in large language models.” Philosophical Transactions B 380, no. 1932 (2025): 20230499.
• Doshi, Fenil R., Talia Konkle, and George A. Alvarez. “A feedforward mechanism for human-like contour integration.” PLOS Computational Biology 21, no. 8 (2025): e1013391.
• Pan, Xu, Jingxuan Fan, Zidi Xiong, Ely Hahami, Jorin Overwiening, and Ziqian Xie. “User-Assistant Bias in LLMs.” arXiv preprint arXiv:2508.15815 (2025).
• Irie, Kazuki, and Samuel J. Gershman. “Fast weight programming and linear transformers: from machine learning to neurobiology.” arXiv preprint arXiv:2508.08435 (2025).
• Yeh, Catherine, Tara Menon, Robin Singh Arya, Helen He, Moira Weigel, Fernanda Viégas, and Martin Wattenberg. “Story Ribbons: Reimagining Storyline Visualizations with Large Language Models.” arXiv preprint arXiv:2508.06772 (2025).
• Ahdritz, Gustaf, and Anat Kleiman. “The SMeL Test: A simple benchmark for media literacy in language models.” arXiv preprint arXiv:2508.02074 (2025).
• Liu, Shuze, Atsushi Kikumoto, David Badre, and Samuel J. Gershman. “Neural and behavioral signatures of policy compression in cognitive control.” Cerebral Cortex (2025): 35(8).
• Martinez, Joel E., Rachel H. Krasner, Laura Rosero, Samuel J. Gershman, and Mina Cikara. “Social group discovery, structure, and stereotype updating.” Journal of Experimental Psychology: General (2025).
• Cohen, Zach, and Jan Drugowitsch. “A unifying theory of receptive field heterogeneity predicts hippocampal spatial tuning.” bioRxiv (2025): 2025-07.
• Geuter, Jonathan, Gregor Kornhardt, Ingimar Tomasson, and Vaios Laschos. “Universal Neural Optimal Transport.” In Forty-second International Conference on Machine Learning. 2025.

Papers are listed by publication/upload date, with the most recent first.

• Kumar, M. Ganesh, Adam Manoogian, William Qian, Cengiz Pehlevan, and Shawn A. Rhoads. “Neurocomputational Underpinnings of Suboptimal Beliefs in Reinforcement Learning Agents.” In 8th Annual Conference on Cognitive Computational Neuroscience.
• Zavatone-Veth, Jacob A., and Cengiz Pehlevan. “A note on the dynamics of extended-context disordered kinetic spin models.” arXiv preprint arXiv:2507.18461 (2025).
• Thasarathan, Harrish, Julian Forsyth, Thomas Fel, Matthew Kowal, and Konstantinos G. Derpanis. “Universal sparse autoencoders: Interpretable cross-model concept alignment.” In Forty-second International Conference on Machine Learning. 2025.
• Keller, T. Anderson. “Flow Equivariant Recurrent Neural Networks.” arXiv preprint arXiv:2507.14793 (2025).
• Sajid, Noor, and Johan Medrano. “Dissociating model architectures from inference computations.” Cognitive Neuroscience (2025): 1-3.
• Song, Yue, T. Anderson Keller, Yisong Yue, Pietro Perona, and Max Welling. “Langevin Flows for Modeling Neural Latent Dynamics.” arXiv preprint arXiv:2507.11531 (2025).
• Zhang, Huaqing, Bingbin Liu, Juno Kim, and Andrej Risteski. “Improving Pathfinding with Anchoring Tokens.” In ICML 2025 Workshop on Methods and Opportunities at Small Scale.
• Overwiening, Jorin, M. Ganesh Kumar, and Haim Sompolinsky. “TeDFA-δ: Temporal integration in deep spiking networks trained with feedback alignment improves policy learning.” In 8th Annual Conference on Cognitive Computational Neuroscience.
• Briden, Julia, Yilun Du, Enrico M. Zucchelli, and Richard Linares. “Compositional Diffusion Models for Powered Descent Trajectory Generation with Flexible Constraints.” In 2025 IEEE Aerospace Conference, pp. 1-19. IEEE, 2025.
• Lu, Yue M., Mary Letey, Jacob A. Zavatone-Veth, Anindita Maiti, and Cengiz Pehlevan. “Asymptotic theory of in-context learning by linear attention.” Proceedings of the National Academy of Sciences 122, no. 28 (2025): e2502599122.
• Carvalho, Wilka, Sam Hall-McMaster, Honglak Lee, and Samuel J. Gershman. “Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machines.” arXiv preprint arXiv:2507.05561 (2025).
• Li, Victoria R., Jenny Kaufmann, Martin Wattenberg, David Alvarez-Melis, and Naomi Saphra. “Can Interpretation Predict Behavior on Unseen Data?” arXiv preprint arXiv:2507.06445 (2025).
• Kong, Zhenglun, Mufan Qiu, John Boesen, Xiang Lin, Sukwon Yun, Tianlong Chen, Manolis Kellis, and Marinka Zitnik. “SPATIA: Multimodal Model for Prediction and Generation of Spatial Cell Phenotypes.” arXiv preprint arXiv:2507.04704 (2025).
• Karuvally, Arjun, Franz Nowak, Anderson T. Keller, Carmen Amo Alonso, Terrence J. Sejnowski, and Hava T. Siegelmann. “Bridging Expressivity and Scalability with Adaptive Unitary SSMs.” arXiv preprint arXiv:2507.05238 (2025).
• Lauditi, Clarissa, Blake Bordelon, and Cengiz Pehlevan. “Transfer Learning in Infinite Width Feature Learning Networks.” arXiv preprint arXiv:2507.04448 (2025).
• Andrade, Jeffery, George Alvarez, and Talia Konkle. “Dissecting sparse circuits to high-level visual categories in deep neural networks.” Journal of Vision 25, no. 9 (2025): 2305-2305.
• Zitnik, Marinka. “AI-enabled drug discovery reaches clinical milestone.” Nature Medicine.
• Balaban, Halely, and Tomer D. Ullman. “The capacity limits of moving objects in the imagination.” Nature Communications 16, no. 1 (2025): 5899.
• Doshi, Fenil R., Thomas Fel, Talia Konkle, and George Alvarez. “Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models.” arXiv preprint arXiv:2507.00493 (2025).
• Doshi, Fenil R., Thomas Fel, Talia Konkle, and George A. Alvarez. “Towards Holistic Vision in Deep Neural Networks: Disentangling Local and Global Processing.” Journal of Vision 25, no. 9 (2025): 2148-2148.
• Prince, Jacob S., Binxu Wang, Akshay V. Jagadeesh, Thomas Fel, Emily Lo, George A. Alvarez, Margaret S. Livingstone, and Talia Konkle. “Feature accentuation along the encoding axes of IT neurons uncovers hidden differences in model-brain alignment.” Journal of Vision 25, no. 9 (2025): 2215-2215.
• Saha, Srijani, George Alvarez, and Talia Konkle. “Edge-based Image Reconstruction Provides a Unified Account of (many) Lightness Illusions.” Journal of Vision 25, no. 9 (2025): 2179-2179.
• Xiang, Yang, Eric Bigelow, Tobias Gerstenberg, Tomer D. Ullman, and Samuel Gershman. “Language models assign responsibility based on actual rather than counterfactual contributions.” In Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 47. 2025.
• Xiang, Yang, and Samuel Gershman. “Modeling intrinsic motivation as reflective planning.” In Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 47. 2025.
• Liu, Shuze, Samuel Gershman, and Bilal Bari. “Quantifying the cost of context sensitivity in decision making.” In Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 47. 2025.
• Wang, Binxu, and Carlos Ponce. “Factorized convolution models for predicting and interpreting neuronal tuning in natural images.” Journal of Vision 25, no. 9 (2025): 2899-2899.
• Hu, Jennifer, Felix Anthony Sosa, and Tomer D. Ullman. “Making Sense of Nonsense.” In Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 47. 2025.
• Alkhadrawi, Adham M., Valeria Peña-Trujillo, Sebastian Gallo-Bernal, Michael S. Gee, Camilo E. Jaimes Cobos, Kyungsu Kim, Saul Langarica, Young-Tak Kim, Teresa Victoria, and Synho Do. “ARANet: Adaptive Resolution Attention Network for Precise MRI-Based Segmentation and Quantification of Fetal Size and Amniotic Fluid Volume.” Journal of Imaging Informatics in Medicine (2025): 1-15.

Papers are listed by publication/upload date, with the most recent first.

• Murthy, Sonia K., Rosie Zhao, Jennifer Hu, Sham Kakade, Markus Wulfmeier, Peng Qian, and Tomer Ullman. “Inside you are many wolves: Using cognitive models to interpret value trade-offs in LLMs.” arXiv preprint arXiv:2506.20666 (2025).
• Fang, Ching, and Kanaka Rajan. “From memories to maps: Mechanisms of in context reinforcement learning in transformers.” arXiv preprint arXiv:2506.19686 (2025).
• Meterez, Alexandru, Depen Morwani, Costin-Andrei Oncescu, Jingfeng Wu, Cengiz Pehlevan, and Sham Kakade. “A Simplified Analysis of SGD for Linear Regression with Weight Averaging.” arXiv preprint arXiv:2506.15535 (2025).
• Hu, Yifan, Frank Liang, Dachuan Zhao, Jonathan Geuter, Varshini Reddy, Craig W. Schmidt, and Chris Tanner. “Entropy-Driven Pre-Tokenization for Byte-Pair Encoding.” arXiv preprint arXiv:2506.15889 (2025).
• Kangaslahti, Sara, Elan Rosenfeld, and Naomi Saphra. “Hidden Breakthroughs in Language Model Training.” arXiv preprint arXiv:2506.15872 (2025).
• Finn, Emma, T. Anderson Keller, Manos Theodosis, and Demba E. Ba. “Origins of Creativity in Attention-Based Diffusion Models.” arXiv preprint arXiv:2506.17324 (2025).
• Bhattacharya, Antara Raaghavi, Isabel Papadimitriou, Kathryn Davidson, and David Alvarez-Melis. “Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles.” arXiv preprint arXiv:2506.13886 (2025).
• Li, Michelle M., Ben Y. Reis, Adam Rodman, Tianxi Cai, Noa Dagan, Ran D. Balicer, Joseph Loscalzo, Isaac S. Kohane, and Marinka Zitnik. “One Patient, Many Contexts: Scaling Medical AI Through Contextual Intelligence.” arXiv preprint arXiv:2506.10157 (2025).
• Singhvi, Divyansh, Diganta Misra, Andrej Erkelens, Raghav Jain, Papadimitriou, Isabel, and Naomi Saphra. “Using Shapley interactions to understand how models use structure.” arXiv preprint arXiv:2403.13106 (2025).
• Simmons-Edler, Riley, Ryan P. Badman, Felix Baastad Berg, Raymond Chua, John J. Vastola, Joshua Lunger, William Qian, and Kanaka Rajan. “Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments.” arXiv preprint arXiv:2506.06981 (2025).
• Geuter, Jonathan, Youssef Mroueh, and David Alvarez-Melis. “Guided Speculative Inference for Efficient Test-Time Alignment of LLMs.” arXiv preprint arXiv:2506.04118 (2025).
• Costa, Valérie, Thomas Fel, Ekdeep Singh Lubana, Bahareh Tolooshams, and Demba Ba. “From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit.” arXiv preprint arXiv:2506.03093 (2025).
• Irie, Kazuki, Morris Yau, and Samuel J. Gershman. “Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers.” arXiv preprint arXiv:2506.00744 (2025).
• Qin, Tian, Core Francisco Park, Mujin Kwun, Aaron Walsman, Eran Malach, Nikhil Anand, Hidenori Tanaka, and David Alvarez-Melis. “Decomposing Elements of Problem Solving: What” Math” Does RL Teach?” arXiv preprint arXiv:2505.22756 (2025).
• Brantley, Kianté, Mingyu Chen, Zhaolin Gao, Jason D. Lee, Wen Sun, Wenhao Zhan, and Xuezhou Zhang. “Accelerating RL for LLM Reasoning with Optimal Advantage Regression.” arXiv preprint arXiv:2505.20686 (2025).
• Mirtaheri, Parsa, Ezra Edelman, Samy Jelassi, Eran Malach, and Enric Boix-Adsera. “Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones.” arXiv preprint arXiv:2505.21825 (2025).
• Li, Kevin, and Marinka Zitnik. “Prompting Decision Transformers for Zero-Shot Reach-Avoid Policies.” arXiv preprint arXiv:2505.19337 (2025).
• Gholamzadeh, Ali, and Noor Sajid. “Model alignment using inter-modal bridges.” arXiv preprint arXiv:2505.12322 (2025).
• Xiang, Yang and Samuel J. Gershman. “Modeling intrinsic motivation as reflective planning.” Proceedings of the 47th Annual Cognitive Science Society. (2025)
• Tolooshams, Bahareh, Sara Matias, Hao Wu, Simona Temereanca, Naoshige Uchida, Venkatesh N. Murthy, Paul Masset, and Demba Ba. “Interpretable deep learning for deconvolutional analysis of neural signals.” Neuron 113, no. 8 (2025): 1151-1168.

Papers are listed by publication/upload date, with the most recent first.

• Goldberg, Adele E., Supantho Rakshit, Jennifer Hu, and Kyle Mahowald. “A suite of LMs comprehend puzzle statements as well as humans.” arXiv preprint arXiv:2505.08996 (2025).
• Velez-Arce, Alejandro, and Marinka Zitnik. “PyTDC: A multimodal machine learning training, evaluation, and inference platform for biomedical foundation models.” arXiv preprint arXiv:2505.05577 (2025).
• Reinhold, Kimberly, Marci Iadarola, Shi Tang, Annabel Chang, Whitney Kuwamoto, Madeline A. Albanese, Senmiao Sun et al. “Striatum supports fast learning but not memory recall.” Nature (2025): 1-10.
• Lee, Jaeeon, and Bernardo L. Sabatini. “From avoidance to new action: the multifaceted role of the striatal indirect pathway.” Nature Reviews Neuroscience (2025): 1-12.
• Li, Kenneth, Yida Chen, Fernanda Viégas, and Martin Wattenberg. “When Bad Data Leads to Good Models.” arXiv preprint arXiv:2505.04741 (2025).
• Shen, Wanxiang, Thinh H. Nguyen, Michelle Min Rui Li, Yepeng Huang, Intae Moon, Nitya Nair, Daniel Marbach, and Marinka Zitnik. “Generalizable AI predicts immunotherapy outcomes across cancers and treatments.” medRxiv (2025): 2025-05.
• Park, Core Francisco, Zechen Zhang, and Hidenori Tanaka. “New News: System-2 Fine-tuning for Robust Integration of New Knowledge.” arXiv preprint arXiv:2505.01812 (2025).
• Xiang, Yang, Eric Bigelow, Tobias Gerstenberg, Tomer Ullman, and Samuel J. Gershman. “Language models assign responsibility based on actual rather than counterfactual contributions.” Proceedings of the 47th Annual Cognitive Science Society. (2025)
• Pan, Xu, Ely Hahami, Zechen Zhang, and Haim Sompolinsky. “Memorization and Knowledge Injection in Gated LLMs.” arXiv preprint arXiv:2504.21239 (2025).
• Avidan, Yehonatan, Qianyi Li, and Haim Sompolinsky. “Unified theoretical framework for wide neural network learning dynamics.” Physical Review E 111, no. 4 (2025): 045310.
• Wu, Jiageng, Bowen Gu, Ren Zhou, Kevin Xie, Doug Snyder, Yixing Jiang, Valentina Carducci et al. “BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text.” arXiv preprint arXiv:2504.19467 (2025).
• Casto, Colton, Hannah Small, Moshe Poliak, Greta Tuckute, Benjamin Lipkin, Agata Wolna, Anila M. D’Mello, and Evelina Fedorenko. “The cerebellar components of the human language network.” bioRxiv (2025): 2025-04.
• Tong, William L., and Cengiz Pehlevan. “Learning richness modulates equality reasoning in neural networks.” CCN Proceedings. (2025).
• Qian, Lechen, Mark Burrell, Jay A. Hennig, Sara Matias, Venkatesh N. Murthy, Samuel J. Gershman, and Naoshige Uchida. “Prospective contingency explains behavior and dopamine signals during associative learning.” Nature Neuroscience (2025): 1-13.
• Kumar, M. Ganesh, Adam Manoogian, Billy Qian, Cengiz Pehlevan, and Shawn A. Rhoads. “Neurocomputational underpinnings of suboptimal beliefs in recurrent neural network-based agents.” bioRxiv (2025): 2025-03.
• Boero, Luis, Hao Wu, Joseph D. Zak, Paul Masset, Farhad Pashakhanloo, Siddharth Jayakumar, Bahareh Tolooshams, Demba Ba, and Venkatesh N. Murthy. “Perception and neural representation of intermittent odor stimuli in mice.” bioRxiv (2025): 2025-02.
• Kumar, M. Ganesh, Blake Bordelon, Jacob A. Zavatone-Veth, and Cengiz Pehlevan. “A Model of Place Field Reorganization During Reward Maximization.” Proceedings of the 42nd International Conference on Machine Learning (ICML). (2025)

Papers are listed by publication/upload date, with the most recent first.

• Zavatone-Veth, Jacob A., Blake Bordelon, and Cengiz Pehlevan. “Summary statistics of learning link changing neural representations to behavior.” arXiv preprint arXiv:2504.16920 (2025).
• Hu, Jennifer, Michael A. Lepori, and Michael Franke. “Linking forward-pass dynamics in Transformers and real-time human processing.” arXiv preprint arXiv:2504.14107 (2025).
• Jha, Kunal, Wilka Carvalho, Yancheng Liang, Simon S. Du, Max Kleiman-Weiner, and Natasha Jaques. “Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination.” arXiv preprint arXiv:2504.12714 (2025).
• Papadimitriou, Isabel, Huangyuan Su, Thomas Fel, Naomi Saphra, Sham Kakade, and Stephanie Gil. “Interpreting the Linear Structure of Vision-language Model Embedding Spaces.” arXiv preprint arXiv:2504.11695 (2025).
• Yu, Yemin, Neil Tenenholtz, Lester Mackey, Ying Wei, David Alvarez-Melis, Ava P. Amini, and Alex X. Lu. “Causal integration of chemical structures improves representations of microscopy images for morphological profiling.” arXiv preprint arXiv:2504.09544 (2025).
• Zhang, Yuyang, Xinhe Zhang, Jia Liu, and Na Li. “Sample Efficient Algorithms for Linear System Identification under Noisy Observations.” arXiv preprint arXiv:2504.09057 (2025).
• Zhao, Rosie, Alexandru Meterez, Sham Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. “Echo chamber: Rl post-training amplifies behaviors learned in pretraining.” arXiv preprint arXiv:2504.07912 (2025).
• van Meegen, Alexander, and Haim Sompolinsky. “Coding schemes in neural networks learning classification tasks.” Nature Communications 16, no. 1 (2025): 3354.
• Qin, Tian, David Alvarez-Melis, Samy Jelassi, and Eran Malach. “To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoning.” arXiv preprint arXiv:2504.07052 (2025).
• Fang, Ada, Zaixi Zhang, Andrew Zhou, and Marinka Zitnik. “ATOMICA: Learning Universal Representations of Intermolecular Interactions.” bioRxiv (2025): 2025-04.  
• Liu, Shuze, Lucy Lai, Samuel J. Gershman, and Bilal A. Bari. “Time and memory costs jointly determine a speed–accuracy trade-off and set-size effects.” Journal of Experimental Psychology: General (2025).
• Gershman, Samuel J., Johannes Bill, and Jan Drugowitsch. “Hierarchical Vector Analysis of Visual Motion Perception.” Annual Review of Vision Science 11 (2025).
• Ahmed, Zergham, Joshua B. Tenenbaum, Christopher J. Bates, and Samuel J. Gershman. “Synthesizing world models for bilevel planning.” arXiv preprint arXiv:2503.20124 (2025).

Papers are listed by publication/upload date, with the most recent first.

• Fish, Sara, Julia Shephard, Minkai Li, Ran I. Shorrer, and Yannai A. Gonczarowski. “EconEvals: Benchmarks and Litmus Tests for LLM Agents in Unknown Environments.” arXiv preprint arXiv:2503.18825 (2025).
• Liu, Shuze, Yang Xiang, and Samuel J. Gershman. “Probabilistic Forecasting Guides Dynamic Decisions.” PsyArXiv. (2025).
• Gao, Shanghua, Richard Zhu, Zhenglun Kong, Ayush Noori, Xiaorui Su, Curtis Ginder, Theodoros Tsiligkaridis, and Marinka Zitnik. “TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Tools.” arXiv preprint arXiv:2503.10970 (2025).
• Tong, William L., and Cengiz Pehlevan. “Learning richness modulates equality reasoning in neural networks.” arXiv preprint arXiv:2503.09781 (2025).
• van der Wal, Oskar, Pietro Lesci, Max Muller-Eberstein, Naomi Saphra, Hailey Schoelkopf, Willem Zuidema, and Stella Biderman. “PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs.” arXiv preprint arXiv:2503.09543 (2025).
• Song, Siyuan, Jennifer Hu, and Kyle Mahowald. “Language Models Fail to Introspect About Their Knowledge of Language.” arXiv preprint arXiv:2503.07513 (2025).
• Wang, Binxu. “An Analytical Theory of Power Law Spectral Bias in the Learning Dynamics of Diffusion Models.” arXiv preprint arXiv:2503.03206. (2025).
• Huang, Yepeng, Xiaorui Su, Varun Ullanat, Ivy Liang, Lindsay Clegg, Damilola Olabode, Nicholas Ho, Bino John, Megan Gibbs, and Marinka Zitnik. “Multimodal AI predicts clinical outcomes of drug combinations from preclinical data.” arXiv preprint arXiv:2503.02781 (2025).
• Geuter, Jonathan, Clément Bonet, Anna Korba, and David Alvarez-Melis. “DDEQs: Distributional Deep Equilibrium Models through Wasserstein Gradient Flows.” arXiv preprint arXiv:2503.01140 (2025).
• Choudhary, Shubham, Paul Masset, and Demba Ba. “Implicit Generative Modeling by Kernel Similarity Matching.” arXiv preprint arXiv:2503.00655 (2025).
• Carvalho, Wilka, and Andrew Lampinen. “Naturalistic Computational Cognitive Science: Towards generalizable models and theories that capture the full range of natural behavior.” arXiv preprint arXiv:2502.20349 (2025).

This issue of the Kempner Research Roundup features publications, preprints, and awards from March 2024 to May 2024.

* indicates equal contribution ^† indicates Kempner community member

Publications

Q-Probe: A Lightweight Approach to Reward Maximization for Language Models
Kenneth Li^†, Samy Jelassi, Hugh Zhang, Sham Kakade, Martin Wattenberg, David Brandfonbrener
ICML (2024)

Feature-selective responses in macaque visual cortex follow eye movements during natural vision
Xiao W, Sharma S, Kreiman G^†, Livingstone MS
Nature Neuroscience (2024)

Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs
Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L. Leavitt, Naomi Saphra^†
ICLR (2024)

TRAM: Bridging Trust Regions and Sharpness Aware Minimization
Tom Sherborne, Naomi Saphra^†, Pradeep Dasigi, Hao Peng
ICLR (2024)

First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models
Naomi Saphra^†, Eve Fleisig, Kyunghyun Cho, Adam Lopez
NAACL (2024)

Place fields organize along goal trajectory with reinforcement learning
M Ganesh Kumar^†, Cengiz Pehlevan^†
Cognitive Computational Neuroscience (2024)

AI-Powered Autonomous Weapons Risk Geopolitical Instability and Threaten AI Research
Riley Simmons-Edler*^†, Ryan Badman*^†, Shayne Longpre, Kanaka Rajan^†
ICML (2024)

The Learning Hypothesis on Spatial Receptive Field Remapping
Henry Kuo, Paul Masset, Blake Bordelon^†, Cengiz Pehlevan^†
Cognitive Computational Neuroscience (2024)

Shades of Zero: Distinguishing impossibility from inconceivability
Jennifer Hu*, Felix Sosa*, Tomer Ullman^†
CogSci (2024)

Traveling Waves Encode The Recent Past and Enhance Sequence Learning
T. Anderson Keller^†, Lyle Muller, Terrence Sejnowski, Max Welling
ICLR (2024)

Clinical Text Datasets for Medical Artificial Intelligence and Large Language Models — A Systematic Review
Jiageng Wu*, Xiaocong Liu*, Minghui Li Li, Wanxin Li, Zichang Su Su, Garay Lucas, Zhiyun Zhang, Yujie Zhang, Qingcheng Zeng, Jie Shen, Changzheng Yuan, Jie Yang†
The New England Journal of Medicine AI

Self-Organising Neural Discrete Representation Learning à la Kohonen
Kazuki Irie*†, Róbert Csordás*, Jürgen Schmidhuber
International Conference on Artificial Neural Networks (2024)

Preprints

Auxiliary task demands mask the capabilities of smaller language models
Jennifer Hu^†, Michael C. Frank

One nose but two nostrils: Learn to align with sparse connections between two olfactory cortices
Bo Liu, Shanshan Qin, Venkatesh Murthy^†, Yuhai Tu

MLPs Learn In-Context
William L. Tong^†, Cengiz Pehlevan^†

Partial observation can induce mechanistic mismatches in data-constrained models of neural dynamics
William Qian^†, Jacob A. Zavatone-Veth, Benjamin S. Ruben, Cengiz Pehlevan^†

Time and memory costs jointly determine a speed-accuracy trade-off and set-size effects
Shuze Liu^†, Lucy Lai, Samuel Joseph Gershman^†, Bilal Abdul Bari

A Dynamical Model of Neural Scaling Laws
Blake Bordelon, Alexander Atanasov, Cengiz Pehlevan^†

News

Haim Sompolinsky attended the Brain Prize ceremony in Copenhagen on May 29, where he was honored as a recipient of the 2024 Brain Prize.

Associate Faculty member Marinka Zitnik was awarded an NSF CAREER award.

Affiliate Faculty member Gabriel Kreiman’s work was featured in the Harvard Medical School Magazine: The limits of computer vision and our own.

This issue of the Kempner Research Roundup features publications, preprints, and awards from December 2023 to February 2024.

* indicates equal contribution
^† indicates Kempner community member

Awards

Affiliate Faculty member Melanie Weber was selected as a 2024 Alfred P. Sloan Fellow in Mathematics!

Publications

Exploring the Promise and Limits of Real-Time Recurrent Learning
Kazuki Irie^†, Anand Gopalakrishnan, Jürgen Schmidhuber
ICLR (2024)

Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs
Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L. Leavitt, Naomi Saphra^†
ICLR (2024)

Dynamic Masking Rate Schedules for MLM Pretraining
Zachary Ankner, Naomi Saphra^†, Davis Blalock, Jonathan Frankle^†, Matthew L. Leavitt
ICLR (2024)

Effective Structural Encodings via Local Curvature Profiles
Lukas Fesser, Melanie Weber^†
ICLR (2024)

Forward learning with top-down feedback: empirical and analytical characterization
Srinivasan Ravi, Mignacco Francesca, Sorboro Martino, Refinetti Maria, Cooper Avi, Kreiman Gabriel^†, Dellaferrera Giorgia
ICLR (2024)

On the hardness of learning under symmetries
Bobak Kiani*, Thien Le*, Hannah Lawrence*, Stefanie Jegelka, Melanie Weber^†
ICLR (2024)

TRAM: Bridging Trust Regions and Sharpness Aware Minimization
Tom Sherborne, Naomi Saphra^†, Pradeep Dasigi, Hao Peng
EACL (2024)

Transformable linkage-based gripper for multi-mode grasping and manipulation
Junghan Kwon*, David Bombara*, Clark Teeple, Joonhaeng Lee, Chuck Hoberman, Robert Wood, Justin Werfel^†
IEEE Robotics and Automation Letters (December 2023)

Tuned compositional feature replays for efficient stream learning
Talbot Morgan, Zawar Rushikesh, Badkundri Rohil, Zhang Mengmi, Kreiman Gabriel^†
IEEE Transactions in Neural Networks and Learning Systems (December 2023)

Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions
Kazuki Irie^†, Róbert Csordás, Jürgen Schmidhuber
EMNLP (2023) [short paper]

Approximating Two-Layer Feedforward Networks for Efficient Transformers
Róbert Csordás, Kazuki Irie^†, Jürgen Schmidhuber
Findings of EMNLP (2023)

Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck
Benjamin L. Edelman^†, Surbhi Goel, Sham Kakade^†, Eran Malach†, Cyril Zhang
NeurIPS (2023) [spotlight]

Combining Behaviors with the Successor Features Keyboard
Wilka Carvalho^†, Andre Saraiva, Angelos Filos, Andrew Kyle Lampinen, Loic Matthey, Richard L. Lewis, Honglak Lee, Satinder Singh, Danilo J. Rezende, Daniel Zoran
NeurIPS (2023)

Contrastive Training of Complex-Valued Autoencoders for Object Discovery
Aleksandar Stanić*, Anand Gopalakrishnan*, Kazuki Irie^†, Jürgen Schmidhuber
NeurIPS (2023)

The Hidden Linear Structure in Score-Based Models and its Application
Binxu Wang^†, John Vastola
Workshop on Diffusion Models @ NeurIPS (2023)

Understanding Learning Dynamics of Neural Representations via Feature Visualization at Scale
Chandana Kuntala, Carlos R Ponce, Deepak Kumar Sharma, Binxu Wang^†
UniReps (Unifying Representations in Neural Models) Workshop @ NeurIPS (2023)

Preprints

Adaptive algorithms for shaping behavior
William L. Tong^†, Anisha Ayer, Venkatesh N. Murthy^†, Gautam Reddy

Q-Probe: A Lightweight Approach to Reward Maximization for Language Models
Kenneth Li^†, Samy Jelassi, Hugh Zhang^†, Sham Kakade^†, Martin Wattenberg, David Brandfonbrener^†

Measuring and Controlling Persona Drift in Language Model Dialogs
Kenneth Li^†, Tianle Liu, Naomi Bashkansky, David Bau, Fernanda Viégas^†, Hanspeter Pfister, Martin Wattenberg^†

A Dynamical Model of Neural Scaling Laws
Blake Bordelon, Alexander Atanasov, Cengiz Pehlevan^†

This inaugural issue of the Kempner Research Roundup highlights publications, preprints, and awards from throughout 2023.

* indicates equal contribution
^† indicates Kempner community member

Awards

Associate Faculty member Cengiz Pehlevan won an NSF Career Award for his project titled Developing Neural Network Theory for Uncovering How the Brain Learns!

NeurIPS 2023 Acceptances

Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency[spotlight]
Owen Queen, Thomas Hartvigsen, Teddy Koker, Huan He, Theodoros Tsiligkaridis, Marinka Zitnik^†

Full-Atom Protein Pocket Design via Iterative Refinement [spotlight]
Zaixi Zhang, Zepu Lu, Zhongkai Hao, Marinka Zitnik^†, Qi Liu

Inference-Time Intervention: Eliciting Truthful Answers from a Language Model [spotlight]
Kenneth Li*^†, Oam Patel*, Fernanda Viégas, Hanspeter Pfister, Martin Wattenberg^†

Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks [spotlight]
Blake Bordelon, Cengiz Phlevan^†

Learning Curves for Heterogeneous Feature-Subsampled Ridge Ensembles
Benjamin S. Ruben, Cengiz Pehlevan^†

Neural circuits for fast Poisson compressed sensing in the olfactory bulb
Jacob A. Zavatone-Veth*, Paul Masset*, William L. Tong*^†, Joseph D. Zak, Venkatesh N. Murthy*^†, Cengiz Pehlevan*^†

Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry
Bariscan Bozkurt, Cengiz Pehlevan^†, Alper T Erdogan

Long Sequence Hopfield Memory
Hamza Tahir Chaudhry*, Jacob A. Zavatone-Veth*, Dmitry Krotov, Cengiz Pehlevan^†

Learning curves for deep structured Gaussian feature models
Jacob A. Zavatone-Veth, Cengiz Pehlevan^†

Cognitive Steering in Deep Neural Networks via Long-Range Modulatory Feedback Connections
Talia Konkle^†, George Alvarez^†, Scaling Data-Constrained Language Models
Niklas Muennighoff, Alexander M. Rush, Boaz Barak^†, Teven Le Scao, Aleksandra Piktus, Nouamane Tazi, Sampo Pyysalo, Thomas Wolf, Colin Raffel

Successor-predecessor intrinsic exploration
Changmin Yu, Neil Burgess, Maneesh Sahani, Samuel J. Gershman^†

Combining Behaviors with the Successor Features Keyboard
Wilka Carvalho†, Andre Saraiva, Angelos Filos, Andrew Kyle Lampinen, Loic Matthey, Richard L. Lewis, Honglak Lee, Satinder Singh, Danilo J. Rezende, Daniel Zoran

Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitation
David Brandfonbrener^†, Ofir Nachum, Joan Bruna

Feature-Learning Networks Are Consistent Across Widths At Realistic Scales
Nikhil Vyas*, Alexander Atanasov*, Blake Bordelon*, Depen Morwani, Sabarish Sainathan^†, Cengiz Pehlevan^†

Dynamics of Temporal Difference Reinforcement Learning
Blake Bordelon, Paul Masset, Henry Kuo, Cengiz Pehlevan^†

Other Publications

Prompting is not a substitute for probability measurements in large language models
Jennifer Hu^†, Roger Levy
EMNLP (2023)

Comparing the Evaluation and Production of Loophole Behavior in Humans and Large Language Models
Sonia K. Murthy^†, Kiera Parece, Sophie Bridgers, Peng Qian, Tomer Ullman^†
Findings of EMNLP (2023)

Emergence of belief-like representations through reinforcement learning
Jay Hennig, Sandra Romero Pinto, Takahiro Yamaguchi, Scott Linderman, Naoshige Uchida, Samuel Gershman^†
PLoS Computational Biology (2023)

Learning Compositional Tasks from Language Instructions
Lajanugen Logeswaran, Wilka Carvalho^†, Honglak Lee
Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI) (2023)

Composing Task Knowledge With Modular Successor Feature Approximators
Wilka Carvalho^†, Angelos Filos, Richard L. Lewis, Honglak Lee, Satinder Singh
ICLR (2023)

Transformable linkage-based gripper for multi-mode grasping and manipulation
Junghan Kwon*, David Bombara*, Clark Teeple, Joonhaeng Lee, Chuck Hoberman, Robert Wood, Justin Werfel†
IEEE Robotics and Automation Letters (2023)

Comparing the Evaluation and Production of Loophole Behavior in Children and Large Language Models
Sonia K. Murthy^†, Sophie Bridgers, Kiera Parece, Elena L. Glassman, Tomer Ullman
First Workshop on Theory of Mind in Communicating Agents, ICML (2023)

Cortical topographic motifs emerge in a self-organized map of object space
Fenil R. Doshi^†, Talia Konkle^†
Science Advances (2023)

The neural architecture of theory-based reinforcement learning
Momchil S. Tomov, Pedro A. Tsividis, Thomas Pouncy, Joshua B. Tenenbaum, Samuel J. Gershman^†
Neuron (2023)

Face-deprived networks show distributed but not clustered face-selective maps
Fenil R. Doshi^†, Talia Konkle^†
Vision Sciences Society (2023)

Feedforward Neural Networks can capture Human-like Perceptual and Behavioral Signatures of Contour Integration
Fenil R. Doshi^†, Talia Konkle^†, George Alvarez^†
Cognitive Computational Neuroscience (CCN) (2023)

High-performance evolutionary algorithms for online neuron control
Binxu Wang^†, Carlos R. Ponce
Proceedings of the Genetic and Evolutionary Computation Conference (2023)

Structure-inducing pre-training
Matthew B. A. McDermott, Brendan Yap, Peter Szolovits & Marinka Zitnik^†
Nature Machine Intelligence (2023)

Coordinated drift of receptive fields in Hebbian/anti-Hebbian network models during noisy representation learning
Shanshan Qin, Shiva Farashahi, David Lipshutz, Anirvan M. Sengupta, Dmitri B. Chklovskii & Cengiz Pehlevan^†
Nature Neuroscience (2023)

What can 5.17 billion regression fits tell us about artificial models of the human visual system?
Colin Conwell, Jacob S. Prince, George A. Alvarez^†, Talia Konkle^†
3rd Workshop on Shared Visual Representations in Human and Machine Intelligence (SVRHM), NeurIPS

Human-like signatures of contour integration in deep neural networks
Fenil R. Doshi^†
Talk presented at Vision Science Society (2022)

Preprints

Chain-Of-Thought Reasoning is a Policy Improvement Operator
Hugh Zhang^†, David C. Parkes

Pruning for Feature-Preserving Circuits in CNNs
Chris Hamblin, Talia Konkle^†, George Alvarez^†

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics
Caleb Weinreb, Mohammed Abdal, Monium Osman^†, Libby Zhang, Sherry Lin, Jonah Pearl, Sidharth Annapragada, Eli Conlin, Winthrop F. Gillis, Maya Jay, Ye Shaokai, Alexander Mathis, Mackenzie Weygandt Mathis, Talmo Pereira, Scott W. Linderman, Sandeep Robert Datta