Kempner Institute Researchers Receive ICML 2026 Outstanding Paper Honorable Mention
One of the top nine papers, among 6,300 conference papers, recognized for technical depth, novelty, and potential for impact in the field of machine learning
The paper’s authors are Binxu Wang, a Kempner Research Fellow; Jacob A. Zavatone-Veth, a Junior Fellow of the Harvard Society of Fellows; and Cengiz Pehlevan, a Kempner associate faculty member.
A paper by Kempner Institute researchers has received an Outstanding Paper Honorable Mention at ICML 2026, the 43rd International Conference on Machine Learning, held July 6–11, 2026, in Seoul, South Korea.
The paper’s authors are Kempner Research Fellow Binxu Wang; Jacob A. Zavatone-Veth, a Junior Fellow of the Harvard Society of Fellows and an affiliate of the Harvard Center for Brain Science; and Kempner associate faculty member Cengiz Pehlevan, associate professor of applied mathematics at the Harvard John A. Paulson School of Engineering & Applied Sciences (SEAS).
The authors were recognized with an Outstanding Paper Honorable Mention for “A Random Matrix Perspective on the Consistency of Diffusion Models,” which was one of nine papers selected for outstanding paper awards or honorable mentions. These awards recognize technical depth, novelty, and potential for impact in the field. This year, ICML received more than 23,900 submissions and accepted just over 6,300 papers.
Research explains why diffusion models can produce consistent outputs
The paper introduces a mathematical framework explaining a striking phenomenon: diffusion models trained on entirely different, non-overlapping slices of a dataset can still generate nearly identical images when handed the same noise seed. Diffusion models build images by denoising — repeatedly stripping away noise while adding detail until a sample resembles the training data — so it is far from obvious why two models that never saw a single shared example should converge on the same output.
The authors trace this consistency to a simple linear effect. Because different data splits share nearly identical broad statistics — chiefly the mean and covariance — models often absorb those shared patterns rather than memorizing individual examples. Two models trained on disjoint data therefore learn to denoise in much the same way, producing consistent samples.
To formalize this, the researchers turn to random matrix theory, a tool for understanding how randomness behaves in high-dimensional systems. Their analysis quantifies how a finite, randomly sampled training set shapes what a model learns. It shows, for instance, why limited data pulls the generated samples toward the dataset’s average: sampling variability effectively “renormalizes” the noise level, causing models to shrink the subtler, low-variance directions in the data — so a face generated from a small training set drifts toward an “average” face, with smoother textures and backgrounds.
The theory also pinpoints what drives the remaining differences between models. Three factors govern how much two models trained on different slices of a dataset disagree: aniosotropy, or variation across feature directions, inhomogeneity, or variation across individual inputs and the overall size of the training set.
The authors derived their results in a simplified linear setting and then validated the predictions on more practical architectures, including UNet and DiT models. Together, the findings offer a clearer account of when diffusion models produce stable, reproducible outputs — and precisely what causes them to diverge.
About the Kempner
The Kempner Institute seeks to understand the basis of intelligence in natural and artificial systems by recruiting and training future generations of researchers to study intelligence from biological, cognitive, engineering, and computational perspectives. Its bold premise is that the fields of natural and artificial intelligence are intimately interconnected; the next generation of artificial intelligence (AI) will require the same principles that our brains use for fast, flexible natural reasoning, and understanding how our brains compute and reason can be elucidated by theories developed for AI. Join the Kempner mailing list to learn more, and to receive updates and news.
PRESS CONTACT:
Deborah Apsel Lang | (617) 495-7993
kempnercommunications@harvard.edu
