Deeper Learning

Predicting future neural activity is a critical step toward achieving real-time, closed-loop neurotechnologies. To this end, we introduce POCO, a unified forecasting model trained on…

• Preprint
• Code

The authors provide a theoretical analysis demonstrating the existence of anytime learning schedules for overparameterized linear regression, and highlight the central role of weight averaging—also known as model merging—in achieving the optimal convergence rates of stochastic gradient descent.

• Preprint
• Code

Despite reaching equal performance success when trained on the same task, artificial neural networks can develop dramatically different internal solutions, much like different students solving the same math problem using completely different approaches. Our study introduces a unified framework to quantify this variability across Recurrent Neural Network (RNN) solutions, which we term solution degeneracy, and analyze what factors shape it across thousands of recurrent networks trained on memory and decision-making tasks.

• Preprint

This work introduces a relational foundation model for neurological discovery and evaluates it through discovery loops that connect AI predictions to experiments in Parkinson’s disease, bipolar disorder, and Alzheimer’s disease.

• PROTON website
• Paper
• Code
• Model

This work explores using video as the primary modality for robot foundation models. Unlike static images, videos naturally encode physical dynamics and semantics of the world, providing a rich prior for physical decision-making.

• Preprint
• Hugging Face
• Project website

From Linear Directions to Convex Geometry

The authors ask the fundamental question: is the linear view of DINOv2 under the Linear Representation Hypothesis (LRH) sufficient to describe how deep vision models organize information? The authors examine the geometry and statistics of the learned concepts themselves and the results suggest that representations are organized beyond linear sparsity alone.

• Interactive demo
• Preprint

The authors offer an interpretability deep dive, examining the most important concepts emerging in one of today’s central vision foundation models, DINOv2.

• Interactive demo
• Preprint

The authors identify a novel phenomenon, Boomerang Distillation, which occurs when distilling a large language model into a smaller one. In this blog post, they describe how Boomerang Distillation can be used to create entire families of LLMs of fine-grained sizes without any training from a single student-teacher pair.

• Preprint
• Code
• Models

The authorsintroduce LOTION, a framework that optimizes a continuous variant of the quantized loss surface while provably preserving all global minima of the original problem.

• Preprint

ToolUniverse is a framework for developing AI agents for science, often referred to as “AI scientists.” It provides an environment where LLMs interact with more than six hundred scientific tools, including machine learning models, databases, and simulators. ToolUniverse standardizes how AI models access and combine these tools, allowing researchers to develop, test, and evaluate AI agents for science.

• Project
• Paper
• Code