Science of AI
What We Do
AI is advancing at an unprecedented pace, leaving significant gaps in our understanding of the fundamental principles fueling its success. The increasing training complexity and size of AI models pose significant challenges for scientists who study the underlying principles at work in ML models. We are working to develop and test new scientific, engineering, and mathematical principles underpinning the numerous technological breakthroughs in AI and deep learning.
Representational learning
Probing the internal representations and computational strategies of artificial neural networks is a pivotal challenge in machine learning that resonates with fundamental questions about natural intelligence. Our exploration goes beyond basic functionality to understand the genesis of these representations and their impact on algorithm design. By unpacking these “black boxes,” we aim to develop neural networks that are not just more reliable and safer for deployment but also transparent enough to anticipate changes in behavior in response to shifts in input distributions, ensuring robustness in varied scenarios.
Research Projects
Infrastructure for Studying LLMs and Generative AI
Our engineers collaborated with Allen AI to create OLMo, an open-source, state-of-the-art LLM and a powerful tool for better understanding the science behind LLMs.
Representational Learning and Mechanistic Interpretability
In a recent blog post and preprint, Kempner researchers explain mechanistic interpretability results using known inductive biases.
Efficiency vs Core Capabilities in LLMs
Transformers are better than State Space Models at copying, according to a new blog post by Kempner researchers. They find improved efficiency of State Space Models sacrifices some core capabilities for modern LLMs.
