Science of AI

What We Do

AI is advancing at an unprecedented pace, leaving significant gaps in our understanding of the fundamental principles fueling its success. The increasing training complexity and size of AI models pose significant challenges for scientists who study the underlying principles at work in ML models. We are working to develop and test new scientific, engineering, and mathematical principles underpinning the numerous technological breakthroughs in AI and deep learning. 

Research Spotlight

Representational learning

Probing the internal representations and computational strategies of artificial neural networks is a pivotal challenge in machine learning that resonates with fundamental questions about natural intelligence. Our exploration goes beyond basic functionality to understand the genesis of these representations and their impact on algorithm design. By unpacking these “black boxes,” we aim to develop neural networks that are not just more reliable and safer for deployment but also transparent enough to anticipate changes in behavior in response to shifts in input distributions, ensuring robustness in varied scenarios.

Research Projects

Infrastructure for Studying LLMs and Generative AI

Our engineers collaborated with Allen AI to create OLMo, an open-source, state-of-the-art LLM and a powerful tool for better understanding the science behind LLMs. 

Read about OLMo

Representational Learning and Mechanistic Interpretability

In a recent blog post and preprint, Kempner researchers explain mechanistic interpretability results using known inductive biases.

Read the blog post

Efficiency vs Core Capabilities in LLMs

Transformers are better than State Space Models at copying, according to a new blog post by Kempner researchers. They find improved efficiency of State Space Models sacrifices some core capabilities for modern LLMs.

Read the blog post