LLM Skills and Metacognition: Scaffolding for New Forms of Learning?

Join us for a talk by Sanjeev Arora, Charles C. Fitzmorris Professor of Computer Science at Princeton University and Director of Princeton Language and Intelligence. This talk is part of the Kempner Seminar Series, a research-level seminar series that covers topics related to the basis of intelligence in natural and artificial systems.

LLMs, especially their recent “reasoning” incarnations, are capable of impressive problem solving. This talk will argue that a key role in this success is their “metacognition” capabilities (“thinking about thinking”), which appears to arise spontaneously in LLMs. We’ll give diverse examples of such metacognition and how to measure it. We argue that it gives insight into how LLM training gives rise to complex capabilities, as well as how these capabilities may be enhanced in future. We give several examples of using LLM metacognition to enhance their learning.

Sanjeev Arora is Charles C. Fitzmorris Professor of Computer Science and Director of Princeton Language and Intelligence, a unit devoted to research and applications of large AI models. He got his Phd from UC Berkeley in 1994 and has been a faculty member at Princeton since then. He has been awarded the ACM Prize in Computing (2011), Fulkerson Prize in Discrete Mathematics (2012), Packard Fellowship, Sloan Fellowship, and the ACM Doctoral Dissertation Prize. He was a plenary speaker at the International Congress of Mathematicians in 2018 and is a member of the National Academy of Science and American Academy of Arts and Sciences.