Some Very Old and Very New Problems in Learning Theory

This talk will discuss two recent theoretical works. In the first part, I will discuss “model collapse”: what happens when models are repeatedly retrained on data that contains synthetic samples generated by previous models. In the second part, I will discuss when and why standard gradient-based learning may fail, and a framework for computational lower bounds for standard SGD-trained neural networks.