Solving overparametrized systems of nonlinear equations

Join us for a talk by Andrea Montanari, John D. and Sigrid Banks Professor in the Department of Statistics and Department of Mathematics at Stanford University. This talk is part of the Kempner Seminar Series, a research-level seminar series on recent advances in the field.

Abstract: I will discuss the problem of solving a system of equations F(x)=0,
for x a d-dimensional unit vectors and D a non-linear map from R^d to R^n
whose components are independent, rotationally invariant Gaussian processes.
We studied this problem under the proportional asymptotics in which n and d
goes to diverge, with their ratio converging to alpha>0. I will present
upper and lower bounds, as well as conjectures about the existence of solutions
and the existence of polynomial-time algorithms to find them.
Finally, I will discuss generalizations of this model, and how these insights
shed light on the optimization landscape of overparametrized neural nets.
[Based on joint work with Eliran Subag.]