| Abstract: |
| Potential functions used in optimizations, dynamics applications, and machine learning etc. can be rather complicated in term of their structures and properties especially in very high dimensions. Due to lacking of knowledge on concrete forms of potential functions in real applications, even the determination of their basic structures and properties is a challenging problem in both mathematical analysis and numerical simulations. This talk presents a probabilistic approach to investigate the landscape of potential functions, including those in high dimensions, by using an appropriate coupling scheme to couple two copies of the overdamped Langevin dynamics of the potential functions. It can be theoretically shown that for potential functions with single or multiple wells, the coupling time distributions admit qualitatively distinct exponential tails in terms of noise magnitudes. In addition, a quantitative characterization of the non-convexity of a multi-well potential function can also be obtained via linear extrapolation. These theoretical findings thus suggest a promising approach to probe the shape of a potential landscape through the coupling time distributions at least numerically. Numerical examples of loss landscapes of neural networks with different sizes will be presented. This talk is mainly based on a recent joint work with Yao Li at UMASS and Molei Tao at Georgia Tech. |
|