CMX Lunch Seminar
Broadly speaking, my research interests are partial differential equations, numerical optimization, and machine learning. I am currently mainly working on deep learning, specifically the mathematical theory behind neural networks, the application of compressed sensing to training sparse neural networks, and the application of deep learning to materials science. Recently, I have taken a particular interest in the approximation theory behind neural networks. In addition, I continue to work on a variety of other projects. These topics include convex optimization and optimization on manifolds, and the application of compressed sensing to electronic structure calculations and signal processing. I am currently working with Professor Jinchao Xu and did my PhD under Professor Russel Caflisch.
We consider the problem of approximating high dimensional functions using shallow neural networks. We begin by introducing natural spaces of functions which can be efficiently approximated by such networks. Then, we derive the metric entropy of the unit balls in these spaces. Drawing upon recent work connecting stable approximation rates to metric entropy, this leads to the optimal approximation rates for the given spaces. Next, we show that higher approximation rates can be obtained by further restricting the function class. In particular, for a restrictive but natural space of functions, shallow networks with ReLU$^k$ activation function achieve an approximation rate of $O(n^{-(k+1)})$ in every dimension. Finally, we discuss the connections between this surprising result and the finite element method.