Caltech Home > PMA Home > Calendar > Information, Geometry, and Physics Seminar
open search form
Wednesday, October 23, 2024
4:00 PM - 5:30 PM
Linde Hall 310

Information, Geometry, and Physics Seminar

An entropic view of machine learning
Akhil Premkumar, Kavli Institute for Cosmological Physics, University of Chicago,

In light of the recent advances in generative modeling, we can view learning as the process of assigning probabilities to plausible outcomes in a high dimensional event space. The miracle of modern AI is that such a task is viable even when the data is sparse in relation to the total volume of possibilities. One wonders why it works, and how far we can stretch this magic---what is the smallest number of samples needed to learn a distribution, and, is it possible to continue learning after the training data has been depleted? I tackle these questions by borrowing ideas from information theory, statistical mechanics, thermodynamics, and optimal transport. I will illustrate some experiments with diffusion models that reinforce our theoretical insights.

For more information, please contact Mathematics Department by phone at 626-395-4335 or by email at [email protected].