The year was marked by the impressive performance of large-scale language models and generative artificial intelligence. These involve large-scale neural networks, which are having a considerable impact in science, industry and services, as well as in teaching. These are generated by sampling probability distributions. The lecture covers the mathematical foundations of this field.
It begins by explaining why high-dimensional modeling is approached in a probabilistic rather than deterministic framework. It is also motivated by the importance of random algorithms and, in particular, the Monte-Carlo method. An important application is statistical inference, which involves calculating conditional probabilities using Monte-Carlo sampling and summation. This introduction gives a brief overview of the field, separating the modeling, sampling and training aspects.
The basis for training is the realization of a family of independent random variables with the same unknown probability distribution. This distribution can be modeled using exponential models, Markov fields or neural networks, with parameters optimized by maximum likelihood. New samples are generated from these models, using accept-reject algorithms, Markov chains or diffusion scoring.
