The use of parsimonious representations is at the heart of the scientific modeling approach, through the philosophical concept of Occam's razor, but parsimony is also fundamental to the construction of low-dimensional models for data processing. The lecture shows the mathematical equivalence of the notions of parsimony, approximation and regularity, defined by linear or non-linear operators.
Parsimonious representations have many applications in data compression, denoising, inverse problem solving and statistical learning. The lecture is devoted to the mathematical properties of parsimonious approximations, using harmonic analysis, Fourier transforms and wavelet bases. High-dimensional approximations using two-layer neural networks are also studied.