Abstract
Typical sets and the asymptotic equipartition theorem are extended to the case of independent variables with continuous values. The properties of conditional entropy and Kullback-Liebler divergence are studied, as well as mutual information and entropy additivity for independent variables. We then analyze the convergence of the entropy of dependent random variables as the number of variables increases. This allows us to define the entropy rate. We demonstrate the asymptotic existence of this entropy rate for a stationary process.
