The idea is to describe a given statistical ensemble with the minimum number of modes. Let u(t) be a random generalized process with t as a parameter (spatial or temporal). We would like to find a deterministic function 
with a structure typical of the members of the ensemble in some sense. In other words, a functional in the following form needs to be maximized :
where 
is the autocorrelation function of u(t) and u(t'), 
is the average or expected value of f(u), 
is a scalar product and the asterisk denotes a complex conjugate. The classical methods of the calculus of variations gives the final result for 
, if R(t,t') is an integrable function
The solution of (2) forms a complete set of a square-integrable orthogonal functions 
with associated eigenvalues 
. It was shown that any ensemble of random generalized functions can be represented by a series of orthonormal functions with random coefficients, the coefficients being uncorrelated with one another:
These functions are the eigenfunctions of the autocorrelation with positive eigenvalues. The eigenvalues are the energy of the various eigenfunctions (modes). Moreover, since the modes were determined by maximizing 
(the energy of a mode), the series (3) converges as rapidly as possible. This means that it gives rise to an optimal set of basis functions from all possible sets.
If the averaging is performed in time domain, then 
(here t is a time parameter) can be represented as follows:
where a's are temporal coefficients and 's are the spatial eigenfunctions or modes.
The transformation (2)-(3) is the Karhunen-Loève (KL) transformation.
Because of discretization of experimental data, a vector form of KL is widely used. In this case u becomes an ensemble of finite-dimensional vectors, the correlation function R is a correlation matrix and the eigenfunctions are called eigenvectors.
On practical grounds, (3) (or(4)) usually is represented only in terms of a finite set of functions,
Briefly, some important properties of KL:
1. The generalized coordinate system defined by the eigenfunctions of the correlation matrix is optimal in the sense that the mean-square error resulting from a finite representation of the process is minimized. That is for any fixed L:
iff are KL eigenfunctions of (2).
2. The random variables appearing in an expansion of the kind given by the equation (3) are orthonormal if and only if the orthonormal functions and the constants are respectively the eigenfunctions and the eigenvalues of the correlation matrix.
3. In addition to the mean-square error minimizing property, the Karhunen-Loève expansion has some additional desirable properties. Of these, the minimum representation entropy property is worth mentioning.
4. Algazi and Sakrison [32] showed that KL expansion is optimal not only in terms of minimizing mean-square error between the signal and it's truncated representation (Property 1), but also minimizes a number of modes to describe the signal for a given error.
Sirovich [10] pointed out that the temporal correlation matrix will yield the same dominant spatial modes, while often giving rise to a much smaller and computationally more tractable eigenproblem - the method of snapshots. Mathematically, for a process u(t,x) instead of finding a spatial two-point correlation matrix 
, where N is a number of spatial points and solving (2) ( 
-matrix), one can compute a temporal correlation 
-matrix 
over spatial averaging,
where M is number of temporal snapshots and calculate 
from 
series as
where 
's are the solutions of the equation 
. Usually 
and the computational cost of finding 's can be reduced dramatically.
The optimality of KL reduces the amount of information required to represent statistically dependent data to a minimum. This crucial feature explains the wide usage of KL in a process of analyzing data.
In [12] the limitations of KL with temporal averaging were discussed. It was shown that in this case the analysis uses only information that is close to a particular final state of the system and thus cannot be used for the system which has a several final states. Also it was pointed out that the analysis de-emphasizes infrequent events, although they could be dynamically very important (burst-like events in a turbulent boundary layer). Alternative averaging techniques were proposed and shown to be more informative in terms of investigating the system dynamics.
