The Karhunen-Loève procedure was proposed independently by Karhunen [1] (1946), Loève [2] (1955). But the method itself is known under a variety of names in different fields: Principal Component or Hotelling Analysis (Hotelling, 1953, [4]), Empirical Component Analysis (Lorenz, 1956, [3]), Quasiharmonic Modes (Brooks el at., 1988, [5]), Proper Orthogonal Decomposition (Lumley, 1967, [6]), Singular Value Decomposition (Golub and Van Loan, 1983, [7]), Empirical Eigenfunction Decomposition (Sirovich, 1987, [10]) and others. Closely related to this technique is factor analysis, which is used in psychology and economics (Harman, 1960, [8]).
From the mathematical point of view KL expansion (2) is nothing else but a transformation which diagonalizes a given matrix R and brings it to a canonical form R=ULV, where L is a diagonal matrix. Therefore the roots of KL actually go into the middle of the last century. A review of the early history of KL expansion can be found in [16]. The mathematical content of KL procedure is therefore classical and is contained in paper by Schmidt [9] (1907).
Because of the large amount of computations required to find the eigenvectors, KL technique was virtually unused until the middle of the century. Radical changes came with the appearance of powerful computers and development of efficient algorithms to compute the eigenfunctions (method of snapshots, [10]). Now KL expansion is used extensively in the fields of detection, estimation, pattern recognition, and image processing as an efficient tool to store random processes, in system controls. KL expansion, known also as Proper Orthogonal Decomposition (POD) is used in connection with stochastic turbulence problems (Lumley, 1967, [6]). In that context, the associated eigenfuntions can be identified with the characteristic eddies of the turbulence field. Also KL expansion appears to be involved in some biological processes, e.g. some particular stimulus configurations defining by the shape, position and orientation of pieces of border between areas of different brightness (which are called edge segments) are the sources of an effective responce in the cells and edge segments are among the first features extracted in the primary visual cortex (Hubel & Wiesel [29]).