Elsevier

Signal Processing

Volume 79, Issue 3, December 1999, Pages 235-249
Signal Processing

A new formulation for the Karhunen–Loeve expansion

https://doi.org/10.1016/S0165-1684(99)00099-7Get rights and content

Abstract

The expansion of a random signal into series of functions, with uncorrelated random variables for decomposition coefficients, appears in many aspects of signal processing. These possibilities are offered by the Karhunen–Loeve expansion. This expansion is rarely used in practice, because we are not able to estimate it quickly and efficiently. After recalling the Karhunen–Loeve expansion for a one-dimensional random signal, we describe a new method to approximate the solutions of the Fredholm integral equation. Then, we propose a new formulation of the Karhunen–Loeve expansion. Next, we extend this new formulation to the case of a two-dimensional random signal. An application to image interpolation is proposed where we compare our results with the results obtained using the classical formulation.

Zusammenfassung

Die Entwicklung eines Zufallssignals in eine Funktionenreihe mit unkorrelierten Zufallsvariablen als Entwicklungskoeffizienten erscheint in vielen Aspekten der Signalverarbeitung. Diese Möglichkeiten werden durch die Karhunen–Loeve-Entwicklung geboten. In der Praxis wird diese Entwicklung selten gebraucht, weil wir sie nicht schnell und effizient schätzen können. Nach einer Wiederholung der Karhunen–Loeve-Entwicklung eines eindimensionalen Zufallssignals beschreiben wir eine neue Methode zur Approximation der Lösungen der Fredholmschen Integralgleichung. Danach schlagen wir eine neue Formulierung für die Karhunen–Loeve-Entwicklung vor. Darauf aufbauend erweitern wir diese neue Formulierung auf den Fall eines zweidimensionalen Zufallssignals. Es wird eine Anwendung auf die Bildinterpolation vorgeschlagen, wobei wir unsere Ergebnisse mit denen der klassischen Formulierung vergleichen.

Résumé

Le problème de la décomposition d'un signal aléatoire en une série de fonctions, avec pour coefficients de décomposition des variables aléatoires décorrélées, apparaı̂t dans de nombreuses questions en traitement du signal. Le développement de Karhunen–Loève offre de telles possibilités, mais il est très rarement utilisé en pratique, car on ne dispose pas d'algorithme de calcul rapide pour l’évaluer efficacement. Après avoir rappelé le principe du développement de Karhunen–Loève dans le cas de signaux mono-dimensionnels, nous présentons une méthode originale pour déterminer les solutions de l’équation intégrale de Fredholm. Nous proposons également une nouvelle écriture pour le développement de Karhunen–Loève, afin qu'il puisse être appliqué en un délai relativement court. Puis, nous étendons cette nouvelle formulation aux cas de signaux bidimensionnels. Une application est proposée dans le cadre de l'interpolation d'images où nos résultats sont comparés avec ceux obtenus par utilisation de la formulation classique.

Introduction

In several domains of signal processing, it may be interesting to provide a second-moment characterization of a random signal in terms of uncorrelated random variables. We can find in the literature such an expansion referred to as the Karhunen–Loeve expansion. The basis functions of this expansion are the eigenfunctions, solutions of the Fredholm integral equation where the kernel is the autocorrelation function of the studied random process. Generally, the problem of finding the solutions of this integral equation is reduced to the question of finding the eigenvectors and the eigenvalues of a covariance matrix. Unfortunately, when we consider the discrete form of this integral equation, the solutions often constitute an insufficient approximation of the real solutions. Furthermore, applying the Karhunen–Loeve expansion in its discrete form can take a very long time. So, the Karhunen–Loeve expansion is rarely used in practice because we do not know how to estimate it quickly and efficiently [11], [12], [16]. We answer this question by proposing a new formulation and an algorithm that is cheaper and more accurate than classical approaches.

First of all, we recall in Section 1 the Karhunen–Loeve expansion in the case of one-dimensional signal. Then, we describe, in Section 2, a new method to approximate solutions of the Fredholm integral equation and we illustrate this method with several experiments. In order to limit memory problems and to quickly obtain a signal expansion, we propose, in Section 3, a new formulation of the Karhunen–Loeve expansion by using the discrete cosine transform. Next, in Section 4, we propose an extension of this new formulation to the case of a two-dimensional signal, in order to apply it in image processing. We finish this article with an example of an application for image interpolation. We propose several experiments to describe the quality of our results and we compare them with the results obtained using the classical formulation.

Section snippets

The Karhunen–Loeve expansion

In this section, we propose the Karhunen–Loeve development in its classical formulation. This development is based on the signal expansion into series of functions with uncorrelated random variables for decomposition coefficients. For this reason, first of all, we explain how to expand a random signal.

Fredholm integral equation, numerical method

It is possible to find in the literature several methods for solving the Fredholm integral equation. Some of them give an analytical expression for the solutions by substituting a differential equation for the integral equation [1], [7], [8], [9], [17]. So, these methods imply the solving of a differential equation. This may be difficult according to the kernel studied. For other methods the principle is to modify the formulation of the integral equation in order to find a matrix system [3],

New formulation of the Karhunen–Loeve expansion

We propose in this section a new formulation for the Karhunen–Loeve expansion by using the discrete cosine transform. In these conditions, we are looking for the expression of the coefficients of the random signal expanded into a series of cosines; we shall reconstruct the approximation of the random signal in the final phase of the treatment [4], [5], [6].

Let S(t) be the signal to be expanded and let Ŝ(t) be the reconstructed signal. We haveŜ(t)=n=1MxnΦn(t)∀t∈[−T;T],where xn are the random

Extension to a two-dimensional random signal

We propose in this section an extension of this new formulation of the Karhunen–Loeve expansion to the case of a two-dimensional signal, in order to apply it in image processing.

An application: interpolation of a two-dimensional random signal

To evaluate the improvement due to our formulation, we propose in this section an example of an application of the Karhunen–Loeve expansion for image interpolation.

Conclusion

We have presented in this paper a new method for finding an analytical approximation for the solutions of the Fredholm integral equation and a new formulation of the Karhunen–Loeve expansion. We have proposed an application for image interpolation. Several experiments have shown that the interpolated images obtained with our formulation present better results than with the classical processing. Indeed, the restoration of the low frequency and the preservation of the contrasts are better and the

References (17)

  • A.B. Baggeroer, State Variables and Communication Theory, The MIT Press, Cambridge, MA, 1970, pp....
  • J.F. Blinn, What's the deal with the DCT?, IEEE Comput. Graphics Appl. 13 (4) (July 1993)...
  • R. Courant, D. Hilbert, Methods of Mathematical Physics, Vol. I, Interscience Publisher, New York, 1961, pp....
  • Ph. Courmontagne, Interpolation d'un signal bidimensionnel en présence de termes perturbateurs, Mémoire de thèse,...
  • Ph. Courmontagne, J.F. Cavassilas, Une nouvelle écriture du développement de Karhunen–Loève, Application à...
  • Ph. Courmontagne, J.F. Cavassilas, An interpolation-filtering method for noise-corrupted images, in: Proceedings 13th...
  • W.B. Davenport, W.L. Root, An Introduction to the Theory of Random Signal and Noise, McGraw-Hill, New York,...
  • C.W. Helstrom, Statistical Theory of Signal Detection, Pergamon Press, 2nd Edition, 1968, pp....
There are more references available in the full text version of this article.

Cited by (7)

  • Stability of Karhunen-Loève expansion for the simulation of Gaussian stochastic fields using Galerkin scheme

    2014, Probabilistic Engineering Mechanics
    Citation Excerpt :

    Thus, the Fredholm integral equation in general has to be solved numerically. Several methods have been developed for this purpose, e.g. fast multipole methods [8,9], wavelet-Galerkin approach [5,10,11], the discrete cosine transform [12], hierarchical matrix technique [13,14], spectral element approximation [15] and the Galerkin procedure [4,16]. In this paper, the Galerkin procedure is chosen for this step since it allows the use of standard FEM technology and the same FE mesh can be used for solving Fredholm integral equation and equations of the physical system.

  • A stochastic approach to nonlinear unconfined flow subject to multiple random fields

    2009, Stochastic Environmental Research and Risk Assessment
  • Using wavelet transform to estimate the eigenfunctions of Karhunen-Loeve expansion

    2004, Proceedings of the International Computer Congress 2004 on Wavelet Analysis and its Applications, and Active Media Technology
View all citing articles on Scopus
View full text