Denoising of degraded facial image sequences based PCA

: One of the most stringent issues in image sequence processing lies in the high dimensionality of the image sequences. This issue is further made worse by the high correlation between the frames, on the one hand, and by the difﬁculty of noisy motion estimation, on the other hand. As a result, the de-noising algorithms are complex and require intense computational workload. To address this issue, we propose a novel algorithm based on the Principal Components Analysis (PCA) for dimension reduction, followed by a spatiotemporal ﬁlter for de-noising the dimensionally-reduced frames. The de-noising process is performed in the image reduced space by applying a temporal ﬁlter for motion compensation combined with a weighted average ﬁlter, and an wiener ﬁlter as a spatial ﬁlter for noise removal. The performance of the proposed method is proven using the Cohen-Kanade facial expressions (CKFE) database tested against different noise levels with PSNR and SSIM as metrics for objective evaluation. The experiments show that the proposed method gives better results than state of the art classical de-noising competitors.


Introduction
Image sequences are often corrupted by noise during both the acquisition and transmission. This degradation can affect the information retrieval and content interpretation. Thus, noise removal is an important preprocessing tool which can facilitate the subsequent processes such as motion tracking. There exist many algorithms for image sequence denoising as reported in the literature. Denoising is implemented in spatialdomain [1], temporal domain [2], or a combination of both, i.e.,spatiotemporal domain [3], that exploit the huge number of correlations presented in images sequence.
Several spatial and temporal filtering methods have been applied to image sequence. As an example, the authors in [4] proposed a spatial method based on multidirectional wiener filter by selecting only the homogeneous directions in order to preserve the important structures of the image. The authors in [5] proposed a temporal image sequence denoising filter based on temporal data blocks, where the filter can both reduce the noise and minimize the artifacts blocks, but can't prevent blur edges. Based on the results given by these methods, it is now admitted that the spatial filter cannot be performed separately since it can generate annoying artifacts and spatial blurring which affect the visual quality. On the other hand, utilizing the temporal filter alone often causes blurring in the motion areas. Therefore, the spatial filter can be used in combination with the temporal filter by exploiting both the spatial correlation between pixels and the temporal correlation that exists between adjacent frames in order to obtain a high quality of denoised image sequence [6].
In [7], an efficient algorithm is proposed for denoising the degraded face image sequence in the PCA domain for recognizing the face presented in sequence. The authors applied a temporal filter that performed motion compensation combined with a weighted average filter, followed by an adaptive spatial filter to the sub-image of small features and the sub-image of large features.
Noise removal is well performed in most of the existing spatio-temporal methods but unfortunately, they suffer from the high correlation between the frames and the high dimensionality of the image sequences. This can lead to blurring the areas with less noise. To overcome this problem, we propose a method based on spatio-temporal 2D Wiener filtering for denoising images and PCA [8] for dimension reduction, resulting in the so-called 2D-WF+PCA method. This paper is organized as follows. In section 2, the proposed method is described. Then, to demonstrate its performance, experimental results are presented in Section 3. Finally, Section 4 concludes this paper.

Proposed method: 2D-WF+PCA
The proposed algorithm uses the PCA method to decorrelate the important information from noise, and to reduce the dimension of image space. Therefore, the calculations workload is reduced. Then, we applied a spatial filter based on 2D wiener filter (2D-WF) and a temporal filter using the motion detector and recursive time averaging to enhance the denoised spatial frames. The various steps of the proposed denoising algorithm are described below.
Step 1: Project previous and current noisy frames in the PCA space, where the denoising process is performed using the reduced frames. The advantage of the PCA method is to discriminate information content from noise.
Step 2: Use PCA transform to obtain the new representation of images. Each frame is denoised via a 2D-WF, owing to its ability to suppress high amounts of noise. After spatial filtering, there remains redundancy in the sequence of images taken in the temporal direction. Therefore, a motion estimator is used to detect motion. If no motion is present between successive frames, an averaged value is used to replace the pixel in the current frame.
Step 3: The temporal filtering of spatially-filtered frames uses a simple block based on motion detection and recursive time averaging that is switched off where motion is detected. The motion detector is applied to control the noise suppression according to the area quality (static or movable). As a result, more noise removal is applied in the static areas and less process in the motion areas.
Step 4: The filtered frames are reconstructed using the PCA inverse transform.

Performance metrics
The performance of this algorithm is evaluated using two objective metrics, the peak signal-to-noise-ratio (PSNR) and the Structural SIMilarity(SSIM) [12], for providing the quantitative evaluation of results. PSNR is the mostly used objective measure in the literature, although it is not well correlated with human visual perception [12]. SSIM is believed to be a better indicator to perceive image quality. The final PSNR and SSIM results for denoised image sequence are computed by averaging their local values of the frames.  Table 1: PNSR values of all algorithms with different Gaussian noise levels.
The PSNR is defined by where r is the maximum intensity of an image, MSE is the mean square error between the distorted and original images.
The SSIM is defined where, µ x , µ y are the average of x and y, respectively; σ x , σ y are the variance of x and y, respectively; cov xy is the covariance of x and y; c1, c2 and c3 are the variables used to stabilize the division with weak denominator.  Table 1 and corresponding Figure 1, and Table 2 and corresponding Figure 2 show the resulting PSNR and SSIM respectively for the proposed method (2D-WF+PCA) in comparison with other methods: 2D-WF, 3D-WF and 3D-knn-f, for three image sequences with six noise levels. As shown by experimental results, a significant improvement is given by the proposed algorithm over its competitors, due to its ability to perform the denoising process in a fewer dimensions space.

(a) Visual results
As shown in Figure 3, the proposed algorithm yields good visual quality of the image sequences over the tested methods presenting the original noisy frame in the first column and the denoised frames obtained by the tested denoising algorithms   in the remaining columns, respectively. It can be seen that the proposed algorithm (third column) is effective at suppressing noise while well preserving the edge and details.

Conclusion
We developed a spatiotemporal algorithm for image sequence denoising, where the PCA transform is used to separate the useful information from noise while reducing the space dimension. Then, each image is filtered by the 2D-WF. The spatial filtering is followed by the temporal filter, where the motion detector is used to control the noise. The experimental results show that our proposed model has achieved higher noise removal gain as compared with the tested algorithms. The quantitative metrics (PSNR and SSIM) and the visual quality demonstrate theses results.