Enhancement of coded video sequences via an adaptive nonlinear post-processing

https://doi.org/10.1016/S0923-5965(02)00105-4Get rights and content

Abstract

Most video data compression standards use block-based motion estimation and the discrete cosine transform. This scheme takes advantage of the local spatial correlation property of images. However, it induces the well-known blocking artifacts, corner outliers, and ringing noise when the image is highly compressed. Conventional 2-D techniques for contrast sharpening cannot be applied to the enhancement of block-coded image sequences, because they emphasize such artifacts that become very visible and annoying in an image sequence. The local spectra and bandwidth of both the noise and the signal vary spatially, and the characteristics of the filters need to be locally adapted.

For this purpose, in this paper we present a computationally simple post-filtering method that enhances the true details, limits the overshoot near sharp edges and attenuates the coding artifacts. Moreover, a mechanism is proposed that allows the behaviour of the filter to change according to the presence and the intensity of the coding artifacts. To this purpose, we also introduce a new distortion measure for blocking and ringing artifacts.

Introduction

Blocking effects and ringing noise affect video sequences processed with many video source coding techniques; the smaller the target bit-rate of the channel, the more the coding artifacts [17].

Many deblocking schemes have been proposed in still image coding such as JPEG under the assumption that blocking artifacts are always located at block boundaries [2], [4], [5], [6], [7]. In video coding, however, the blocking artifacts of the previous frame can be propagated to the current frame, and can be located at any position in a block due to motion-compensated prediction [17]. Therefore, simple block boundary smoothing is not good enough to remove blocking artifacts appearing in video coding. Moreover, a low-pass effect is also needed inside the macroblock to reduce ringing and motion compensation mismatch effects. Iterative methods based on projection on a convex set [12], [16], or mean-field annealing techniques [9], or multichannel temporal regularization techniques [1] are complex and often not adequate for real-time video coding.

In this paper, we present a technique based on unsharp masking (UM) for noise smoothing and edge enhancing. The general approach is the one used in [13], but the presented method has several new important features. For example, it takes into account the amount of the coding artifacts and the fact that the blocking artifacts can be located at any position in a video sequence. In fact, the proposed method does not need any information about the position and the size of the blocks.

This paper is organized as follows. Section 2 describes in detail the proposed algorithm; Section 3 presents the adopted distortion measure and its use in the enhancement operator. In Section 4 some experimental results are introduced, while Section 5 gives some final considerations.

Section snippets

The enhancement algorithm

The block diagram of the proposed algorithm is depicted in Fig. 1. Let s(n,m) be the input signal; the enhanced signal u(n,m) results from the contribution of three terms,u(n,m)=s(n,m)+λ[zx(n,m)cx(n,m)+zy(n,m)cy(n,m)],wherezx(n,m)=2s(n,m)−s(n,m−1)−s(n,m+1),zy(n,m)=2s(n,m)−s(n−1,m)−s(n+1,m),respectively, are the output of a Laplacian filter applied horizontally and vertically to the input image. As explained later, the parameter λ is used to adjust the enhancement intensity.

From Fig. 1, it can

Distortion measure for blocking and ringing artifacts

In order to define a metric for blockiness, various methods have been proposed in the literature. The mean square difference of slope (MSDS) across a block boundary is introduced as a metric by Minami and Zakhor [7]; a block discontinuity measure is defined by Jeon [4] as the sum of squared pixel differences over the four block boundaries. Kuo and Hsieh [5] observed that blocking artifacts are not visible in all blocks; they define the visibility V of a block as the sum of the absolute pixel

Experimental results

In this section, we examine the performances of the proposed method and compare them to those of other algorithms in the literature. As usual when dealing with sharpening techniques, the most reliable quality criterion is the visual appearance; it is well known that a quantitative analysis of the enhancement is not easy because no ideal images are available for comparison and any measure should take into account the complex behavior of human vision.

The proposed algorithms have been tested on a

Conclusions

In this paper, a new post-processing algorithm for the enhancement of coded video has been presented. It is able to enhance the significant details, it limits the overshoots near sharp edges and attenuates the coding artifacts. Its behaviour ranges nonlinearly from a low-pass filter in uniform areas to a strong high-pass one in detailed areas.

The algorithm includes a mechanism that allows to change its effect according to the amount of coding artifacts. To this purpose, a new distortion measure

References (17)

  • G. Ramponi

    A cubic unsharp masking technique for contrast enhancement

    IEEE Trans. Signal Process.

    (1998)
  • M. Yuen et al.

    A survey of hybrid MC/DPCM/DCT video coding distortions

    Signal Processing

    (1998)
  • M.G. Choi et al.

    Multichannel regularized recovery of compressed video sequences

    IEEE Trans. Circuits Systems II: Analog Digital Signal Process.

    (2001)
  • Y. Feng Hsu et al.

    A new adaptive separable median filter for removing blocking effects

    IEEE Trans. Consumer Electronics

    (1993)
  • A.K. Jain

    Fundamentals of Digital Image Processing

    (1989)
  • B. Jeon et al.

    Blocking artifacts reduction in image coding based on minimum block discontinuity criterion

    SPIE Visual Communications Image Process.

    (1995)
  • C.J. Kuo et al.

    Adaptive post-processor for block encoded images

    IEEE Trans. Circuits Systems Video Technol.

    (1995)
  • S. Marsi et al.

    A simple algorithm for the reduction of blocking artifacts in images and its implementation

    IEEE Trans. Consumer Electronics

    (1998)
There are more references available in the full text version of this article.

Cited by (5)

View full text