Reduced reference stereoscopic image quality assessment using digital watermarking

https://doi.org/10.1016/j.compeleceng.2014.06.007Get rights and content

Highlights

  • A novel RR-SIQA algorithm based on digital watermarking technique is proposed.

  • Dither modulation (DM) watermarking method is used.

  • The proposed algorithm avoids the necessity of an ancillary channel in the deployment of RR-SIQA.

  • Good consistency with subjective quality scores.

Abstract

In this paper, a novel reduced-reference stereoscopic image quality assessment (RR-SIQA) algorithm is proposed by means of an unconventional use of watermarking technique. Watermarking techniques are usually employed for authenticity verification and copyright protection. Here, watermarking technique is adopted to provide a new approach for RR-SIQA. Firstly, the features of image are extracted in reorganized discrete cosine transform domain, and then embedded into the stereoscopic image as invisible hidden information. In order to improve the reliability of the watermarking, some channel coding techniques are applied before the process of embedding watermark. At the receiver, the watermark can be decoded and used to measure the quality of the distorted stereoscopic image. The proposed algorithm overcomes the limitations of other existing methods that require an auxiliary channel. Experimental results illustrate that the proposed algorithm has a good consistency with subjective quality scores, and can reflect the visual perception of stereoscopic image effectively.

Introduction

Three-dimensional (3D) video system can provide more immersive and realistic enhancement in end-user experience, compared with traditional two-dimensional (2D) video services [1], [2]. With the proliferation of sophisticated multimedia communication services, automatic monitoring of stereoscopic image/video quality is very important for content and service providers as it can help improving end-users’ satisfaction. Human eyes are the final receivers of stereoscopic image, therefore, the most reliable and straightforward methods for measuring perceptual quality of stereoscopic image are the subjective evaluation. But in most cases, these processes are quite complex, expensive and time-consuming [3]. Therefore, accurate and efficient objective quality assessment methods allowing for automatically evaluating stereoscopic image/video quality are highly desirable [4].

Similar to classification of mono-scopic image quality assessment (MIQA) methods, stereoscopic image quality assessment (SIQA) methods can be categorized into Full-Reference (FR), No-Reference (NR) and reduced-reference (RR)-SIQA methods according to the availability of the reference signal. In order to assess the perceptual quality of distorted stereoscopic image, the FR methods require the whole reference stereoscopic image, which is supposed to be artifact free and of perfect quality. Peak signal to noise ratio (PSNR) has been used as a representative FR method for a very long time, owing to its simple formulation, clear physical meaning, and easy optimization despite having not good correlation with human visual perception. In recent years, the FR-SIQA field has been made significant progress [5], [6], [7], [8], [9]. Benoit et al. [5] proposed a SIQA method by fusing 2D image quality assessment (IQA) techniques and depth/disparity information. IJsselsteijn et al. [6] proposed higher level evaluation concepts (naturalness and viewing experience) that are sensitive to both 2D image quality and stereoscopic depth. You et al. [7] proposed a FF-SIQA algorithm for predicting crosstalk perception. With the research of binocular vision, Zhou et al. [8] proposed a binocular fusion process based SIQA method, and Lin and Wu [9] proposed a perceptual FF-SIQA by considering binocular Integration Behaviors. However, in many practical scenarios where the reference stereoscopic image is not always available, NR-SIQA is required as a particularly difficult task. NR-SIQA methods attempt to measure the perceptual quality of stereoscopic image inferred by a set of special distortion types, in which the prior knowledge of the distortions can be used [10], [11]. However, the performances of those algorithms have not been thoroughly evaluated on a comprehensive and large dataset.

To provide a compromise between the FR- and NR-SIQA methods, RR-SIQA methods involve sending or supplying some partial of information about the reference along with the distorted stereoscopic image that is useful in quality evaluation. In this manner, a stereoscopic image is sent to the receiver by transmission channel, which may introduce distortions in the received stereoscopic image. By contrast, RR-SIQA methods can usually achieve more accurate assessment than NR-SIQA methods, benefitting from some known features extracted from the reference stereoscopic image. Thus, RR-SIQA methods can be useful in multimedia communications over wired or wireless networks, where the full information of the reference stereoscopic image is often not available.

Some RR-MIQA researches had been reported [11], [12], [13]. Wang and Simonceli [11] proposed a wavelet-domain natural image statistic metric (WNISM). Narwaria et al. [12] proposed a RR-MIQA approach based on the phase and magnitude of the 2-D discrete Fourier transform. Ma et al. [13] proposed a RR-MIQA method by depicting the sub-band statistical characteristics. Moreover, other RR-MIQA methods based on watermarking technologies were proposed [14], [15]. At the beginning, watermarking techniques were mainly proposed for applications such as certification, copyright protection in the past dozen years [16], [17]. Nowadays, watermarking based MIQA is becoming an active research topic. Wang et al. [14] proposed a watermarking-based RR-MIQA method, where watermark is embedded into the reference image in discrete wavelet transform (DWT) domain by using a quantization method. Samee and GÖtze [15] proposed a watermarking-based RR-MIQA algorithm for channel distortion (distortion specific algorithm). At the sender, fragile watermarks were embedded in image. At the receiver, according to the errors between the extracted and original watermarks, the quality of image was estimated. Nevertheless, strictly speaking, these watermarking-based MIQA methods are not of reduced-reference because no RR extracted features concerning either the reference image or the distorted image are actually utilized in the quality assessment process.

Subsequently, RR-SIQA methods had also been made some progresses. Hewage and Martini [18] proposed a RR-SIQA method for color plus depth based 3D video transmission. In those methods, RR features extracted at the sender were sent to the receiver through an ancillary channel. However, this restricts the applications of the RR-SIQA methods because an ancillary channel may be expensive or inconvenient to be provided. An alternative solution would be to send the partial information in the same channel as stereoscopic images being transmitted, which is highly desirable.

In this paper, we propose a new watermarking-based RR-SIQA algorithm, in which features of the reference stereoscopic image are extracted and embedded as watermark. When a distorted version of such a stereoscopic image is received, the end-users can decode the hidden watermark information and use them to evaluate the quality of the distorted stereoscopic image. Extensive experiments based on the MPCL symmetric distortion database [3] and LIVE 3D IQA database (phase I) [4] against the subjective scores have been conducted to demonstrate the effectiveness of the proposed watermarking-based RR-SIQA algorithm.

The rest of the paper is organized as follows. Section 2 describes the proposed watermarking-based RR-SIQA algorithms in detail, including reorganized discrete cosine transform (RDCT), feature extraction, watermark embedding and quality assessment. Section 3 gives experimental results and performance comparisons. Finally, the conclusion is given in Section 4.

Section snippets

The proposed watermarking based RR-SIQA algorithm

Fig. 1 shows the framework of the proposed watermarking based RR-SIQA algorithm (here, it will be called for short as the proposed algorithm), including feature extraction, watermark embedding, encoding, distortion process, decoding, watermark extraction and quality analysis system. At the sender, the features of reference stereoscopic image are extracted in RDCT domain, these features are quantized into 8-bit precision. To further improve the reliability of the quantized features, two error

Stereoscopic image quality databases

Two databases are used to test the proposed algorithm and compare it with other MIQA and SIQA algorithms. The test databases are described as follows.

  • (1)

    The MPCL database in Ningbo University [3] contains 12 reference stereoscopic images altered with five types of symmetrical distortions at different distortion levels. Specifically, 60, 60, 60, 60 and 72 distorted stereoscopic images with different levels of JPEG, JPEG2000 (JP2K), Gaussian Blur (Gblur), White Noise (WN) and H.264 distortions are

Conclusion

In this paper, a novel reduced-reference stereoscopic image quality assessment (RR-SIQA) algorithm has been proposed by incorporating properties of the reorganized discrete cosine transform strategy, Watson contrast sensitivity masking, stereoscopic structure and watermarking technique. Comparing with the existing full-reference and reduced-reference image quality assessment methods, the proposed algorithm has a good consistency with subjective quality scores as demonstrated on the LIVE 3D

Acknowledgements

This research is supported by the Natural Science Foundation of China (Grant Nos. U1301257, 61171163, 61271270, 61271021 and 61311140262), the Zhejiang Provincial Natural Science Foundation of China (Grant No. LY13F050005), the Innovative Scientific Research Project for Postgraduates in Zhejiang Province (Grant No. YK2011049), the Research Foundation of Education Department of Zhejiang Province (Grant No. Y201224839), and the Zhejiang University of Science and Technology startup fund. It was

Wujie Zhou received the B.S. degree from Hunan University of Science and Technology, Xiangtan, China, in 2006, and the M.S. Degree from Guangdong University of Technology, Guangzhou, China, in 2009, and received his Ph.D. degree at Ningbo University, Ningbo, China, in 2014. His research interests include multimedia signal processing and communication.

References (26)

  • J.Y. You et al.

    Assessment of stereoscopic crosstalk perception

    IEEE Trans Multimedia

    (2012)
  • W.J. Zhou et al.

    PMFS: a perceptual modulated feature similarity metric for stereoscopic image quality assessment

    IEEE Signal Process Lett

    (2014)
  • Y.H. Lin et al.

    Quality assessment of stereoscopic 3D image compression by binocular integration behaviors

    IEEE Trans Image Process

    (2014)
  • Cited by (0)

    Wujie Zhou received the B.S. degree from Hunan University of Science and Technology, Xiangtan, China, in 2006, and the M.S. Degree from Guangdong University of Technology, Guangzhou, China, in 2009, and received his Ph.D. degree at Ningbo University, Ningbo, China, in 2014. His research interests include multimedia signal processing and communication.

    Gangyi Jiang received his M.S. Degree from Hangzhou University in 1992, and received his Ph.D. Degree from Ajou University, Korea, in 2000. He is now a Professor in Faculty of Information Science and Engineering, Ningbo University, China. His research interests mainly include digital video compression, multi-view video coding, etc.

    Mei Yu received her M.S. Degree from Hangzhou Institute of Electronics Engineering, China, in 1993, and Ph.D. Degree from Ajou University, Korea, in 2000. She is now a Professor in Faculty of Information Science and Engineering, Ningbo University, China. Her research interests include image/video coding and video perception.

    Zhongpeng Wang received his M.S. Degree from Harbin Institute of Technology, China, in 1992, and received his Ph.D. Degree from Bejing University of Posts and Telecommunications in China, in 2004. He is now an Associate Professor in School of Information and Electronic Engineering, Zhejiang University of Science and Technology, China. His research interests include digital signal processing and wireless communication.

    Zongju Peng received his B.S. Degree from Sichuan Normal College, China, in 1995, and M.S. Degree from Sichuan University, China, in 1998, and received his Ph.D. Degree from Institute of Computing Technology, Chinese Academy of Science, in 2010. He is now an Associate Professor in Faculty of Information Science and Engineering, Ningbo University, China. His research interests include image/video compression.

    Feng Shao received his B.S. and Ph.D. Degrees from Zhejiang University, Hangzhou, China, in 2002 and 2007, respectively, all in Electronic Science and Technology. He is currently an Associate Professor in Faculty of Information Science and Engineering, Ningbo University, China. His research interests include 3D video coding, 3D quality assessment, and image perception, etc.

    Reviews processed and recommended for publication to the Editor-in-Chief by Associate Editor Dr. Eduardo Cabal-Yepez.

    View full text