Abstract
3D image quality assessment (3D-IQA) plays an important role in 3D multimedia applications. In recent years, convolutional neural networks (CNN) have been widely used in various images processing tasks and achieve excellent performance. In this paper, we propose a blind 3D-IQA metric based on multi-scale feature learning by using multi-column convolutional neural networks (3D-IQA-MCNN). To address the problem of limited 3D-IQA dataset size, we take patches from the left image and right image as input and use the full-reference (FR) IQA metric to approximate a reference ground-truth for training the 3D-IQA-MCNN. Then we put the patches from left image and right image into the pre-trained 3D-IQA-MCNN and obtain two quality feature vectors based on multi-scale. Finally, by regressing the quality feature vectors onto the subjective mean opinion score (MOS), the visual quality of 3D images is predicted. Experimental results show that the proposed method achieves high consistency with human subjective assessment and outperforms several state-of-the-art 3D-IQA methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Chen, L., Zhao, J.: Robust contourlet-based watermarking for depth-image-based rendering 3D images. In: 2016 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Nara, pp. 1–4 (2016)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
You, J., et al.: Perceptual quality assessment for stereoscopic images based on 2D image quality metrics and disparity analysis. In: Proceedings of the International Workshop Video Processing Quality Metrics Consumer Electronics, pp. 1–6 (2010)
Benoit, A., et al.: Quality assessment of stereoscopic images. EURASIP J. Image Video Process. 2008, 1–13 (2009)
Tam, W.J., Speranza, F., Yano, S., Shimono, K., Ono, H.: Stereoscopic 3D-TV: visual comfort. IEEE Trans. Broadcast. 57(2), 335–346 (2011)
Lebreton, P., Raake, A., Barkowsky, M., Le Callet, P.: Evaluating depth perception of 3D stereoscopic videos. IEEE J. Sel. Top. Signal Process. 6(6), 710–720 (2012)
Shao, F., Lin, W., Wang, S., Jiang, G., Yu, M.: Blind image quality assessment for stereoscopic images using binocular guided quality lookup and visual codebook. IEEE Trans. Broadcast. 61(2), 154–165 (2015)
Gu, K., et al.: No-reference stereoscopic IQA approach: from nonlinear effect to parallax compensation. J. Elect. Comput. Eng 2012(pt.3), 436031.1–436031.12 (2012)
Gu, K., Zhai, G., Lin, W., Yang, X., Zhang, W.: No-reference image sharpness assessment in autoregressive parameter space. IEEE Trans. Image Process. 24(10), 3218–3231 (2015)
Su, C., Cormack, L.K., Bovik, A.C.: Oriented correlation models of distorted natural images with application to natural stereopair quality evaluation. IEEE Trans. Image Process. 24(5), 1685–1699 (2015)
Oh, H., Ahn, S., Kim, J., Lee, S.: Blind deep S3D image quality evaluation via local to global feature aggregation. IEEE Trans. Image Process. 26(10), 4923–4936 (2017)
Chen, M., Cormack, L.K., Bovik, A.C.: No-reference quality assessment of natural stereopairs. IEEE Trans. Image Process. 22(9), 3379–3391 (2013)
Sazzad, Z.M., et al.: Objective no-reference stereoscopic image quality prediction based on 2D image features and relative disparity. Adv. Multimed. 2012(8), 1–16 (2012)
Lin, Y., Wu, J.: Quality assessment of stereoscopic 3D image compression by binocular integration behaviors. IEEE Trans. Image Process. 23(4), 1527–1542 (2014)
Ciregan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, pp. 3642–3649 (2012)
Shuai, Y., Wang, Y., Peng, Y., Xia, Y.: Accurate image super-resolution using cascaded multi-column convolutional neural networks. In: 2018 IEEE International Conference on Multimedia and Expo (ICME 2018), pp. 1–6, 23–27 July (2018)
Mass, A.L., et al.: Rectifier nonlinearities improve neural network acoustic models. In: ICMLW, vol. 30, no. 1 (2013)
Moorthy, A.K., et al.: Subjective evaluation of stereoscopic image quality. Signal Process. Image Commun. 28(8), 870–883 (2013)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, pp. 1026–1034 (2015)
Pascanu, R., et al.: On the difficulty of training recurrent neural networks. In: ICML, pp. 1310–1318 (2013)
Chen, M.-J., et al.: Full-reference quality assessment of stereopairs accounting for rivalry. Signal Process. Image Commun. 28(9), 1143–1155 (2013)
Gorley, P., et al.: Stereoscopic image quality metrics and compression. In: Proceedings of the SPIE, vol. 6803 (2008)
Acknowledgment
This work was supported by Natural Science Foundation of China under Grant No. 61671283, 61301113.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, Y., Yuan, S., Xia, Y., An, P. (2020). Blind 3D Image Quality Assessment Based on Multi-scale Feature Learning. In: Zhai, G., Zhou, J., Yang, H., An, P., Yang, X. (eds) Digital TV and Wireless Multimedia Communication. IFTC 2019. Communications in Computer and Information Science, vol 1181. Springer, Singapore. https://doi.org/10.1007/978-981-15-3341-9_22
Download citation
DOI: https://doi.org/10.1007/978-981-15-3341-9_22
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-3340-2
Online ISBN: 978-981-15-3341-9
eBook Packages: Computer ScienceComputer Science (R0)