Skip to main content
SpringerLink
Log in

Learning cascade regression for super-resolution image quality assessment

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Super-resolution (SR) image quality assessment (SRIQA) is a fundamental topic in the literature of SR domain. Most existing SR methods usually adopt full reference (FR) metrics which need the original images as reference, to evaluate the performance of different SR algorithms and the quality of resultant SR images. However, in practice the original HR images are not available for FR-SRIQA. Therefore, it is particularly meaningful to develop a no-reference (NR) SRIQA metric to assess SR algorithms. In this paper, a novel NR-SRIQA metric comprised of a cascade two-layer regression model is presented. The newly proposed method first employs three different kinds of perceptual statistical features to measure the degradation of SR images. Then a cascade two-layer regression framework with three independent AdaBoost Decision Tree Regression (DTR) models and one ridge regression model, is developed to build the mapping relationship from the obtained statistical features to the corresponding subjective quality scores in a coarse-to-fine manner. Thorough evaluation experiments on the benchmark database proposed by Ma et al. (Comput Vis Image Understand 158:1–16 2017) confirm that the proposed NR-SRIQA metric is capable of yielding more consistent perceptual quality assessment on SR images than other state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data Availability

Data will be made available on request

References

  1. Ma C, Yang C, Yang X, Yang M (2017) Learning a no-reference quality metric for single-image super-resolution. Comput Vis Image Understand 158:1–16

    Article  Google Scholar 

  2. Zhang K, Tao D, Gao X, Li X, Xiong Z (2015) Learning multiple linear mappings for efficient single image super-resolution. IEEE Trans Image Process 24(3):846–861

    Article  MathSciNet  MATH  Google Scholar 

  3. Wang Z, Chen J, Hoi SC (2020) Deep learning for image super-resolution: A survey. IEEE Trans Pattern Anal Machine Intell 43(10):3365–3387

    Article  Google Scholar 

  4. Xiang T, Yang Y, Guo S (2020) Blind night-time image quality assessment: Subjective and objective approaches. IEEE Trans Multimed 22(5):1259–1272

    Article  Google Scholar 

  5. Wang S, Gu K, Zhang X, Lin W, Ma S, Gao W (2018) Reduced-reference quality assessment of screen content images. IEEE Trans Circuits Syst Video Technol 28(1):1–14

    Article  Google Scholar 

  6. Bo Y, Bahetiyaer B, Chenxi M, Ke L, Weimin T (2019) Deep objective quality assessment driven single image super-resolution. IEEE Trans Multimedia 21(11):2957–2971

    Article  Google Scholar 

  7. Sun W, Liao Q, Xue J, Zhou F (2018) SPSIM: A superpixel-based similarity index for full-reference image quality assessment. IEEE Trans Image Process 27(9):4232–4244

    Article  MathSciNet  MATH  Google Scholar 

  8. Khosravi MH, Hassanpour H (2020) Blind quality metric for contrastdistorted images based on eigendecomposition of color histograms. IEEE Trans Circuits Syst Video Technol 30(1):48–58

    Article  Google Scholar 

  9. Liu Y, Gu K, Zhang Y, Li X, Zhai G, Zhao D, Gao W (2020) Unsupervised blind image quality evaluation via statistical measurements of structure, naturalness, and perception. IEEE Trans Circuits Syst Video Technol 30(4):929–943

    Article  Google Scholar 

  10. Yan Q, Gong D, Zhang Y (2019) Two-stream convolutional networks for blind image quality assessment. IEEE Trans Image Process 28(5):2200–2211

    Article  MathSciNet  Google Scholar 

  11. Po L, Liu M, Yuen WYF, Li Y, Xu X, Zhou C, Wong PHW, Lau KW, Luk H (2019) A novel patch variance biased convolutional neural network for no-reference image quality assessment. IEEE Trans Circuits Syst Video Technol 29(4):1223–1229

    Article  Google Scholar 

  12. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  13. Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: A feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386

    Article  MathSciNet  MATH  Google Scholar 

  14. Sheikh HR, Bovik AC, Veciana GD (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14(12):2117–2128

    Article  Google Scholar 

  15. Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444

    Article  Google Scholar 

  16. Yu K, Meng Z, Sun M (2013) Image quality assessment: a reduced reference algorithm for the super-resolution reconstruction image. In: Proc IEEE Int Conf Instrum Meas Comput Commun and Control, pp 171–175

  17. Gao X, Gao F, Tao D, Li X (2013) Universal blind image quality assessment metrics via natural scene statistics and multiple kernel learning. IEEE Trans Neural Netw Learn Syst 24(12):2013–2026

    Article  Google Scholar 

  18. Saad M, Bovik AC, Charrier C (2012) Blind image quality assessment: A natural scene statistics approach in the DCT domain. IEEE Trans Image Process 21(8):3339–3352

    Article  MathSciNet  MATH  Google Scholar 

  19. Chen Y, Zhao Y, Li S, Zuo W, Jia W, Liu X (2020) Blind quality assessment for cartoon images. IEEE Trans Circuits Syst Video Technol 30(9):3282–3288

    Article  Google Scholar 

  20. Zhou W, Wang Z, Chen Z (2021) Image super-resolution quality assessment: Structural fidelity versus statistical naturalness. In: 2021 13th International Conference on Quality of Multimedia Experience (QoMEX), pp 61–64 . IEEE

  21. Jiang Q, Liu Z, Gu K, Shao F, Zhang X, Liu H, Lin W (2022) Single image superresolution quality assessment: a real-world dataset, subjective studies, and an objective metric. IEEE Trans Image Process 31:2279–2294

    Article  Google Scholar 

  22. Wang Y, Zhou L, Yu B, Wang L, Zu C, Lalush DS, Lin W, Wu X, Zhou J, Shen D (2018) 3D auto-context-based locality adaptive multi-modality gans for pet synthesis. IEEE Trans Med Imaging 38(6):1328–1339

    Article  Google Scholar 

  23. Luo Y, Zhou L, Zhan B, Fei Y, Zhou J, Wang Y, Shen D (2022) Adaptive rectification based adversarial network with spectrum constraint for high-quality pet image synthesis. Med Image Anal 77:102335

    Article  Google Scholar 

  24. Hou H, Jin Q, Zhang G, Li Z (2022) Ct image quality enhancement via a dual-channel neural network with jointing denoising and super-resolution. Neurocomputing 492:343–352

    Article  Google Scholar 

  25. Zhang W, Ma K, Yan J, Deng D, Wang Z (2020) Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Trans Circuits Syst Video Technol 30(1):36–47

    Article  Google Scholar 

  26. Zhang T, Zhang K, Xiao C, Xiong Z, Lu J (2022) Joint channel-spatial attention network for super-resolution image quality assessment. Appl Intell 1–15

  27. Zhang Y, Gao X, He L, Lu W, He R (2019) Blind video quality assessment with weakly supervised learning and resampling strategy. IEEE Trans Circuits Syst Video Technol 29(8):2244– 2255

  28. Zhang W, Fan Z (2014) A no-reference contourlet-decomposition-based image quality assessment method for super-resolution reconstruction. In: Proc Int Soc Optics and Photonics, vol 9273, p 927326

  29. Huang H, Yu J, Sun D (2012) A no-reference SVD-based image quality assessment method for super resolution reconstruction. Comput-Aided Design Comput Graph 24(9):1204–1210

    Google Scholar 

  30. Fang Y, Zhang C, Yang W, Liu J, Guo Z (2018) Blind visual quality assessment for image super-resolution by convolutional neural network. Multimed Tools Appl 77(22):29829–29846

    Article  Google Scholar 

  31. Narwaria M, Lin W (2010) Objective image quality assessment based on support vector regression. IEEE Trans Neural Netw 21(3):515–519

    Article  Google Scholar 

  32. Moorthy AK, Bovik AC (2011) Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Trans Image Process 20(12):3350–3364

    Article  MathSciNet  MATH  Google Scholar 

  33. Zhang K, Zhu D, Jing J, Gao X (2019) Learning a cascade regression for no-reference super-resolution image quality assessment. In: Proc IEEE Int Conf Image Process, pp 450–453

  34. Portilla J, Simoncelli EP (2000) A parametric texture model based on joint statistics of complex wavelet coefficients. Int J Comput Vis 40(1):49–70

    Article  MATH  Google Scholar 

  35. Wainwright MJ, Simoncelli EP (2000) Scale mixtures of gaussians and the statistics of natural images. In: Adv Neural Inf Process Syst, pp 855–861

  36. Breiman L, Friedman JH, Olshena RA, Stone C (1984) Classification and regression trees. IEEE Trans Image Process 37(15):237–251

    Google Scholar 

  37. Drucker H (1997) Improving regressors using boosting techniques. Proc Int Confer on Machine Learning 97:107–115

    Google Scholar 

  38. Williams S (1996) Pearson’s correlation coefficient. In: J New Zealand Medical vol 109, p 38

  39. Sheskin DJ (2007) Spearman’s rank-order correlation coefficient. In: Handbook of parametric and nonparametric statistical procedures, pp 1353–1370

  40. Abdi H (2007) The kendall rank correlation coefficient. Encyclopedia of measurement and statistics. Sage, Thousand Oaks, CA, pp 508–510

    Google Scholar 

  41. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708

    Article  MathSciNet  MATH  Google Scholar 

  42. Liu L, Liu B, Huang H, Bovik AC (2014) No-reference image quality assessment based on spatial and spectral entropies. Signal Process: Image Commun 29(8):856–863

    Google Scholar 

  43. Zhang L, Zhang L, Bovik AC (2015) A feature-enriched completely blind image quality evaluator. IEEE Trans Image Process 24(8):2576–2591

    Article  MathSciNet  MATH  Google Scholar 

  44. Kang L, Ye P, Li Y, Doermann D (2014) Convolutional neural networks for no-reference image quality assessment. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 1733–1740

  45. Yan Q, Gong D, Zhang Y (2018) Two-stream convolutional networks for blind image quality assessment. IEEE Trans Image Process 28(5):2200–2211

    Article  MathSciNet  Google Scholar 

  46. Su S, Yan Q, Zhu Y, Zhang C, Ge X, Sun J, Zhang Y (2020) Blindly assess image quality in the wild guided by a self-adaptive hyper network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3667–3676

  47. Li H, Zhang K, Niu Z, Shi H (2022) C\(^{2}\)MT: A credible and class-aware multi-task transformer for SR-IQA. IEEE Signal Process Lett 29:2662–2666

    Article  Google Scholar 

  48. Timofte R, Smet VD, Gool LV (2016) Seven ways to improve examplebased single image super resolution. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 1865–1873

  49. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  50. Yang J, Lin Z, Cohen S (2013) Fast image super-resolution based on in-place example regression. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 1059–1066

  51. Dong W, Zhang L, Shi G, Wu X (2011) Image deblurring and superresolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans Image Process 20(7):1838–1857

    Article  MathSciNet  MATH  Google Scholar 

  52. Shan Q, Li Z, Jia J, Tang C (2008) Fast image/video upsampling. Trans Graph 27:153–11537

    Article  Google Scholar 

  53. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: Proc IEEE Int Conf Comput Vis, pp 349–356

  54. Timofte R, Smet V, Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: Proc IEEE Int Conf Comput Vis, pp 1920–1927

  55. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Eur Conf Comput Vis, pp 184–199

  56. Yang S, Lin G, Jiang Q, Lin W (2019) A dilated inception network for visual saliency prediction. IEEE Trans Multimed 22(8):2163–2176

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61971339, Grant 62061047, and Grant 61471161, in part by the Textile Intelligent Equipment Information and Control Innovation Team of Shaanxi Innovation Ability Support Program under Grant 2021TD-29, in part by the Textile Intelligent Equipment Information and Control Innovation Team of Shaanxi Innovation Team of Universities, in part by the Natural Science Foundation of Xinjiang Uygur Autonomous Region under Grant 2020D01C157, and in part by the Key Project of the Natural Science Foundation of Shaanxi Province under Grant 2018JZ6002

Author information

Authors and Affiliations

  1. School of Electronics and Information, Xi’an Polytechnic University, Xi’an, 710048, China

    Xing Quan, Kaibing Zhang & Dandan Fan

  2. Shaanxi Key Laboratory of Clothing Intelligence, the School of Computer Science, Xi’an Polytechnic University, 710048, Xi’an, China

    Kaibing Zhang & Jinguang Chen

  3. Yan’an Vocational and Technical College, Yan’an, 716000, China

    Danni Zhu

  4. School of Medical Engineering and Technology, Xinjiang Medical University, Urumqi, 830054, China

    Yanting Hu

Authors
  1. Xing Quan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaibing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Danni Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dandan Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yanting Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jinguang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Xing Quan: Writing - original draft, Revised version preparation. Kaibing Zhang: Methodology-Proponents of major academic ideas and supervision. Danni Zhu: Writing-Response to the reviewers’ comments. Dandan Fan: Experiment- Data processing, Figure plotting. Yanting Hu: Writing-review and editing. Jinguang Chen: Writing-Polishing the English presentation.

Corresponding author

Correspondence to Kaibing Zhang.

Ethics declarations

Ethical and informed consent for data used

This paper does not contain any studies with human participants or animals performed by any of the authors

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Quan, X., Zhang, K., Zhu, D. et al. Learning cascade regression for super-resolution image quality assessment. Appl Intell 53, 27304–27322 (2023). https://doi.org/10.1007/s10489-023-04905-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04905-w

Keywords

Search

Navigation