Skip to main content
SpringerLink
Log in

Fast image super-resolution with the simplified residual network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, the image super-resolution (SR) methods based on residual learning have obtained remarkable quality performance. However, the current residual-learning methods have low computational performance and slow convergence rate. In this paper, we propose a high-efficiency two-level residual network to make the network learn more useful high-frequency information. Only 5 convolution layers in the LR space are used in our residual network, and no parameters are introduced in the other layers. Compared with the long training time up to several hours or days of previous deep residual networks, our simplified network can make the training time reduce to half an hour. Besides, our simplified network achieves satisfactory quality performance. The evaluation on the public datasets shows that our method can process SR of ultra-high definition (UHD) videos in real-time (more than 24 frames per second) on a generic graphical processing unit (GPU).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp 275–282

  2. Dai T, Cai J, Zhang Y, Xia ST, Zhang L (2019) Second-order attention network for single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR

  3. Dong C, Chen CL, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European Conference on Computer Vision, ECCV, pp 184–199

  4. Dong C, Chen CL, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European Conference on Computer Vision, ECCV, pp 391–407

  5. Freedman G, Fattal R (2011) Image and video upscaling from local self-examples. ACM Trans Graph 30(2):1–11

    Article  Google Scholar 

  6. Gao X, Zhang K, Tao D, Li X (2012) Image super-resolution with sparse neighbor embedding. IEEE Trans Image Process 21(7):3194–3205

    Article  MathSciNet  Google Scholar 

  7. He K, Zhang X, Ren S (2016) Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR

  8. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp 1646–1654

  9. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp 1637–1645

  10. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp 5835–5843

  11. Ledig C, Theis L, Huszar F et al (2016) Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp 105–114

  12. Lim B, Son S, Kim H et al (2017) Enhanced deep residual networks for single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp 1132–1140

  13. Lu X, Ma C, Ni B (2018) Deep regression tracking with shrinkage loss. In: European Conference on Computer Vision, ECCV

  14. Lu X, Wang W, Ma C et al (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR

  15. Lu X, Wang W, Shen J, Tai YW et al (2020) Learning video object segmentation from unlabeled videos. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR

  16. Pablo A, Michael M, Charless F, Jitendra M (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  17. Shi W, Caballero J, Huszr F et al (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp 1874–1883

  18. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp 2790–2798

  19. Timofte R, Smet VD, Gool LV (2013) Anchored neighborhood regression for fast example-based super-resolution. In: IEEE International Conference on Computer Vision, ICCV, pp 1920–1927

  20. Timofte R, Smet VD, Gool LV (2014) A+: Adjusted anchored neighborhood regression for fast super-resolution. In: Asian Conference on Computer Vision, ACCV, pp 111–126

  21. Tong CS, Leung KT (2007) Super-resolution reconstruction based on linear interpolation of wavelet coefficients. Multidim Syst Signal Process 18(2):153–171

    Article  MathSciNet  Google Scholar 

  22. Wang Z, Yang Y, Wang Z, Chang S, Yang J, Huang TS (2015) Learning super-resolution jointly from external and internal examples. IEEE Trans Image Process 24(11):4359–4371

    Article  MathSciNet  Google Scholar 

  23. Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deeply improved sparse coding for image super-resolution. In: IEEE International Conference on Computer Vision, ICCV, pp 370–378

  24. Yang J, Wang Z, Lin Z, Cohen S, Huang T (2012) Coupled dictionary training for image super-resolution. IEEE Trans Image Process 21(8):3467–3478

    Article  MathSciNet  Google Scholar 

  25. Yang J, Wright J, Huang TS, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR

  26. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

  27. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR

  28. Zhou D (2010) An edge-directed bicubic interpolation algorithm. In: International Congress on Image and Signal Processing, pp 1186–1189

Download references

Acknowledgements

We would like to thank Prof. Kim Munchurl in Korea Advanced Institute of Science and Technology (KAIST) for the initial inspiration of this work. This work is supported by the Project of High-level Talents Research Foundation of Jinling Institute of Technology (No. jit-b-201802), the General Program of Natural Science Foundation of the Jiangsu Higher Education Institutions of China (No. 19KJB520007), the Shandong Provincial Natural Science Foundation (Grant No. ZR2019PF023), the Project of Shandong Province Higher Educational Science and Technology Program under grant (No. J17KB184) and the Science and Technology Development Plan Project of Weifang City (No. 2019GX005).

Author information

Authors and Affiliations

  1. School of Computer Engineering, Jinling Institute of Technology, Nanjing, Jiangsu, 211169, China

    Chunmeng Wang

  2. School of Computer Science and Technology, Shandong University of Finance and Economics, 250014, Jinan, Shandong, China

    Lingqiang Ran

  3. Media and Communication College, Weifang University, Weifang, Shandong, 261061, China

    Chen He

Authors
  1. Chunmeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingqiang Ran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chen He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunmeng Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Ran, L. & He, C. Fast image super-resolution with the simplified residual network. Multimed Tools Appl 80, 4327–4339 (2021). https://doi.org/10.1007/s11042-020-09954-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09954-8

Keywords

Search

Navigation