Skip to main content
SpringerLink
Log in
Book cover

International Conference on Neural Information Processing

ICONIP 2017: Neural Information Processing pp 217–226Cite as

Sharp and Real Image Super-Resolution Using Generative Adversarial Network

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10636))

Abstract

Recent studies have achieved great progress on accuracy and speed of single image super-resolution (SISR) based on neural networks. Most current SISR methods use mean squared error (MSE) loss as objective function. As a result, they can get high peak signal-to-noise ratios (PSNR) which are however not in full agreement with the visual qualities by experiments, and thus the output from these methods could be prone to blurry and over-smoothed. Especially at large upscaling factors, the output images are perceptually unsatisfactory in general. In this paper, we firstly propose a novel residual network architecture based on generative adversarial network (GAN) for image super-resolution (SR), which is capable of inferring photo-realistic images for 4\(\times \) upscaling factors. Perceptual loss is applied as the objective function to make output image sharper and more real. In addition, we adopt some tricks to preprocess the input dataset and use improved techniques to train the generator and discriminator separately, which are proved to be effective for the result. We validate our GAN-based approach on CelebA dataset with mean opinion score (MOS) as performance measure. The results demonstrate that the proposed approach performs better than previous methods.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vis. Appl. 25(6), 1423–1468 (2014)

    Article  Google Scholar 

  2. Yang, C.-Y., Ma, C., Yang, M.-H.: Single-image super-resolution: a benchmark. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 372–386. Springer, Cham (2014). doi:10.1007/978-3-319-10593-2_25

    Google Scholar 

  3. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), 24–26 June 2008, Anchorage, Alaska, USA (2008)

    Google Scholar 

  4. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  5. Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp. 1874–1883 (2016)

    Google Scholar 

  6. He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). doi:10.1007/978-3-319-46493-0_38

    Chapter  Google Scholar 

  7. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A.C., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 8–13 December 2014, Montreal, Quebec, Canada, pp. 2672–2680 (2014)

    Google Scholar 

  8. Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)

    Article  Google Scholar 

  9. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June, pp. 1646–1654 (2016)

    Google Scholar 

  10. Sønderby, C.K., Caballero, J., Theis, L., Shi, W., Huszár, F.: Amortised MAP inference for image super-resolution. CoRR abs/1610.04490 (2016)

    Google Scholar 

  11. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  12. Sun, J., Sun, J., Xu, Z., Shum, H.: Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans. Image Process. 20(6), 1529–1542 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  13. Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013, pp. 1059–1066 (2013)

    Google Scholar 

  14. Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)

    Article  Google Scholar 

  15. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). doi:10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  16. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). doi:10.1007/978-3-319-10593-2_13

    Google Scholar 

  17. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  18. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp. 1637–1645 (2016)

    Google Scholar 

  19. Ledig, C., Theis, L., Huszar, F., Caballero, J., Aitken, A.P., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-realistic single image super-resolution using a generative adversarial network. CoRR abs/1609.04802 (2016)

    Google Scholar 

  20. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR abs/1511.06434 (2015)

    Google Scholar 

  21. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6–11 July 2015, pp. 448–456 (2015)

    Google Scholar 

  22. Zeiler, M.D., Taylor, G.W., Fergus, R.: Adaptive deconvolutional networks for mid and high level feature learning. In: IEEE International Conference on Computer Vision, ICCV 2011, Barcelona, Spain, 6–13 November 2011, pp. 2018–2025 (2011)

    Google Scholar 

  23. Odena, A., Dumoulin, V., Olah, C.: Deconvolution and checkerboard artifacts. Distill 1(10), e3 (2016)

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (grants No. 61672133, No. 61502080 and No. 61632007) and the Fundamental Research Funds for the Central Universities (grants No. ZYGX2015J058 and No. ZYGX2014Z007).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Center for Future Media, University of Electronic Science and Technology of China, Chengdu, 611731, China

    Dongyang Zhang, Jie Shao, Gang Hu & Lianli Gao

Authors
  1. Dongyang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lianli Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Shao .

Editor information

Editors and Affiliations

  1. Guangdong University of Technology, Guangzhou, China

    Derong Liu

  2. Guangdong University of Technology, Guangzhou, China

    Shengli Xie

  3. South China University of Technology, Guangzhou, China

    Yuanqing Li

  4. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Dongbin Zhao

  5. King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

    El-Sayed M. El-Alfy

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Zhang, D., Shao, J., Hu, G., Gao, L. (2017). Sharp and Real Image Super-Resolution Using Generative Adversarial Network. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10636. Springer, Cham. https://doi.org/10.1007/978-3-319-70090-8_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70090-8_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70089-2

  • Online ISBN: 978-3-319-70090-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation