Skip to main content
SpringerLink
Log in

BliMSR: Blind Degradation Modelling for Generating High-Resolution Medical Images

  • Conference paper
  • First Online:
Medical Image Understanding and Analysis (MIUA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14122))

Included in the following conference series:

  • 351 Accesses

Abstract

A persisting problem with existing super-resolution (SR) models is that they cannot produce minute details of anatomical structures, pathologies, and textures critical for proper diagnosis. This is mainly because they assume specific degradations like bicubic downsampling or Gaussian noise, whereas, in practice, the degradations can be more complex and hence need to be modelled “blindly”. We propose a novel attention-based GAN model for medical image super-resolution that models the degradation in a data-driven agnostic way (“blind”) to achieve better fidelity of diagnostic features in medical images. We introduce a new ensemble loss in the generator that boosts performance and a spectral normalisation in the discriminator to enhance stability. Experimental results on lung CT scans demonstrate that our model, BliMSR, produces super-resolved images with enhanced details and textures and outperforms recent competing models, including a diffusion model for generating super-resolution images, thus establishing a state-of-the-art. The code is available at https://github.com/Samiran-Dey/BliMSR.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chaudhari, A.S., et al.: Super-resolution musculoskeletal MRI using deep learning. Magn. Reson. Med. 80, 2139–2154 (2018). https://doi.org/10.1002/mrm.27178

    Article  Google Scholar 

  2. Lim, B., et al.: Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE (2017). https://doi.org/10.1109/cvprw.2017.151

  3. Chen, Y., et al. Efficient and accurate MRI super-resolution using a generative adversarial network and 3D multi-level densely connected network. arXiv:1803.01417 (2018)

  4. Chen, C., et al.: Quantitative imaging of peripheral trabecular bone microarchitecture using MDCT. Med. Phys. 45, 236–249 (2017). https://doi.org/10.1002/mp.12632

    Article  Google Scholar 

  5. Mahapatra, D., et al.: Image super-resolution using progressive generative adversarial networks for medical image analysis. Comput. Med. Imaging Graph. 71, 30–39 (2019). https://doi.org/10.1016/j.compmedimag.2018.10.005

    Article  Google Scholar 

  6. Mariana-Iuliana, G., et al.: Multimodal multi-head convolutional attention with various Kernel sizes for medical image super-resolution. In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2194–2204. https://doi.org/10.1109/WACV56688.2023.00223

  7. Zeng, G.Y., et al.: MedSRGAN: medical images super-resolution using generative adversarial networks. Multimed. Tools Appl. 79, 21815–21840 (2020). https://doi.org/10.1007/s11042-020-08980-w

    Article  Google Scholar 

  8. Li, H., et al.: SRDiff: single image super-resolution with diffusion probabilistic models. Neurocomputing 479, 47–59 (2022). https://doi.org/10.1016/j.neucom.2022.01.029

    Article  Google Scholar 

  9. Wolterink, J.M., et al.: Generative adversarial networks for noise reduction in low-dose CT. IEEE Trans. Med. Imaging 36, 2536–2545 (2017). https://doi.org/10.1109/tmi.2017.2708987

    Article  Google Scholar 

  10. Gu, J., et al.: Blind super-resolution with iterative Kernel correction. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2019). https://doi.org/10.1109/cvpr.2019.00170

  11. Park, J., et al.: Computed tomography super-resolution using deep convolutional neural network. Phys. Med. Biol. 63(14), 145011 (2018). https://doi.org/10.1088/1361-6560/aacdd4

    Article  Google Scholar 

  12. Christian, L., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105–114 (2017). https://doi.org/10.1109/CVPR.2017.19

  13. Lin, Z., et al.: Why spectral normalization stabilizes GANs: analysis and improvements (2020)

    Google Scholar 

  14. Cheon, M., Kim, J.-H., Choi, J.-H., Lee, J.-S.: Generative adversarial network-based image super-resolution using perceptual content losses. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11133, pp. 51–62. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11021-5_4

    Chapter  Google Scholar 

  15. Sajjadi, M.S.M., et al.: EnhanceNet: single image super-resolution through automated texture synthesis. In: 2017 IEEE International Conference on Computer Vision (CCV). IEEE (2017). https://doi.org/10.1109/iccv.2017.481

  16. Rad, M.S., et al.: SROBB: targeted perceptual loss for single image super-resolution. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2710–2719 (2019)

    Google Scholar 

  17. Armato, S.G., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. Phys. 38, 915–931 (2011). https://doi.org/10.1118/1.3528204

    Article  Google Scholar 

  18. Wang, X., et al.: Real-ESRGAN: training real-world blind super-resolution with pure synthetic data. In: International Conference on Computer Vision Workshops (ICCVW) (2021)

    Google Scholar 

  19. Chenyu, Y., et al.: CT super-resolution GAN constrained by the identical, residual, and cycle learning ensemble (GAN-circle). IEEE Transactions on Medical Imaging (2019). https://doi.org/10.1109/TMI.2019.2922960

  20. Zhang, K., et al.: Learning a single convolutional super-resolution network for multiple degradations. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3271 (2018)

    Google Scholar 

  21. Wang, Z., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  22. Bing, X., Zhang, W., et al.: Medical image super resolution using improved generative adversarial networks. IEEE Access 7, 145030–145038 (2019). https://doi.org/10.1109/ACCESS.2019.2944862

  23. Dong, C., Loy, C.C., et al.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016). https://doi.org/10.1109/tpami.2015.2439281

  24. de González, A.B., Darby, S.: Risk of cancer from diagnostic x-rays: estimates for the UK and 14 other countries. Lancet 363(9406), 345–351 (2004). https://doi.org/10.1016/s0140-6736(04)15433-0

  25. Gravel, P., Beaudoin, G., De Guise, J.A.: A method for modeling noise in medical images. IEEE Trans. Med. Imaging 23, 1221–1232 (2004). https://doi.org/10.1109/TMI.2004.832656

    Article  Google Scholar 

  26. Hany, M.: Chest CT-scan images dataset: CT-scan images with different types of chest cancer. www.kaggle.com/datasets/mohamedhanyyy/chest-ctscan-images (2020)

  27. Kim, D., et al.: Progressive face super-resolution via attention to facial landmark (2019)

    Google Scholar 

  28. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1646–1654 (2016). https://doi.org/10.1109/CVPR.2016.182

  29. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1637–1645 (2016). https://doi.org/10.1109/CVPR.2016.181

  30. Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind” image quality analyzer. IEEE Signal Process. Lett. 20, 209–212 (2013). https://doi.org/10.1109/LSP.2012.2227726

  31. Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006). https://doi.org/10.1109/TIP.2005.859378

    Article  Google Scholar 

  32. Tai, Y., et al.: MemNet: a persistent memory network for image restoration. In: 2017 EEE International Conference on Computer Vision (ICCV). IEEE (2017). https://doi.org/10.1109/iccv.2017.486

  33. Tong, T., Li, G., et al.: Image super-resolution using dense skip connections. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4809–4817 (2017). https://doi.org/10.1109/ICCV.2017.514

  34. Wang, X., Yu, K., et al.: Recovering realistic texture in image super-resolution by deep spatial feature transform. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE (2018). https://doi.org/10.1109/cvpr.2018.00070

  35. Wang, X., Yu, K., et al.: ESRGAN: enhanced super-resolution generative adversarial networks (2018)

    Google Scholar 

  36. Yang, Q., Yan, P., et al.: Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Trans. Med. Imaging 37, 1348–1357 (2018). https://doi.org/10.1109/TMI.2018.2827462

  37. Yu, H., Liu, D., Shi, H., et al.: Computed tomography super-resolution using convolutional neural networks. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3944–3948 (2017). https://doi.org/10.1109/ICIP.2017.8297022

  38. Zhang, L., Dai, H., Sang, Yu.: Med-SRNet: GAN-based medical image super-resolution via high-resolution representation learning. Comput. Intell. Neurosci. 2022, 1–9 (2022). https://doi.org/10.1155/2022/1744969

    Article  Google Scholar 

  39. Zhang, R., Isola, P., et al.: The unreasonable effectiveness of deep features as a perceptual metric. In: CVPR (2018)

    Google Scholar 

  40. Zhang, Y., Tian, Y., et al.: Residual dense network for image super-resolution. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE (2018). https://doi.org/10.1109/cvpr.2018.00262

Download references

Acknowledgement

We acknowledge Indo-Swedish DBT-Vinnova project, BT/PR41025/Swdn/135/9/ 2020, for supporting this research. TC is funded through the Turing-Roche strategic partnership, who are sponsoring the conference registration and attendance costs. TC is also affiliated with Linacre College, University of Oxford through a non-stipendiary EPA Cephalosporin fellowship.

Author information

Authors and Affiliations

  1. School of Mathematical and Computational Sciences, IACS, Kolkata, India

    Samiran Dey & Partha Basuchowdhuri

  2. Department of Electronics and Telecommunication Engineering, IIEST, Howrah, India

    Debasis Mitra

  3. Department of Electrical Engineering, Uppsala University, Uppsala, Sweden

    Robin Augustine

  4. Computer Science and Engineering Department, Jadavpur University, Kolkata, India

    Sanjoy Kumar Saha

  5. The Alan Turing Institute and University College London, London, UK

    Tapabrata Chakraborti

  6. Linacre College, University of Oxford, Oxford, UK

    Tapabrata Chakraborti

Authors
  1. Samiran Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Partha Basuchowdhuri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Debasis Mitra
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robin Augustine
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sanjoy Kumar Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tapabrata Chakraborti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Samiran Dey or Tapabrata Chakraborti .

Editor information

Editors and Affiliations

  1. University of Aberdeen, Aberdeen, UK

    Gordon Waiter

  2. University of Aberdeen, Aberdeen, UK

    Tryphon Lambrou

  3. University of Aberdeen, Aberdeen, UK

    Georgios Leontidis

  4. University of Aberdeen, Aberdeen, UK

    Nir Oren

  5. University of Aberdeen, Aberdeen, UK

    Teresa Morris

  6. University of Aberdeen, Aberdeen, UK

    Sharon Gordon

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dey, S., Basuchowdhuri, P., Mitra, D., Augustine, ., Saha, S.K., Chakraborti, T. (2024). BliMSR: Blind Degradation Modelling for Generating High-Resolution Medical Images. In: Waiter, G., Lambrou, T., Leontidis, G., Oren, N., Morris, T., Gordon, S. (eds) Medical Image Understanding and Analysis. MIUA 2023. Lecture Notes in Computer Science, vol 14122. Springer, Cham. https://doi.org/10.1007/978-3-031-48593-0_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-48593-0_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-48592-3

  • Online ISBN: 978-3-031-48593-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation