Skip to main content

Advertisement

SpringerLink
Log in

Microscopic image super resolution using deep convolutional neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, deep convolutional neural networks (CNNs) have achieved excellent results in single image super resolution (SISR). Owing to the strength of deep CNNs, it gives promising results compared to state-of-the-art learning based models on natural images. Therefore, deep CNNs techniques have also been successfully applied to medical images to obtain better quality images. In this study, we present the first multi-scale deep CNNs capable of SISR for low resolution (LR) microscopic images. To achieve the difficulty of training deep CNNs, residual learning scheme is adopted where the residuals are explicitly supervised by the difference between the high resolution (HR) and the LR images and HR image is reconstructed by adding the lost details into the LR image. Furthermore, gradient clipping is used to avoid gradient explosions with high learning rates. Unlike the deep CNNs based SISR on natural images where the corresponding LR images are obtained by blurring and subsampling HR images, the proposed deep CNNs approach is tested using thin smear blood samples that are imaged at lower objective lenses and the performance is compared with the HR images taken at higher objective lenses. Extensive evaluations show that the superior performance on SISR for microscopic images is obtained using the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ayas S, Ekinci M (2014) Random forest-based tuberculosis bacteria classification in images of zn-stained sputum smear samples. SIViP 8(1):49–61

    Article  Google Scholar 

  2. Bauschke HH, Borwein JM (1996) On projection algorithms for solving convex feasibility problems. SIAM Rev 38(3):367–426

    Article  MathSciNet  Google Scholar 

  3. Bevilacqua M, Roumy A, Guillemot C, Morel MLA (2012) Low-complexity single image super resolution based on nonnegative neighbor embedding. In: Proceedings of the 23rd British machine vision conference, pp 1–10

  4. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004. IEEE, vol 1, pp I–I

  5. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  6. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision. Springer, pp 391–407

  7. Duchon CE (1979) Lanczos filtering in one and two dimensions. J Appl Meteorol 18(8):1016–1022

    Article  Google Scholar 

  8. Eaton-Rosen Z, Bragman F, Ourselin S, Cardoso MJ (2018) Improving data augmentation for medical image segmentation. In: International conference on medical imaging with deep learning, pp 1–3

  9. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

  10. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 315–323

  11. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  12. Hussain Z, Gimenez F, Yi D, Rubin D (2017) Differential data augmentation techniques for medical imaging classification tasks. In: AMIA annual symposium proceedings, American Medical Informatics Association, pp 979

  13. Irani M, Peleg S (1993) Motion analysis for image enhancement: resolution, occlusion, and transparency. J Vis Commun Image Represent 4(4):324–335

    Article  Google Scholar 

  14. Jain V, Seung S (2009) Natural image denoising with convolutional networks. In: Advances in neural information processing systems, pp 769–776

  15. Jan Z, Khan A, Sajjad M, Muhammad K, Rho S, Mehmood I (2018) A review on automated diagnosis of malaria parasite in microscopic blood smears images. Multimed Tools Appl 77(8):9801–9826

    Article  Google Scholar 

  16. Keys R (1981) Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process 29(6):1153–1160

    Article  MathSciNet  Google Scholar 

  17. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  18. Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1637–1645

  19. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate superresolution. In: IEEE conference on computer vision and pattern recognition, vol 2, pp 5

  20. Lewis JP (1995) Fast template matching. In: Vision interface, vol 95, pp 15–19

  21. Li M, Nguyen TQ (2008) Markov random field model-based edge-directed image interpolation. IEEE Trans Image Process 17(7):1121–1128

    Article  MathSciNet  Google Scholar 

  22. Li Q, Cai W, Wang X, Zhou Y, Feng DD, Chen M (2014) Medical image classification with convolutional neural network. In: 2014 13th international conference on control automation robotics & vision (ICARCV). IEEE, pp 844–848

  23. Li X, Orchard MT (2001) New edge-directed interpolation. IEEE Trans Image Process 10(10):1521–1527

    Article  Google Scholar 

  24. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JA, Van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  Google Scholar 

  25. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814

  26. Nie L, Wang X, Zhang J, He X, Zhang H, Hong R, Tian Q (2017) Enhancing micro-video understanding by harnessing external sounds. In: Proceedings of the 2017 ACM on multimedia conference. ACM, pp 1192–1200

  27. Organization WH (2017) Global tuberculosis report 2017. World Health Organization

  28. Organization WH (2017) World malaria report 2017. World Health Organization

  29. Pascanu R, Mikolov T, Bengio Y (2012) Understanding the exploding gradient problem. arXiv:12115063

  30. Pascanu R, Mikolov T, Bengio Y (2013) On the difficulty of training recurrent neural networks. In: International Conference on Machine Learning, pp 1310–1318

  31. Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  32. Rivenson Y, Göröcs Z, Günaydin H, Zhang Y, Wang H, Ozcan A (2017) Deep learning microscopy. Optica 4(11):1437–1443

    Article  Google Scholar 

  33. Schultz RR, Stevenson RL (1996) Extraction of high-resolution frames from video sequences. IEEE Trans Image Process 5(6):996–1011

    Article  Google Scholar 

  34. Shi H, Ward R (2002) Canny edge based image expansion. In: IEEE international symposium on circuits and systems, 2002. ISCAS 2002. IEEE, vol 1, pp I–I

  35. Song X, Feng F, Liu J, Li Z, Nie L, Ma J (2017) Neurostylist: neural compatibility modeling for clothing matching. In: Proceedings of the 2017 ACM on multimedia conference. ACM, pp 753–761

  36. Song X, Feng F, Han X, Yang X, Liu W, Nie L (2018) Neural compatibility modeling with attentive knowledge distillation. arXiv:180500313

  37. Spanhol FA, Oliveira LS, Petitjean C, Heutte L (2016) Breast cancer histopathological image classification using convolutional neural networks. In: 2016 international joint conference on neural networks (IJCNN). IEEE, pp 2560–2567

  38. Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the IEEE international conference on computer vision, pp 1920–1927

  39. Timofte R, De Smet V, Van Gool L (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. In: Asian conference on computer vision. Springer, pp 111–126

  40. Tsai DM, Lin CT (2003) Fast normalized cross correlation for defect detection. Pattern Recogn Lett 24(15):2625–2631

    Article  Google Scholar 

  41. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612

    Article  Google Scholar 

  42. Wang Q, Ward RK (2007) A new orientation adaptive interpolation method. IEEE Trans Image Process 16(4):889–900

    Article  MathSciNet  Google Scholar 

  43. Wang SH, Tang C, Sun J, Yang J, Huang C, Phillips P, Zhang YD (2018) Multiple sclerosis identication by 14-layer convolutional neural network with batch normalization, dropout, and stochastic pooling. Front Neurosci 12:1–11

    Google Scholar 

  44. WebMicroscope (2019) http://fimm.webmicroscope.net/Research/Momic/mamic

  45. Xie J, Xu L, Chen E (2012) Image denoising and inpainting with deep neural networks. In: Advances in neural information processing systems, pp 341–349

  46. Yang J, Wright J, Huang T, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008

  47. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  48. Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238

    Article  Google Scholar 

  49. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: International conference on curves and surfaces, Springer

  50. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155

    Article  MathSciNet  Google Scholar 

  51. Zhang YD, Pan C, Sun J, Tang C (2018) Multiple sclerosis identification by convolutional neural network with dropout and parametric relu. J Comput Sci 28:1–10

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Karadeniz Technical University, 61080, Trabzon, Turkey

    Selen Ayas & Murat Ekinci

Authors
  1. Selen Ayas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Murat Ekinci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Selen Ayas.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ayas, S., Ekinci, M. Microscopic image super resolution using deep convolutional neural networks. Multimed Tools Appl 79, 15397–15415 (2020). https://doi.org/10.1007/s11042-019-7397-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7397-7

Keywords

Search

Navigation