Skip to main content
SpringerLink
Log in

Supervised Dictionary Learning with Smooth Shrinkage for Image Denoising

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The dictionary-based method is an important approach to image denoising. In the existing methods, the dictionary is either pre-defined or learned adaptively from the data under certain constraints such as sparsity or orthogonality, which often leads to an approximation solution. In this paper, we propose a novel supervised dictionary learning model with smooth shrinkage for image denoising. By incorporating the dictionary learning into the denoising target, our model is trained in a task-driven fashion without the need of explicit constraints. We analyze the proposed model theoretically and show that it tends to learn sparse and orthogonal dictionaries, which is further verified empirically. Experimental results on four different noise levels demonstrate the effectiveness of our model both quantitatively and visually in comparison with the classical dictionary-based denoising methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://www.cs.ubc.ca/~schmidtm/Software/minConf.html.

References

  1. Agostinelli F, Hoffman M, Sadowski P, Baldi P (2014) Learning activation functions to improve deep neural networks. arXiv preprint arXiv:1412.6830

  2. Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  MATH  Google Scholar 

  3. Alain G, Bengio Y (2014) What regularized auto-encoders learn from the data-generating distribution. J Mach Learn Res 15(1):3563–3593

    MathSciNet  MATH  Google Scholar 

  4. Atto AM, Pastor D, Mercier G (2008) Smooth sigmoid wavelet shrinkage for non-parametric estimation. In: International conference on acoustics, speech, and signal processing, pp 3265–3268

  5. Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. Adv Neural Inf Process Syst 19:153

    Google Scholar 

  6. Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560

    Article  Google Scholar 

  7. Cormen TH, Leiserson CE, Rivest RL, Stein C (2001) Introduction to algorithms, vol 6. MIT press, Cambridge

    MATH  Google Scholar 

  8. Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630

    Article  MathSciNet  MATH  Google Scholar 

  9. Donoho DL, Johnstone JM (1994) Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3):425–455

    Article  MathSciNet  MATH  Google Scholar 

  10. Elad M (2010) Sparse and redundant representations: from theory to applications in signal and image processing. Springer, Berlin

    Book  MATH  Google Scholar 

  11. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745

    Article  MathSciNet  Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2015a) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  13. He YC, Lu HT, Huang L, Shi XH (2015b) Non-negative matrix factorization with pairwise constraints and graph laplacian. Neural Process Lett 42(1):167–185

    Article  Google Scholar 

  14. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  15. Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  16. Hyvärinen A (1999) Sparse code shrinkage: denoising of nongaussian data by maximum likelihood estimation. Neural Comput 11(7):1739–1768

    Article  Google Scholar 

  17. Hyvärinen A, Hoyer PO (2001) A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vis Res 41(18):2413–2423

    Article  Google Scholar 

  18. Krishnan D, Fergus R (2009) Fast image deconvolution using hyper-Laplacian priors. In: Bengio Y, Schuurmans D, Lafferty JD, Williams CKI, Culotta A (eds) Advances in neural information processing systems. Curran Associates, Inc., pp 1033–1041

  19. Le QV, Karpenko A, Ngiam J, Ng AY (2011) ICA with reconstruction cost for efficient overcomplete feature learning. In: Shawe-Taylor J, Zemel RS, Bartlett PL, Pereira F, Weinberger KQ (eds) Advances in neural information processing systems. Curran Associates, Inc., pp 1017–1025

  20. LeCun YA, Bottou L, Orr GB, Müller KR (2012) Efficient backprop. In: Montavon G, Orr GB, Müller KR (eds) Neural networks: tricks of the trade: Second edition. Springer, Berlin, Heidelberg, pp 9–48. doi:10.1007/978-3-642-35289-8_3

  21. Liu W, Zha ZJ, Wang Y, Lu K, Tao D (2016) \( p \)-Laplacian regularized sparse coding for human activity recognition. IEEE Trans Ind Electron 63(8):5120–5129

    Google Scholar 

  22. Mairal J, Ponce J, Sapiro G, Zisserman A, Bach FR (2009) Supervised dictionary learning. In: Koller D, Schuurmans D, Bengio Y, Bottou L (eds) Advances in neural information processing systems. Curran Associates, Inc., pp 1033–1040

  23. Mairal J, Bach F, Ponce J (2012) Task-driven dictionary learning. IEEE Trans Pattern Anal Mach Intell 34(4):791–804

    Article  Google Scholar 

  24. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: International conference on computer vision, vol 2. pp 416–423

  25. Ngiam J, Coates A, Lahiri A, Prochnow B, Le QV, Ng AY (2011) On optimization methods for deep learning. In: International Conference on Machine Learning, pp 265–272

  26. Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis Res 37(23):3311–3325

    Article  Google Scholar 

  27. Pati YC, Rezaiifar R, Krishnaprasad P (1993) Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Twenty-seventh asilomar conference on signals, systems and computers, pp 40–44

  28. Portilla J, Strela V, Wainwright MJ, Simoncelli EP (2003) Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Trans Image Process 12(11):1338–1351

    Article  MathSciNet  MATH  Google Scholar 

  29. Rubinstein R, Elad M (2014) Dictionary learning for analysis-synthesis thresholding. IEEE Trans Signal Process 62(22):5962–5972

    Article  MathSciNet  Google Scholar 

  30. Rubinstein R, Peleg T, Elad M (2013) Analysis K-SVD: a dictionary-learning algorithm for the analysis sparse model. IEEE Trans Signal Process 61(3):661–677

    Article  MathSciNet  Google Scholar 

  31. Shang L, Cao FW (2006) Adaptive denoising using a modified sparse coding shrinkage method. Neural Process Lett 24(2):153–162

    Article  Google Scholar 

  32. Sprechmann P, Litman R, Yakar TB, Bronstein A, Sapiro G (2013) Efficient supervised sparse analysis and synthesis operators. In: Annual conference on neural information processing systems

  33. Tomczak JM (2015) Learning informative features from restricted boltzmann machines. Neural Process Lett 44:1–16

    Google Scholar 

  34. Wang X, Wang C, Shang L, Sun ZL (2016) Dispersion constraint based non-negative sparse coding model. Neural Process Lett 43(2):603–609

    Article  Google Scholar 

  35. Wang Z, Huang Y, Luo X, Wang L, Luo S (2011) Learning topographic representations of nature images with pairwise cumulant. Neural Process Lett 34(2):155–175

    Article  Google Scholar 

  36. Wu F, Jing XY, Yue D (2016) Multi-view discriminant dictionary learning via learning view-specific and shared structured dictionaries for image classification. Neural Process Lett pp 1–18

  37. Zeng X, Bian W, Liu W, Shen J, Tao D (2015) Dictionary pair learning on Grassmann manifolds for image denoising. IEEE Trans Image Process 24(11):4556–4569

    Article  MathSciNet  Google Scholar 

  38. Zhao H, Luo J, Huang Z, Nagumo T, Murayama J, Zhang L (2015) Statistically adaptive image denoising based on overcomplete topographic sparse coding. Neural Process Lett 41(3):357–369

    Article  Google Scholar 

Download references

Acknowledgements

The work was supported by the National Natural Science Foundation of China (Grant No. 91420302), the National Basic Research Program of China (Grant No. 2015CB856004) and the Key Basic Research Program of Shanghai (Grant No. 15JC1400103).

Author information

Authors and Affiliations

  1. Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China

    Keting Zhang & Liqing Zhang

Authors
  1. Keting Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liqing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liqing Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, K., Zhang, L. Supervised Dictionary Learning with Smooth Shrinkage for Image Denoising. Neural Process Lett 47, 535–548 (2018). https://doi.org/10.1007/s11063-017-9665-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-017-9665-8

Keywords

Search

Navigation