Skip to main content
SpringerLink
Log in

A structure-self-organizing DBN for image recognition

  • S.I. : ATCI 2020
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

A structural self-organized DBN (S-DBN) is proposed in this paper to improve the ability of feature learning in unsupervised training. In S-DBN, the strategy of dropout is designed for unsupervised learning to reduce inner cooperation between feature detectors. Then, the regularization-reinforced transfer function is put forward, in order to further reduce the insignificant weights, and to raise the abilities of feature learning and generation. The fast training method of contrastive divergence is designed, and backpropagation is used in supervised training. Finally, two experiments on regression and classification using MNIST show that S-DBN has better generation and faster convergence rate than other methods, in particular, in regression experiment, the proposed model beats traditional DBN by 1.50%; in image classification, the proposed model achieves smaller testing error in much less computing time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Prakken H, Sartor G (2015) Law and logic: a review from an argumentation perspective. Artif Intell 227:214–245

    Article  MathSciNet  Google Scholar 

  2. Hinton.G.E, Salakhutdinov.R.R. Reducing the dimensionality of data with neural networks. Science, 2006. 313(5786): 504–507.

  3. LeCun Y, Bengio Y, Hinton GE (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  4. Hinton GE, Osindero S, Teh Y (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554

    Article  MathSciNet  Google Scholar 

  5. Deselaers T, Hasan S, Bender O (2009) A deep learning approach to machine transliteration. In: Proceedings of the 4th workshop on statistical machine translation, pp 233–241

  6. Kunlei Z, Xuewen C (2014) Large-scale deep belief nets with MapReduce. Access IEEE 2:395–403

    Article  Google Scholar 

  7. Yoshua B, Pascal L, Dan P, Hugo L (2007) Greedy layer-wise training of deep networks. In: Conference on neural information processing systems, pp 153–160

  8. Shenhua G, Yuting Z, Kui J et al (2015) Single sample face recognition via learning deep supervised autoencoders. IEEE Trans Inform Forensics Secur 10:2108–2118

    Article  Google Scholar 

  9. Qiao J, Pan G, Han H (2019) A regularization-reinforced DBN for digital recognition. Nat Comput 18(4):721–733

    Article  MathSciNet  Google Scholar 

  10. Dahl GE, Dong Y, Deng L (2011) Large vocabulary continuous speech recognition with context-dependent DBN-HMMS. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, pp 4688–4691

  11. Shenghua Z, Yan L, Bin L et al (2015) Query-oriented unsupervised multi-document summarization via deep learning model. Expert Syst Appl 42:8146–8155

    Article  Google Scholar 

  12. Ren Z, Furao S, Jinxi Z (2014) A model with fuzzy granulation and deep belief networks for exchange rate forecasting. In: IEEE international joint conference on neural networks, pp 366–373

  13. Zhang S, Bao Y, Zhou P, et al. (2014) Improving deep neural networks for LVCSR using dropout and shrinking structure In: 2014 IEEE international conference on acoustics, speech and signal processing, pp 6849–6853

  14. M. Honegger (2018) Shedding light on black box machine learning algorithms: development of an axiomatic framework to assess the auality of aethods that explain individual predictions, arXiv preprint https://arXiv.org/1808.05054

  15. Yoshua B (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127

    Article  Google Scholar 

  16. Guangyuan P, Junfei Q, Wei C, et al. (2014) An improved RBM based on Bayesian regularization. In: IEEE international joint conference on neural networks, pp 2935–2939

  17. Längkvist M, Karlsson L, Loutfi A (2014) A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recognit Lett 42(6):11–24

    Article  Google Scholar 

  18. Jurgen S (2014) Deep learning in neural networks: an overview. Technical Report IDSIA-03-14, pp 1–88

  19. Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D (2018) A survey of methods for explaining black box models. ACM Comput Surv (CSUR) 51(5):93

    Google Scholar 

  20. Chen D, Lv J, Yi Z (2018) Graph regularized restricted Boltzmann machine. IEEE Trans Neural Netw Learn Syst 29(6):2651–2659

    Article  MathSciNet  Google Scholar 

  21. Yokoyama Y, Katsumata T, Yasuda M (2019) Restricted Boltzmann machine with multivalued hidden variables: a model suppressing over-fitting. Rev Socionetw Strateg 13:1–14

    Article  Google Scholar 

  22. Jia LI, David HE, Yong QU (2019) Diagnosis of gear early pitting faults using PSO optimized deep neural network. J Northeast Univ (Natural Science) 40(7):974–979

    Google Scholar 

  23. Deepa SN, Baranilingesan I (2018) Optimized deep learning neural network predictive controller for continuous stirred tank reactor. Comput Electr Eng 71:782–797

    Article  Google Scholar 

  24. Nair H, Tan C, Zeng M, Mengshoel OJ, Shen JP (2019) AttriNet: learning mid-level features for human activity recognition with deep belief networks. In: ACM international joint conference on pervasive and ubiquitous computing, pp 510–517

  25. Yang H, Hu B, Pan X, Yan S, Feng Y, Zhang X (2017) Deep belief network-based drug identification using near infrared spectroscopy. J Innov Opt Health Sci 10(2):545–551

    Article  Google Scholar 

  26. Voosen P (2017) How AI detectives are cracking open the black box of deep learning. Science 357(6346):22–28

    Article  MathSciNet  Google Scholar 

  27. Jun-Fei Q, Guang-yuan P, Hong-gui H (2015) The design and application of continuous deep belief network. Acta Automatica Sinica 12(41):2138–2146

    Google Scholar 

  28. Tan B, Wong A, Bizheva K (2018) Enhancement of morphological and vascular features in OCT images using a modified Bayesian residual transform. Biomed Opt Express 9(5):2394–2406

    Article  Google Scholar 

  29. Guo J, Wang L, Hou B, Lv J, Yuan Y, Mao Q (2019) Two-phase probabilistic collaborative representation-based classification. Expert Syst Appl 133:9–20

    Article  Google Scholar 

  30. [online] deeplearning.net. 2019

  31. Hinton GE, Srivastava N, Krizhevsky A (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint https://arXiv.org/1207.0580

Download references

Acknowledgments

This work was funded by Beijing Postdoctoral Science Foundation under Grant ZZ-2019-65, Chaoyang District Postdoctoral Science Foundation under Grant 2019ZZ-45 and Beijing Municipal Education Commission under Grant KM201811232016.

Funding

This work was funded by Beijing Postdoctoral Science Foundation under Grant ZZ-2019-65, Chaoyang District Postdoctoral Science Foundation under Grant 2019ZZ-45 and Beijing Municipal Education Commission under Grant KM201811232016.

Author information

Authors and Affiliations

  1. Beijing Information Science and Technology University, Beijing, China

    Qili Chen

  2. Electronic Engineering, Chinese University of Hong Kong, Hong Kong, China

    Guangyuan Pan

  3. Shenzhen Garry Intelligent Technology Co. Ltd, Shenzhen, China

    Guangyuan Pan

Authors
  1. Qili Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guangyuan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, Qili Chen and Guangyuan Pan; methodology, Qili Chen and Guangyuan Pan; software, Qili Chen and Guangyuan Pan; validation, Qili Chen jinjin Jiang and Guangyuan Pan; formal analysis, Guangyuan Pan; investigation, Guangyuan Pan; resources, Guangyuan Pan; data curation, Guangyuan Pan; writing—original draft preparation, Qili Chen jinjin Jiang and Guangyuan Pan; writing—review and editing, Qili Chen jinjin Jiang and Guangyuan Pan; visualization, Qili Chen jinjin Jiang and Guangyuan Pan; supervision, Guangyuan Pan; project administration, Guangyuan Pan; funding acquisition, Qili Chen. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Guangyuan Pan.

Ethics declarations

Conflict of interest

All the authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Q., Pan, G. A structure-self-organizing DBN for image recognition. Neural Comput & Applic 33, 877–886 (2021). https://doi.org/10.1007/s00521-020-05262-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05262-2

Keywords

Search

Navigation