Skip to main content
SpringerLink
Log in

Evolving Deep Forest with Automatic Feature Extraction for Image Classification Using Genetic Programming

  • Conference paper
  • First Online:
Parallel Problem Solving from Nature – PPSN XVI (PPSN 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12269))

Included in the following conference series:

Abstract

Deep forest is an alternative to deep neural networks to use multiple layers of random forests without back-propagation for solving various problems. In this study, we propose a genetic programming-based approach to automatically and simultaneously evolving effective structures of deep forest connections and extracting informative features for image classification. First, in the new approach we define two types of modules: forest modules and feature extraction modules. Second, an encoding strategy is developed to integrate forest modules and feature extraction modules into a tree and the search strategy is introduced to search for the best solution. With these designs, the proposed approach can automatically extract image features and find forests with effective structures simultaneously for image classification. The parameters in the forest can be dynamically determined during the learning process of the new approach. The results show that the new approach can achieve better performance on the datasets having a small number of training instances and competitive performance on the datasets having a large number of training instances. The analysis of evolved solutions shows that the proposed approach uses a smaller number of random forests over the deep forest method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)

    Article  Google Scholar 

  2. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  3. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  4. Khan, A., Sohail, A., Zahoora, U., Qureshi, A.S.: A survey of the recent architectures of deep convolutional neural networks. arXiv preprint arXiv:1901.06032 (2019)

  5. Wang, Y., Yao, Q., Kwok, J., Ni, L.: Few-shot learning: a survey. arXiv preprint arXiv:1904.05046 (2019)

  6. Zhou, Z.H., Feng, J.: Deep forest: towards an alternative to deep neural networks. In: Proceedings of International Joint Conferences on Artificial Intelligence, pp. 3553–3559 (2017)

    Google Scholar 

  7. Al-Sahaf, H., et al.: A survey on evolutionary machine learning. J. Roy. Soc. NZ 49(2), 205–228 (2019)

    Article  Google Scholar 

  8. Sun, Y., Xue, B., Zhang, M., Yen, G.G.: Evolving deep convolutional neural networks for image classification. IEEE Trans. Evol. Comput. 24, 394–407 (2019)

    Article  Google Scholar 

  9. Bi, Y., Xue, B., Zhang, M.: An evolutionary deep learning approach using genetic programming with convolution operators for image classification. In: Proceedings of IEEE Congress on Evolutionary Computation, pp. 3197–3204 (2019)

    Google Scholar 

  10. Baioletti, M., Milani, A., Santucci, V.: Learning Bayesian networks with algebraic differential evolution. In: Auger, A., Fonseca, C.M., Lourenço, N., Machado, P., Paquete, L., Whitley, D. (eds.) PPSN 2018. LNCS, vol. 11102, pp. 436–448. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99259-4_35

    Chapter  Google Scholar 

  11. Chen, B., Wu, H., Mo, W., Chattopadhyay, I., Lipson, H.: Autostacker: a compositional evolutionary learning system. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 402–409 (2018)

    Google Scholar 

  12. Bi, Y., Xue, B., Zhang, M.: Genetic programming with a new representation to automatically learn features and evolve ensembles for image classification. IEEE Trans. Cybern., 15 p. (2020). https://doi.org/10.1109/TCYB.2020.2964566

  13. Bi, Y., Xue, B., Zhang, M.: An automated ensemble learning framework using genetic programming for image classification. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 365–373 (2019)

    Google Scholar 

  14. Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)

    MATH  Google Scholar 

  15. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  16. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 886–893 (2005)

    Google Scholar 

  17. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)

    Article  Google Scholar 

  18. Liu, L., et al.: Deep learning for generic object detection: a survey. arXiv preprint arXiv:1809.02165 (2018)

  19. Zhou, Z.H.: Ensemble Methods: Foundations and Algorithms. Chapman and Hall/CRC, Boca Raton (2012)

    Book  Google Scholar 

  20. Sagi, O., Rokach, L.: Ensemble learning: a survey. Wiley Interdisc. Rev.: Data Min. Knowl. Discov. 8(4), 1–19 (2018)

    Google Scholar 

  21. Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45014-9_1

    Chapter  Google Scholar 

  22. Young, S., Abdou, T., Bener, A.: Deep super learner: a deep ensemble for classification problems. In: Bagheri, E., Cheung, J.C.K. (eds.) Canadian AI 2018. LNCS (LNAI), vol. 10832, pp. 84–95. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-89656-4_7

    Chapter  Google Scholar 

  23. Ding, C., Tao, D.: Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 1002–1014 (2017)

    Article  Google Scholar 

  24. Zhou, M., Zeng, X., Chen, A.: Deep forest hashing for image retrieval. Pattern Recogn. 95, 114–127 (2019)

    Article  Google Scholar 

  25. Zhu, G., Hu, Q., Gu, R., Yuan, C., Huang, Y.: ForestLayer: efficient training of deep forests on distributed task-parallel platforms. J. Parallel Distrib. Comput. 132, 113–126 (2019)

    Article  Google Scholar 

  26. Zhang, Y.L., et al.: Distributed deep forest and its application to automatic detection of cash-out fraud. ACM Trans. Intell. Syst. Technol. 10(5), 1–19 (2019)

    Google Scholar 

  27. Bi, Y., Xue, B., Zhang, M.: An effective feature learning approach using genetic programming with image descriptors for image classification [research frontier]. IEEE Comput. Intell. Mag. 15(2), 65–77 (2020)

    Article  Google Scholar 

  28. Montana, D.J.: Strongly typed genetic programming. Evol. Comput. 3(2), 199–230 (1995)

    Article  Google Scholar 

  29. Samaria, F.S., Harter, A.C.: Parameterisation of a stochastic model for human face identification. In: Proceedings of 1994 IEEE Workshop on Applications of Computer Vision, pp. 138–142 (1994)

    Google Scholar 

  30. Lee, K.C., Ho, J., Kriegman, D.J.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 5, 684–698 (2005)

    Google Scholar 

  31. Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 524–531 (2005)

    Google Scholar 

  32. Mallikarjuna, P., Targhi, A.T., Fritz, M., Hayman, E., Caputo, B., Eklundh, J.O.: THE KTH-TIPS2 database, pp. 1–10. Computational Vision and Active Perception Laboratory, Stockholm, Sweden (2006)

    Google Scholar 

  33. LeCun, Y., Cortes, C., Burges, C.J.: The mnist database (1998). http://yann.lecun.com/exdb/mnist

  34. Krizhevsky, A., Nair, V., Hinton, G.: The cifar-10 dataset, no. 55 (2014). http://www.cs.toronto.edu/kriz/cifar.html

  35. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)

    Article  MathSciNet  Google Scholar 

  36. Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016)

    Google Scholar 

  37. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images. Technical report, Citeseer (2009)

    Google Scholar 

  38. Ba, J., Caruana, R.: Do deep nets really need to be deep? In: Advances in Neural Information Processing Systems, pp. 2654–2662 (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Engineering and Computer Science, Victoria University of Wellington, Wellington, 6140, New Zealand

    Ying Bi, Bing Xue & Mengjie Zhang

Authors
  1. Ying Bi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying Bi .

Editor information

Editors and Affiliations

  1. Leiden University, Leiden, The Netherlands

    Thomas Bäck

  2. Leiden University, Leiden, The Netherlands

    Mike Preuss

  3. Leiden University, Leiden, The Netherlands

    André Deutz

  4. Sorbonne University, Paris, France

    Hao Wang

  5. Sorbonne University, Paris, France

    Carola Doerr

  6. Leiden University, Leiden, The Netherlands

    Michael Emmerich

  7. University of Münster, Münster, Germany

    Heike Trautmann

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bi, Y., Xue, B., Zhang, M. (2020). Evolving Deep Forest with Automatic Feature Extraction for Image Classification Using Genetic Programming. In: Bäck, T., et al. Parallel Problem Solving from Nature – PPSN XVI. PPSN 2020. Lecture Notes in Computer Science(), vol 12269. Springer, Cham. https://doi.org/10.1007/978-3-030-58112-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58112-1_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58111-4

  • Online ISBN: 978-3-030-58112-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation