Skip to main content
SpringerLink
Log in
Book cover

Canadian Conference on Artificial Intelligence

Canadian AI 2017: Advances in Artificial Intelligence pp 27–38Cite as

A Global Search Approach for Inducing Oblique Decision Trees Using Differential Evolution

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10233))

Abstract

This paper describes the application of a Differential Evolution based approach for inducing oblique decision trees in a global search strategy. By using both the number of attributes and the number of class labels in a dataset, this approach determines the size of the real-valued vector utilized for encoding the set of hyperplanes used as test conditions in the internal nodes of an oblique decision tree. Also a scheme of three steps to map the linear representation of candidate solutions into feasible oblique decision trees is described. Experimental results obtained show that this approach induces more accurate classifiers than those produced by other proposed induction methods.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Highest values for each dataset are in bold.

References

  1. Agapitos, A., O’Neill, M., Brabazon, A., Theodoridis, T.: Maximum margin decision surfaces for increased generalisation in evolutionary decision tree learning. In: Silva, S., Foster, J.A., Nicolau, M., Machado, P., Giacobini, M. (eds.) EuroGP 2011. LNCS, vol. 6621, pp. 61–72. Springer, Heidelberg (2011). doi:10.1007/978-3-642-20407-4_6

    Chapter  Google Scholar 

  2. Barros, R.C., Basgalupp, M.P., Carvalho, A., Freitas, A.A.: A survey of evolutionary algorithms for decision-tree induction. IEEE Trans. Syst. Man Cybern.-Part C: Appl. Rev. 42(3), 291–312 (2012). doi:10.1109/TSMCC.2011.2157494

    Article  Google Scholar 

  3. Basgalupp, M.P., Barros, R.C., de Carvalho, A.C., Freitas, A.A.: Evolving decision trees with beam search-based initialization and lexicographic multi-objective evaluation. Inf. Sci. 258, 160–181 (2014). doi:10.1016/j.ins.2013.07.025

    Article  MathSciNet  Google Scholar 

  4. Bot, M.C.J., Langdon, W.B.: Improving induction of linear classification trees with genetic programming. In: Whitley, L.D., Goldberg, D.E., Cantú-Paz, E., Spector, L., Parmee, I.C., Beyer, H.G. (eds.) GECCO-2000, pp. 403–410. Morgan Kaufmann, Burlington (2000)

    Google Scholar 

  5. Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Taylor & Francis, Abingdon (1984)

    MATH  Google Scholar 

  6. Cantú-Paz, E., Kamath, C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Trans. Evol. Comput. 7(1), 54–68 (2003). doi:10.1109/TEVC.2002.806857

    Article  Google Scholar 

  7. Das, S., Abraham, A., Konar, A.: Automatic clustering using an improved differential evolution algorithm. IEEE Trans. Syst. Man Cybern.-Part A: Syst. Hum. 38(1), 218–237 (2008). doi:10.1109/tsmca.2007.909595

    Article  Google Scholar 

  8. De Falco, I.: Differential evolution for automatic rule extraction from medical databases. Appl. Soft Comput. 13(2), 1265–1283 (2013). doi:10.1016/j.asoc.2012.10.022

    Article  Google Scholar 

  9. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7(Dec), 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  10. Durillo, J.J., Nebro, A.J.: jMetal: a Java framework for multiobjective optimization. Adv. Eng. Softw. 42(10), 760–771 (2011). doi:10.1016/j.advengsoft.2011.05.014

    Article  Google Scholar 

  11. Espejo, P.G., Ventura, S., Herrera, F.: A survey on the application of genetic programming to classification. IEEE Trans. Syst. Man Cybern.-Part C: Appl. Rev. 40(2), 121–144 (2010). doi:10.1109/TSMCC.2009.2033566

    Article  Google Scholar 

  12. García, S., Derrac, J., Triguero, I., Carmona, C.J., Herrera, F.: Evolutionary-based selection of generalized instances for imbalanced classification. Knowl.-Based Syst. 25(1), 3–12 (2012). doi:10.1016/j.knosys.2011.01.012

    Article  Google Scholar 

  13. Geetha, K., Baboo, S.S.: An empirical model for thyroid disease classification using evolutionary multivariate Bayesian prediction method. Glob. J. Comput. Sci. Technol. 16(1), 1–9 (2016)

    Google Scholar 

  14. Hawkins, D.M.: The problem of overfitting. ChemInform 35(19) (2004). doi:10.1002/chin.200419274

  15. Hothorn, T., Hornik, K., Zeileis, A.: Unbiased recursive partitioning: a conditional inference framework. J. Comput. Graph. Stat. 15(3), 651–674 (2006). doi:10.1198/106186006x133933

    Article  MathSciNet  Google Scholar 

  16. Hyafil, L., Rivest, R.L.: Constructing optimal binary decision trees is NP-complete. Inf. Process. Lett. 5(1), 15–17 (1976). doi:10.1016/0020-0190(76)90095-8

    Article  MathSciNet  MATH  Google Scholar 

  17. Kennedy, H.C., Chinniah, C., Bradbeer, P., Morss, L.: The contruction and evaluation of decision trees: a comparison of evolutionary and concept learning methods. In: Corne, D., Shapiro, J.L. (eds.) AISB EC 1997. LNCS, vol. 1305, pp. 147–161. Springer, Heidelberg (1997). doi:10.1007/BFb0027172

    Chapter  Google Scholar 

  18. Krętowski, M., Grześ, M.: Evolutionary learning of linear trees with embedded feature selection. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, pp. 400–409. Springer, Heidelberg (2006). doi:10.1007/11785231_43

    Chapter  Google Scholar 

  19. Leema, N., Nehemiah, H.K., Kannan, A.: Neural network classifier optimization using differential evolution with global information and back propagation algorithm for clinical datasets. Appl. Soft Comput. 49, 834–844 (2016). doi:10.1016/j.asoc.2016.08.001

    Article  Google Scholar 

  20. Li, J., Ding, L., Li, B.: Differential evolution-based parameters optimisation and feature selection for support vector machine. Int. J. Comput. Sci. Eng. 13(4), 355–363 (2016)

    Article  Google Scholar 

  21. Lichman, M.: UCI Machine Learning Repository (2013). University of California, Irvine. http://archive.ics.uci.edu/ml

  22. Liu, K.H., Xu, C.G.: A genetic programming-based approach to the classification of multiclass microarray datasets. Bioinformatics 25(3), 331–337 (2009). doi:10.1093/bioinformatics/btn644

    Article  Google Scholar 

  23. Lopes, R.A., Freitas, A.R.R., Silva, R.C.P., Guimarães, F.G.: Differential evolution and perceptron decision trees for classification tasks. In: Yin, H., Costa, J.A.F., Barreto, G. (eds.) IDEAL 2012. LNCS, vol. 7435, pp. 550–557. Springer, Heidelberg (2012). doi:10.1007/978-3-642-32639-4_67

    Chapter  Google Scholar 

  24. Murthy, S.K., Kasif, S., Salzberg, S., Beigel, R.: OC1: a randomized algorithm for building oblique decision trees. In: Proceedings of AAAI 1993, vol. 93, pp. 322–327 (1993)

    Google Scholar 

  25. Plagianakos, V.P., Tasoulis, D.K., Vrahatis, M.N.: A review of major application areas of differential evolution. In: Chakraborty, U.K. (ed.) Advances in Differential Evolution. SCI, vol. 143, pp. 197–238. Springer, Heidelberg (2008). doi:10.1007/978-3-540-68830-38

    Chapter  Google Scholar 

  26. Quinlan, J.R.: Simplifying decision trees. Int. J. Hum.-Comput. Stud. 27(3), 221–234 (1987). doi:10.1006/ijhc.1987.0321

    Google Scholar 

  27. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, Burlington (1993)

    Google Scholar 

  28. Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997). doi:10.1023/A:1008202821328

    Article  MathSciNet  MATH  Google Scholar 

  29. Strobl, C., Malley, J., Tutz, G.: An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychol. Methods 14(4), 323–348 (2009). doi:10.1037/a0016973

    Article  Google Scholar 

  30. Tušar, T.: Optimizing accuracy and size of decision trees. In: ERK-2007, pp. 81–84 (2007)

    Google Scholar 

  31. Veenhuis, C.B.: Tree based differential evolution. In: Vanneschi, L., Gustafson, S., Moraglio, A., Falco, I., Ebner, M. (eds.) EuroGP 2009. LNCS, vol. 5481, pp. 208–219. Springer, Heidelberg (2009). doi:10.1007/978-3-642-01181-8_18

    Chapter  Google Scholar 

  32. Vukobratović, B., Struharik, R.: Evolving full oblique decision trees. In: CINTI 2015, pp. 95–100. IEEE (2015). doi:10.1109/CINTI.2015.7382901

  33. Wang, P., Tang, K., Weise, T., Tsang, E.P.K., Yao, X.: Multiobjective genetic programming for maximizing ROC performance. Neurocomputing 125, 102–118 (2014). doi:10.1016/j.neucom.2012.06.054

    Article  Google Scholar 

  34. Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Burlington (2005)

    MATH  Google Scholar 

Download references

Acknowledgments

This work has been supported by the Mexican Government (CONACyT FOMIX-DICC project No. TAB-2014-C01-245876 and the PROMEP-SEP project No. DSA/103.5/15/6409).

Author information

Authors and Affiliations

  1. Departamento de Sistemas y Computación, Instituto Tecnológico de Veracruz, Veracruz, Mexico

    Rafael Rivera-Lopez

  2. División Académica de Informática y Sistemas, Universidad Juárez Autónoma de Tabasco, Cunduacán, Mexico

    Juana Canul-Reich

Authors
  1. Rafael Rivera-Lopez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juana Canul-Reich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juana Canul-Reich .

Editor information

Editors and Affiliations

  1. University of Regina, Regina, Saskatchewan, Canada

    Malek Mouhoub

  2. University of Montreal, Montreal, Québec, Canada

    Philippe Langlais

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Rivera-Lopez, R., Canul-Reich, J. (2017). A Global Search Approach for Inducing Oblique Decision Trees Using Differential Evolution. In: Mouhoub, M., Langlais, P. (eds) Advances in Artificial Intelligence. Canadian AI 2017. Lecture Notes in Computer Science(), vol 10233. Springer, Cham. https://doi.org/10.1007/978-3-319-57351-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-57351-9_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-57350-2

  • Online ISBN: 978-3-319-57351-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation