Skip to main content
SpringerLink
Log in

Enhancing extended belief rule-based systems for classification problems using decomposition strategy and overlap function

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Multi-class and multi-attribute are two important features of classification problems and have different effects on the requirements and performance of the classifier. Decomposition strategy and overlap function are two effective ways to enhance the performance of classifiers, because the former decomposes a complex multi-class problem into multiple simple sub-problems; the latter uses various functions to specify the conjunctive relationship of input variables in a multi-attribute problem. Extended belief rule-based system (EBRBS) is an advanced rule-based system that has been widely used in classification problems. In order to apply decomposition strategies and overlap functions to enhance the performance of EBRBSs, the present work focuses on the investigative research and comparative evaluation of the commonly used one-versus-one (OVO) decomposition strategy and five common overlap functions to improve the performance of EBRBSs on multi-class and multi-attribute problems. More specifically, three typical kinds of EBRBSs, namely original EBRBS (O-EBRBS), EBRBS with dynamic rule activation (DRA-EBRBS), and a latest EBRBS for big data (Micro-EBRBS), are selected to conduct extensive experimental studies on twenty classification problems. To best of our knowledge, this present work is the first time to provide a meaningful and useful study in revealing the potential capability of the EBRBSs with decomposition strategy and overlap function for multi-class and multi-attribute problems. Experimental results demonstrate that the square product overlap function and the OVO strategy can enhance the performance of EBRBSs over others for twenty classification problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Jiang Z, Bian Z, Wang S (2020) Multi-view local linear KNN classification: theoretical and experimental studies on image classification. Int J Mach Learn Cybern 11:525–543. https://doi.org/10.1007/s13042-019-00992-9

    Article  Google Scholar 

  2. Yan W, Sun Q, Sun J, Li Y (2020) Semi-supervised learning framework based on statistical analysis for image set classification. Pattern Recogn 107:1–15. https://doi.org/10.1016/j.patcog.2020.107500

    Article  Google Scholar 

  3. Fan X, Hu S, He J (2019) A dynamic selection ensemble method target recognition based on clustering and randomized reference classifier. Int J Mach Learn Cybern 10:515–525. https://doi.org/10.1007/s13042-017-0732-2

    Article  Google Scholar 

  4. Volna E, Kotyrba M (2017) Enhanced ensemble-based classifier with boosting for pattern recognition. Appl Math Comput 310:1–14. https://doi.org/10.1016/j.amc.2017.04.019

    Article  MathSciNet  MATH  Google Scholar 

  5. Zhang DG, Wang JX, Fan HR, Zhang T, Gao JX, Peng Y (2020) New method of traffic flow forecasting based on quantum particle swarm optimization strategy for intelligent transportation system. Int J Commun Syst 34(1):1–20. https://doi.org/10.1002/dac.4647

    Article  Google Scholar 

  6. Liu XH, Zhang DG, Yan HR, Cui YY, Chen L (2019) A new algorithm of the best path selection based on machine learning. IEEE Access 7:126913–126928. https://doi.org/10.1109/ACCESS.2019.2939423

    Article  Google Scholar 

  7. Galar M, Fernández A, Barrenechea E, Bustince H, Herrera F (2011) An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recogn 44:1761–1776. https://doi.org/10.1016/j.patcog.2011.01.017

    Article  Google Scholar 

  8. Liu Y, Bi JW, Fan ZP (2017) A method for multi-class sentiment classification based on an improved one-vs-one (OVO) strategy and the support vector machine (SVM) algorithm. Inf Sci 394–395:38–52. https://doi.org/10.1016/j.ins.2017.02.016

    Article  Google Scholar 

  9. Zhang T, Zhang DG, Yang HR, Qiu JN, Gao JX (2021) A new method of data missing estimation with FNN-based tensor heterogeneous ensemble learning for internet of vehicle. Neurocomputing 420:98–110. https://doi.org/10.1016/j.neucom.2020.09.042

    Article  Google Scholar 

  10. Zhang DG, Liu S, Liu XH, Zhang T, Cui YY (2018) Novel dynamic source routing protocol (DSR) based on genetic algorithm-bacterial foraging optimization (GA-BFO). Int J Commun Syst 31(18):1–20. https://doi.org/10.1002/dac.3824

    Article  Google Scholar 

  11. Yang LH, Liu J, Wang YM, Martinez L (2021) A micro-extended belief rule-bases system for big data multi-class classification problems. IEEE Trans Syst Man Cybern Syst 51(1):420–440. https://doi.org/10.1109/TSMC.2018.2872843

    Article  Google Scholar 

  12. Zhang DG, Ge H, Zhang T, Cui YY, Liu XH, Mao GQ (2019) New multi-hop clustering algorithm for vehicular ad hoc networks. IEEE Trans Intell Transp Syst 20(4):1517–1530. https://doi.org/10.1109/TITS.2018.2853165

    Article  Google Scholar 

  13. Gomez D, Rodriguez JT, Montero J, Bustince H, Barrenechea E (2016) n-Dimensional overlap functions. Fuzzy Sets Syst 287:57–75. https://doi.org/10.1016/j.fss.2014.11.023

    Article  MathSciNet  MATH  Google Scholar 

  14. De Miguel L, Gómez D, Rodríguez JT, Montero J, Bustince H, Dimuro GP, Sanz JA (2019) General overlap functions. Fuzzy Sets Syst 372:81–96. https://doi.org/10.1016/j.fss.2018.08.003

    Article  MathSciNet  MATH  Google Scholar 

  15. Calzada A, Liu J, Wang H, Kashyap A (2015) A new dynamic rule activation method for extended belief rule-based systems. IEEE Trans Knowl Data Eng 27(4):880–894. https://doi.org/10.1109/TKDE.2014.2356460

    Article  Google Scholar 

  16. Zhu HZ, Xiao MQ, Zhao X, Tang XL, Yang LH, Kang WJ, Liu ZZ (2020) A structure optimization method for extended belief-rule-based classification system. Knowl Based Syst 203:1–15. https://doi.org/10.1016/j.knosys.2020.106096

    Article  Google Scholar 

  17. Fang WJ, Gong XT, Liu GG, Wu YJ, Fu YG (2020) A balance adjusting approach of extended belief-rule-based system for imbalanced classification problem. IEEE Access 8:41201–41212. https://doi.org/10.1109/ACCESS.2020.2976708

    Article  Google Scholar 

  18. Liu J, Martinez L, Calzada A, Wang H (2013) A novel belief rule base representation, generation and its inference methodology. Knowl Based Syst 53:129–141. https://doi.org/10.1016/j.knosys.2013.08.019

    Article  Google Scholar 

  19. Cui YY, Zhang DG, Zhang T, Chen L, Piao MJ, Zhu HL (2020) Novel method of mobile edge computation offloading based on evolutionary game strategy for IoT devices. AEU Int J Electron C 118:1–10. https://doi.org/10.1016/j.aeue.2020.153134

    Article  Google Scholar 

  20. Yang LH, Wang YM, Su Q, Fu YG, Chin KS (2016) Multi-attribute search framework for optimizing extended belief rule-based systems. Inf Sci 370–371:159–183. https://doi.org/10.1016/j.ins.2016.07.067

    Article  Google Scholar 

  21. Yang LH, Wang YM, Fu YG (2018) A consistency analysis-based rule activation method for extended belief-rule-based systems. Inf Sci 445–446:50–65. https://doi.org/10.1016/j.ins.2018.02.059

    Article  MathSciNet  Google Scholar 

  22. Zhang A, Gao F, Yang M, Bi WH (2020) A new rule reduction and training method for extended belief rule base based on DBSCAN algorithm. Int J Approx Reason 119:20–39. https://doi.org/10.1016/j.ijar.2019.12.016

    Article  MathSciNet  MATH  Google Scholar 

  23. Yang LH, Wang YM, Lan YX, Chen L, Fu YG (2017) A data envelopment analysis (DEA)-based method for rule reduction in extended belief-rule-based systems. Knowl Based Syst 123:174–187. https://doi.org/10.1016/j.knosys.2017.02.021

    Article  Google Scholar 

  24. Alcala-Fdez J, Fernández A, Luengo J, Derrac J, García S, Sanchez L, Herrera F (2011) KEEL data-mining software tool-data set repository, integration of algorithms and experimental analysis framework. J Multiple Valued Logic Soft Comput 17:255–287. https://doi.org/10.1016/j.jlap.2009.12.002

    Article  Google Scholar 

  25. García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining—experimental analysis of power. Inf Sci 180:2044–2064. https://doi.org/10.1016/j.ins.2009.12.010

    Article  Google Scholar 

  26. Fernández A, Calderon M, Barrenechea E, Bustince H, Herrera F (2010) Solving multi-class problems with linguistic fuzzy rule based classification systems based on pairwise learning and preference relations. Fuzzy Sets Syst 161:3064–3080. https://doi.org/10.1016/j.fss.2010.05.016

    Article  MathSciNet  MATH  Google Scholar 

  27. Elkano M, Galar M, Sanz J, Fernández A, Barrenechea E, Herrera F, Bustince H (2015) Enhancing multiclass classification in FARC-HD fuzzy classifier, on the synergy between n-dimensional overlap functions and decomposition strategies. IEEE Trans Fuzzy Syst 23(5):1562–1580. https://doi.org/10.1109/TFUZZ.2014.2370677

    Article  Google Scholar 

  28. Garcia LPF, Saez JA, Luengo J, Lorena AC, de Carvalho ACPLF, Herrera F (2015) Using the One-vs-One decomposition to improve the performance of class noise filters via an aggregation strategy in multi-class classification problems. Knowl Based Syst 90:153–164. https://doi.org/10.1016/j.knosys.2015.09.023

    Article  Google Scholar 

  29. Zhang Z, Krawczyk B, Garcìa S, Rosales-Pérez A, Herrera F (2016) Empowering one-vs-one decomposition with ensemble learning for multi-class imbalanced data. Knowl Based Syst 106:251–263. https://doi.org/10.1016/j.knosys.2016.05.048

    Article  Google Scholar 

  30. Liu JY, Jia BB (2020) Combining one-vs-one decomposition and instance-based learning for multi-class classification. IEEE Access. https://doi.org/10.1109/ACCESS.2020.3034448

    Article  Google Scholar 

  31. Asmus TC, Dimuro GP, Bedregal B, Sanz JA Jr, Pereira S, Bustince H (2020) General Interval-valued overlap functions and interval-valued overlap indices. Inf Sci 527:27–50. https://doi.org/10.1016/j.ins.2020.03.091

    Article  MathSciNet  MATH  Google Scholar 

  32. Arqub OA (2017) Adaptation of reproducing kernel algorithm for solving fuzzy Fredholm—Volterra integrodifferential equations. Neural Comput Appl 28:1591–1610. https://doi.org/10.1007/s00521-015-2110-x

    Article  Google Scholar 

  33. Arqub OA, Al-Smadi M (2020) Fuzzy conformable fractional differential equations: novel extended approach and new numerical solutions. Soft Comput 24(16):12501–12522. https://doi.org/10.1007/s00500-020-04687-0

    Article  Google Scholar 

  34. Chen JQ, Mao GQ, Li CL, Zhang DG (2020) A topological approach to secure message dissemination in vehicular networks. IEEE Trans Intell Transp Syst 21(1):135–148. https://doi.org/10.1109/TITS.2018.2889746

    Article  Google Scholar 

  35. Zhang DG, Wu H, Zhao PZ, Liu XH, Cui YY, Chen L, Zhang T (2020) New approach of multi-path reliable transmission for marginal wireless sensor network. Wirel Netw 26(2):1503–1517. https://doi.org/10.1007/s11276-019-02216-y

    Article  Google Scholar 

  36. Yang JB, Liu J, Wang J, Sii HS, Wang HW (2006) Belief rule-base inference methodology using the evidential reasoning approach—RIMER. IEEE Trans Syst Man Cybern Part A Syst Hum 36(2):266–285. https://doi.org/10.1109/TSMCA.2005.851270

    Article  Google Scholar 

  37. Kahraman HT, Sagiroglu S, Colak I (2013) The development of intuitive knowledge classifier and the modeling of domain dependent data. Knowl Based Syst 37:283–295. https://doi.org/10.1016/j.knosys.2012.08.009

    Article  Google Scholar 

  38. Ye FF, Yang LH, Wang YM, Chen L (2020) An environmental pollution management method based on extended belief rule base and data envelopment analysis under interval uncertainty. Comput Ind Eng 144:1–15. https://doi.org/10.1016/j.cie.2020.106454

    Article  Google Scholar 

  39. Wang YM, Ye FF, Yang LH (2020) Extended belief rule based system with joint learning for environmental governance cost prediction. Ecol Ind 111:1–14. https://doi.org/10.1016/j.ecolind.2020.106070

    Article  Google Scholar 

  40. Wang YM, Yang JB, Xu DL (2006) Environmental impact assessment using the evidential reasoning approach. Eur J Oper Res 174(3):1885–1913. https://doi.org/10.1016/j.ejor.2004.09.059

    Article  MATH  Google Scholar 

  41. Friedman J (1996) Another approach to polychotomous classification Technical Report, Department of Statistics, Stanford University

  42. Hüllermeier E, Vanderlooy S (2010) Combining predictions in pairwise classification: an optimal adaptive voting strategy and its relation to weighted voting. Pattern Recogn 43(1):128–142. https://doi.org/10.1016/j.patcog.2009.06.013

    Article  MATH  Google Scholar 

  43. Elkano M, Galar M, Sanz J, Bustince H (2016) Fuzzy Rule-Based Classification Systems for multi-class problems using binary decomposition strategies: on the influence of n-dimensional overlap functions in the Fuzzy Reasoning Method. Inf Sci 332:94–114. https://doi.org/10.1016/j.ins.2015.11.006

    Article  Google Scholar 

  44. Hüllermeier E, Brinker K (2008) Learning valued preference structures for solving classification problems. Fuzzy Sets Syst 159(18):2337–2352. https://doi.org/10.1016/j.fss.2008.01.021

    Article  MathSciNet  MATH  Google Scholar 

  45. Moreno-Torres JG, Saez JA, Herrera F (2012) Study on the impact of partition-induced dataset shift on k-fold cross-validation. IEEE Trans Neural Netw Learn Syst 23(8):1304–1312. https://doi.org/10.1109/TNNLS.2012.2199516

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Nos. 72001043 and 61773123), the Natural Science Foundation of Fujian Province of China (No. 2020J05122), the Humanities and Social Science Foundation of the Ministry of Education of China (No. 20YJC630188), the Social Science Planning Fund Project of Fujian Province of China (No. FJ2019C032), the Chengdu International Science Cooperation Project (No. 2020-GH02-00064-HZ), and the Spanish Ministry of Economy and Competitiveness through the Spanish National Research Project PGC2018-099402-B-I00.

Author information

Authors and Affiliations

  1. Decision Sciences Institute, Fuzhou University, Fuzhou, People’s Republic of China

    Long-Hao Yang & Ying-Ming Wang

  2. Key Laboratory of Spatial Data Mining and Information Sharing of Ministry of Education, Fuzhou University, Fuzhou, People’s Republic of China

    Ying-Ming Wang

  3. School of Computing, Ulster University, Northern Ireland, UK

    Long-Hao Yang, Jun Liu, Hui Wang & Luis Martínez

  4. Department of Computer Science, University of Jaén, Jaén, Spain

    Long-Hao Yang & Luis Martínez

Authors
  1. Long-Hao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying-Ming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Luis Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying-Ming Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, LH., Liu, J., Wang, YM. et al. Enhancing extended belief rule-based systems for classification problems using decomposition strategy and overlap function. Int. J. Mach. Learn. & Cyber. 13, 811–837 (2022). https://doi.org/10.1007/s13042-021-01355-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01355-z

Keywords

Search

Navigation