Abstract
Among the several attempts to improve the Naive Bayes (NB) classifier, the Aggregating One-Dependence Estimators (AODE) has proved to be one of the most attractive, considering not only the low error it provides but also its efficiency. AODE estimates the corresponding parameters for every SPODE (Superparent-One-Dependence Estimators) using each attribute of the database as the superparent, and uniformly averages them all. Nevertheless, AODE has properties that can be improved. Firstly, the need to store all the models constructed leads to a high demand on space and hence, to the impossibility of dealing with problems of high dimensionality; secondly, even though it is fast, the computational time required for the training and the classification time is quadratic in the number of attributes. This is specially significant in the classification time, as it is frequently carried out in real time. In this paper, we propose the HODE classifier as an alternative approach to AODE in order to alleviate its problems by estimating a new variable (the hidden variable) as a superparent besides the class, whose main objective is to gather all the dependences existing in the AODE models. The results obtained show that this new algorithm provides similar results in terms of accuracy with a reduction in classification time and space complexity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Langley, P., Iba, W., Thompson, K.: An Analysis of Bayesian Classifiers. In: 10th national conference on artificial intelligence, pp. 223–228. AAAI Press, Menlo Park (1992)
Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. John Wiley & Sons Inc., Chichester (1973)
Domingos, P., Pazzani, M.: Beyond independence: Conditions for the optimality of the simple Bayesian classifier. In: 13th International Conference on Machine Learning, pp. 105–112. Morgan Kaufmann, Italy (1996)
Langley, P., Sage, S.: Induction of selective Bayesian classifiers. In: 10th Conference on Uncertainty in Artificial Intelligence, pp. 399–406. Morgan Kaufmann, San Francisco (1994)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. J. Mach. Learn. 29(2-3), 131–163 (1997)
Kohavi, R.: Scaling Up the Accuracy of Naive-Bayes Classifiers: a Decision-Tree Hybrid. In: 2nd International Conference on Knowledge Discovery and Data Mining, pp. 202–207 (1996)
Elkan, C.: Boosting and Naive Bayesian learning, Technical report, Dept. Computer Science and Eng., Univ. of California, San Diego (1997)
Webb, G.I., Boughton, J.R., Wang, Z.: Not So Naive Bayes: Aggregating One-Dependence Estimators. J. Mach. Learn. 58(1), 5–24 (2005)
Sahami, M.: Learning limited dependence Bayesian classifiers. In: 2nd International Conference on Knowledge Discovery in Databases, pp. 335–338. AAAI Press, Menlo Park (1996)
Yang, Y., Korb, K., Ting, K.-M., Webb, G.I.: Ensemble Selection for SuperParent-One-Dependence Estimators. In: Zhang, S., Jarvis, R. (eds.) AI 2005. LNCS, vol. 3809, pp. 102–112. Springer, Heidelberg (2005)
Zheng, F., Webb, G.I.: A Comparative Study of Semi-naive Bayes Methods in Classification Learning. In: Simoff, S.J., Williams, G.J., Galloway, J., Kolyshkina, I. (eds.) 4th Australasian Data Mining Conference (AusDM 2005), pp. 141–156. University of Technology, Sydney (2005)
Zheng, Z., Webb, G.I.: Lazy Learning of Bayesian Rules. J. Mach. Learn. 41(1), 53–84 (2000)
Keogh, E., Pazzani, M.: Learning Augmented Bayesian Classifiers: A Comparison of Distribution-based and Classification-based Approaches. In: 7th Int’l Workshop on AI and Statistics, Florida, pp. 225–230 (1999)
Dempster, A., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data via the EM Algorithm. J. R. Stat. Soc. Ser. B-Stat. Methodol. 39, 1–38 (1977)
Langseth, H., Nielsen, T.D.: Classification using hierarchical Naïve Bayes models. J. Mach. Learn. 63(2), 135–159 (2006)
Lowd, D., Domingos, P.: Naive Bayes models for probability estimation. In: 22nd international conference on Machine learning, pp. 529–536. ACM, Bonn (2005)
Cheeseman, P., Stutz, J.: Bayesian classification (AutoClass): theory and results. In: Advances in knowledge discovery and data mining, pp. 153–180. AAAI Press, Menlo Park (1996)
Akaike, H.: A Bayesian analysis of the minimum AIC procedure. Ann. Inst. Statist. Math. 30A, 9–14 (1978)
Collection of Datasets available from the Weka Official HomePage, University of Waikato, http://www.cs.waikato.ac.nz/ml/weka/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Flores, M.J., Gámez, J.A., Martínez, A.M., Puerta, J.M. (2009). HODE: Hidden One-Dependence Estimator. In: Sossai, C., Chemello, G. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2009. Lecture Notes in Computer Science(), vol 5590. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02906-6_42
Download citation
DOI: https://doi.org/10.1007/978-3-642-02906-6_42
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02905-9
Online ISBN: 978-3-642-02906-6
eBook Packages: Computer ScienceComputer Science (R0)