Abstract
Markov blanket discovery plays an important role in both Bayesian network induction and feature selection for classification tasks. In this paper, we propose the Dynamic Ordering-based Search algorithm (DOS) for learning a Markov blanket of a domain variable from statistical conditional independence tests on data. The new algorithm orders conditional independence tests and updates the ordering immediately after a test is completed. Meanwhile, the algorithm exploits the known independence to avoid unnecessary tests by reducing the set of candidate variables. This results in both efficiency and reliability advantages over the existing algorithms. We theoretically analyze the algorithm on its correctness and empirically compare it with the state-of-the-art algorithm. Experiments show that the new algorithm achieves computational savings of around 40% on multiple benchmarks while securing similar or even better accuracy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)
Tsamardinos, I., Brown, L.E., Aliferis, C.F.: The max-min hill-climbing bayesian network structure learning algorithm. Machine Learning 65(1), 31–78 (2006)
Zeng, Y., Poh, K.L.: Block learning bayesian network structure from data. In: Proceedings of the Fourth International Conference on Hybrid Intelligent Systems (HIS 2004), pp. 14–19 (2004)
Zeng, Y., Hernandez, J.C.: A decomposition algorithm for learning bayesian network structures from data. In: Washio, T., Suzuki, E., Ting, K.M., Inokuchi, A. (eds.) PAKDD 2008. LNCS (LNAI), vol. 5012, pp. 441–453. Springer, Heidelberg (2008)
Zeng, Y., Xiang, Y., Hernandez, J.C., Lin, Y.: Learning local components to understand large bayesian networks. In: Proceedings of The Ninth IEEE International Conference on Data Mining (ICDM), pp. 1076–1081 (2009)
Koller, D., Sahami, M.: Toward optimal feature selection. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 284–292 (1996)
Margaritis, D., Thrun, S.: Bayesian network induction via local neighborhoods. Advances in Neural Information Processing Systems 12, 505–511 (1999)
Tsamardinos, I., Aliferis, C.F., Statnikov, A.R.: Algorithms for large scale markov blanket discovery. In: Proceedings of the Sixteenth International Florida Artificial Intelligence Research Society Conference, pp. 376–381 (2003)
Tsamardinos, I., Aliferis, C.: Towards principled feature selection: Relevancy, filters and wrappers. In: Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics (2003)
Tsamardinos, I., Aliferis, C., Statnikov, A.: Time and sample efficient discovery of markov blankets and direct causal relations. In: KDD, pp. 673–678 (2003)
Aliferis, C., Tsamardinos, I., Statnikov, A.: Hiton: A novel markov blanket algorithm for optimal variable selection. In: Proceedings of American Medical Informatics Association Annual Symposium (2003)
Pena, J.M., Nilsson, R., Bjorkegren, J., Tegner, J.: Towards scalable and data efficient learning of markov boundaries. International Journal of Approximate Reasoning 45(2), 211–232 (2007)
Fu, S., Desmarais, M.C.: Fast markov blanket discovery algorithm via local learning within single pass. In: Proceedings of the Twenty-First Canadian Conference on Artificial Intelligence, pp. 96–107 (2008)
Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley-Interscience, New York (2006)
Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. MIT Press, Cambridge (2000)
Loughry, J., van Hemert, J., Schoofs, L.: Efficiently enumerating the subsets of a set. Department of Mathematics and Computer Science, University of Antwerp, RUCA, Belgium, pp. 1–10 (2000)
Yaramakala, S., Margaritis, D.: Speculative markov blanket discovery for optimal feature selection. In: Proceedings of the Fifth IEEE International Conference on Data Mining, pp. 809–812 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zeng, Y., He, X., Xiang, Y., Mao, H. (2011). Dynamic Ordering-Based Search Algorithm for Markov Blanket Discovery. In: Huang, J.Z., Cao, L., Srivastava, J. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2011. Lecture Notes in Computer Science(), vol 6635. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20847-8_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-20847-8_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20846-1
Online ISBN: 978-3-642-20847-8
eBook Packages: Computer ScienceComputer Science (R0)