Skip to main content
SpringerLink
Log in

Duplicate Detection for Bayesian Network Structure Learning

  • Special Feature
  • Published:
New Generation Computing Aims and scope Submit manuscript

Abstract

We address the well-known score-based Bayesian network structure learning problem. Breadth-first branch and bound (BFBnB) has been shown to be an effective approach for solving this problem. Duplicate detection is an important component of the BFBnB algorithm. Previously, an external sorting-based technique was used for delayed duplicate detection (DDD). We propose a hashing-based technique for DDD and a bin packing algorithm for minimizing the number of external memory files and operations. We also give a structured duplicate detection approach which completely eliminates DDD. Importantly, these techniques ensure the search algorithms respect any given memory bound. Empirically, we demonstrate that structured duplicate detection is significantly faster than the previous state of the art in limited-memory settings. Our results show that the bin packing algorithm incurs some overhead, but that the overhead is offset by reducing I/O when more memory is available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. In this work, we use “memory” to refer to fast-access storage, such as RAM; by “external memory,” we mean storage with slower access, such as hard disks and network storage.

  2. “Appendix A” includes all of the acronyms we use and briefly describes each of them.

  3. “Appendix B” includes all of the notation we use in this work.

  4. The problem can also be defined as a maximization using non-positive local scores.

  5. In particular, we assume each node is written to disk at least once. We do not consider efficient implementations which avoid this. Thus, \(n_\mathrm{w} \ge n_\mathrm{u}\).

  6. We sometimes refer to an abstract node as one node in the abstract state space; other times, we use it to refer to the collection of “concrete” nodes in the original space. The meaning should be clear from context.

  7. For this analysis, we do not consider the load factor of the hash table.

  8. Our code is publicly available at https://bitbucket.org/bmmalone/urlearning-cpp.

  9. The strategy is optimal in that it minimizes the number of files. We solve the optimization problem using an integer linear programming formulation.

  10. We use a sigificance cutoff of \(p=0.05\) to determine significance.

References

  1. Bartlett, M., Cussens, J.: Integer linear programming for the Bayesian network structure learning problem. Artif. Intell. (2015) (In press)

  2. Chickering, D.M.: Learning Bayesian networks is NP-complete. In: Learning from Data: Artificial Intelligence and Statistics V, pp. 121–130. Springer, Berlin (1996)

  3. Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9, 309–347 (1992)

    MATH  Google Scholar 

  4. de Campos, C.P., Ji, Q.: Efficient learning of Bayesian networks using constraints. J. Mach. Learn. Res. 12, 663–689 (2011)

    MathSciNet  MATH  Google Scholar 

  5. de Campos, L.M., Huete, J.F.: A new approach for learning belief networks using independence criteria. Int J Approx. Reason. 24(1), 11–37 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  6. Edelkamp, S., Schrödl, S.: Heuristic Search: Theory and Applications. Morgan Kaufmann, San Francisco (2012)

  7. Fan, X., Yuan, C., Malone, B.: Tightening bounds for Bayesian network structure learning. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence (2014)

  8. Garey, M.R., Johnson, D.S.: Computers and intractability: a guide to the theory of np-completeness. W. H. Freeman and Co., New York (1979)

    MATH  Google Scholar 

  9. Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20, 197–243 (1995)

    MATH  Google Scholar 

  10. Jahnsson, N., Malone, B., Myllymäki, P.: Hashing-based hybrid duplicate detection for Bayesian network structure learning. In: Advanced Methodologies for Bayesian Networks Workshop, vol. 9505 of LNAI, pp. 46–60. Springer, Berlin (2015)

  11. Johnson, D.: Near-Optimal Bin Packing Algorithms. PhD thesis, Massachusetts Institute of Technology (1973)

  12. Koivisto, M., Sood, K.: Exact Bayesian structure discovery in Bayesian networks. J. Mach. Learn. Res. 5, 549–573 (2004)

    MathSciNet  MATH  Google Scholar 

  13. Korf, R.E.: A new algorithm for optimal bin packing. In: Proceedings of the 18th AAAI Conference on Artificial Intelligence (2002)

  14. Korf, R.E.: Best-first frontier search with delayed duplicate detection. In: Proceedings of the 19th AAAI Conference on Artifical Intelligence (2004)

  15. Korf, R.E.: Linear-time disk-based implicit graph search. J. ACM 55(6), 26:1–26:40 (2008)

  16. Malone, B., Järvisalo, M., Myllymäki, P.: Impact of learning strategies on the quality of Bayesian networks: an empirical evaluation. In: Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence (2015)

  17. Malone, B., Kangas, K., Järvisalo, M., Koivisto, M., Myllymäki, P.: Predicting the hardness of learning Bayesian networks. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence (2014)

  18. Malone, B., Yuan, C.: Evaluating anytime algorithms for learning optimal Bayesian networks. In: Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence (2013)

  19. Malone, B., Yuan, C.: A depth-first branch and bound algorithm for learning optimal Bayesian networks. In: Graph Structures for Knowledge Representation and Reasoning, vol. 8323 of LNCS, pp. 111–122. Springer (2014)

  20. Malone, B., Yuan, C., Hansen, E., Bridges, S.: Improving the scalability of optimal Bayesian network learning with external-memory frontier breadth-first branch and bound search. In: Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (2011)

  21. Ott, S., Imoto, S., Miyano, S.: Finding optimal models for small gene networks. In: Proceedings of the Pacific Symposium on Biocomputing (2004)

  22. Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)

  23. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach. Pearson Education, London (2003)

  24. Silander, T., Myllymäki, P.: A simple approach for finding the globally optimal Bayesian network structure. In: Proceedings of the 22nd Conference on Uncertainty in Artificial Intelligence (2006)

  25. Silander, T., Roos, T., Kontkanen, P., Myllymäki, P.: Factorized normalized maximum likelihood criterion for learning Bayesian network structures. In: Proceedings of the 4th European Workshop on Probabilistic Graphical Models (2008)

  26. Spirtes, P., Glymour, C., Schemes, R.: Causation, Prediction, and Search, 2nd edn. The MIT Press, Cambridge (2000)

  27. Suzuki, J.: Learning Bayesian belief networks based on the MDL principle: an efficient algorithm using the branch and bound technique. IEICE Trans. Inf. Syst. E82-D(2), 356–367 (1999)

  28. Tamada, Y., Imoto, S., Miyano, S.: Parallel algorithm for learning optimal Bayesian network structure. J. Mach. Learn. Res. 12, 2437–2459 (2011)

    MathSciNet  MATH  Google Scholar 

  29. Teyssier, M., Koller, D.: Ordering-based search: a simple and effective algorithm for learning Bayesian networks. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (2005)

  30. Tian, J.: A branch-and-bound algorithm for MDL learning Bayesian networks. In: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (2000)

  31. Van Beek, P., Hoffmann, H.-F.: Machine learning of Bayesian networks using constraint programming. In: Proceedings of the 21st Conference on Principles and Practice of Constraint Programming (2015)

  32. Yuan, C., Malone, B.: An improved admissible heuristic for finding optimal Bayesian networks. In: Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence (2012)

  33. Yuan, C., Malone, B.: Learning optimal Bayesian networks: a shortest path perspective. J. Artif. Intell. Res. 48, 23–65 (2013)

    MathSciNet  MATH  Google Scholar 

  34. Zhou, R., Hansen, E.A.: Sparse-memory graph search. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence (2003)

  35. Zhou, R., Hansen, E.A.: Structured duplicate detection in external-memory graph search. In: Proceedings of the 19th AAAI Conference on Artificial Intelligence (2004)

  36. Zhou, R., Hansen, E.A.: Breadth-first heuristic search. Artif. Intell. 170, 385–408 (2006)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Helsinki, Helsinki, Finland

    Niklas Jahnsson

  2. Section of Bioinformatics and Systems Cardiology, Department of Internal Medicine III and Klaus Tschira Institute for Integrative Computational Cardiology, University of Heidelberg, Heidelberg, Germany

    Brandon Malone

  3. Helsinki Institute for Information Technology, University of Helsinki, Helsinki, Finland

    Petri Myllymäki

Authors
  1. Niklas Jahnsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brandon Malone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Petri Myllymäki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brandon Malone.

Appendices

Appendix A: Acronyms

This appendix gives all of the acronyms used in this paper (Table 1).

Table 1 All of the acronyms used in this paper
Full size table

Appendix B: Notation

This appendix gives all of the mathematical notation used in this paper (Table 2).

Table 2 All of the mathematical notation used in this paper
Full size table

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jahnsson, N., Malone, B. & Myllymäki, P. Duplicate Detection for Bayesian Network Structure Learning. New Gener. Comput. 35, 47–67 (2017). https://doi.org/10.1007/s00354-016-0004-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00354-016-0004-9

Keywords

Search

Navigation