Skip to main content
SpringerLink
Log in

Approximations of Bayesian networks through KL minimisation

  • Special Issue
  • Published:
New Generation Computing Aims and scope Submit manuscript

Abstract

Exact inference in large, complex Bayesian networks is computationally intractable. Approximate schemes are therefore of great importance for real world computation. In this paper we consider an approximation scheme in which the original Bayesian network is approximated by another Bayesian network. The approximating network is optimised by an iterative procedure, which minimises the Kullback-Leibler divergence between the two networks. The procedure is guaranteed to converge to a local minimum of the Kullback-Leibler divergence. An important question in this scheme is how to choose the structure of the approximating network. In this paper we show how redundant structures of the approximating model can be pruned in advance. Simulation results of model optimisation are provided to illustrate the methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Pearl, J.,Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann Publishers, Inc., 1988.

  2. Jensen, F. V.,An introduction to Bayesian networks, UCL Press, 1996.

  3. Castillo, E., Gutierrez, J. M. and Hadi A. S.,Expert Systems and Probabilistic Network Models, Springer, 1997.

  4. Cooper, G. F., “The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks.”Artificial Intelligence, 42, pp. 393–405, 1990.

    Article  MATH  MathSciNet  Google Scholar 

  5. Shwe, M. A., Middleton, B., Heckerman, D. E., Henrion, M., Horvitz, E. J., Lehman, H. P. and Cooper, G. F., “Probabilistic Diagnosis Using a Reformulation of the Internist-1 QMR Knowledge Base,”Methods of Information in Medicine, 30, pp. 241–55, 1991.

    Google Scholar 

  6. Saul, L. K., Jaakkola, T. and Jordan, M. I., “Mean Field Theory for Sigmoid Belief Networks,”Journal of Artificial Intelligence Research, 4, pp 61–76, 1996.

    MATH  Google Scholar 

  7. Jaakkola, T. S. and Jordan, M. I., “Variational Probabilistic Inference and the QMR-DT Network,”Journal of Artificial Intelligence Research, 10, pp. 291–322, 1999.

    MATH  Google Scholar 

  8. Jordan, M. I., editor. “Learning in Graphical Models”,volume 89, NATO ASI, Series D: Behavioural and Social Sciences, Kluwer, 1998.

  9. Barber, D. and Wiegerinck, W., “Tractable Variational Structures for Approximating Graphical Models,” In Kearns, M. S., Solla, S. A. and Cohn, D. A., editors,Advances in Neural Information Processing Systems, volume 11. MIT Press, 1999.

  10. Wiegerinck, W. and Barber, D., “Mean Field Theory Based on Belief Networks for Approximate Inference,” In Niklasson, L., Bodén, M. and Ziemke, T., editors,ICANN’98: Proceedings of the 8th International Conference on Artificial Neural Networks, Skövde, Sweden, 2–4 September 1998,2, pp. 499–504, London, Springer, 1998.

    Google Scholar 

  11. Whittaker, J.,Graphical Models in Applied Multivariate Statistics. Wiley, Chichester, 1990.

    MATH  Google Scholar 

  12. Lauritzen, S. L. and Spiegelhalter, D. J., “Local Computations with Probabilties on Graphical Structures and their Application to Expert Systems,”J. Royal Statistical society B, 50, pp. 154–227, 1988.

    MathSciNet  Google Scholar 

  13. Neal, R., “Connectionist Learning of Belief Networks,”Artificial Intelligence, 56, pp. 71–113, 1992.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. RWCP Theoretical Foundation SNN University of Nijmegen, Geert Grooteplein Noord 21, 6525, EZ Nijmegen, The Netherlands

    Wim Wiegerinck & Bert Kappen

Authors
  1. Wim Wiegerinck
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bert Kappen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wim Wiegerinck.

Additional information

Wim Wiegerinck, Ph.D.: He has a postdoctoral position at Theoretical Foundation SNN, University of Nijmegen, the Netherlands. He obtained his M.S. in physics from the University of Amsterdam, the Netherlands in 1988 and Ph.D. degree in physics from the University of Nijmegen, the Netherlands in 1996. His current research interest is in theory and applications of graphical models and artificial neural networks.

Bert Kappen, Ph.D.: He studied particle physics in Groningen, the Netherlands and completed his PhD in this field in 1987 at the Rockefeller University in New York. From 1987 to 1989 he worked as a scientist at the Philips Research Laboratories in Eindhoven, the Netherlands. Presently, he is associate professor of physics at SNN University of Nijmegen, conducting research in artificial and biological neural networks.

About this article

Cite this article

Wiegerinck, W., Kappen, B. Approximations of Bayesian networks through KL minimisation. New Gener Comput 18, 167–175 (2000). https://doi.org/10.1007/BF03037595

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03037595

Keywords

Search

Navigation