Abstract
Most real-world machine learning problems have both statistical and relational aspects. Thus learners need representations that combine probability and relational logic. Markov logic accomplishes this by attaching weights to first-order formulas and viewing them as templates for features of Markov networks. Inference algorithms for Markov logic draw on ideas from satisfiability, Markov chain Monte Carlo and knowledge-based model construction. Learning algorithms are based on the conjugate gradient algorithm, pseudo-likelihood and inductive logic programming. Markov logic has been successfully applied to problems in entity resolution, link prediction, information extraction and others, and is the basis of the open-source Alchemy system.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Becker, S., Le Cun, Y.: Improving the convergence of back-propagation learning with second order methods. In: Proceedings of the 1988 Connectionist Models Summer School, San Mateo, CA, pp. 29–37. Morgan Kaufmann, San Francisco (1989)
Besag, J.: Statistical analysis of non-lattice data. The Statistician 24, 179–195 (1975)
Braz, R., Amir, E., Roth, D.: Lifted first-order probabilistic inference. In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, Edinburgh, UK, pp. 1319–1325. Morgan Kaufmann, San Francisco (2005)
Collins, M.: Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In: Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, Philadelphia, PA, pp. 1–8. ACL (2002)
Damien, P., Wakefield, J., Walker, S.: Gibbs sampling for Bayesian non-conjugate and hierarchical models by auxiliary variables. Journal of the Royal Statistical Society, Series B 61 (1999)
De Raedt, L., Dehaspe, L.: Clausal discovery. Machine Learning 26, 99–146 (1997)
Della Pietra, S., Della Pietra, V., Lafferty, J.: Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 380–392 (1997)
Dietterich, T.: Experience with Markov logic networks in a large AI system. In: Probabilistic, Logical and Relational Learning - Towards a Synthesis, number 05051 in Dagstuhl Seminar Proceedings. Internationales Begegnungs- und Forschungszentrum für Informatik (IBFI), Dagstuhl, Germany (2007)
Friedman, N., Getoor, L., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, Stockholm, Sweden, pp. 1300–1307. Morgan Kaufmann, San Francisco (1999)
Genesereth, M.R., Nilsson, N.J.: Logical Foundations of Artificial Intelligence. Morgan Kaufmann, San Mateo (1987)
Getoor, L., Taskar, B. (eds.): Introduction to Statistical Relational Learning. MIT Press, Cambridge (2007)
Gilks, W.R., Richardson, S., Spiegelhalter, D.J.: Markov Chain Monte Carlo in Practice. Chapman and Hall, London, UK (1996)
Grenager, T., Klein, D., Manning, C.D.: Unsupervised learning of field segmentation models for information extraction. In: Proceedings of the Forty-Third Annual Meeting on Association for Computational Linguistics, Ann Arbor, Michigan, pp. 371–378. Association for Computational Linguistics (2005)
Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)
Jaimovich, A., Meshi, O., Friedman, N.: Template based inference in symmetric relational markov random fields. In: Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence, Vancouver, Canada, AUAI Press (2007)
Kautz, H., Selman, B., Jiang, Y.: A general stochastic approach to solving problems with hard and soft constraints. In: Gu, D., Du, J., Pardalos, P. (eds.) The Satisfiability Problem: Theory and Applications, pp. 573–586. American Mathematical Society, New York (1997)
Kersting, K., De Raedt, L.: Towards combining inductive logic programming with Bayesian networks. In: Proceedings of the Eleventh International Conference on Inductive Logic Programming, Strasbourg, France, pp. 118–131. Springer, Heidelberg (2001)
Kok, S., Domingos, P.: Learning the structure of Markov logic networks. In: Proceedings of the Twenty-Second International Conference on Machine Learning, Bonn, Germany, pp. 441–448. ACM Press, New York (2005)
Kok, S., Domingos, P.: Statistical predicate invention. In: Proceedings of the Twenty-Fourth International Conference on Machine Learning, Corvallis, OR, pp. 433–440. ACM Press, New York (2007)
Kok, S., Sumner, M., Richardson, M., Singla, P., Poon, H., Lowd, D., Domingos, P.: The Alchemy system for statistical relational AI. Technical report, Department of Computer Science and Engineering, University of Washington, Seattle, WA (2007), http://alchemy.cs.washington.edu
Kushmerick, N.: Wrapper induction: Efficiency and expressiveness. Artificial Intelligence 118(1-2), 15–68 (2000)
Lavrač, N., Džeroski, S.: Inductive Logic Programming: Techniques and Applications. Ellis Horwood, Chichester (1994)
Lawrence, S., Bollacker, K., Giles, C.L.: Autonomous citation matching. In: Proceedings of the Third International Conference on Autonomous Agents, ACM Press, New York (1999)
Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Mathematical Programming 45(3), 503–528 (1989)
Lloyd, J.W.: Foundations of Logic Programming. Springer, Berlin, Germany (1987)
Lowd, D., Domingos, P.: Efficient weight learning for Markov logic networks. In: Proceedings of the Eleventh European Conference on Principles and Practice of Knowledge Discovery in Databases, Warsaw, Poland, pp. 200–211. Springer, Heidelberg (2007)
Lowd, D., Domingos, P.: Recursive random fields. In: Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, Hyderabad, India, AAAI Press, Menlo Park (2007)
Lunn, D.J., Thomas, A., Best, N., Spiegelhalter, D.: WinBUGS – a Bayesian modeling framework: Concepts, structure, and extensibility. Statistics and Computing 10, 325–337 (2000)
Matuszek, C., Witbrock, M.: Personal communication (2006)
McCallum, A.: Efficiently inducing features of conditional random fields. In: Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence, Acapulco, Mexico, Morgan Kaufmann, San Francisco (2003)
Mihalkova, L., Mooney, R.: Bottom-up learning of Markov logic network structure. In: Proceedings of the Twenty-Fourth International Conference on Machine Learning, Corvallis, OR, pp. 625–632. ACM Press, New York (2007)
Møller, M.: A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 6, 525–533 (1993)
Muggleton, S.: Stochastic logic programs. In: De Raedt, L. (ed.) Advances in Inductive Logic Programming, pp. 254–264. IOS Press, Amsterdam, Netherlands (1996)
Neville, J., Jensen, D.: Dependency networks for relational data. In: Proceedings of the Fourth IEEE International Conference on Data Mining, Brighton, UK, pp. 170–177. IEEE Computer Society Press, Los Alamitos (2004)
Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (2006)
Pasula, H., Marthi, B., Milch, B., Russell, S., Shpitser, I.: Identity uncertainty and citation matching. In: Advances in Neural Information Processing Systems 14, MIT Press, Cambridge (2002)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)
Pearlmutter, B.: Fast exact multiplication by the Hessian. Neural Computation 6(1), 147–160 (1994)
Poole, D.: First-order probabilistic inference. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, pp. 985–991. Morgan Kaufmann, San Francisco (2003)
Poon, H., Domingos, P.: Sound and efficient inference with probabilistic and deterministic dependencies. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, Boston, MA, pp. 458–463. AAAI Press, Menlo Park (2006)
Poon, H., Domingos, P.: Joint inference in information extraction. In: Proceedings of the Twenty-Second National Conference on Artificial Intelligence, Vancouver, Canada, pp. 913–918. AAAI Press, Menlo Park (2007)
Quinlan, J.R.: Learning logical definitions from relations. Machine Learning 5, 239–266 (1990)
Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77, 257–286 (1989)
Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62, 107–136 (2006)
Riedel, S., Klein, E.: Genic interaction extraction with semantic and syntactic chains. In: Proceedings of the Fourth Workshop on Learning Language in Logic, Bonn, Germany, pp. 69–74. IMLS (2005)
Robinson, J.A.: A machine-oriented logic based on the resolution principle. Journal of the ACM 12, 23–41 (1965)
Roth, D.: On the hardness of approximate reasoning. Artificial Intelligence 82, 273–302 (1996)
Sha, F., Pereira, F.: Shallow parsing with conditional random fields. In: Proceedings of the 2003 Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics, Association for Computational Linguistics (2003)
Shewchuck, J.: An introduction to the conjugate gradient method without the agonizing pain. Technical Report CMU-CS-94-125, School of Computer Science, Carnegie Mellon University (1994)
Singla, P., Domingos, P.: Discriminative training of Markov logic networks. In: aaai05, Pittsburgh, PA, pp. 868–873. AAAI Press, Menlo Park (2005)
Singla, P., Domingos, P.: Entity resolution with Markov logic. In: Proceedings of the Sixth IEEE International Conference on Data Mining, Hong Kong, pp. 572–582. IEEE Computer Society Press, Los Alamitos (2006)
Singla, P., Domingos, P.: Memory-efficient inference in relational domains. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, Boston, MA, AAAI Press, Menlo Park (2006)
Singla, P., Domingos, P.: Markov logic in infinite domains. In: Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence, Vancouver, Canada, pp. 368–375. AUAI Press (2007)
Srinivasan, A.: The Aleph manual. Technical report, Computing Laboratory, Oxford University (2000)
Wasserman, S., Faust, K.: Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge (1994)
Wei, W., Erenrich, J., Selman, B.: Towards efficient sampling: Exploiting random walk strategies. In: aaai04, San Jose, CA, AAAI Press, Menlo Park (2004)
Wellman, M., Breese, J.S., Goldman, R.P.: From knowledge bases to decision models. Knowledge Engineering Review 7 (1992)
Wellner, B., McCallum, A., Peng, F., Hay, M.: An integrated, conditional model of information extraction and coreference with application to citation matching. In: Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence, Banff, Canada, pp. 593–601. AUAI Press (2004)
Yedidia, J.S., Freeman, W.T., Weiss, Y.: Generalized belief propagation. In: Leen, T., Dietterich, T., Tresp, V. (eds.) Advances in Neural Information Processing Systems 13, pp. 689–695. MIT Press, Cambridge (2001)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Domingos, P., Kok, S., Lowd, D., Poon, H., Richardson, M., Singla, P. (2008). Markov Logic. In: De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S. (eds) Probabilistic Inductive Logic Programming. Lecture Notes in Computer Science(), vol 4911. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78652-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-78652-8_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-78651-1
Online ISBN: 978-3-540-78652-8
eBook Packages: Computer ScienceComputer Science (R0)