Abstract
Learning from “structured examples” is necessary in a number of settings, including inductive logic programming. Here we analyze a simple learning problem in which examples have non-trivial structure: specifically, a learning problem in which concepts are strings over a fixed alphabet, examples are deterministic finite automata (DFAs), and a string represents the set of all DFAs that accept it. We show that solving this “dual” DFA learning problem is hard, under cryptographic assumptions. This result implies the hardness of several other more natural learning problems, including learning the description logic CLASSSIC from subconcepts, and learning arity-two “determinate” function-free Prolog clauses from ground clauses. The result also implies the hardness of two formal problems related to the area of “programming by demonstration”: learning straightline programs over a fixed operator set from input-output pairs, and learning straightline programs from input-output pairs and “partial traces”.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Angluin, D. (1987). Learning regular sets from queries and counterexamples. Information and Control, 75:87–106.
Angluin, D. (1988). Learning with hints (extended abstract). In Proceedings of the 1988 Workshop on Computational Learning Theory, Boston, Massachusetts.
Beck, H., Gala, H., & Navathe, S. (1989). Classification as a query processing technique in the CANDIDE semantic model. In Proceedings of the Data Engineering Conference, pages 572–581, Los Angeles, California.
Biermann, A. (1978). The inference of regular lisp programs from examples. IEEE Transactions on Systems, Man and Cybernetics, 8(8).
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. (1989). Classifying learnable concepts with the Vapnik-Chervonenkis dimension. Journal of the Association for Computing Machinery, 36(4):929–965.
Boppana, R. & Sipser, M. (1990). The complexity of finite functions. In Handbook of Theoretical Computer Science, pages 758–804. Elsevier.
Borgida, A. (1992). Description logics are not just for the flightless-birds: a new look at the utility and foundations of description logics. Technical Report DCS-TR-295, Rutgers University Department of Computer Science.
Borgida, A. & Patel-Schneider, P. F. (1994). Asemantics and complete algorithm for subsumption in the CLASSIC description logic. Journal of Artificial Intelligence Research, 1:277–308.
Cohen, W. W. (1993). Cryptographic limitations on learning one-clause logic programs. In Proceedings of the Tenth National Conference on Artificial Intelligence, Washington, D.C.
Cohen, W.W. (1995). Pac-learning recursive logic programs: negative results. Journal of AI Research, 2:541–573.
Cohen, W.W. & Hirsh, H. (1992). Learnability of description logics. In Proceedings of the Fifth AnnualWorkshop on Computational Learning Theory, Pittsburgh, Pennsylvania. ACM Press.
Cohen, W. W. & Hirsh, H. (1994a). The learnability of description logics with equality constraints. Machine Learning, 17(2/3).
Cohen, W. W. & Hirsh, H. (1994b). Learning the CLASSIC description logic: Theoretical and experimental results. In Principles of Knowledge Representation and Reasoning: Proceedings of the Fourth International Conference (KR94). Morgan Kaufmann.
Cohen, W. W. & Hirsh, H. (1995). Corrigendum for “learnability of description logics”. In Proceedings of the Eighth Annual Workshop on Computational Learning Theory, Santa Cruz, California. ACM Press.
Cohen, W. W. & Page, C. D. (1995). Polynomial learnability and inductive logic programming: Methods and results. New Generation Computing, 13(3).
Cypher, A., editor (1993). Watch what I do: Programming by demonstration. The MIT Press, Cambridge, Massachusetts.
De Raedt, L., editor (1995). Advances in Inductive Logic Programming. IOS Press.
Devanbu, P., Brachman, R. J., Selfridge, P., & Ballard, B. (1991). LaSSIE: A knowledge-based software information system. Communications of the ACM, 35(5).
Dietterich, T. G., Lathrop, R. H., & Lozano-Perez, T. (1997). Solving the multiple-instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1–2):31–71.
Dietterich, T. G., London, B., Clarkson, K., & Dromey, G. (1982). Learning and inductive inference. In Cohen, P. and Feigenbaum, E. A., editors, The Handbook of Artificial Intelligence, Volume III.William Kaufmann, Los Altos, CA.
Džeroski, S., Muggleton, S., & Russell, S. (1992). Pac-learnability of determinate logic programs. In Proceedings of the 1992 Workshop on Computational Learning Theory, Pittsburgh, Pennsylvania.
Ergün, F., Kumar, S. R., & Rubinfeld, R. (1995). On learning bounded-width branching programs. In Proceedings of the Eighth Annual ACM Conference on Computational Learning Theory, Santa Cruz, CA. ACM Press.
Frazier, M. & Pitt, L. (1996). Classic learning. To appear in Machine Learning.
Freund, Y., Kearns, M., Ron, D., Rubinfeld, R., Schapire, R., & Sellie, L. (1993). Efficient learning of typical finite automata from random walks. In Proceedings of the 25th ACM Symposium on the Theory of Computing, pages 315–324. ACM Press.
Hopcroft, J. E. & Ullman, J. D. (1979). Introduction to Automata Theory, Languages, and Computation. Addison-Wesley.
Horváth, T., Sloan, R., & Tuŕan, G. (1997).Learning logic programs by using the product homomorphism method. In Proceedings of the Tenth Annual ACM Conference on Computational Learning Theory, Vanderbilt, Tennessee. ACM Press.
Kearns, M. & Valiant, L. (1989). Cryptographic limitations on learning Boolean formulae and finite automata. In 21th Annual Symposium on the Theory of Computing. ACM Press.
Kharitonov, M. (1992). Cryptographic lower bounds on the learnability of boolean functions on the uniform distribution. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, Pittsburgh, Pennsylvania. ACM Press.
Kietz, J.-U. (1993). Some computational lower bounds for the computational complexity of inductive logic programming. In Proceedings of the 1993 European Conference on Machine Learning, Vienna, Austria.
Lloyd, J. W. (1987). Foundations of Logic Programming: Second Edition. Springer-Verlag.
MacGregor, R. M. (1991). The evolving technology of classification-based knowledge representation systems. In Sowa, J., editor, Principles of semantic networks: explorations in the representation of knowledge. Morgan Kaufmann.
Mays, E., Apte, C., Griesmer, J., & Kastner, J. (1987). Organizing knowledge in a complex financial domain. IEEE Expert, pages 61–70.
Muggleton, S. & De Raedt, L. (1994). Inductive logic programming: Theory and methods. Journal of Logic Programming, 19/20(7):629–679.
Muggleton, S. & Feng, C. (1992). Efficient induction of logic programs. In Inductive Logic Programming. Academic Press.
Page, C. D. & Frisch, A. M. (1992). Generalization and learnability: A study of constrained atoms. In Inductive Logic Programming. Academic Press.
Pitt, L. & Valiant, L. (1988). Computational limitations on learning from examples. Journal of the ACM, 35(4):965–984.
Pitt, L. & Warmuth, M. (1990). Prediction-preserving reducibility. Journal of Computer and System Sciences, 41:430–467.
Quinlan, J. R. (1990). Learning logical definitions from relations. Machine Learning, 5(3).
Summers, P. D. (1977). Amethodology for LISP program construction from examples. Journal of the Association for Computing Machinery, 24(1):161–175.
Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27(11).
Woods, W. A. & Schmolze, J. G. (1992). The KL-ONE family. Computers And Mathematics With Applications, 23(2-5).
Wright, J., Weixelbaum, E., Vesonder, G., Brown, K., Palmer, S., Berman, J., & Moore, H. (1993). A knowledgebased configurator that supports sales engineering and manufacturing and AT&T network systems. AI Magazine, 14:69–80.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Cohen, W.W. Hardness Results for Learning First-Order Representations and Programming by Demonstration. Machine Learning 30, 57–87 (1998). https://doi.org/10.1023/A:1007406511732
Issue Date:
DOI: https://doi.org/10.1023/A:1007406511732