Abstract
We consider the problem of learning from a fallible expert that answers all queries about a concept, but often gives incorrect answers. The expert can also be thought of as a truth table describing the concept which has been partially corrupted. In order to learn the underlying concept with arbitrarily high precision, we would like to use its structure in order to correct most of the incorrect answers. We assume that the expert's errors are uniformly and independently distributed, occur with any fixed probability strictly smaller than 1/2, and are persistent. In particular, we present a polynomial time algorithm using membership queries for correcting and learning fallible Deterministic Finite Automata under the uniform distribution.
Article PDF
Similar content being viewed by others
References
Alon, N., & Spencer, J. H. (1991). The Probabilistic Method. Wiley Interscience.
Angluin, D. (1981). A note on the number of queries needed to identify regular languages. Information and Control, 51, 76–87.
Angluin, D. (1987). Learning regular sets from queries and counterexamples. Information and Computation, 75, 87–106.
Angluin, D., Frazier, M., & Pitt, L. (1992). Learning conjunctions of Horn Clauses. Machine Learning, 9, 147–164.
Angluin, D., Hellerstein, L., & Karpinski, M. (1993). Learning read-once formulas with queries. Journal of the Association for Computing Machinery, 48, 185–210.
Angluin, D., & Laird, P. (1988). Learning from noisy examples. Machine Learning, 2, 343–370.
Angluin, D., & Slonim, D. (1994). Randomly fallible teachers: Learning monotone DNF with an incomplete membership oracle. Machine Learning, 14, 7–26.
Blum, L., Luby, M., & Rubinfeld, R. (1993). Self-Testing/Correcting with applications to numerical problems. Journal of Computer and System Sciences, 47, 549–595.
Blumer, E., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1987). Occam's Razor. Information Processing Letters, 27, 377–380.
Chernoff, H. (1952). A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Annals of the Mathematical Statistics, 23, 493–507.
Dean, T., Angluin, D., Basye, K., Engelson, S., Kaelbling, L., Kokkevis, E., & Maron, O. (1992). Inferring finite automata with stochastic output functions and an application to map learning. Proceedings of the 10th National Conference on Artificial Intelligence, (pp. 208–214). San Jose, CA: AAAI Press.
Fischer, P., & Simon, H. (1992). On learning ring-sum-expansions. SIAM Journal on Computing, 21, 181–192.
Freund, Y., Kearns, M. J., Ron, D., Rubinfeld, R., Schapire, R. E., & Sellie, L. (1993). Efficient learning of typical finite automata from random walks. Proceedings of the 25th Annual ACM Symposium on Theory of Computing, (pp. 315–324). San-Diego, CA: ACM Press.
Goldman, S. A., Kearns, M. J., & Schapire, R. E. (1993). Exact identification of read-once formulas using fixed points of amplification functions. SIAM Journal on Computing, 22, 705–726.
Goldman, S. A., & Mathias, H. D. (1992). Learning k-term DNF formulas with an incomplete membership oracle. Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, (pp. 85–92). Pittsburgh, PA: Morgan Kaufmann.
Helmbold, D., Sloan, R., & Warmuth, M. K. (1992). Learning integer lattices. SIAM Journal on Computing, 21, 240–266.
Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58, 13–30.
Hopcroft, J. E. (1971). An n log n algorithm for minimizing the states in a finite automaton. The Theory of Machines and Computations, 189–196. Academic Press, New-York.
Huffman, D. A. (1954). The synthesis of sequential switching circuits. J. Franklin Institute, 257, 161–190, 275–303.
Kearns, M. J. (1990). The Computational Complexity of Machine Learning. MIT Press.
Kearns, M. J. (1993). Efficient noise-tolerant learning from statistical queries. Proceedings of the 25th Annual ACM Symposium on Theory of Computing, (pp. 392–401). San-Diego: ACM Press.
Kearns, M. J., & Li, M. (1993). Learning in the presence of malicious errors. Siam Journal on Computing, 22, 807–837.
Kearns, M. J., & Schapire, R. E. (1990) Efficient distribution-free learning of probabilistic concepts. Proceedings of the 31st Annual Symposium on Foundations of Computer Science, (pp. 382–391). St. Louis, MO: IEEE Computer Society Press. To appear in JCSS.
Kearns, M. J., & Valiant, L. G. (1994). Cryptographic limitations on learning boolean formulae and finite automata. Journal of the Association for Computing Machinery, 41, 67–95.
Laird, P. (1988). Learning From Good Data and Bad. Kluwer Academic Publishers.
Lipton, R. (1991). New directions in testing. Distributed Computing and Cryptography, DIMACS Series in Discrete Math and Theoretical Computer Science, American Mathematical Society, 2, 191–202.
Moore, E. F. (1956). Gedanken experiments on sequential machines. Automata Studies, 129–153. Princeton Univ. Press, Princeton, N.J.
Pitt, L., & Warmuth, M. K. (1990) Prediction-preserving reducibility. Journal of Computer and System Sciences, 41, 430–467.
Rivest, R., & Schapire, R. E. (1987). Diversity-based inference of finite automata. Proceedings of the 28th IEEE Symposium on Foundations of Computer Science, (pp. 78–87). Los Angeles, CA: IEEE Computer Science Press. To appear in JACM.
Rivest, R., & Schapire, R. E. (1993). Inference of finite automata using homing sequences. Information and Computation, 103, 299–347.
Ron, D., & Rubinfeld, R. (1993). Learning fallible finite state automata. Proceedings of the 6th Annual ACM Workshop on Computational Learning Theory, (pp. 218–227). Santa-Cruz, CA: Morgan Kaufmann.
Sakakibara, Y. (1991). On learning from queries and counterexamples in the presence of noise. Information Processing Letters, 37, 279–284.
Sakakibara, Y., & Siromoney, R. (1992). A noise model on learning sets of strings. Proceedings of the 5th Annual Workshop on Computational Learning Theory (pp. 295–302). Pittsburgh, PA: Morgan Kaufmann.
Schapire, R. E. (1991). The Design and Analysis of Efficient Learning Algorithms. MIT Press.
Shackelford, G., & Volper, D. (1988). Learning k-DNF with noise in the attributes. Proceedings of the 1988 Workshop on Computational Learning Theory, (pp. 97–103). Cambridge, MA: Morgan Kaufmann.
Sloan, R. H. (1988). Types of noise in data for concept learning. Proceedings of the 1988 Workshop on Computational Learning Theory, (pp. 91–96). Cambridge MA: Morgan Kaufmann.
Sloan, R. H. (1989). Computational Learning Theory: New Models and Algorithms. Doctoral dissertation, Department of Computer Science, Massachusetts Institute of Technology, Cambridge, MA. Issued as MIT/LCS/TR-448.
Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27, 1134–1142.
Valiant, L. G. (1985) Learning disjunctions of conjunctions. Proceedings of the 9th International Joint Conference on Artificial Intelligence, (pp. 560–566). Los Angeles, CA: Morgan Kaufmann.
Rights and permissions
About this article
Cite this article
Ron, D., Rubinfeld, R. Learning Fallible Deterministic Finite Automata. Machine Learning 18, 149–185 (1995). https://doi.org/10.1023/A:1022899313248
Issue Date:
DOI: https://doi.org/10.1023/A:1022899313248