Hava T. Siegelmann
ABSTRACT
This paper deals with finite networks which consist of interconnections of synchronously evolving processors. Each processor updates its state by applying a “sigmoidal” scalar nonlinearity to a linear combination of the previous states of all units. We prove that one may simulate all Turing Machines by rational nets. In particular, one can do this in linear time, and there is a net made up of about 1,000 processors which computes a universal partial-recursive function. Products (high order nets) are not required, contrary to what had been stated in the literature. Furthermore, we assert a similar theorem about non-deterministic Turing Machines. Consequences for undecidability and complexity issues about nets are discussed too.
- 1.N. Alon, A.K. Dewdney, T.J. Oft, "Efficient simulation of finite automata by neural nets," J. A.C.M. 38 (1991): 495-514. Google Scholar
Digital Library
- 2.J. Berstel, C. Reutenauer, Rational Series and Their Languages, Springer-Verlag, Berlin, 1988. Google ScholarDigital Library
- 3.L, Blum, M. Shub, and S. Smale, "On a theory of computation and complexity over the real numbers: NP completeness, recursive functions, and universal machines," Bull. A.M.S. 21(1989): 1-46.Google ScholarCross Ref
- 4.S. Franklin, M. Garzon, "Neural computability," in Progress In Neural Networks, Vol 1)(O. M. Omidvat, ed.), Ablex, Norwood, NJ, (1990): 128-144.Google Scholar
- 5.M. Garzon, S. Franklin, "Neural computability II," in Proc. 3rd Int. Joint Conf. Neural Networks (1989): I, 631-637.Google Scholar
- 6.C.L. Giles, D. Chen, C.B. Miller, H.H. Chen, G.Z. Sun, Y.C. Lee, "Second-order recurrent neural networks for grammatical inference," Proceedings of the International Joint Conference on Neural Networks, Seattle, Washington, IEEE Publication, col. 2 (1991): 273-278.Google Scholar
- 7.R. I-Iartley, H. Szu, "A comparison of the computational power of neural network models," in Proc. IEEE Conf. Neural Networks (1987): III. 17-22.Google Scholar
- 8.J.E. Hopcroft, and J.D. Ullman, Introduction to Automata Theory, Languages, and Computation, Addison-Wesley, 1979. Google ScholarDigital Library
- 9.S.C. Kleene, "Representation of events in nerve nets and finite automata," in Shannon, C.E., and J. McCarthy, eds., Automata Studies, Princeton Univ. Press 1956: 3-41.Google Scholar
- 10.W. Maass, G. Schnitger, E.D. Sontag, "On the computational power of sigmoid versus boolean threshold circuits," Proc. of the 3~nd Annual Syrup. on Foundations of Computer Science (1991): 767-776. Google ScholarDigital Library
- 11.C.M. Marcus, tLM. Westervelt, "Dynamics of iterated-map neural networks," Phys. Rev. Ser. A 40(1989): 3355-3364.Google ScholarCross Ref
- 12.W.S. McCulloch, W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bull. Math. Biophys. 5(1943): 115-133.Google ScholarCross Ref
- 13.M.L. Minsky, Computation: Finite and Infinite Machines, Prentice Hall, Engelwood Cliffs, 1967. Google ScholarDigital Library
- 14.J.B. Pollack, On Connectionist Models of Natural Language Processing, Ph.D. Dissertation, Computer Science Dept, Univ. of Illinois, Urbana, 1987.Google Scholar
- 15.J.H. Reif, J.D. Tygar, A. Yoshida "The computability and complexity of optical beam tracing," Proe. of the 31st Annual Syrup. on Foundations of Computer Science (1990): 106-114.Google Scholar
- 16.R. Schwarzschild, E.D. Sontag, "Algebraic theory of sign-linear systems," in Proceedings of the Automatic Control Conference, Boston, MA, June (1991): 799-804.Google Scholar
- 17.C.E. S annon, ~ universal turing machine with two internal states," in Shannon, C.E., and J. Mc- Carthy, eds., Automata Studies, Princeton Univ. Press 1956: 157-165.Google Scholar
- 18.H.T. Siegelmann, E.D. Sontag, "Analog computation, neural networks, and circuits," submitted.Google Scholar
- 19.E.D. Sontag, Mathematical Control Theory: Deterministic Finite Dimensional Systems, Springer, New York, 1990. Google ScholarDigital Library
- 20.E.D. Sontag, "Feedforward nets for interpolation and classification," J. Comp. $yst. Set., to appear. Google ScholarDigital Library
- 21.G.Z. Sun, H.H. Chen, Y.C. Lee, and C.L. Giles, "Turing equivalence of neural networks with second order connection weights," in Int. Jt. Conf. Neural Nets, Seattle, 1991:II,357-.Google Scholar
Index Terms
- On the computational power of neural nets
Recommendations
On the Computational Power of Neural Nets
This paper deals with finite size networks which consist of interconnections of synchronously evolving processors. Each processor updates its state by applying a "sigmoidal" function to a linear combination of the previous states of all units. We prove ...
The expressive power of time Petri nets
We investigate expressiveness questions for time Petri nets (TPNs) and some of their most useful extensions. We first introduce generalised time Petri nets (GTPNs) as an abstract model that encompasses variants of TPNs such as self modifications and ...
Non Equivalence between Time Petri Nets and Time Stream Petri Nets
PNPM '99: Proceedings of the The 8th International Workshop on Petri Nets and Performance ModelsIt had been shown that Merlin's Time Petri Nets are a special case of Time Stream Petri Nets. In this paper, we show that it does not exist a time equivalence between both models. We extend first this result to the nonequivalence of Timed Automata and ...
Comments