Skip to main content
SpringerLink
Log in
Book cover

International Conference on Unconventional Computation and Natural Computation

UCNC 2015: Unconventional Computation and Natural Computation pp 144–156Cite as

Expressive Power of Non-deterministic Evolving Recurrent Neural Networks in Terms of Their Attractor Dynamics

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9252))

Abstract

We introduce a model of nondeterministic hybrid recurrent neural networks – made up of Boolean input and output cells as well as internal sigmoid neurons, and equipped with the possibility to have their synaptic weights evolve over time, in a nondeterministic manner. When subjected to some infinite input stream and some specific synaptic evolution, the networks necessarily exhibit some attractor dynamics in their Boolean output cells, and accordingly, recognize some specific neural \(\omega \) -languages. The expressive power of these networks is measured via the topological complexity of their underlying neural \(\omega \)-languages. In this context, we prove that the two models of rational-weighted and real-weighted nondeterministic hybrid neural networks are computationally equivalent, and recognize precisely the set of all analytic neural \(\omega \)-languages. They are therefore strictly more expressive than the nondeterministic Büchi and Muller Turing machines.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    By contrast, a deterministic Ev-RNN has only one possible evolution for its synaptic weights, and hence corresponds to a nondeterministic Ev-RNN where the set E is reduced to a singleton.

  2. 2.

    The results of the paper hold equally true even with E taken as \(\mathbf {\Pi ^0_2}\).

  3. 3.

    We recall that the preimage by a Baire class 1 function of a set in \(\mathbf {\Sigma ^0_n}\) (resp. \(\mathbf {\Pi ^0_n}\)) is in \(\mathbf {\Sigma }^\mathbf{0}_{\mathbf{n+1}}\) (resp. \(\mathbf {\Pi }^\mathbf{0}_{\mathbf{n+1}}\)).

References

  1. Cabessa, J.: Interactive evolving recurrent neural networks are super-Turing. In: Filipe, J., Fred, A.L.N. (eds.) Proceedings of ICAART, pp. 328–333. SciTePress (2012)

    Google Scholar 

  2. Cabessa, J., Siegelmann, H.T.: Evolving recurrent neural networks are super-Turing. In: Proceedings of IJCNN 2011, pp. 3200–3206. IEEE (2011)

    Google Scholar 

  3. Cabessa, J., Siegelmann, H.T.: The computational power of interactive recurrent neural networks. Neural Comput. 24(4), 996–1019 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  4. Cabessa, J., Siegelmann, H.T.: The super-Turing computational power of plastic recurrent neural networks. Int. J. Neural. Syst. 24(8), 1450029 (2014)

    Article  Google Scholar 

  5. Cabessa, J., Villa, A.E.P.: Computational capabilities of recurrent neural networks based on their attractor dynamics. In: Proceedings of IJCNN 2015. IEEE (to appear, 2015) (accepted)

    Google Scholar 

  6. Cabessa, J., Villa, A.E.P.: A hierarchical classification of first-order recurrent neural networks. In: Dediu, A.-H., Fernau, H., Martín-Vide, C. (eds.) LATA 2010. LNCS, vol. 6031, pp. 142–153. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  7. Cabessa, J., Villa, A.E.P.: The expressive power of analog recurrent neural networks on infinite input streams. Theor. Comput. Sci. 436, 23–34 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Cabessa, J., Villa, A.E.P.: Recurrent neural networks - a natural model of computation beyond the Turing limits. In: Rosa, A.C., et al., (ed.) Proceedings of IJCCI 2012, pp. 594–599. SciTePress (2012)

    Google Scholar 

  9. Cabessa, J., Villa, A.E.P.: The super-turing computational power of interactive evolving recurrent neural networks. In: Mladenov, V., Koprinkova-Hristova, P., Palm, G., Villa, A.E.P., Appollini, B., Kasabov, N. (eds.) ICANN 2013. LNCS, vol. 8131, pp. 58–65. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  10. Cabessa, J., Villa, A.E.P.: An attractor-based complexity measurement for boolean recurrent neural networks. PloS ONE 9(4), e94204+ (2014)

    Article  Google Scholar 

  11. Cabessa, J., Villa, A.E.P.: Interactive evolving recurrent neural networks are super-turing universal. In: Wermter, S., Weber, C., Duch, W., Honkela, T., Koprinkova-Hristova, P., Magg, S., Palm, G., Villa, A.E.P. (eds.) ICANN 2014. LNCS, vol. 8681, pp. 57–64. Springer, Heidelberg (2014)

    Google Scholar 

  12. Kechris, A.S.: Classical Descriptive Set Theory. Graduate Texts in Mathematics, vol. 156. Springer-Verlag, New York (1995)

    MATH  Google Scholar 

  13. Kleene, S.C.: Representation of events in nerve nets and finite automata. In: Shannon, C., McCarthy, J. (eds.) Automata Studies, pp. 3–41. Princeton University Press, Princeton, NJ (1956)

    Google Scholar 

  14. McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115–133 (1943)

    Article  MathSciNet  MATH  Google Scholar 

  15. Minsky, M.L.: Computation: Finite and Infinite Machines. Prentice-Hall Inc., Englewood Cliffs (1967)

    MATH  Google Scholar 

  16. Neumann, J.V.: The Computer and the Brain. Yale University Press, New Haven (1958)

    MATH  Google Scholar 

  17. Perrin, D., Pin, J.-E.: Infinite Words - Automata, Semigroups, Logic and Games. Pure and Applied Mathematics, vol. 141. Elsevier, San Diego (2004)

    MATH  Google Scholar 

  18. Siegelmann, H.T.: Neural Networks and Analog Computation: Beyond the Turing Limit. Birkhauser Boston Inc., Cambridge (1999)

    Book  MATH  Google Scholar 

  19. Siegelmann, H.T., Sontag, E.D.: Analog computation via neural networks. Theor. Comput. Sci. 131(2), 331–360 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  20. Siegelmann, H.T., Sontag, E.D.: On the computational power of neural nets. J. Comput. Syst. Sci. 50(1), 132–150 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  21. Staiger, L.: \(\omega \)-languages. In: Rozenberg, G., Salomaa, A. (eds.) Handbook of Formal Languages: Beyond Words, vol. 3, pp. 339–387. Springer-Verlag, New York (1997)

    Chapter  Google Scholar 

  22. Thomas, W.: Automata on infinite objects. In: van Leeuwen, J. (ed.) Handbook of Theoretical Computer Science: Formal Models and Semantics, vol. B, pp. 133–192. Elsevier and MIT Press, Amsterdam (1990)

    Google Scholar 

  23. Turing, A.M.: Intelligent machinery. Technical report, National Physical Laboratory, Teddington, UK (1948)

    Google Scholar 

  24. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., Aertsen, A.: Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature 373(6514), 515–518 (1995)

    Article  Google Scholar 

  25. Villa, A.E.P., Tetko, I.V., Hyland, B., Najem, A.: Spatiotemporal activity patterns of rat cortical neurons predict responses in a conditioned task. Proc. Natl. Acad. Sci. U.S.A. 96(3), 1106–1111 (1999)

    Article  Google Scholar 

  26. Wagner, K.: On \(\omega \)-regular sets. Inf. Control 43(2), 123–177 (1979)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory of Mathematical Economics, University of Paris 2 – Panthéon-Assas, 4 Rue Blaise Desgoffe, 75006, Paris, France

    Jérémie Cabessa

  2. Department of Information Systems, Faculty of Business and Economics, University of Lausanne, CH-1015, Lausanne, Switzerland

    Jacques Duparc

Authors
  1. Jérémie Cabessa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacques Duparc
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jérémie Cabessa .

Editor information

Editors and Affiliations

  1. University of Auckland, Auckland, New Zealand

    Cristian S. Calude

  2. University of Auckland, Auckland, New Zealand

    Michael J. Dinneen

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Cabessa, J., Duparc, J. (2015). Expressive Power of Non-deterministic Evolving Recurrent Neural Networks in Terms of Their Attractor Dynamics. In: Calude, C., Dinneen, M. (eds) Unconventional Computation and Natural Computation. UCNC 2015. Lecture Notes in Computer Science(), vol 9252. Springer, Cham. https://doi.org/10.1007/978-3-319-21819-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-21819-9_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-21818-2

  • Online ISBN: 978-3-319-21819-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation