Skip to main content
SpringerLink
Log in

Online Training for High-Performance Analogue Readout Layers in Photonic Reservoir Computers

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Reservoir computing is a bio-inspired computing paradigm for processing time-dependent signals. The performance of its hardware implementation is comparable to state-of-the-art digital algorithms on a series of benchmark tasks. The major bottleneck of its implementations is the readout layer, based on slow offline post-processing. Few analogue solutions have been proposed, but all suffered from noticeable decrease in performance due to added complexity of the setup. Here, we propose the use of online training to solve these issues. We study the applicability of this method using numerical simulations of an experimentally feasible reservoir computer with an analogue readout layer. We also consider a nonlinear output layer, which would be very difficult to train with traditional methods. We show numerically that online learning allows to circumvent the added complexity of the analogue layer and obtain the same level of performance as with a digital layer. This work paves the way to high-performance fully analogue reservoir computers through the use of online training of the output layers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Jaeger H, Haas H. Science 2004;304:78.

    Article  CAS  PubMed  Google Scholar 

  2. Maass W., Natschläger T., Markram H. Neural comput 2002;14:2531.

    Article  PubMed  Google Scholar 

  3. Lukoševičius M., Jaeger H. Comp Sci Rev 2009;3:127.

    Article  Google Scholar 

  4. Triefenbach F., Jalalvand A., Schrauwen B., Martens J. P. Adv Neural Inf Process Syst 2010;23: 2307.

    Google Scholar 

  5. Meftah B., Lézoray O., Benyettou A. Cogn Comput 2016;8(2):237.

    Article  Google Scholar 

  6. Malik Z. K., Hussain A., Wu J. Cogn Comput 2014;6(3):595.

    Article  Google Scholar 

  7. Scardapane S., Uncini A. Cogn Comput 2016:1–11.

  8. 2006. The 2006/07 forecasting competition for neural networks computational intelligence. http://www.neural-forecasting-competition.com/NN3/.

  9. Arsenault H. Optical processing and computing: Elsevier; 2012.

  10. Appeltant L., Soriano M. C., Van der Sande G., Danckaert J., Massar S., Dambre J., Schrauwen B., Mirasso C. R., Fischer I. Nat Commun 2011;2:468.

  11. Paquot Y., Duport F., Smerieri A., Dambre J., Schrauwen B., Haelterman M., Massar S. Sci Rep 2012;2:287.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Larger L., Soriano M., Brunner D., Appeltant L., Gutiérrez J. M., Pesquera L., Mirasso C. R., Fischer I. Opt Express 2012;20:3241.

    Article  CAS  PubMed  Google Scholar 

  13. Martinenghi R., Rybalko S., Jacquot M., Chembo Y. K., Larger L. Phys Rev Let 2012;108: 244101.

    Article  Google Scholar 

  14. Duport F., Schneider B., Smerieri A., Haelterman M., Massar S. Opt Express 2012;20:22783.

    Article  PubMed  Google Scholar 

  15. Brunner D., Soriano M. C., Mirasso C. R., Fischer I. Nat Commun 2012;4:1364.

    Article  Google Scholar 

  16. Vinckier Q., Duport F., Smerieri A., Vandoorne K., Bienstman P., Haelterman M., Massar S. Optica 2015;2(5):438.

    Article  Google Scholar 

  17. Vandoorne K., Mechet P., Van Vaerenbergh T., Fiers M., Morthier G., Verstraeten D., Schrauwen B., Dambre J., Bienstman P. Nat Commun 2014;5:3541.

  18. Woods D., Naughton T. J. Nat Phys 2012;8(4):257.

    Article  CAS  Google Scholar 

  19. Smerieri A., Duport F., Paquot Y., Schrauwen B., Haelterman M., Massar S. 2012:944–952.

  20. Duport F., Smerieri A., Akrout A., Haelterman M., Massar S. Sci Rep 2016;6:22381.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Vinckier Q., Bouwens A., Haelterman M., Massar S. Opt Soc Amer 2016:SF1F.1.

  22. Bottou L. In: Online learning and neural networks: Cambridge University Press; 1998. http://leon.bottou.org/papers/bottou-98x.

  23. Shalev-Shwartz S. Found Trends Mach Learn 2012;4(2):107.

    Article  Google Scholar 

  24. Antonik P., Duport F., Hermans M., Smerieri A., Haelterman M., Massar S. IEEE Trans Neural Netw Learn Syst 2016;PP(99):1.

    Article  Google Scholar 

  25. Rodan A., Tino P. IEEE Trans Neural Netw 2011;22:131.

    Article  PubMed  Google Scholar 

  26. Arfken G. B. Mathematical methods for physicists. Orlando, FL: Academic Press; 1985.

    Google Scholar 

  27. Bishop CM. Pattern recognition and machine learning: Springer; 2006.

  28. Press W. H., Flannery B. P., Teukolsky S. A., Vetterling W.T. 1986. Numerical recipes: the art of scientific computing.

  29. Bottou L. In: Advanced lectures on machine learning: Springer Verlag; 2004, pp. 146–168.

  30. Mathews V. J., Lee J. In: SPIE’s 1994 International Symposium on Optics, Imaging, and Instrumentation: International Society for Optics and Photonics; 1994, pp. 317–327.

  31. Horowitz P., Hill W. The art of electronics: Cambridge University Press; 1980.

  32. Tikhonov A. N., Goncharsky A., Stepanov V., Yagola A. G., Vol. 328. Numerical methods for the solution of ill-posed problems. Netherlands: Springer; 1995.

  33. Soriano M. C., Ortín S., Brunner D., Larger L., Mirasso C. R., Fischer I., Pesquera L. Opt Express 2013;21(1):12.

    Article  CAS  PubMed  Google Scholar 

  34. Soriano M. C., Ortín S., Keuninckx L., Appeltant L., Danckaert J., Pesquera L., Van der Sande G. IEEE Trans Neural Netw Learn Syst 2015;26(2):388.

  35. Antonik P., Hermans M., Duport F., Haelterman M., Massar S. In: SPIE’s 2016 Laser Technology and Industrial Laser Conference 2016;9732:97320B.

  36. Bauduin M., Vinckier Q., Massar S., Horlin F. In: 2016 IEEE 17th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC): IEEE; 2016, pp. 1–5.

  37. Antonik P., Hermans M., Haelterman M., Massar S., Vol. 9948. APNNS’s 23th International Conference on Neural Information Processing: LNCS; 2016, pp. 318–325.

Download references

Author information

Authors and Affiliations

  1. Laboratoire d’Information Quantique, Université libre de Bruxelles, Avenue F. D. Roosevelt 50, CP 224, Brussels, Belgium

    Piotr Antonik & Serge Massar

  2. Service OPERA-Photonique, Université libre de Bruxelles, Avenue F. D. Roosevelt 50, CP 194/5, Brussels, Belgium

    Marc Haelterman

Authors
  1. Piotr Antonik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marc Haelterman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Serge Massar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Piotr Antonik.

Ethics declarations

Funding

This study was funded by the Interuniversity Attraction Poles program of the Belgian Science Policy Office (grant IAP P7-35 “photonics@be”), by the Fonds de la Recherche Scientifique FRS-FNRS and by the Action de Recherche Concertée of the Académie Universitaire Wallonie-Bruxelles (grant AUWB-2012-12/17-ULB9).

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Antonik, P., Haelterman, M. & Massar, S. Online Training for High-Performance Analogue Readout Layers in Photonic Reservoir Computers. Cogn Comput 9, 297–306 (2017). https://doi.org/10.1007/s12559-017-9459-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-017-9459-3

Keywords

Search

Navigation