Abstract
This chapter presents an experiment that was not originally planned as part of my thesis. The project was set up when Michiel Hermans joined our team in 2015 with an idea of implementing the backpropagation training algorithm (more on that in Sect. 3.2) in hardware, using our opto-electronic reservoir computer (see Sect. 1.2.4) with one slight modification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hermans, Michiel, Piotr Antonik, Marc Haelterman, and Serge Massar. 2016. Embodiment of learning in electro-optical signal processors. Physical Review Letters 117: 128301.
Rumelhart, David E., James L. McClelland, and PDP Research Group. 1986. Parallel distributed processing: explorations in the microstructure of cognition. In Learning internal representations by error propagation, vol. 1, 318–362. Cambridge, MA, USA: MIT Press.
Werbos, Paul J. 1988. Generalization of backpropagation with application to a recurrent gas market model. Neural Networks 1 (4): 339–356.
LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521 (7553): 436–444.
Hermans, Michiel, Joni Dambre, and Peter Bienstman. 2015. Optoelectronic systems trained with backpropagation through time. IEEE Transactions on Neural Networks and Learning Systems 26 (7): 1545–1550.
Hermans, Michiel, Miguel Soriano, Joni Dambre, Peter Bienstman, and Ingo Fischer. 2015. Photonic delay systems as machine learning implementations. JMLR 16: 2081–2097.
Hermans, Michiel, Michaël Burm, Thomas Van Vaerenbergh, Joni Dambre, and Peter Bienstman. 2015. Trainable hardware for dynamical computing using error backpropagation through physical media. Nature Communications 6: 6729.
Ikeda, Kensuke, and Kenji Matsumoto. 1987. High-dimensional chaotic behavior in systems with time-delayed feedback. Physica D: Non-Linear Phenomena 29 (1): 223–235.
Nesterov, Yurii. 1983. A method of solving a convex programming problem with convergence rate O (1/k2). Soviet Mathematics Doklady 27 (2): 372–376.
Sutskever, Ilya, James Martens, George Dahl, and Geoffrey Hinton. 2013. On the importance of initialization and momentum in deep learning. In Proceedings of the 30th international conference on machine learning (ICML-13), 1139–1147.
Paquot Yvan, François Duport, Anteo Smerieri, Joni Dambre, Benjamin Schrauwen, Marc Haelterman, and Serge Massar. 2012. Optoelectronic reservoir computing. Scientific Reports 2: 287.
Zimmermann, Hubert. 1980. OSI reference model–the ISO model of architecture for open systems interconnection. IEEE Transactions on Communications 28 (4): 425–432.
Garofolo, John, S., and NIST. 1993. TIMIT Acoustic-phonetic continuous speech corpus. Linguistic Data Consortium.
Triefenbach, Fabian, Azarakhsh Jalalvand, Benjamin Schrauwen, and Jean-Pierre Martens. 2010. Phoneme recognition with large hierarchical reservoirs. Advances in Neural Information Processing Systems 23: 2307–2315.
Triefenbach, Fabian, Kris Demuynck, and Jean-Pierre Martens. 2014. Large vocabulary continuous speech recognition with reservoir-based acoustic models. IEEE Signal Processing Letters 21 (3): 311–315.
Vinckier, Quentin, François Duport, Anteo Smerieri, Kristof Vandoorne, Peter Bienstman, Marc Haelterman, and Serge Massar. 2015. High-performance photonic reservoir computer based on a coherently driven passive cavity. Optica 2 (5): 438–446.
Hermans, Michiel, and Benjamin Schrauwen. 2012. Infinite sparse threshold unit networks. In Proceedings of the international conference on artificial neural networks, 612–619.
Bishop, Christopher M. 2006. Pattern recognition and machine learning. Springer.
Singh, Satnam. 2011. Using the Virtex-6 Embedded Tri-Mode Ethernet MAC Wrapper v1.4 with the ML605 Board. http://blogs.msdn.microsoft.com/satnam_singh/2011/02/11/using-the-virtex-6-embedded-tri-mode-ethernet-mac-wrapper-v1-4-with-the-ml605-board/.
Brunner, Daniel, Miguel C Soriano, Claudio R Mirasso, and Ingo Fischer. 2013. Parallel photonic information processing at gigabyte per second data rates using transient states. Nature Communications 4: 1364.
Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. 2011. Deep sparse rectifier neural networks. In 14th international conference on artificial intelligence and statistics, vol. 15 (106), p. 275.
Shi, Bingxue, and Chun Lu. 2002. Generator of neuron transfer function and its derivative. US Patent 6429699.
Vandoorne, Kristof, Pauline Mechet, Thomas Van Vaerenbergh, Martin Fiers, Geert Morthier, David Verstraeten, Benjamin Schrauwen, Joni Dambre, and Peter Bienstman. 2014. Experimental demonstration of reservoir computing on a silicon photonics chip. Nature Communications 5: 3541.
Smerieri, Anteo, François Duport, Yvan Paquot, Benjamin Schrauwen, Marc Haelterman, and Serge Massar. 2012. Analog readout for optical reservoir computers. In Advances in neural information processing systems, 944–952.
Duport, François, Anteo Smerieri, Akram Akrout, Marc Haelterman, and Serge Massar. 2016. Fully analogue photonic reservoir computer. Scientific Reports 6: 22381.
Vinckier, Quentin, Arno Bouwens, Marc Haelterman, and Serge Massar. 2016. Autonomous all-photonic processor based on reservoir computing paradigm. In Conference on lasers and electro-optics. Optical society of America. SF1F.1.
Waldrop, M.Mitchell. 2016. The chips are down for Moore’s law. Nature 530: 144–147.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Antonik, P. (2018). Backpropagation with Photonics. In: Application of FPGA to Real‐Time Machine Learning. Springer Theses. Springer, Cham. https://doi.org/10.1007/978-3-319-91053-6_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-91053-6_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91052-9
Online ISBN: 978-3-319-91053-6
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)