Efficient digital implementation of the sigmoid function for reprogrammable logic
Efficient digital implementation of the sigmoid function for reprogrammable logic
- Author(s): M.T. Tommiska
- DOI: 10.1049/ip-cdt:20030965
For access to this article, please select a purchase option:
Buy article PDF
Buy Knowledge Pack
IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.
Thank you
Your recommendation has been sent to your librarian.
- Author(s): M.T. Tommiska 1
-
-
View affiliations
-
Affiliations:
1: Signal Processing Laboratory, Helsinki University of Technology, Finland
-
Affiliations:
1: Signal Processing Laboratory, Helsinki University of Technology, Finland
- Source:
Volume 150, Issue 6,
November 2003,
p.
403 – 411
DOI: 10.1049/ip-cdt:20030965 , Print ISSN 1350-2387, Online ISSN 1359-7027
Special attention must be paid to an efficient approximation of the sigmoid function in implementing FPGA-based reprogrammable hardware-based artificial neural networks. Four previously published piecewise linear and one piecewise second-order approximation of the sigmoid function are compared with SIG-sigmoid, a purely combinational approximation. The approximations are compared in terms of speed, required area resources and accuracy measured by average and maximum error. It is concluded that the best performance is achieved by SIG-sigmoid.
Inspec keywords: field programmable gate arrays; piecewise linear techniques; logic design; combinatorial mathematics; neural nets
Other keywords:
Subjects: Logic and switching circuits; Combinatorial mathematics; Logic design methods; Neural nets (theory); Neural computing techniques; Digital circuit design, modelling and testing; Combinatorial mathematics
References
-
-
1)
- K. Basterretxea , J.M. Tarela , I. del Campo . Digital design of sigmoid approximator for artificial neural networks. Electron. Lett. , 1 , 35 - 37
-
2)
- U. Meyer–Baese . (2001) Digital signal processing with field-programmable gate arrays.
-
3)
- G.E. Moore . Cramming more components onto integrated circuits. Electron. , 8 , 114 - 117
-
4)
- Krips, M., Lammert, T., Kummert, A.: `FPGA implementation of a neural network for a real-time hand tracking system', Proc. 1st IEEE Int. Workshop on Electronic Design, Test and Applications (DELTA), 29–31 January 2002, Christchurch, New Zealand, p. 313–317.
-
5)
- J.L. Holt , J-N. Hwang . Finite precision error analysis of neural network hardware implementations. IEEE Trans. Comput. , 3 , 281 - 290
-
6)
- B. Parhami . (2000) Computer arithmetic: Algorithms and hardware designs.
-
7)
- M. Zhang , S. Vassiliadis , J.C. Delgado-Frias . Sigmoid generators for neural computing using piecewise approximations. IEEE Trans. Comput. , 9 , 1045 - 1049
-
8)
- K.M. Sammut , R. Jones S. . Implementing nonlinear activation functions in neural network emulators. Electron. Lett. , 12 , 1037 - 1038
-
9)
- P.D. Moerland , E. Fiesler . (1997) Neural network adaptations to hardware implementations, Handbook of neural computation.
-
10)
- H. Amin , K.M. Curtis , B.R. Hayes–Gill . Piecewise linear approximation applied to nonlinear function of a neural network. IEE Proc. Circuits, Devices Sys. , 6 , 313 - 317
-
11)
- Maya, S., Reynoso, R., Torres, C., Arias–Estrada, M.: `Compact spiking neural network implementation in FPGA', Proc. 10th Int. Conf. on Field Programmable Logic and Applications (FPL), August 2000, Villach, Austria, p. 270–276.
-
12)
- J.M. Zurada . (1992) Introduction to artificial neural systems.
-
13)
- K. Basterretxea , J.M. Tarela , N. Mastorakis . (2001) Approximation of sigmoid function and the derivative for artificial neurons, Advances in neural networks and applications.
-
14)
- P. Ashenden . (2000) The designer's guide to VHDL.
-
15)
- Faiedh, H., Gafsi, Z., Besbes, K.: `Digital hardware implementation of sigmoid function and its derivative for artificial neural networks', Proc. 13th Int. Conf. on Microelectronics, Oct. 2001, Rabat, Morocco, p. 189–192.
-
16)
- D.D. Gajski . (1997) Principles of digital design.
-
17)
- W.V. Quine . A way to simplify truth functions. Am. Math. Mon. , 627 - 631
-
18)
- M.J.S. Smith . Application-specific integrated circuits.
-
19)
- D.J. Myers , R.A. Hutchinson . Efficient implementation of piecewise linear activation function for digital VLSI neural networks. Electron. Lett. , 24 , 1662 - 1663
-
20)
- H.K. Kwan . Simple sigmoid-like activation function suitable for digital hardware implementation. Electron. Lett. , 15 , 1379 - 1380
-
21)
- Synplify Pro®, User Guide and Tutorial, Synplicity Inc., 2002.
-
22)
- APEX II programmable logic device family datasheet, www.altera.com/literature/ds/ds_ap2.pdf, accessed on 20th August 2003.
-
23)
- E.J. McCluskey . Minimization of boolean functions. Bell Sys. Tech. J. , 5 , 1417 - 1444
-
24)
- Omondi, A.R., Rajapakse, J.C.: `Neural networks in FPGAs', Proc. 9th Int. Conf. on Neural Information Processing (ICONIP), 18-22 November 2002, vol. 2, Sgaore, p. 954–959.
-
25)
- Alippi, C., Storti–Gajani, G.: `Simple approximation of sigmoidal functions: realistic design of digital neural networks capable of learning', Proc. IEEE Int. Symp. on Circuits and Systems, 11–14 June 1991, Sgaore, p. 1505–1508.
-
26)
- M.R. Dagenais , V.K. Agarwal , N.C. Rumin . McBoole, a new procedure for exact logic minimization. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. , 1 , 229 - 238
-
1)