Abstract
Complex-valued neural networks (CVNNs) bring in nonholomorphic functions in two ways: (i) through their loss functions and (ii) the widely used activation functions. The derivatives of such functions are defined in Wirtinger calculus. In this paper, we derive two popular algorithms—the gradient descent and the Levenberg-Marquardt (LM) algorithm—for parameter optimization in the feedforward CVNNs using the Wirtinger calculus, which is simpler than the conventional derivation that considers the problem in real domain. While deriving the LM algorithm, we solve and use the result of a least squares problem in the complex domain,\(\|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}\), which is more general than the \(\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}\). Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
van Trees, H.L.: Optimum Array Processing. Wiley Interscience, New York (2002)
Hirose, A.: Complex-Valued Neural Networks. Springer, Heidelberg (2006)
Calhoun, V.D., Adali, T., Pearlson, G.D., van Zijl, P.C.M., Pekar, J.J.: Independent component analysis of fMRI data in the complex domain. Magnetic Resonance in Medicine 48(1), 180–192 (2002)
Stuber, G.L.: Principles of Mobile Communication. Kluwer, Boston (2001)
Helstrom, C.W.: Elements of Signal Detection and Estimation. Prentice Hall, New Jersey (1995)
Nitta, T.: An Extension of the Back-Propagation Algorithm to Complex Numbers. Neural Networks 10(8), 1391–1415 (1997)
Kim, T., Adali, T.: Approximation by Fully-Complex Multilayer Perceptrons. Neural Computation 15(7), 1641–1666 (2003)
Wirtinger, W.: Zur Formalen Theorie der Funktionen von mehr Complexen Veränderlichen. Mathematische Annalen 97, 357–375 (1927) (in German)
Brandwood, D.H.: Complex Gradient Operator and its Application in Adaptive Array Theory. IEE Proceedings F (Communications, Radar and Signal Processing) 130(1), 11–16 (1983)
van den Bos, A.: Complex Gradient and Hessian. IEEE Proceedings (Vision, Image and Signal Processing) 141(6), 380–382 (1994)
Bouboulis, P., Theodoridis, S.: Extension of Wirtinger’s Calculus to Reproducing Kernel Hilbert Spaces and the Complex Kernel LMS. IEEE Trans. on Signal Processing 59(3), 964–978 (2011)
Li, H., Adali, T.: Algorithms for Complex ML ICA and Their Stability Analysis Using Wirtinger Calculus. IEEE Trans. on Signal Processing 58(12), 6156–6167 (2010)
Li, H., Adali, T.: Complex-Valued Adaptive Signal Processing Using Nonlinear Functions. EURASIP Journal on Advances in Signal Processing 2008, 1–9 (2008)
Marquardt, D.: An Algorithm for Least-Squares Estimation of Nonlinear Parameters. SIAM Journal on Applied Mathematics 11(2), 431–441 (1963)
Hagan, M., Menhaj, M.: Training Feedforward Networks with the Marquardt Algorithm. IEEE Trans. on Neural Networks 5(6), 989–993 (1994)
Remmert, R.: Theory of Complex Functions. Springer, New York (1991)
Kreutz-Delgado, K.: The Complex Gradient Operator and the CR-Calculus, http://dsp.ucsd.edu/~kreutz/PEI05.htm
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Amin, M.F., Amin, M.I., Al-Nuaimi, A.Y.H., Murase, K. (2011). Wirtinger Calculus Based Gradient Descent and Levenberg-Marquardt Learning Algorithms in Complex-Valued Neural Networks. In: Lu, BL., Zhang, L., Kwok, J. (eds) Neural Information Processing. ICONIP 2011. Lecture Notes in Computer Science, vol 7062. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24955-6_66
Download citation
DOI: https://doi.org/10.1007/978-3-642-24955-6_66
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24954-9
Online ISBN: 978-3-642-24955-6
eBook Packages: Computer ScienceComputer Science (R0)