An Improved Training Method for Feed-Forward Neural Networks

Abstract

In many fields of signal processing feed-forward neural networks, especially multilayer perceptron neural networks, are used as approximators. We suggest to state the weight adaptation process (training) as an optimization procedure solving a conventional nonlinear regression problem. Thus the presented theory can easily be adapted to any similar problem.

Recently it has been shown that second order methods yield to a fast decrease of the training error for small and medium scaled neural networks. Especially Marquardt’s algorithm is well known for its simplicity and high robustness.

In this paper we show an innovative approach to minimize the training error. We will demonstrate that an extension of Marquardt’s algorithm, i.e. the adaptation of the increasing/decreasing factor, leads to much better convergence properties than the original formula. Simulation results illustrate excellent robustness concerning the initial values of the weights and less overall computational costs.