A method for improving the real-time recurrent learning algorithm

doi:10.1016/S0893-6080(05)80126-1

Neural Networks

Volume 6, Issue 6, 1993, Pages 807-821

https://doi.org/10.1016/S0893-6080(05)80126-1 Get rights and content

Abstract

Williams and Zipser (1989) proposed two analogue learning algorithms for fully recurrent networks. The first method is an exact gradient-following algorithm for problems where data consists of epochs. The second method, called the Real-Time Recurrent Learning (RTRL) algorithm, uses data described by a temporal stream of inputs and outputs, without time marks or epochs. In this paper we describe a new implementation of this RTRL algorithm. This improved implementation makes it possible to increase the performance of the learning algorithm during the training phase by using some a priori knowledge about the temporal necessities of the problem. The reduction of the computational expense of the training enables the use of this algorithm for more complex problems. Some simulations of a process control task demonstrate the properties of this algorithm.

References (16)

BressloffP.C. et al.
Discrete time leaky integrator network with synaptic noise
Neural Networks
(1991)
FetzE.E. et al.
Neural network models of the primate motor system
RohwerR.
The “Moving Target” training algorithm
WilliamsR.J. et al.
Experimental analysis of the Real-time Recurrent Learning Algorithm
Connection Science
(1989)
AlmeidaL.B.
Back propagation in perceptrons with feedback
DecortisF. et al.
Temporal dimension in cognitive models
DockxK. et al.
Adding a temporal dimension to expert systems working in a real-time environment. CC-AI The Journal for the Integrated Study of Artificial Intelligence, Cognitive Science, and Applied Epistemology
(1992)
GrossbergR.
Some Networks that can learn, remember, and reproduce any number of complicated space-time patterns I
Journal of Mathematics and Mechanics
(1969)

There are more references available in the full text version of this article.

Cited by (27)

Learning in fully recurrent neural networks by approaching tangent planes to constraint surfaces
2012, Neural Networks
In this paper we present a new variant of the online real time recurrent learning algorithm proposed by Williams and Zipser (1989). Whilst the original algorithm utilises gradient information to guide the search towards the minimum training error, it is very slow in most applications and often gets stuck in local minima of the search space. It is also sensitive to the choice of learning rate and requires careful tuning. The new variant adjusts weights by moving to the tangent planes to constraint surfaces. It is simple to implement and requires no parameters to be set manually. Experimental results show that this new algorithm gives significantly faster convergence whilst avoiding problems like local minima.
On-line Gauss-Newton-based learning for fully recurrent neural networks
2005, Nonlinear Analysis, Theory, Methods and Applications
In this paper we propose a novel, Gauss–Newton-based variant of the Real Time Recurrent Learning (RTRL) algorithm by Williams and Zipser (Neural Comput. 1 (1989) 270–280) for on-line training of Fully Recurrent Neural Networks. The new approach stands as a robust and effective compromise between the original, gradient-based RTRL (low computational complexity, slow convergence) and Newton-based variants of RTRL (high computational complexity, fast convergence). By gathering information over time in order to form Gauss–Newton search vectors, the new learning algorithm, GN–RTRL, is capable of converging faster to a better quality solution than the original algorithm. Experimental results reflect these qualities of GN–RTRL, as well as the fact that GN–RTRL may have in practice lower computational cost in comparison, again, to the original RTRL.
Intracranial pressure model in intensive care unit using a simple recurrent neural network through time
2004, Neurocomputing
Citation Excerpt :
Although the RTRL algorithm has great power and generality, it has the disadvantage of being computationally very expensive. In spite of several modifications of RTRL [4,45] to reduce the computational expense, it is still complicated when dealing with complex problems. Therefore, the partially RNNs, also called SRN, involve treating a neural network as a simple dynamical system in which previous states are made available as additional input to any layer, i.e., feedback connections are organized in strict topologies such as hidden to hidden, output to output feedback.
This paper aims to establish a patient's intracranial pressure (ICP) model in neurosurgical intensive care unit (NICU) using neural network. Non-invasive physiological signals from patients including mean arterial pressure (MAP), heart rate (HR), end-tidal of carbon dioxide (EtCO₂) and regional cerebral oxygenation (rSO₂) were measured. However, ICP remains ill-defined, complicated and non-linear because it is affected by many predictable and unpredictable factors. Our study employs the structure of recurrent network to develop a modified neural network algorithm called a simple recurrent neural network through time (SRNNTT). The proposed recurrent neural network combines Elman architecture of the simple recurrent network structure and backpropagation through time. In order to demonstrate the performance of the proposed model, four kinds of neural-network classifiers have been tested on Mackey–Glass differential-delay equation which is a chaotic time series signal. Finally, we used this SRNNTT model to build the ICP model using data from six head-injured patients. Although the accuracy of the ICP model is still far from ideal, the methodology used non-invasive vital signs (i.e., MAP, HR, EtCO₂, and rSO₂) to predict an invasive, dangerous and expensive signal (i.e., ICP) has achieved this monitoring system more safely and flexibly in NICU.
On the improvement of the real time recurrent learning algorithm for recurrent neural networks
1999, Neurocomputing
This paper reviews different approaches to improving the real time recurrent learning (RTRL) algorithm and attempts to group them into common frameworks. The characteristics of sub-grouping strategy, mode exchange RTRL, and cellular genetic algorithms are discussed. The relationships between these algorithms are highlighted and their time complexities and convergence capability are compared. The learning algorithms are applied to train recurrent neural networks in an attempt to solve a long-term dependency problem, to model the Hénon map, and to predict the chaotic intensity pulsations of an NH₃ laser. The results show that the original RTRL algorithm achieves the lowest error among the gradient-based algorithms, but it requires the longest training time; whereas the sub-grouping strategy uses the shortest training time but its convergence capability is the poorest. The results also demonstrate that the cellular genetic algorithm is an alternative means of training recurrent neural networks when the gradient-based methods fail to find an acceptable solution.
A conjugate gradient learning algorithm for recurrent neural networks
1999, Neurocomputing
The real-time recurrent learning (RTRL) algorithm, which is originally proposed for training recurrent neural networks, requires a large number of iterations for convergence because a small learning rate should be used. While an obvious solution to this problem is to use a large learning rate, this could result in undesirable convergence characteristics. This paper attempts to improve the convergence capability and convergence characteristics of the RTRL algorithm by incorporating conjugate gradient computation into its learning procedure. The resulting algorithm, referred to as the conjugate gradient recurrent learning (CGRL) algorithm, is applied to train fully connected recurrent neural networks to simulate a second-order low-pass filter and to predict the chaotic intensity pulsations of NH₃ laser. Results show that the CGRL algorithm exhibits substantial improvement in convergence (in terms of the reduction in mean squared error per epoch) as compared to the RTRL and batch mode RTRL algorithms.
Adaptive networks for physical modeling
1998, Neurocomputing
This paper presents an original link between neural networks theory and mechanical modeling networks. The problem is to find the parameters characterizing mechanical structures in order to reproduce given mechanical behaviors. Replacing “neural” units with mechanically based units and applying classical learning algorithms dedicated to supervised dynamic networks to these mechanical networks allows us to find the parameters for a physical model. Some new variants of real-time recurrent learning (RTRL) are also introduced, based on mechanical principles.
The notion of interaction during learning is discussed at length and the results of tests are presented. Instead of the classical {machine learning system, environment} pair, we propose to study the {machine learning system, human operator, environment} triplet.
Experiments have been carried out in simulated scenarios and some original experiments with a force-feedback device are also described.

View all citing articles on Scopus

View full text

Original ContributionA method for improving the real-time recurrent learning algorithm

Abstract

Neural Networks

Connection Science

Back propagation in perceptrons with feedback

Temporal dimension in cognitive models

Adding a temporal dimension to expert systems working in a real-time environment. CC-AI The Journal for the Integrated Study of Artificial Intelligence, Cognitive Science, and Applied Epistemology

Some Networks that can learn, remember, and reproduce any number of complicated space-time patterns I

Journal of Mathematics and Mechanics

Original Contribution
A method for improving the real-time recurrent learning algorithm