Artificial neural network based generalized storage–yield–reliability models using the Levenberg–Marquardt algorithm

https://doi.org/10.1016/j.jhydrol.2005.10.033Get rights and content

Abstract

Generalised storage–yield–reliability models are developed using multi-layer perceptrons artificial neural networks (ANNs), trained using the Levenberg–Marquardt algorithm. These ANNs provide, for the first time, generalised models for simultaneously predicting within-year and over-year storage capacities, given the yield, reliability and readily obtainable streamflow statistics. The training, validation and testing of the models used time series data from 18 streams located in different parts of the world, which were carefully selected so that they nearly cover the range of flow variability observed in world streams. The performance of the models was very good. Further comparison of the ANN models with existing regression models revealed that the latter are marginally better; however, given that the regression models require the over-year capacity to be known a priori, the ANN models are more generic and should be preferred.

Introduction

Reservoir planning involves the determination of storage capacity of the reservoir, which will satisfy the demand with an acceptable level of reliability. This information can be provided by developing the storage–yield–reliability (S–Y–R) relationship for the site under investigation. The development of the S–Y–R function is generally achieved by carrying out a sequential analysis of time series data using one of a variety of traditional techniques such as behaviour simulation and the Sequent Peak Algorithm (see McMahon and Adeloye, 2005). However, more recently, other techniques have been developed which by-pass the sequential analysis of time series data. Because they do not require the sequential analysis of time series data, these generalised techniques, as they are commonly known (see Vogel and Stedinger, 1987) are much faster to implement, which makes them very useful during preliminary design stage, and are potentially more suited to application at ungauged sites as described by Adeloye et al. (2003). However, these desirable aspects of having generalised S–Y–R models are marred by the fact that nearly all the existing ones only predict the over-year capacity; in situations where significant within-year capacity exists, such as when the annual flow variability is low or the level of development is low, the existing generalised models will underestimate the total storage requirements.

Tentative solutions to this problem have included subjectively adjusting predicted over-year capacity estimate for within-year contribution (e.g. Hardison, 1965) and using more formal regression equations relating total (i.e. over-year plus within-year) capacity to the over-year capacity (e.g. Adeloye et al., 2003). However, both of these are limited in scope because they assume that the over-year storage is known a priori. In addition, the procedure by Hardison (1965) used data from the USA and hence may not be valid elsewhere. Finally, Adeloye et al. (2003) assumed a linear relationship between capacity and input factors such as the demand, reliability and coefficient of variation (Cv) of annual flows, whereas the true relationship is non-linear.

The aim of this study was to develop ANN-based generalized S–Y–R models for reservoirs, which will predict both the within-year and over-year capacities simultaneously. ANNs are most suitable for this task because they are capable of mapping most non-linear functions, which S–Y–R functions are, without specifying explicitly the form of the functions (Shamseldin, 1997). According to Haykin (1994) an artificial neural network is “a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two aspects: knowledge is acquired by the network through a learning process and interneuron connection strengths known as synaptic weights are used to store knowledge.”

In Section 2, a brief introduction to ANNs is given together with a short review of its numerous applications in hydrology and water resources. This is then followed by a description of the methodology used in the study. Finally, the results are presented and discussed.

Section snippets

Artificial neural networks methodology

The theory and mathematical basis of ANNs have been described excellently elsewhere (see Haykin, 1994); so, no attempt will be made to repeat them in detail here. In a manner akin to the human brain, ANNs learn from experience and the knowledge gained through such experience is used in arriving at prediction decisions when presented with new inputs. The experiential knowledge is achieved normally by training the network, using input–output exemplar pairs to determine synaptic weights. The

Methodology

The scope of the study was accomplished by the following specific objectives: first, S–Y–R relationships were developed using the modified Sequent Peak Algorithm (SPA) in order to determine the output data for the training and testing of the neural networks. Secondly, several neural network models were designed and compared to find the best one for the solution of the problem. Thirdly, the selected neural networks were compared with the recently developed regression models by Adeloye et al.

Development of artificial neural networks

Among all the neural networks paradigms available, a feedforward multilayer perceptron with a single hidden layer was considered to be the best choice for this study. Feedforward multilayer preceptrons have been shown to have a computational superiority in comparison to other paradigms (see Hornik et al., 1989). The activation functions chosen were the sigmoid hyperbolic tangent function in the hidden layer and the linear function in the output layer. The hyperbolic tangent sigmoid function was

Determination of the network architecture

The decision on input variables and output variables for a neural network is in general problem dependent. Therefore, the output variables were constrained by the purpose of the study and they were the over-year storage as a ratio of the mean annual runoff Ka/MAR, the total storage as a ratio of the mean annual runoff Kt/MAR and the within-year capacity as a ratio of the mean annual runoff d/MAR.

In the first part of the study, 13 models were developed to simultaneously predict these three

ANN models for three outputs

In this first part, thirteen models each with three output variables but different combinations of the input variables were developed, namely:

  • Model 1:

    [Ka/MAR, Kt/MAR, d/MAR]=f (Cv2, Rt, D, max(Cvmonthly), range(Mmr), min(Mmr), L)

  • Model 2:

    [Ka/MAR, Kt/MAR, d/MAR]=f (Cv2, Rt, D, max(Cvmonthly), range(Mmr), min(Mmr))

  • Model 3:

    [Ka/MAR, Kt/MAR, d/MAR]=f (Cv2, Rt, D, max(Cvmonthly), min(Mmr),)

  • Model 4:

    [Ka/MAR, Kt/MAR, d/MAR]=f (Cv2, Rt, D, max(Cvmonthly), range(Mmr))

  • Model 5:

    [Ka/MAR, Kt/MAR, d/MAR]=f (Cv2, Rt, D, range(Mmr), min(Mmr))

  • Model 6:

    [Ka/MAR, Kt

Choice of the optimal generalised S–Y–R model

In this study two sets of neural networks were designed: the first set contains thirteen models, which simultaneously predict Ka/MAR, Kt/MAR, and d/MAR, while the second set contains 10 models, which predict just Ka/MAR and Kt/MAR. The first ten models in the first set have the same input variables as the models in the second set, making therefore a comparison among them straightforward.

Fig. 3 compares the average correlation coefficient r for these first 10 models in the two sets during the

Conclusion

Twenty-three multi-layer, feed-forward artificial neural network models were designed and trained using the early stopping approach with the Levenberg–Marquardt back-propagation algorithm. The scope was to investigate the use of ANN for simultaneously predicting within-year and over-year reservoir capacities, and so provide generalised models for use in certain situations such as during preliminary planning analysis or analysis at ungauged sites. Input variables for the models were selected

Acknowledgements

The data used in this study were made available to T.A. McMahon (University of Melbourne) through the assistance of a number of individuals including Alan Gustard (Centre for Ecology, Wallingford, UK), Jim Slack (US Geological Survey), Stefan van Biljon (Department of Water Affairs, Republic of South Africa) and Murray Peel (University of Melbourne, Australia). We would like to thank Professor McMahon for the permission to use the data. Other data were collected with the collaboration of

References (39)

  • Artificial neural networks in hydrology. II: hydrological applications

    Journal of Hydrologic Engineering

    (2000)
  • C. Bishop

    Neural Networks for Pattern Recognition

    (1995)
  • A. Cancelliere et al.

    A neural network approach for deriving irrigation reservoir operating rules

    Water Resources Management

    (2002)
  • Y.M. Chang et al.

    Comparison of static-feedforward and dynamic-feedback neural networks for rainfall-runoff modelling

    Journal of Hydrology

    (2004)
  • H. Demuth et al.

    Neural Network Toolbox for use with Matlab

    (1998)
  • M.B. Fiering

    Estimates of resilience indices by simulation

    Water Resources Research

    (1982)
  • I. Flood et al.

    Neural networks in civil engineering. I: principles and understanding

    Journal of Computing in Civil Engineering

    (1994)
  • I. Flood et al.

    Neural networks in civil engineering. II: systems and application

    Journal of Computing in Civil Engineering

    (1994)
  • Freni, G., La Loggia, G., Termini, D., Vitellaro, P., 2004. A water demand model by means of the artificial neural...
  • Cited by (79)

    • Employing artificial neural network for effective biomass prediction: An alternative approach

      2022, Computers and Electronics in Agriculture
      Citation Excerpt :

      This algorithm accumulates the advantages of the gradient descent methods and the Gauss-Newton algorithm and at the same time this technique is able to overcome disadvantages of the standard backpropagation algorithm (Zhou et al., 2018; Özçelik et al., 2019) which has been extensively used in the area of forest and environmental modeling research. The disadvantages of the standard backpropagation algorithm are instability of the constructed system, existence of the strong possibility for the process to capture local minima, and the slow convergence (Adeloye and Munari, 2006; Wilamowski and Yu, 2010; Scanlan and Mulvaney, 2013; Wu and Ji, 2015). The value of the combination coefficient μ represents the algorithm’s step and its value is of vital importance for the functioning and efficiency of the algorithm because it defines the stability and the speed of convergence of the LM algorithm.

    • Artificial neural network and molecular modeling for assessing the adsorption performance of a hybrid alginate-based magsorbent

      2021, Journal of Molecular Liquids
      Citation Excerpt :

      The training of the network (FF-ANN[4:5:1]) was performed using the Levenberg–Marquardt back-propagation algorithm (LM-BP) [34,37]. This back-propagation method (Levenberg-Marquardt) was employed because it proved to have the fastest convergence on networks that contain up to a few hundreds of weights [37]. Likewise, previous studies demonstrated that Levenberg–Marquardt algorithm can be the best choice owing to the significant training performance and a high degree of correlation between targets and network outputs [36–41].

    View all citing articles on Scopus
    View full text