Novel design of weighted differential evolution for parameter estimation of Hammerstein-Wiener systems

Graphical abstract


Introduction
The block-structured models have emerged as a potential research topic in the domain of nonlinear parameter estimation, characterized by connections of linear dynamic and nonlinear static subsystems. Among the block-oriented structures, the Hammerstein and Wiener models are the most well-known, simple, and effective configurations with vast applications in chemical engineering, biomedical engineering, control systems, signal processing, and power electronics [1][2][3]. The Hammerstein model constitutes a static nonlinear block followed by linear dynamic block and the Wiener model comprises of a linear dynamic cascaded with a static nonlinearity [4]. The union of both the Hammerstein and the Wiener system generates the Hammerstein-Wiener model, which consists of one dynamic linear block sandwiched between two static nonlinear blocks having applications in all fields of science and engineering including nonlinear industrial processes [5], controls [6], signal processing [7], and instrumentation [8]. Several parameter estimation procedures have been formulated for identification of the Hammerstein-Wiener model, mainly including one-shot set-membership method [9], subspace method [10], blind approach [11], over parametrization [12], recursive least square algorithm [13], maximum likelihood method [14] iterative method [15] multi-signal-based method [16] and fractional approach [17]. All these proposed algorithms are deterministic procedures being broadly employed for the parameter estimation of Hammerstein-Wiener models having their own advantages, applications, and shortcomings but stochastic solvers based on evolutionary heuristics procedures have not yet been explored for the efficient parameter estimation of Hammerstein-Wiener models. Nature inspired heuristics based stochastic solvers have been extensively explored for constrained and unconstrained optimization problems arising in physical systems [18][19][20][21] including plasma physics [22], astrophysics [23], atomic physics [24], electrical power systems [25], electrical machines [26], quantum mechanics [27], electronic devices [28], signal processing [29], electric circuits [30], nanofluidic systems [31], energy [32], computer virus models [33], biomedical engineering [34], thermodynamics [35] supply chain management [36], scheduling problem [37], and finance [38]. Furthermore, the evolutionary algorithms have also been applied for parameter estimation of block-oriented models including controlled autoregressive [39], controlled autoregressive moving average systems [40], Hammerstein [41,42], wiener [43] and feedback nonlinear systems [44]. These applications motivated authors to exploit evolutionary computing heuristics as an alternate, accurate, and reliable parameter estimation technique for Hammersteinwiener systems and their fractional variants developed on similar pattern as reported in [45,46]. The potential features of the designed scheme are listed below: A new application of evolutionary heuristic paradigms based on Weighted Differential Evolution is introduced for accurate parameter estimation of nonlinear Hammerstein-Wiener systems and comparative analysis with counterparts to prove the worth and efficacy.
The designed scheme is effectively implemented to estimate the model parameters in terms of measured input and output data, as well as, the internal variables associated with the prior estimations of the subsequent measures. The robustness, stability and convergence of the designed metaheuristic paradigm are established through decision variables of the Hammerstein-Wiener systems of different lengths corrupted with process noise scenarios. Statistical observations on measure on central tendency and variance further prove the efficacy of the designed methodology WDE as precise, efficient, stable, robust and alternate computing platform for system identification of HWM for controlled autoregressive scenarios.
Remaining of the article is structured as follows: In Section 2, the parameter estimation model of Hammerstein-wiener systems, problem formulation, explanation of designed methodology is given. In Section 3 the different models of Hammerstein -wiener system and their simulation results are provided in detail. While last section summarizes the study including conclusion and future recommendations.

Design methodology
In this section, two-stage method is employed to estimate the parameters of Hammerstein-Wiener system, in the first phase, the mathematical models of the Hammerstein-Wiener systems are developed along with the cost function definition. In the second phase, evolutionary computing algorithms as optimization techniques are described to estimate the input as well as output nonlinear blocks parameters and noise model parameters of the Hammerstein-Wiener systems.
wðtÞ ¼ ½1 À AðzÞwðtÞ þ BðzÞvðtÞ Using (3) and (8), sðtÞ can be defined as: Substituting (9) in (7), we have: The parameter vector for the Hammerstein-Wiener system from (10) is given as: The Thiel's inequality coefficient index is employed for the error function formulation of Hammerstein-Wiener models as: here yðt i Þ stands for desired output for i th observation andŷðt i Þ for estimated response of the actual output, while l represents the total number of instances. The estimated response yðt l Þ for with respective information vector w T ðt l Þ is mathematically written as: For the ideal case of parameter estimation, the estimated output y approach its optimal y as e ! 0:

Optimization procedure
Metaheuristic evolutionary strategies like WDE and GAs that are proposed in this work for parameter estimation of Hammerstein-Wiener model are briefly explained here.
WDE is a latest bi-population algorithm from the family of evolutionary heuristics developed to solve nearly all real-valued unimodal and multimodal optimization problems [47]. WDE is capable of efficiently finding evolutionary search direction. Additionally, in WDE population diversity remain stable and does not decline swiftly which leads to effective searches in next upcoming iterations. A few benchmark problems include GPS network adjustment problem, Pressure-vessel, Speed-reducer Welded-beam design [47], and camera calibration [48].
GAs introduced by Holland [49], is a stochastic, effective, and broadly used evolutionary computation algorithm developed to solve real-valued numerical optimization problems [49]. GAs is easy implemented, robust, simple, efficient and reliable global search algorithm that uses three basic operators, like crossover, mutation and selection for generating new efficient population with better fitness [50]. The individuals with better fitness are less likely to trap in local minima. Few recent applications of GAs include electrical circuits [50], supply chain [51], lungs cancer [52] and wind speed prediction [53].
Efficacy of WDE, and GAs are the inspirations to the authors to use these evolutionary heuristics for finding optimal parameters of Hammerstein-Wiener system. Flow chart with procedural steps of GAs is shown in Fig. 3. Furthermore, the detailed stepwise procedures of WDE for Hammerstein-Wiener model is given as pseudocode form in Table 1. The designed methodology of the proposed work is illustrated in Fig. 2.

Performance indices
In this study, four performance operators i.e., MAE hwh , MWD hwh , RMSE hwh , and TIC hwh are utilized to validate the performance of the proposed evolutionary algorithms. These performance operators along with their mathematical description with respect to true and estimated parameters are provided in this section.
MAE hwh is mathematically written as: MWD hwh is mathematically written as: RMSE hwh is defined as: TIC hwh mathematical formulation is as follows: The magnitude of these performance metrics should approach zero for an ideal model.

Results and discussion
In this section, results of the experiments are discussed for two different examples of Hammerstein-Wiener system on the basis of different length of parameter vector and noise variances conducted though evolutionary computing heuristics WDE and GAs.

Model: I
In this Hammerstein-Wiener model, five unknown entities in the parameter vector are taken for estimation with polynomial type nonlinearity in both input and output typt nonlinearity and is mathematical form as: wðtÞ ¼ BðzÞ AðzÞ f lðtÞ ½ ; sðtÞ ¼ wðtÞ þ tðtÞ;  The objective function of nonlinear nonlinera Wiener model example 1 is formulated as described in equation (22) with K = 20 and N = 6 as: BðzÞ ¼ 0:15z À1 À 0:35z À2 ; vðtÞ ¼ 0:5lðtÞ þ 0:18l 2 ðtÞ À 0:15l 3 ðtÞ; yðtÞ ¼ sðtÞ þ 0:15s 2 ðtÞ: Likewise, the objective function of nonlinear nonlinear Wiener model example 2 is given as: In these two Hammerstein Wiener models, input signal is taken as randomly generated signal of zero mean and unit variance, while noise is also a random signal with mean zero and constant variance. The parameter estimation of Hammerstein Wiener models is performed through the wellknown evolutionary computational heuristics i.e., WDE and GAs for optimization of fitness functions for 20 snap shots. The results of proposed scheme based on learning curves along with absolute error analysis are given in Fig. 4. The iterative convergence graphs of fitness in case of WDE for Model-I and II are presented in subfigs. 4(a) and (d), respectively while for GAs the learning curves are shown in subfig. 4(g) for Model-I, 80db noise, subfig. 4(j) Model-I, 60db noise, subfigs. 4(m-o) and Model-II, 80db, 60db and 30db noise levels, respectively. It is observed that both the algorithms are convergent but convergence of WDE is slightly superior than GAs. Comparison based on fitness values are also shown here in subfig 5(b) and 5 (e) for Model I and II, respectively, while normalizing error plots comparisons are presented in subfig 5(h) and 5(k) for Model I and II respectively. In these plots, it can be observed that fitness values achieved by WDE is higher than GAs, also it can be seen that fitness valus decreases as the noise level increases. Along with this comparison, absolute errors (AEs) for the two Hammerstein-Wiener model for the three noise scenarios are shown in graphical form in subfig 5(f) and 5(i) for Model I and II respectively in case of WDE and in subfig 5(c) and 5(f) for Model I and II respectively in case of WDE while in subfig 5(i) and 5(l) for Model I and II respectively in case of GAs. The AE magnitudes are found in the range of 10 À9 , 10 À7 , 10 À4 for noise levels 80db, 60db and 30db with WDE, and 10 À7 , 10 À5 , 10 À4 for noise levels 80db, 60db and 30db with GAs for Model-I. while almost similar trend is found for GAs. Consistent accuracy is found for both the algorithms while slight degradration in the accuracy is observed with an increase in the noise levels for the proposed schemes. Fig. 5.
Analysis of accuracy of the designed scheme is performed for 100 iterations for the two models of Hammerstein-Wiener system and results based on fitness values are plotted in semilogrithmic style for better analysis in Fig. 6. It can be observed that the respective magnitudes found close to 10 À9 , 10 À7 , 10 À4 with noise levels 80db, 60db and 30db in case of Model-I, and 10 À5 , 10 À4 , 10 À3 with noise levels 80db, 60db and 30db for in case of Model-II with WDE. Almost similar trend is found for GAs. Very small high fitness values proves the accuracy of the scheme.
The efficacy and relaiblility of the proposed evolutionary heuristics is validated through performance indices i.e., error function, MWD hwh , MAE hwh , RMSE hwh , and TIC hwh . Results in term of the best run based on minimum fitness, as well as the complexity measures based on time, generations consumed, and function counts are listed in Table 2. It is quite clear that the MWD hwh noticed close to 10 À3 and 10 À2 for Model I with WDE and GAs, while in case of Model II, magnitudes are found close to 10 À2 and 10 À1 , for Model II with WDE and GAs. The more values close to zeros of the performance indices proves the consistency and precision of the proposed evolutionary algorithms.
Comparison via different performance indices i.e., error function, MWD hwh , MAE hwh , RMSE hwh ; and TIC hwh are employed to further examine the accuracy of the designed schemes and shown in pictoral form in Fig. 7 for WDE and GAs, respectively. The MAE hwh magnitudes for both the models are shown in graphical form for 100 independent runs in subfig. 7(a), and subfig. 7 (b), for Model I and II, respectively. In case of WDE and in subfig. 8 (a), and subfig. 8 (b), for Model I and II, respectively. In case of GAs. It can be seen from the figures that the MAE values for WDE in case of Model I and II are found in the range of 10 À3 to 10 À1 for noise levels 80db, 60db and 30db. In order to further examine the precision, results as the plots of histogram are investigated based on RMSE values for both WDE and GAs are shown in subfigs. 6 (c-f), and subfigs. 7 (c-f), respectively. In addition to histogram graphs, empirical Cumulative Distribution Function graphs are also plotted in terms of MWD magnitudes for Hammerstein Wiener models with both designed schemes and are shown in subfigs. 6 (g-j), and subfigs.   7 (g-j), respectively, for WDE and GAs, respectively. In subfigs. 6 (kl), and subfigs. 7 (k-l), comparative bar graphs are ploted both with WDE and GAs, respectively for Model I and II. In order to endorse the accuracy further, stacked bar graphical illustrations are also shown in terms of TIC values for Model I and II in subfigs. 6 (mn), and subfigs. 7 (m-n), for WDE and GAs, respectively. and similar trend is found for GAs outcomes. These all graphical illustrations validate the consistent accuracy of the two proposed heuristic strategies for the parameter estimation of Hammerstein Wiener systems. Further analysis of precision on 100 independent runs of the algorithms is carried out through the statistical performance measures via mean, best, and worst values of fitness and statistical outcomes for the two proposed evoltuionary heuristics are listed in Tables 3 and 4 for respective Models I and II with three noise level added in Hammerstein Wiener system. It can be observed that with the increase in the noise level, there is a decrease in the performance of the WDE and GAs, and same is the trend with the also with the increase in the unknown poarameters of the model as less degree of freedom make parameter estimation problem more stiff.
Complexity analysis is computed for the proposed evolutionary algorithms WDE and GAs via average time spent, mean generations executed and average times fitness functions are executed during optimization for finding optimal parameters of Hammerstein Wiener models. Computational complexity was analysed for 100 independent runs of both evolutionary schemes and are listed in Table. 5. It can be seen that the values of average time executed, generations completed and functions evaluation are around 0.35 ± 0.004, 200, 8040 for WDE, and 1.13 ± 0.096, 600, 24,040 for GAs, in case of Model I. It can be seen that among these two evolutionary heuristic algorithm, WDE is relatively less computationaly complex than that of GAs. Also with the increase in the dimensionality of the Hammerstein Wiener model, the computational complexity of WDE and GAs rises. All the computational work is performed on computer station, having Intel(R) Core (TM) i7-4770 CPU @3.40 GHz processor, 8 GB RAM.

Conclusions and future recommendations
In this study a novel application of evolutionary heuristic paradigm based on Weighted Differential Evolution and Genetic Algorithms are exploited for accurate parameter estimation of nonlinear Hammerstein-Wiener systems with various noise scenarios and number of unknown elements in the parameter vector. Experimental results prove that both algorithms are reasonably convergent and accurate however the performance of WDE is relatively better by means of precision and complexity indices. Results through statistics validate that proposed evolutionary algorithms are quite efficient but performance of the algorithms declines as the noise level increases. Comparative analysis via different performance measuring indices i.e., MWD hwh , MAE hwh , RMSE hwh , and TIC hwh also validate the consistency of the designed procedures. Furthermore, computational complexity of GAs is found more than WDE based on time consumed, iterations completed, and function counts. Also, with the increase in the length of the parameter vector of the Hammerstein Wiener model, the complexity of WDE and GAs increases and same trend is observed with the increase in the noise levels. The novel designs evolutionary heuristics are indeed effective algorithms for parameter estimation problems of blockoriented models.
In future, the newly introduced nature inspired heuristics [27,50,[54][55][56][57] like firefly, gravitational search optimization algorithm, bat algorithm, ant bee colony optimization and their recently introduced fractional variants can be good alternatives to boost the accuracy of the proposed Hammerstein Wiener structures.

Compliance with Ethics Requirements
Human and animal rights statements: All the authors of the manuscript declared that there is no research involving human participants and/or animal.
Informed consent: All the authors of the manuscript declared that there is no material that required informed consent.
Data Availability: My manuscript has no data associated with it.