Neural Network Modeling of Hydrodynamics Processes

Many of the computational methods for equation solving can be considered as methods of weighted residuals (MWR), based on the assumption of analytical idea for basic equation solving. Test function type determines MWR specific variety including collocation methods, least squares (RMS) and Galerkin’s method. MWR algorithm realization is basically reduced to nonlinear programming which is solved by minimizing the total equations residual by selecting the parameters of test solution. In this case, accuracy of solving using the MWR is defined by approximating test function properties, along with degree of its conformity with its initial partial differential equations, representing a continuum solution of mathematical physics equations. On fig. 1, computing artificial neural network (ANN) is presented in graphic form, illustrating process of intra-network computations. The input signals or the values of input

variables are distributed and "move" along the connections of the corresponding input together with all the neurons of hidden layer. The signals may be amplified or weakened by being multiplied by corresponding coefficient (weight or connection). Signals coming to a certain neuron within the hidden layer are summed up and subjected to nonlinear transformation using so-called activation function. The signals further proceed to network outputs that can be multiple. In this case the signal is also multiplied by a certain weight value, i.e. sum of neuron output weight values within the hidden layer as a result of neural network operation. Artificial neural networks of similar structure are capable for universal approximation, making possible to approximate arbitrary continuous function with any required accuracy. To analyze ANN approximation capabilities, perceptron with single hidden layer (SLP) was chosen as a basic model performing a nonlinear transformation from input space to output space by using the formula (Bishop, 1995): where n  xRis network input vector, comprised of j x values; q -the neuron number of the single hidden layer; s  wR -all weights and network thresholds vector; ij w-weight entering the model nonlinearly between j-m input and i-m neuron of the hidden layer; i voutput layer neuron weight corresponding to the i-neuron of the hidden layer; 0 , i bbthresholds of neurons of the hidden layer and output neuron; f σ -activation function (in our case the logistic sigmoid is used). ANN of this structure already has the universal approximation capability, in other words it gives the opportunity to approximate the arbitrary analog function with any given accuracy. The main stage of using ANN for resolving of practical issues is the neural network model training, which is the process of the network weight iterative adjustment on the basis of the learning set (sample)   i-example error. For training purposes the statistically distributed approximation algorithms may be used based on the back error propagation or the numerical methods of the differentiable function optimization.

Neuronet's method of weighted residuals for computer simulation of hydrodynamics problems
Let us consider that a certain equation with exact solution () y x for non-numeric value y s equation (3) presents an arbitrary x s within the learning sample.
We have L(y)=R with substitution of approximate solution (1) into equation (3), with R as equation residual. R is continuous function R=f(w,x), being a function of SLP inner www.intechopen.com parameters. Thus, ANN training under outlet functional is composed of inner parameters definition using trial solution (1) for meeting the equation (3) goal and its solution is realized through the corresponding modification of functional quality equation (2) training. Usually total squared error at net outlets is presented as an objective function at neural net training and an argument is the difference between the resulted 's' net outlet and the real value that is known a priori. This approach to neural net utilization is generally applied to the problems of statistical set transformation along with definition of those function values unknown a priori (net outlet) from argument (net inlet). As for simulation issues, they refer to mathematical representation of the laws of physics, along with its modification to be applied practically. It is usually related to necessity for developing a digital description of the process to be modeled. Under such conditions we will have to exclude the a priori known computation result from the objective function and its functional task. Objective function during the known law simulation, therefore, shall only be defined by inlet data and law simulated: Use of neuronet's method of weighted residuals (NMWR) requires having preliminary systematic study for each specific case, aimed at: 1) defining the number of calculation nodes (i.e. the calculation grid size); 2) defining number of neurons within the network, required for obtaining proper approximation power; 3) choosing initial approximations for training neural network test solution ; 4) selecting additional criteria in the goal function for training procedure regularization in order to avoid possible solution non-uniformity; 5) analyzing the possibilities for applying multi-criteria optimization algorithms to search neural network solution parameters (provided that several optimization criteria are available). Artificial neural network used for hydrodynamic processes studying is presented by two fundamentally different approaches. The first is the NMWR used for direct differential hydrodynamics equations solution. The NMWR description and its example realization for Navier-Stokes equations solution is presented in papers (Kretinin, 2006;Kretinin et al., 2008). These equations describe the 2D laminar isothermal flow of viscous incompressible liquid. In the paper (Stogney & Kretinin, 2005), the NMWR is used for simulating flows within a channel with permeable wall. Neural network solution results of hydrodynamic equations for the computational zone consisting of two sub-domains are presented below. One is rotating, while another is immobile. In this case, for NMWR algorithm realization specifying the conjugate conditions at the two sub-domains border is not required. In the second approach, neural network structures are applied to computational experiment results approximation obtained by using traditional methods of computational hydrodynamics and for obtaining of hydrodynamic processes multifactor approximation models. This approach is illustrated by hydrodynamics processes neural network modeling in pipeline in the event of medium leakage through the wall hole.

NMWR application: Preliminary studying
There are specific ANN training programs such as STATISTICA NEURAL NETWORKS or NEURAL TOOLBOX in the medium of MATLAB, adjusting the parameters of the network www.intechopen.com to the known values of the objective function within the given points of its definitional domain. Using these packages in our case, therefore, does not seem possible. At the same time, many of optimization standard methods work well for ANN training, e.g. the conjugate gradients methods, or Newton, etc. To solve the issue of ANN training, we shall use the Russian program IOSO NS 1.0 (designed by prof. I.N. Egorov (Egorov et al., 1998), see www.IOSOTech.com) realizing the algorithm of indirect optimization method based on self-organizing. This program allows minimizing the mathematical model given algorithmically and presented as "black box", i.e. as external file module which scans its values from running variable file generated by optimization program, then calculates objective function value and records it in the output file, addressed in turn by optimization program. It is therefore sufficient for computer program forming, realizing calculations using the required neural network, where the input data will be network internal parameters (i.e. weights, thresholds); on the output, however, there'll be value of required equation sum residual based on accounting area free points. Let us suppose that the objective function 2 y x  is determined within the interval   0;1 . It is necessary to define parameters of ANN perceptron type with one hidden layer, consisting of 3 neurons to draw the near-objective function with given accuracy, computed in 100 accounting points i x evenly portioned in determination field. Computer program for computing network sum residual depending on its parameters can be as follows (Fortran) Using nonlinear optimization universal program products for ANN training is limited to neural networks of the simplest structure, for dimension of optimization tasks solved by data packages does not normally exceed 100; however, it frequently forms 10-20 independent variables due to the fact that efficiency of neural network optimization methods generally falls under the greater dimensions of the nonlinear programming free task. On the other hand, the same neural network training optimization methods prove efficient under much greater dimensions of vector independent variables. Within the framework of given functioning, the standard program codes of neural network models are applied, using the well-known optimization procedures, e.g. Levenberg-Markardt or conjugate gradients -and the computing block of trained neural network with those obtained by the analytical expressions for objective function of the training anti-gradient components, which in composition of the equation under investigation acts as a "teacher" is designed.

Computing algorithm of minimization of neural network decision
Let us consider perceptron operation with one hidden layer from N neuron and one output (1). As training objective function, total RMS error (4) will be considered. The objective function shall be presented as a complex function from neural network parameters; components of its gradient shall be calculated using complex function formula. Network output, therefore, is calculated by the following formula: where x -vector of inputs, s -number of point in training sample, (x) -activation function, w j -weights of output neuron, j -number of neuron in hidden layer. For activation functions, logistic sigmoid will be considered Here b j -threshold of j-number neuron of hidden layer; the function t j (x,b j ) , however, has where v i -neuron weight of hidden layer.
While training on each iterations (the epoch) we shall correct the parameters of ANN toward the anti-gradient of objective function -E(v,w,b), which components are presented in the following form: Thereby, we have got all the components of the gradient of the objective function of minimization, comparatively which iterations will be consecutively realized in accordance with the general formula Here w is vector of current values of network weights and thresholds.

Using NMWR for hydrodynamics equations solving
Parameter optimization of neural network trial solutions is achieved by applying several optimization strategies and by subsequently choosing the maximum effective one (see Cloete & Zurada, 2000). First strategy is to apply totality of effective gradient methods "starting" from various initial points. The other strategy is to apply structural-parametrical optimization to ANN training; this method is based on indirect statistic optimization method on self-organizing basis or parameter space research (see: Egorov et al., 1998;. Any versions for multi-criterion search of several equations system solution are based on different methods of generating multiple solutions, satisfying Pareto conditions. Choosing candidate solution out of Pareto-optimal population must be based on analysis of hydrodynamic process and is similar to identification procedure of mathematical model. In any case, procedure of multi-criterion optimization comes to solving single-criterion problems, forming multiple possible solutions. At the same time particularities of some computational approaches of fluid dynamics allows using iteration algorithms, where on each step solution at only one physical magnitude is generated.

Modeling flows -the first step
The computational procedure described below is analogous to MAC method (Fletcher, 1991), investigating possibility of NMWR application based on neural net trial functions.

Laplace equation solution
Computational capabilities of the developed algorithm can be illustrated by the example of the solution of Navier-Stokes momentum equations, describing two-dimensional isothermal flows of viscous incompressible fluid. On the first stage we will be using this algorithm for Laplace equation solution www.intechopen.com where ij w -hidden layer neurons weights, then analytical expressions for the second speed potential derivatives can be calculated using the following formula Therefore, trial solution (1)  Efficiency of searching of neuronet learning solution parameters depends on problem dimension, i.e. weights and perceptron thresholds variable adjusted quantity. The more significant is neurons quantity in trial solution, the higher is ANN approximate capacity; however, achieving high approximation accuracy is more complicated. At the same time, neuron quantity depends not only on simulated function complexity, but also on calculation nodes quantity in which the residual equation is calculated. It is known that generally points' quantity increase in statistical set used for neural network construction is followed by increase in necessary neurons network (Galushkin, 2002;Galushkin, 2007) quantity. Consequently, the dense calculation grids application results in nonlinear programming problems; while applying rare calculation grids, it is necessary to check the solution realization between calculation nodes, i.e. there is a problem of learning solution procedure standardization. In the neuronet solution reception context on known equation, it is convenient using traditional additive parameter of training neural model quality -a control error which is calculated on the set of additional calculation nodes between calculation grid nodes. Number of these additional calculation grid nodes can be much more significant, and they should cover the whole calculation area, because the nodes number increase with control error on known network parameters does not result in essential computing expenses growth. Hence, referring to learning solution neuronet parameters reception, there exists an issue of solving twice-criterion problem of nonlinear optimization along with minimizing simultaneously both summary residual in control points, or the control error can appear as a restriction parameter, in the limited set of calculation nodes and in this case the neural network solution parameters reception is reduced to the conditional nonlinear optimization problem. At the first stage, residual distribution of the current equation (5)  and v y     . As a whole, the received neural network solution satisfies the equation (5) except for calculation nodes group, for example, in the input border right point vicinity, due to a sudden change of the boundary conditions in this point. In areas with the solution insufficient exactness we will place the calculation nodes additional quantity using the www.intechopen.com following algorithm. Let us formulate the Cohonen neural network with three inlet variables presented by the coordinates of available computation nodes x and y, and also the equations (5) residual value in these nodes, along with the required cluster centers quantity equal to the additional nodes quantity. The cluster center coordinates which will generally be placed in areas with the learning solution low precision (Prokhorov et al., 2001) we will consider additional computation nodes coordinates. The number of these additional nodes in each case is different and defined by iterations, until the decision error does not accept comprehensible value. As a result of the additional formation of received neural network learning solution using additional computation nodes, it turned out to be possible to increase the solution local accuracy in the point B vicinity while maintaining the accuracy high in all other points.

Fig. 4. Formation of additional computation nodes for Laplace equation solution
Therefore, not only has the computing experiment proven reception opportunity of the general neural network solution in the calculation area, but also defined coordinate calculation logic of computational nodes for increasing the accuracy of neural network initial equation solution. Let us study a reception opportunity of the Poisson equation solution using an irregular computational grid, i.e. equation total residual with solutions (1) will be calculated in nodes located in the casual image or certain algorithm, which use has not been connected with the necessity of computational grid coordination and computational area borders.

Poisson equation
Let us study a neural network solution precision on irregular calculation scales for Poisson equation This equation is particularly used for calculating the pressure distribution as well as for time iterations organization at the Navier-Stokes equations solution by pseudo-non-stationary algorithms (Fletcher, 1991). For the solution we shall use an irregular calculation grid, because, in contrast to fluid dynamics classical numerical methods, it does not result in the neutral network learning functions algorithm complication. Meanwhile, advantages using calculation nodes located in calculation area for the complex geometry study current are obvious. The decision is defined by the equations (15) (16) corresponding to these units values Δ. Fig. 6 (а) presents formation re sults of the calculation grid and the speed distribution on the pseudo-non-stationary algorithm first iterative step of the Navier-Stokes equation solution. Here it was possible to receive an exact neural network solution for the whole calculation area without using additional set of calculation nodes.
Let us now study am incompressible fluid internal flow within a channel with a stream turning ( fig. 3). Navier-Stokes equation system describing two-dimensional isothermal flows of the viscous incompressible fluid (Fletcher, 1991):  (17) www.intechopen.com Here again, w is the vector of all the weights and thresholds of the net. In this case the amount of q neurons in the trial solutions remains the same for each decision variable set. This is the parameter on which depend approximated capabilities of neural net trial solution. Result of computational algorithm functioning should be achievement of necessary accuracy level of solution at q minimum value.  (17) (12) solution is calculated with the received scale or with rectangular scale, or by means of random numbers generator, or by using the Sobol -Statnikov generator of LP  (Statnikov & Matusov, 2002); then, distribution solving equation residual in the grid nodes; third, the additional multitude of calculation nodes is generated with using the Cohonen network; then, if the precision is not achieved, points 2 and 3 are realized and the additional calculation components quantity grows until exact neural network solution for the whole calculation area is found; further, pseudo-nonstationary algorithm iterations are organized by using the equations (23)-(24) where equation (16) (Kretinin et al., 2010). On fig. 6 (а-c), changing dynamics of calculation components during realizing various moments of time of equations (23)-(24) algorithm is shown, the speed vector distribution reorganization during the transition from the equation (12) solution to the equations (17)-(19) solution is also illustrated. Finally, analytical solution (as neural network function) for Navier-Stokes equations (17)-(19) systems within the channel with stream turning at Re 100  which is expressed by the formula (23) on last iterative layer and neural network dependence of the pressure distribution on this layer Momentum equations For flows in rotating domain ( fig. 7), the equations for conservation of mass and momentum are written for the relative velocity formulation, where f i in right hand side is given by The absolute velocity formulation is used in the non-rotating domain, and f i =0. Thus, the standard k   turbulence model is used.
Let us consider the neural network functions ,,, ( ,,) iN N upk f xy   w as equations solution, where w-all weights and network thresholds vector that assure summary quadratic residuals minimums for each equation in optional totality of computation nodes coordinating each neural network solution learning iteration, are generated using the random number generator. Neuronet learning solution parameter search process efficiency depends on problem dimension, i.e. weights and perceptron thresholds varying adjusted quantity. The more significant is neurons quantity in trial solution, the higher is ANN approximate capacity; however, achieving high approximation accuracy is more complicated. www.intechopen.com

Fig. 7. Rotating Domain
At the same time, neuron quantity depends not only on simulated function complexity, but also on calculation nodes quantity in which the residual equation is calculated. It is known that generally points' quantity increase in statistical set used for neural network construction is followed by increase in necessary neurons network (Galushkin, 2002;Galushkin, 2007) quantity. Consequently, the dense calculation grids application results in nonlinear programming problems; while applying rare calculation grids, it is necessary to check the solution realization between calculation nodes, i.e. there is a problem of learning solution procedure standardization. In the neuronet solution reception context on known equation, it is convenient using traditional additive parameter of training neural model quality -a control error which is calculated on the set of additional calculation nodes between calculation grid nodes. The number of these additional calculation grid nodes can be much more important and they should cover all calculation area because the nodes number increase in which the control error on known network parameters does not result in essential computing expenses growth. Hence, at the learning solution neuronet parameters reception there is a problem of solving the multi-criterion problem of the nonlinear optimization and to minimize simultaneously both the summary residual in control points, or the control error can appear as a restriction parameter, in the limited set of calculation nodes and in this case the neural network solution parameters reception is reduced to the conditional nonlinear optimization problem.

Modeling leakage in a fuel transfer pipeline
Method of leakage zonal location (Zverev & Lurie, 2009), which generalizes the known method of leakage detection by hydraulic gradient line salient point, can be used to define of pipeline leakage position. When using the "base" zone location method variant, it is supposed that in case of stationary process, hydraulic gradients 1 i and 2 i before and after the leakage are constant and can be designed by the known values of liquid charge at the ends of controllable site. I.e., the distribution indignation of hydrodynamic parameters is not taken into account in comparison with the established flow which arises because of the environment outflow influence through the leakage aperture. It results in speed profile deformation within the cross sections of the pipeline near the leakage and the nonlinear dependence of hydraulic gradient in the function of distance from position of leakage downwards the stream. In order for pipe cross section speed distribution of the pipe to become appropriate for the established flow, the distance up to 40-60 calibres from leakage coordinate у x can be required. Defining nonlinear dependence of hydraulic gradient function near the leakage is possible either based on special experimental researches, or based on the numerical decision of hydrodynamics equations in three-dimensional statement. The given function can be added to the algorithm of zone location methode to reduce the leakage position coordinate definition error. One pipeline section of 300 m length and diameter where 8 uu    -the dynamic speed, y -the distance from the wall,  -the kinematic viscosity.

www.intechopen.com
At the statement of boundary conditions on the input, mass charge 1 m  or the speed 1 u appropriate to this charge is set. In the grid units belonging to the leakage aperture, liquid outflow speed у u is appointed. The given parameter at numerical researches varies for the modeling leakages of various intensity. For any point of the calculating area, any operational value of pressure x   of all the hydrodynamical parameters are equaled to zero, for the concerning of which differences of pressure will be calculated, is set. On the output from the calculating area the conditions of the established current are set, i.e. the derivatives modeling of the boundary layer the standard wall functions for parameters of turbulence k and ε (Fletcher, 1991) are used. The equations discretization is effectuated on the base of finite volumes method in the combination with hexagonal grid. On fig. 10 the formation of the grid in the calculating sections of the pipeline is represented.

Conclusion
Results presented in this chapter concern one of many applications of neural network learning functions to the mathematical physics problems solution, which is solution of hydrodynamics equations. Using learning functions of neural network allows to exclude the errors in solving differential equations caused by derivatives discretization and the borders low precision representation. Numerical algorithm of Navier-Stokes equations system solution differing from known fluid dynamics methods by the possibility of computational grid arbitrary formationis developed. The formation technique of the additional nodes file depending on learning solution local accuracy is offered.
The formation of the finite-elemental models for all extended pipelines and the numerical decision of equations of liquid movement for modeling the hydrodynamics processes now is limited to computer resources as by the quantity of the final elements of model calculated grid, so as by the time of finding of the decision. Even with such model, its use for system functioning operative analysis is quite doubtful. However, the numerical models of hydrodynamics processes can be used for the formation of information databases based on the neural network computing architecture which, after "training", have high speed of calculation. Neural networks are the universal approximation tool of multivariate nonlinear dependences, capable "to be arranged" under appearing of the new information of the researched process, i.e. they can serve as an intellectual tool for monitoring, which is constantly filled up and clarified. Thus, the introduction of neural algorithms in the approved and well-recommending methods of pipeline leakage detection is expedient for increased accuracy and the efficiency of accepted decisions. The constant evolution of the calculation capacity of the modern computers implies in a permanent effort to adjust the existing numerical codes, or to create new codes following new points of view, aiming to adequately simulate fluid flows and the related transport of physical properties. Additionally, the continuous improving of laboratory devices and equipment, which allow to record and measure fluid flows with a higher degree of details, induces to elaborate specific experiments, in order to shed light in unsolved aspects of the phenomena related to these flows. This volume presents conclusions about different aspects of calculated and observed flows, discussing the tools used in the analyses. It contains eighteen chapters, organized in four sections: 1) Smoothed Spheres, 2) Models and Codes in Fluid Dynamics, 3) Complex Hydraulic Engineering Applications, 4) Hydrodynamics and Heat/Mass Transfer. The chapters present results directed to the optimization of the methods and tools of Hydrodynamics.