Multivariate statistical inference in a radial basis function neural network
Introduction
Manufacturing processes are the activities where changes are made to the product's characteristics. The most important processes are machining, casting, welding and forming (Chandrasekaran, Muralidhar, Murali, & Dixit, 2010). These processes are under a constant trend of change due to the incorporation of new materials and methods in the manufacturing of products and components. Due to this, it is useful to have models for production control and decision-making to represent the process based on the behavior explained by the model. For example, the adjustment of parameters and variables is commonly made by "trial and error", generating costs for defective parts, less production time or material costs (Benyounis & Olabi, 2008).
There exists classic methods used to support in decision making such as linear regression, a method for analyzing and modeling the relationship between variables (Montgomery, Peck, & Vinning, 2012). Linear regression is used by many authors for prediction in different cases, e. g. in Benatia, Carrasco, and Florens (2017) and Sun et al., 2017, Lemos and Kalivas, 2017, Souza et al., 2017. Modeling between variables to analyze the manufacturing processes can be complemented with an Analysis of Variance (ANOVA) to know that the process variability is right described by the model and to know which process input affects this variability (Rencher, 2012). Morando et al., 2017, Zhou et al., 2018 and Chakraborty and Chowdhury (2017) are examples of the use of this application. However, to ensure that the conclusions found with the ANOVA are close to reality, it is important that the model fulfil assumptions about residuals. The authors do not always mention these assumptions.
One problem is which require analyzing two or more responses, but most authors do not mention the correlation between all responses, therefore generating one independent model for each response. For example, a welding process generates a certain depth and diameter in the weld penetration, an EDM generates a range of material removed and some roughness on the surface (Mollah & Pratihar, 2008), a machining CNC also generates roughness in one piece and it is useful to measure the energy consumed in the process (Ahilan, Kumanan, Sivakumaran, & Edwin, 2013).
In the cases above mentioned, it is common to use univariate models in the analysis of manufacturing processes that generate more than one response modeling once for each output. For example, Ugrasen, Ravindra, Naveen Prakash, and Keshavamurthy (2014) developed a prediction model for a wire EDM machining process with three responses. Through Analysis of Variance (ANOVA) for each response, the variable with more influence to optimize the process was determined, but the authors do not mention the relationship among responses. Jiang, Zhang, Sun, Zhou, Jiaxin, and Luhong (2014) applies a model to analyze a polymer curing process with two responses: viscosity and curing time, using an ANOVA for each response to determine the variability explained by the model. Jiang et al. (2014) mentioned that the normality assumption must be fulfilled, however, the other assumptions are not mentioned, nor if there is correlation between the two responses. A third example can be seen in Siddaiah, Singh, and Mastanaiah (2017), where a welding process was modeled with four input parameters and four responses. The authors mentioned a multivariate regression analysis, but they used an ANOVA.
Another problem occurs in processes with incomplete information, with complex data or when a linear regression is not enough to analyze the process. In these cases, must be found alternatives to statistical methods to avoid trial and error practices that generate costs in the process.
Intelligent systems are models that can be used to reduce the manufacturing costs. Such methods have several applications, and in recent years, scientists have been increasing their interest in manufacturing applications. They are based on human behavior, and were developed to make decisions on complex problems. They are an alternative to the classic statistical methods or mathematical models.
Neural networks are intelligent systems based on the biological process of the human brain. They have applications in science and engineering, mainly in interpolation, pattern recognition and classification problems (Rajendra & Shankar, 2015). Neural networks are a useful method for establishing relationships among variables, analyzing complex data or incomplete information (Shahlaei, Bahrami, Abdolmaleki, Sadrjavadi, & Bagher, 2015). The most used network is the Back-Propagation (BP) however it has the disadvantage of slow convergence and its ease to stunt in a local minimum (Rajendra & Shankar, 2015).
Another kind of neural network applied in the manufacturing industry is the Radial Basis Function (RBF), whose main characteristic is that its response depends on the distance of the input values to a fixed point called center or centroid. The Radial Basis Function is applied in the network activation, which allows it to be based on local characteristics of the input space for fast learning reducing the computing costs required by other methods.
Genetic Algorithms (GA) are another intelligent system, which can be combined with RBF networks (Gomes & Canedo, 2015; Jiang et al., 2014), mainly to optimize the centers of the neural network and thus, decrease the instability caused by random methods.
The main application of RBF networks is prediction. There have been good results in their application in different cases by several authors in manufacturing processes. One example can be seen in Mollah et al. (2008), where a RBF with a BP and GA were applied to estimate the network centers in two manufacturing processes. They conclude that the GA finds better results than the RP. However, both processes have more than one response and it must be determined if the centers’ estimate would be the same in the multivariate case.
The GA can also be used to optimize the model (Zhao, Hsu, Chang, & Li, 2016). In the multivariate case the optimization could be multi-objective. It has been shown that Particle Swarm Optimization obtains good results too (Chen, Zhu, He, & Yu, 2017).
In recent years, modeling using neural networks and statistical methods is growing (Laha, Ren, & Suganthan, 2015). This is due to the easiness of modeling complex problems, helping to model multivariate or multi-objective processes such as in the cases of Hadavandi, Shahrabi, and Shamshirband (2015), where they propose an alternative method based on neural networks to solve multi-objective, high-dimensional regression problems.
We can found a number of prediction examples in manufacturing processes in Murphy and Thomas (2017), Dutra et al., 2016, Laha et al., 2015, Miao et al., 2017, Yang et al., 2017, and Hadavandi et al. (2015), and examples in casting processes in Schafföner et al., 2016, Timelli et al., 2016 and Patel, Krishna, and Parappagoudar (2016). And we can see some applications of the MANOVA in Marini, Beer, Walters, Villiers, and Walczak (2017) and Vencato et al., 2017, Zhang et al., 2017.
The problem statement is to improve the analysis of the manufacturing processes considering the relationship between several dependent variables.
Some modelling approaches are regression and analysis of variance, but they do not consider the correlation of all responses, causing problems for decision-making based on these models.
This paper propose the application of multivariate statistical analysis in a Radial Basis Function neural network, taking into account the statistical significance between independent and dependent variables and their correlation. Verification of the assumptions for this analysis are fulfilled.
Thus, the multivariate statistical analysis in the Radial Basis Function is a good approach to analyze the process based on correlated variables: it satisfies the assumptions required for this analysis, it determines the process variation, and it also determines the most important variable that had influence in the process used to evaluate the permanent mold casting process.
In the process analysis some questions arise: which variable affects several responses at the same time? Does the model explains the process adequately? In order to respond such questions, the use of the ANOVA is suggested. However, there are statistical assumptions that should be taken into account to get correct conclusions through the analysis of variance. But how would these assumptions be affected by the existence of correlation in several outputs? If there is no significant relationship between responses, analyzing them independently can be helpful. But, if such responses are correlated, the way to arrive to the best conclusions would be to analyze the variables jointly in multivariate methods.
Then the problem is to analyze the manufacturing processes considering several variables of input and output, so generate a model that represents adequately the process to make decisions in the actual process based on the behavior described by the model.
It is common to use univariate models in the analysis of manufacturing processes that generate more than one response modeling once for each output. For example, Ugrasen et al. (2014) developed a prediction model based on BP network for a wire EDM machining process with three responses, and they used the Analysis of Variance (ANOVA) for each response, the variable with more influence to optimize the process was determined, but the authors do not mention the relationship among responses, are not mentioned the assumptions required by ANOVA to arrive at adequate conclusions. These cases are very common in the literature. In this paper, it is suggested to use of the Multivariate Analysis of Variance (MANOVA) to consider the relationship between responses and thus to reach better conclusions, provided that the assumptions that this analysis requires are met.
Considering the relationship between dependent variables, we are considering the multivariate analysis, increases the complexity in relation to the univariate analysis since it seeks to relate two or more independent variables with two or more dependent variables at the same time.
Common approaches ignore the relationship of several dependent variables and a model is constructed for each response, but by not considering that the changes in one response affect in the other they reach erroneous conclusions. ANOVA is also very common in the literature, however, in many cases it is not mentioned the fulfillment of the assumptions that this analysis requires, this also affects the conclusions.
Therefore, the objective of this paper is to suggest a method to analyze a process with several responses considering the relationship among all variables.
Section snippets
Radial basis function neural network
The RBF network, in its most basic form, involves three different layers (Broomhead & Lowe, 1988). The first layer applies the input patterns and connects the network to its environment. The second layer is the only hidden layer in the network and it applies the Radial Basis Function. The output layer is linear, and applies the responses of the network to the activation pattern. The input-output multivariate relationship is given by:
Where yij is the i, j − th
Application
Details for the proposed method are:
- -
Use the redesigned RBF for building empirical models.
- -
Understand how the method is used to estimate the parameters in the RBF model.
- -
Analyze residuals to determine if the Multivariate RBF model adequately fits to the data or to see if any under lying assumptions are violated (apply Eq. 9 to 22).
- -
Test statistical hypotheses and construct the Multivariate Analysis of Variance (MANOVA, apply Eq. 23 to 33).
- -
Apply the RBF redesigned to make a prediction (applying
Conclusions
Due to the difficulties of considering only one process response or generating one model for each response, and the need to describe the manufacturing processes considering all the important variables together, the present work proposes a general methodology by RBF network for the analysis of a permanent mold casting process with three inputs: metal temperature, mold temperature and tilt; and three responses: section A, section B and weight. The central composite design was used to obtain the
References (40)
- et al.
Modeling and prediction of machining quality in CNC turning process using intelligent hybrid decision making tools
Applied soft computing
(2013) - et al.
Optimization of different welding process using statistical and numerical approaches – A reference guide
Advances in engineering-Software
(2008) - et al.
Towards ‘h-p adaptive’ generalized ANOVA
Computer Methods in Applied Mechanics and Engineering
(2017) - et al.
Neural network of Gaussian radial basis function applied to the problem of identification of nuclear accidents in a PWR nuclear power plant
Annals of Nuclear Energy
(2015) - et al.
A novel Boosted-neural network ensemble for modeling multi-target regression problems
Engineering Applications of Artificial Intelligence
(2015) - et al.
Modeling and optimization for curing of polymer flooding using an artificial neural network and a genetic algorithm
Journal of the Taiwan Institute of Chemical Engineers
(2014) - et al.
Modeling of steelmaking process with effective machine learning techniques
Expert systems with applications
(2015) - et al.
Leveraging multiple linear regression for wavelength selection
Chemometrics and Intelligent Laboratory Systems
(2017) - et al.
Multivariate analysis of variance of designed chromatographic data. A case study involving fermentation of rooibos tea
Journal of Chromatography A
(2017) - et al.
ANOVA method applied to proton exchange membrane fuel cell ageing forecasting using an echo state network
Mathematics and Computers in Simulation
(2017)
Improved complex-valued Radial Basis Function (ICRBF) neural networks on multiple crack identification
Applied Soft Computing
Functional composites based on refractories produced by pressure slip casting
Journal of the European Ceramic Society
Application of unfolded principal component analysis-radial basis function neural network for determination of celecoxib in human serum by three-dimensional excitation-emission matrix fluorescence
Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy
A parametrized approach for linear regression of interval data
Knowledge-Based Systems
Greedy method for robust linear regression
Neurocomputing
Influence of process parameters and Sr addition on the microstructure and casting defects of LPDC A356 alloy for engine blocks
Journal of Materials Science & Technology
Process optimization and estimation of machining performances using artificial neural network in wire EDM
Procedia Materials Science
Concurrent validity and cross-validation of the Brunel lifestyle physical activity Questionnaire
Journal of Science and Medicine in Sport
Linear hypothesis testing in high-dimensional one-way MANOVA
Journal of Multivariate Analysis
A genetic algorithm for the multi-objective optimization of mixed-mode assembly line based on the mental work load
Engineering Applications of Artificial Intelligence
Cited by (24)
Adaptive hybrid intelligent MPPT controller to approximate effectual wind speed and optimal rotor speed of variable speed wind turbine
2020, ISA TransactionsCitation Excerpt :The developed system has an ability to forecast the wind speed based on haulage parameters [20]. The hybrid intelligent techniques have been implemented as they apprehend the non-linear relationships between input/output data and shows an appropriate estimation of wind speed rather than the simple intelligent technique [22–32]. Thus, it is most essential to develop a hybrid intelligent control strategy that effectively performs in both constant as well as in variable wind speed conditions.
An accurate and stable RBF method for solving partial differential equations
2019, Applied Mathematics LettersCitation Excerpt :The RBF technique is charming and has been widely applied to various fields in science and engineering [1–5] thanks to its high accuracy and ability to deal with scattered data in high-dimensional spaces.
Short-term passenger flow prediction under passenger flow control using a dynamic radial basis function network
2019, Applied Soft Computing JournalReview of the most common chemometric techniques in illicit drug profiling
2019, Forensic Science InternationalCitation Excerpt :MLP usually has one or more hidden layers, while RBF usually has one. This single hidden layer allows for faster computation when compared to other methods [116]. Other ANN structures include probabilistic neural networks (PNN), which have an additional layer (summation layer) between the input and hidden layers [117].
A New Space-Time RBF Method for Reaction Diffusion and Wave Propagation Problems
2024, Mechanisms and Machine ScienceArtificial intelligence (AI) in biomedical research: discussion on authors’ declaration of AI in their articles title
2023, European Radiology Experimental