MGP–CC: a hybrid multigene GP–Cuckoo search method for hot rolling manufacture process modelling

ABSTRACT Maintaining high level of quality in hot rolling manufacturing processes is very challenging problem to keep competitiveness in the iron and steel industrial market. Monitoring the quality of the output product helps enhancing the product outcomes, increase the company profit and improve the economic growth of the country. In this paper, we propose a new hybrid approach based on multigene genetic programming (MGP) and Cuckoo search (CS) algorithms for developing three rigorous models for roll force, torque and slab temperature in the hot rolling industrial process at the Ereg˜li Iron and Steel Factory in Turkey. MGP is a robust variation of the standard genetic programming (GP) algorithm while CS is a new nature-inspired metaheuristic search algorithm. The performance of the developed models is evaluated and compared with those obtained for the standard MGP and GP approaches.


Introduction
The problem of modelling the hot rolling industrial process is complex and challenging due to the highly dynamic and nonlinear relation between the involved variables in the process such as, friction coefficient, yield stress, disturbances, etc. Moreover, the data sets collected to predict the performance of the process is often noisy and multi-variate (Moussaoui, Selaimia, & Abbassi, 2006). This makes getting accurate measurement of the rolling force/torque very difficult task. However, accurate and reliable models for the rolling force, torque and temperature are vital in designing pass online schedules and maintaining high quality of the production.
Steel market industry has witnesses a high competition every year. Thus, maintaining exceptional level of production quality is essential to reserve a high market share (Cowling, 2003;Liberatore, 1989). In hot rolling strip production in particular, the demand for higher quality products motivates the need for quality control techniques such as reliable and accurate prediction models of the rolling force/torque and temperature (Cowling, 2003;Feng, Liu, Luo, Tang, & Liu, 2008;Kirihata, Maccagno, & Jonas, 1998;Kwak, Kim, Park, Lee, & Hwang, 2000;Schroder, 2003). Typical mathematical models have some limitation since some factors are difficult to be integrated CONTACT Hossam Faris hossam.faris@ju.edu.jo such as the reduction rate and the chemical composition.
A typical solution to come around this problem in some rolling processes is to adopt fragmented look-up tables. However, this solution has some disadvantages such as the large size of the generated look-up tables and the lack of effective interpolation capability. In literature, many authors proposed quality monitoring models for the hot rolling process based on different machine learning techniques such as artificial neural networks (ANN) and fuzzy logic in order to overcome the problems mentioned earlier (Barrios, Torres-Alvarado, & Cavazos, 2012;Bouhouche, Yahi, Hocine, & Bast, 2008;Faris, Sheta, & Öznergiz, 2013;Gorni, 1997;Öznergiz, Özsoy, Delice, & Kural, 2009;Rong, Dan, & Yi, 2005;Sheta, Braik, Öznergiz, Ayesh, & Masud, 2013;Tiensuu, Juutilainen, & Röning, 2011 ). However, the disadvantage of most of these techniques is the lack of interpritability and considered as black box models. For that reasons authors in  and Sheta and Faris (2014) investigated the application of Symbolic Regression approach using genetic programming (GP) for predicting features of a hot rolling process including roll force, roll torque, and slab temperature.
One of the difficulties and weaknesses in GP is finding the numeric constants that form the GP model along with the set of arithmetic functions (Koza, 1992). In standard GP, numeric constants are generated randomly. Therefore, various studies in the literature proposed different ways to solve this problem. Some of the effective approaches proposed by researchers is the hybridization between GP and metaheuristic algorithms. Two main types of these approaches could be distinguished: the co-evolution approaches and the relay approaches. One example of the co-evolution approaches can be found in Mukherjee and Eppstein (2012) where authors used differential evolution to co-evolution of the coefficients in GP. On the other hand, relay approaches perform optimization of the constants in a separated task when GP finishes evolving the structure of the Symbolic Regression model. An example of this approach is proposed in Alonso, Montaa, and Borges (2009) where evolution strategies is applied to optimize GP constants.
In this paper, we propose a new hybrid relay approach based on combining multigene genetic programming (MGP) and Cuckoo search (CS) algorithm for developing more accurate symbolic regression models for a hot rolling industrial process. MGP and CS are executed separately in sequence. MGP is applied first for finding the structure of the symbolic regression models then CS is used for optimizing the numeric constants in these models. MGP is an evolutionary computation algorithm and a robust variation of the standard GP while CS is a new nature-inspired metaheuristic search algorithm. The objective of the proposed approach is to simulate the behaviour of different features of a hot rolling manufacturing process of the Eregli Iron and Steel Factory in Turkey. Three models for the force, torque and slab temperature in the plate mill are accurately developed and compared to those obtained by MGP and the standard GP approaches.
The paper is organized as follow. Section 2 describes the hot rolling process in Eregli Iron and Steel Factory. Section 4 introduces the MGP approach and its main processes. Section 5 gives a brief description of the CS algorithm and explains its application for tuning the generated MGP models. Section 6 describes the data collected for the purpose of the experiments in this study. In Section 7, we list the evaluation criteria used to evaluate the final developed models. Finally, Section 8 presents the experiments settings and discusses the obtained results.

Problem description
The hot-rolling mill plant in Eregli Iron and Steel Factory is shown in Figure 1. The plant consists of two slab furnaces, pre-rolling mill, edger, reversible mill, seven strip rolling stands, a cooling system, a hot leveller, and a shearing system. The plant has also a data acquisition and a  Steel strips with a thickness of 15-16 mm can be produced in the rolling mill plant. In a normal production cycle, each slab passes 5 times in forward and backward directions in the reversible mill. In this plant, the dimensions of slabs are monitored continuously during every passes with an X-ray system, the temperature of slab with a pyrometer, roll force and torque with four load cells placed along the mill. But averages of these measured values for each pass are used for identification. The chemical composition of the low-carbon steels used in this study is given in Table 1.

Genetic programming
GP is an evolutionary algorithm-based methodology for automatically solving problems inspired by biological evolution (Koza, 1991(Koza, , 1994. GP has many advantages when used to model nonlinear systems (Espejo, Ventura, & Herrera, 2010;Hussein, Sheta, & Abd-Elwahab, 2001;Kotanchek, Smits, & Kordon, 2003;Sheta et al., 2013). They include: • Flexibility: is one of the main advantages of GP over many other heuristic approaches. That is because the general structure of the tree representation of GP individuals can fits a wide range of problems. Moreover, using the terminal set and the function set can do both, represent knowledge and perform computation. • Interpretability: some modelling and prediction techniques are considered as black box methods since they generate models which are hard to understand such techniques traditionally include ANNs, SVMs. Unlike those techniques, tree models generated by GP are more simple, easier to evaluate and their complexity is controllable by setting the maximum GP tree depth and length.
Therefore, GP has been applied successfully to a large number of manufacturing and industrial processes. Some of these applications can be found in Hussein et al. (2001), Hussian, Sheta, Kamel, Telbany, and Abdelwahab (2000), , , Sheta and Faris (2014) and Faris (2013). In particular for the case of hot rolling process, the results in  and Sheta and Faris (2014) show that the developed GP mathematical models are very competitive taking into consideration their simplicity and interpretability.

Multigene genetic programming
Multigene GP is a robust variation of the standard GP (Searson, Leahy, & Willis, 2010). Multigene GP can be applied to evolving a population of trees (possible solutions) while minimizing some error criteria. For a system with x input of dimension n × m to produce a model output y with dimension n × 1, we could produce a tree structure which introduce the mathematical relationship: where n is the number of observations taken and m is the number of input variables. In multigene GP, prediction of the output variableŷ is resulted by a weighted output for each trees/genes in the multigene individual plus a bias term. Each tree is a function of zero or more of the N input variables x 1 , . . . , x N . The main processes of the Multigene GP algorithm flow chart is shown in Figure 3.

Initial population and representation
The evolutionary cycle of the algorithm starts by generating random individuals (possible solution or model) each of which consists of M randomly generated trees. A Multigene GP model can be represented in the form shown in Equation (2). A multigene regression model can be written as:ŷ where α 0 represents the bias or offset term while α 1 , . . . , α M are the gene weights and M is the number of genes (i.e. trees) which constitute the available individual.
The weights (i.e. regression coefficients) are automatically determined by a least squares (LS) procedure for each multigene individual. In multigene symbolic regression each symbolic model is represented by number of GP  trees weighted by linear combination. Each tree is considered as a 'gene' by itself. An example of multigene model is shown in Figure 2. The presented model can be introduced mathematically as given in Equation (3).
Each symbolic model will have set of linear coefficients such as α 0 , α 1 , . . .. These parameters will be estimated ordinary LS techniques. This shows the powerful nature of multigene GP since it combines the classical linear regression theory with the ability to capture nonlinear behaviour based the evolutionary process. It was reported that there are many advantages of multigene symbolic regression than standard GP approach for symbolic regression (Hinchliffe et al., 1996).
A multigene symbolic models are consisted of a function set and a terminal set. The function set is likely to contain of a number of simple arithmetic operators functions and/or and nonlinear mathematical functions such as the ones given in Equation (4).
The function F is combined with the terminal set to help the algorithm to develop suitable tree structure which represent a model for the problem. This multigene symbolic model has the advantages that it is linear in the parameters with respect to the coefficients α 0 , α 1 and α 2 .

Fitness evaluation
The root mean square error (RMSE) is used as a fitness function in order to evaluate GP individuals. RMSE can be described by Equation (5).
where y andŷ are the actual and the estimated values based on proposed model and n is the number of measurements used in the experiments, respectively.

Tournament selection
A selection mechanism consists of two components: a selection scheme and a selection pressure control strategy. This strategy is usually critical in designing a selection mechanism and has been widely studied by the evolutionary computation community (Legg, Hutter, & Kumar, 2004). Tournament selection is a beneficial and robust selection mechanism (Miller, Sutton, & Werbos, 1995). The selection pressure of tournament selection changed according to the tournament size. The more the number of individuals are selected the higher the resulting selection pressure.

Crossover
During the evolutionary process, genes are generated using crossover operator. In our case, we used two point crossover. Crossover makes exchange of genes between individuals. A two point high level crossover adopted in Searson et al. (2010) is used. The example in Equations (6)-(8) shows the operation.
• Individuals for crossover • Selected genes for crossover • Generated individual

Mutation
Mutation in multigene GP operates in a similar fashion to standard GP evolutionary process. A random gene is selected then a normal sub-tree mutation is performed.
The new generated gene replaces the parent gene in the new generation.

Coefficients estimation
In this process, the linear coefficients α 0 , α 1 , . . . , α M of each individual in the population are estimated using ordinary LS techniques based on the training data. In order to save the computation time. This operator is applied on a small fraction of each generation. By default it is set to 5%.

Optimizing multigene GP models by CS
CS is one of the recent and powerful metaheuristic optimization algorithms (Yang & Deb, 2009. CS is inspired by the natural brood of Cuckoo birds and the Lévy flight behaviour of some birds species and fruit flies. The CS algorithm is designed mainly on the following three assumptions (Yang & Deb, 2009): (1) A Cuckoo bird selects randomly a nest and put one egg (solution) in it. Therefore, every nest forms one candidate solution.
(2) Some nests that has the best quality solutions will be passed to the next generation. (3) The number of available host nests is fixed, and the ratio that a host discovers a cuckoo egg can be approximated by a fraction p a of the n nests being neglected and replaced by new random nests (solutions) at their positions.
New solutions x t+1 i are generated using a Lévy flights as shown in Equation (9). Lévy flights are performed by random walks with step-lengths are drawn from a Lévy distribution, a heavy-tailed probability distribution, which has an infinite variance with an infinite mean, Equation (10).
In Equation (9), α > 0 is the step size which should be related to the scales of the problem of interest. In most cases, α is supposed equals 1. The product ⊕ denotes entry-wise multiplications.
In this paper, the fitness function used in CS algorithm is the same as MGP which is the RMSE criteria. While nests are encoded as sequence of the coefficients αs that connects the trees in the final MGP developed models shown in Figure 4. The goal is to find the best coefficients that minimize the RMSE of the MGP model. CS algorithm enjoys some advantages that make it very competitive to other metaheuristic algorithm. Such advantages include: • CS has only one parameters to be tuned beside the population size which is the discovery rate P a (Yang & Deb, 2009). • In contrast to other metaheuristic approaches that use uniform distributions or Gaussian to generate new explorative moves in the search space, CS performs random walk via Lévy flight which is more efficient in exploring the search space if the search space is large enough (Yang & Deb, 2009).
The CS algorithm works as described by Algorithm 1 where s is the step size and P a is the probability of discovering the egg. E t is derived form Lèvy Flight distribution which is a random walk with step length depending on the current state plus a transition probability (Yang & Deb, 2009).

Hot rolling data set
The hot rolling collected data set contains 640 collected from 128 different slabs by a General Electric's data acquisition system. The collected measurements were practically measured for three sub-processes: roll force f, roll torque G and slab temperature T of the rolling process. The thickness and width distribution of the data ranged from 31.68 to 168.6 mm and 948.76 to 1457.26 mm, respectively. In order to compare our obtained results with other reported results from the literature Sheta, Öznergiz, Abdelrahman, & Babuska, 2009), same data set partitioning for the training and testing was adopted, 500 for training and 140 for testing.
Each of force f, the torque G and the slab temperature T has six input variables. The inputs and output of the proposed model structures can be summarized as follows: • Inputs to force f and torque G models: u 1 , u 2 , u 3 , u 4 , u 5 and u 6 inputs stand for: entry temperature (T i ), width (W s ), carbon equivalent (C e ), gauge (h i ), draft (i) and roll diameter (R), respectively. The output of each sub-process is stated as M Force , M Torque and given in Equation (11).
• Input to slab temperature T model: u 1 , u 2 , u 3 , u 4 , u 5 and u 6 inputs stand for: Entry temperature (T i ), width (W s ), carbon equivalent (C e ), gauge (h i ), torque (G i ), power (E i ), respectively. The output of each sub-process is stated as M Temp and given in Equation (12).

Performance evaluation
Other performance criteria are used to evaluate the quality of the developed Multigene GP models. The set of criteria used are given as follows: (1) Variance-accounted-for (VAF) (2) Mean squares error (MSE) (3) Mean absolute error (MAE): where y andŷ are the actual and the estimated values based on proposed model and n is the number of measurements used in the experiments, respectively.

Experimental results
In our experiments we use the GPTIPS Tool 1 for developing Multigene GP models for rolling force, torque and slab temperature. The parameters of the application are tuned as listed in Table 2. The best results are obtained using seven genes in each individual. The function set used to combine the terminals in GP individuals is F = {+, −, /, * , sqrt, sin, cos}.

5:
Get a cuckoo randomly by Lévy flights evaluate its fitness F i

6:
Choose a nest among n (say,j) randomly 7: Keep the best solutions/nests 12:

Rank the solutions/nests and find the current best
13: Pass the current best solutions to the next generation 14: end while 15: end  In Figures 5-7 we show the convergence of MGP toward the best obtained force torque and temperature models, respectively. Figures 8-10 show the actual and estimated force, torque and temperature values based on the developed MGP models.
In order to enhance the accuracy of the developed MGP models, CS algorithm is used to tune the parameters of these models. This approach will be referred to as MGP-CS. CS is applied for 500 cycles. The actual and estimated force, torque and temperature values based on the developed MGP-CS models are shown in Figures 11-13, respectively. The developed MGP and MGP-CS models are evaluated using VAF, MSE and MAE; and compared with the previously obtained results for the standard GP approach in  and Sheta and Faris (2014). Based on the comparison results shown in Tables 3-5 we can notice that MGP achieved better results than the standard GP approach in the force and torque cases while there      was no improvement in the temperature case. Moreover, combining CS algorithm with MGP in the MGP-CS approach generated the best quality models for the roll force, torque and temperature since it increased the VAF measurement and decreased MAE and MSE for all testing cases.
In Table 4, there is an enhancement also on the roll torque by decreasing the MAE and MSE values for the testing case. However, in the case of slab temperature which is shown in Table 5 we can notice that the performance is almost the same compared to the standard GP approach.
Finally, it can be concluded that the MGP-CS modelling approach has a promising potential in improving the quality of the roll force and torque prediction models in the hot rolling process. This can be explained by the advantage of Multigene GP that combines the classical linear regression theory with the ability to capture nonlinear behaviour based the evolutionary process.

Conclusion
In this paper, MGP is combined with a CS algorithm to develop estimation models for a Hot-Rolling industrial process. Three sub models are proposed for the process; the rolling force, torque and slab temperature. The developed MGP-CS models are evaluated and compared with the standard GP models based on different evaluation criteria. Experimental data collected from the Eregli Iron and Steel Factory in Turkey was used for the verification of the developed models. Compared to MGP and standard GP, MGP-CS models showed better performance results for rolling force, torque and temperature compared to those obtained for the other approaches. This advantage can support implementing higher quality models efficiently in iron and steel manufacturing plants.

Note
1. GPTIPS Tool is a free, open source MATLAB toolbox for performing symbolic regression by GP.