Keywords

1 Introduction

Steel mills are highly involved in improving mechanical and creep resistance under conditions of high temperatures and long service times in the materials used to produce steels, among them martensitic steels of high percentage in chromium [1, 2]. Since long service times under creep conditions will produce creep damage in steel used to steam turbine in power plants, this improvement can enhance the efficiency of these plants. Optimizing of creep damage evolution can improve mechanical properties until fracture, prolonging the service life of equipment operating [3,4,5].

The purpose of this work is to generate knowledge that allows a better understanding of the different elements influence in the alloy and in the metallurgical mechanisms, allowing to improve martensitic steels materials creep properties. This knowledge will allow easier development of new advanced steels.

Specifically, steels with a content of 9–12% Cr are studied, focusing this study to improve creep resistance [6]. In this approach, several models will be developed to predict the short-long-term creep behavior of new steels based on previous creep behaviors of similar materials.

These models use a previous knowledge of the influence of each composition element of the alloy and the thermal treatment on the fracture time, applied to predict this life expectancy. These models have a direct application in the high-chromium steels studied in this work, but it is also useful for a wide range of steels such as the rest of stainless steels, microalloys, high strength and new generation steels.

Several techniques of linear regression were applied to predict material fracture life, specifically 9–12% Cr martensitic steels based on modified P92 steels. Also, to obtain a clear knowledge of the influence of each feature that define the composition of the material, the thermal pretreatment applied to the material, and the creep test conditions on the final material fracture life. In order to get this purpose, a study of the literature [7,8,9,10,11,12,13,14,15,16,17,18,19] focus on creep test of Cr martensitic steels were performed, obtaining a dataset with 344 instances that was used to train and test the models. Then, several techniques like data pre-processing, outlier detection, analysis of variance, analysis of covariance, analysis of correlation, multivariate data visualization, and principal components analysis were applied. This step studies the influence of each attribute with the output, the material fracture life, to finally select the most significant variables to perform the most accurate models to solve the proposed problem. And then several linear regression techniques were used and validated to build and select an accurate linear model that improves the amount of knowledge of the creep behavior of 9–12% Cr martensitic steels.

Linear regression algorithms, simple (simple linear regression) and generalized (generalized linear regression), with and without using enhanced by gradient boosting techniques (enhanced linear regression and enhanced generalized linear regression) were applied in this work.

These kind of methodologies that applies data analysis techniques and machine learning algorithms is gaining interest in industrial engineering fields [20,21,22] where a mathematical model that represents the problem can be helpful in the decision making. And within of the possible engineering problems, this methodology can be used for the development of new materials (advanced modified P92 steels) improving final properties and reducing time and money during the process.

2 Materials

In this work, short-term and long-term testing were studied. From this study, 344 instances based on these testing were contemplated based on significant information of the composition of the material, the previous thermal treatment performed on the material, and the parameters that control the creep test. More specifically the studied features were:

  • Composition of the material: weight percent (wt%) of the most applied elements of the chemical composition of modified P92 type steel according to ASTM A335 (C, Mn, Si, Cr, Ni, Mo, P, S, W, Nb, V, N, Al, B, Co, Cu).

  • Heat treatment of this steel before the creep test: temperature (NormTemp) and time (NormTime) for normalizing, and temperature (TempTemp) and time (TempTime) for tempering.

  • Control parameters of creep test: Temperature (Temp) and strain (Strain) applied on creep test performance.

  • Time to fracture (TimeFractPoint), that is the predicted value based on the rest of features.

Figure 1 shows the relationships between the variables under study.

Fig. 1.
figure 1

Correlation between independent variables and against the dependent variable.

Full size image

3 Methodology

Based on the high number of variables and the complexity of the studied problem (prediction of fracture time of P92 steels or similar materials), the proposed methodology is applied in two steps, first a multivariate analysis and later a linear regression. With the final goal of predicting the material fracture time, but with an easy to understand model that, at the same time, could show information of the influence of the independent variables on the fracture time.

3.1 Multivariate Analysis

A multivariate analysis was performed to simplify the final models, building simpler models and as result, making a better generalization of the problem to improve the results in the validation stage. This analysis was conducted in five steps in order to detect wrong instances and redundant features:

  • Data pre-processing.

  • Outlier detection [23].

  • Analysis of correlation, variance and covariance [24].

  • Multivariate data visualization.

  • Principal component analysis [25, 26].

3.2 Simple Linear Regression

A probabilistic model of the expected value of an output variable is built based on several independent variables by means of a simple multivariate linear regression (LR) technique [27, 28] based on Eq. 1.

$$ \upeta = \upbeta_{1} {\text{x}}_{1} + \upbeta_{2} {\text{x}}_{2} + \ldots + \upbeta_{\text{n}} {{\rm x}}_{\text{n}} + \upvarepsilon $$
(1)

Where \( \upeta \) is the dependent variable; \( {\text{x}}_{\text{i}} \) are the independent variables; \( \upbeta_{\text{i}} \) are the weight of each independent variable and indicates the influence of these variables on the dependent variable; and finally \( \upvarepsilon \) is the bias based on a Gaussian distribution.

In this case, fracture time of the material is predicted based on 16 features that define the composition of the material, 4 features that define the steel heat treatment, and 2 features that define the creep test control.

3.3 Enhanced Linear Regression

Models based on simple linear regression algorithms can be improved applying gradient boosting [29, 30], combining the results of several simple linear regression models according to a cost function. This technique is call enhanced linear regression (LRB) and calculates and uses the obtained residuals from one model evaluation to train another model, that together make decrease the error according to a squared error loss function. This process is repeated until the training convergence is achieved [31]. The training stage tunes two significant parameters of the algorithm: the number of repetitions (mstop) and the shrinkage factor (shrink), to reduce a possible overtraining.

3.4 Generalized Linear Regression

Another applied technique is the generalized linear regression (GLM), which is based in a flexible generalization of the simple linear regression. In this technique, a linking function allows the optimization of the regression reducing the residual error but assuming different distributions [31,32,33,34,35,36]. Beforehand, the assumption of different distributions can built more accurate models in a not linear problem, like the case of short and long-term creep tests.

The application of the generalized regression algorithm is based on three components: a prediction based on a simple linear regression (Eq. 1); a linking function that is reversible and transforms the variables according to Eq. 2; and a random component based on the chosen distribution (Gaussian, binomial, gamma, etc.), that based on the independent features, specifies the possible distribution of the dependent feature, η.

$$ \upeta = {\text{g}}^{ - 1} \left( \upmu \right) = {\text{g}}^{ - 1} \left( {\upvarepsilon + \upbeta_{1} {\text{x}}_{1} + \upbeta_{2} {\text{x}}_{2} + \cdots + \upbeta_{\text{n}} {{\rm x}}_{\text{n}} } \right) $$
(2)

To apply this technique, it was assumed that the distribution of the original features of the dataset studied in this work were Gaussian, and that subsequently the three link functions shown in Table 1 were applied.

Table 1. Link functions that were applied on this technique
Full size table

3.5 Enhanced Generalized Linear Regression

As in the enhanced simple linear regression, models based on generalized linear regression technique can be improved using boosting techniques. In this case, L2 boosting technique [37, 38] was applied to maximize the accuracy minimizing the descending gradient function error. The boosted generalized linear regression algorithm (GLRB) [39,40,41,42] obtains the residuals in an iterative way according to a squared error loss function until the training convergence is achieved. Also, the training stage tunes two significant parameters of the algorithm: the number of repetitions (mstop) and the number of variables (based on the Akaike criterion), to reduce a possible overtraining and to simplify the model.

3.6 Validation Method

A validation method to obtain comparable results from the built models was performed. This method started with a normalization between 0 and 1 of the instances that formed the dataset. The benefits of this normalization were two: first that the range of the weight that each feature had in the output variable were equal in such a way that the impact of the features in the prediction could be observed in an easy way; and second that the final accuracy could be improved. Later, a training-testing simple validation method was conducted, where an 80% of the original dataset was randomly selected to build and train the models and the remaining 20% of the original dataset were used to test and validate the models. In the training stage, trying to avoid overtraining, 10-fold cross validation was performed. This cross validation allows having a more trustful error to make a preliminary selection of the obtained models from the parameters tuning. In this way, the most accurate models during training stage were selected to be tested and validated with the testing dataset. And finally based on these last results, the model with lower error in the testing stage was selected.

The work conducted in this study was performed using the statistical software tool R x64 v3.4.1 [43].

3.7 Accuracy Criteria

Models accuracy must be measured in order to evaluate the performance of the model prediction. There are several criteria to measure this accuracy when models predict a numeric feature. Some of the most used criteria are computational validation errors, that calculate the relation between the values predicted by the model and the real values measured during the performance of creep tests. In this case, the following criteria were applied:

  • Mean absolute error (MAE) (Eq. 3)

    $$ {\text{MAE}} = \frac{1}{n}\mathop \sum \limits_{k = 1}^{d} \left| {m_{k} - p_{k} } \right| $$
    (3)

  • Root mean squared error (RMSE) (Eq. 4)

    $$ {\text{RMSE}} = \sqrt {\frac{1}{\text{n}}\mathop \sum \limits_{{{\text{k}} = 1}}^{\text{d}} \left( {{\text{m}}_{\text{k}} - {\text{p}}_{\text{k}} } \right)^{2} } $$
    (4)

  • Correlation coefficient (CORR) (Eq. 5)

    $$ {\text{CORR}} = \frac{{\mathop \sum \nolimits_{{{\text{k}} = 1}}^{\text{d}} \frac{{\left( {{\text{p}}_{\text{k}} - {\overline{\text{p}}}} \right) \cdot \left( {{\text{m}}_{\text{k}} - {\overline{\text{m}}}} \right)}}{{{\text{n}} - 1}}}}{{\sqrt {\mathop \sum \nolimits_{{{\text{k}} = 1}}^{\text{d}} \frac{{\left( {{\text{p}}_{\text{k}} - {\overline{\text{p}}}} \right)^{2} }}{{{\text{n}} - 1}}} \cdot \mathop \sum \nolimits_{{{\text{k}} = 1}}^{\text{d}} \frac{{\left( {{\text{m}}_{\text{k}} - {\overline{\text{m}}}} \right)^{2} }}{{{\text{n}} - 1}}}} $$
    (5)

Where \( {\text{d}} \) is the number of instances used from the database to validate the model, \( {\text{m}} \) are the real values, \( {\text{p}} \) are the predicted values, \( \overline{\text{m}} = \frac{1}{\text{n}}\mathop \sum \nolimits_{{{\text{k}} = 1}}^{\text{d}} {{\rm m}}_{\text{k}} \) and \( \overline{\text{p}} = \frac{1}{\text{d}}\mathop \sum \nolimits_{{{\text{k}} = 1}}^{\text{d}} {{\rm p}}_{\text{k}} \).

4 Results and Discussion

The dataset of 9–12% Cr martensitic steels based on modified P92 standard specification was formed from a set of creep experiments obtaining from the related literature. The studied features and ranges were selected according to the proposed in the standard specification of the rule. Previously to apply the validation method, a multivariate analysis was performed on the dataset. Then, based on the results of the multivariate analysis the final dataset was set, and like it was commented previously the dataset was split in two new datasets, the training dataset and the testing dataset. Using the training dataset, the models were built applying 10-fold cross validation process combined with a tuning of the most significant parameters of each algorithm. This process allows to optimize the final accuracy that subsequently will be tested and validate using the testing dataset. One example of the results obtained during the training process, using cross validation and parameter tuning is shown in Fig. 2.

Fig. 2.
figure 2

RMSE obtained during the training stage for the boosted linear regression algorithm. Boosting iterations and shrinkage are tuned.

Full size image

After the analysis of this parameter tuning, the final results obtained during the training stage are listed in Table 2.

Table 2. Results obtained during training and testing stage
Full size table

After the selection of the parameter values to be used in each algorithm during the training stage to improve the accuracy criteria, the models were built. And then were tested with the 20% of the instances that were split previously from the initial dataset and were never used on the training stage. The validation with this testing dataset gives a better idea of the generalization of the problem since these instances were not used to built the models. The results obtained during the testing and validation stages are shown in Table 2.

In this case, the model that gets the most accurate results during the training stage gets the most accurate results during the testing stage. That means that a possible overfitting during the training stage is avoided using 10-fold cross validation, and also that the results obtaining during the validation indicate that the model based on GLM algorithm has an accurate generalization of the problem. That shows that the prediction of the fracture time on modified P92 steels is reliable according to the obtained error.

Then, the model based on the GLM algorithm gets the best performance during all the proposed stages. For this reason, the linear regression model that is defined in Eq. 6 can be considered like an accurate predictor of the material fracture life expectancy, with a RMSE of 6.75% and a correlation of 59.91% based on the testing stage results.

$$ \begin{aligned} {\text{g}}\left( {\text{TimeFractPoint}} \right) &=\, 27.0059 - 14.9172 \cdot {\text{C}} + 6.0532 \cdot {\text{Mn}} - 13.4758 \cdot {\text{Si}} \\ & + \, 13.6218 \cdot {\text{Cr}} - 8.7845 \cdot {\text{Ni}} - 15.7346 \cdot {\text{Mo}} + 4.7883 \cdot {\text{P}} \\ & - \, 2.6898 \cdot {\text{S}} - 11.5633 \cdot {\text{W}} + 1.0100 \cdot {\text{Nb}} + 2.7595 \cdot {\text{V}} \\ & -\, 7.8088 \cdot {\text{N}} - 9.5136 \cdot {\text{Al}} - 0.8844 \cdot {\text{B}} - 10.0950 \cdot {\text{Co}} \\ & -\, 0.8578 \cdot {\text{Cu}} - 1.8980 \cdot {\text{NormTemp}} - 6.5165 \cdot {\text{NormTime}} \\ & + \, 3.1603 \cdot {\text{TempTemp}} + 11.1225 \cdot {\text{TempTime}} - 13.4885 \\ & \cdot\, {\text{Temp}} - 17.4532 \cdot {\text{Strain}} \\ \end{aligned} $$
(6)

5 Conclusions

The results show that linear modeling techniques can predict some material properties, such as fracture time in creep conditions, with high accuracy. And not only a good prediction can be obtained, also and based on the join of multivariate data analysis techniques and linear model prediction a hidden knowledge of the process can be obtained. In this case, obtained models predict fracture time of 9–12% Cr martensitic steels with low errors of approximately a RMSE of 6%. Also the most influential variables in the process have been detected, and their weight in the process was determined. Therefore, it can be concluded that these techniques are useful in industrial problems, such as the one presented, and of significant help for developers of new materials in order to improve the final properties of the product.