Investigation of ANN Architecture for Predicting Load-Carrying Capacity of Castellated Steel Beams

Castellated steel beams (CSB) are an attractive option for the steel construction industry thanks to outstanding advantages, such as the ability to exceed large span, lightweight, and allowing flexible arrangement of the technical pipes through beams. In addition, the complex localized and global failures characterizing these structural members have led researchers to focus on the development of efficient design guidelines. )is paper aims to propose an artificial neural network (ANN) model with optimal architecture to predict the load-carrying capacity of CSB with a scheme of the simple beam bearing load located at the center of the beam.)e ANNmodel is built with 9 input variables, which are essential parameters equivalent to the geometrical properties and mechanical properties of the material, such as the overall depth of the castellated beam, the vertical projection of the inclined side of the opening, the web thickness, the flange width, the flange thickness, the width of web post at middepth, the horizontal projection of inclined side of the opening, the minimumweb yield stress, and the minimum flange yield stress.)e output variable is the load-carrying capacity of the CSB.With the optimal ANN architecture [9-1-1] containing one hidden layer, the performance of the ANN model is evaluated based on statistical criteria such as R, RMSE, and MAE. )e results show that the optimal ANN model is a highly effective predictor of the load-carrying capacity of the CSB with the best value of R � 0.989, RMSE� 3.328, and MAE� 2.620 for the testing part.)e ANNmodel seems to be the best algorithm of machine learning for predicting the CSB loadcarrying capacity.


Introduction
In modern construction, steel structures are used for abundant structures, including heavy industrial buildings, high-rise buildings, equipment support systems, infrastructure, bridges, towers, and racking systems [1]. e steel structure has numerous advantages such as large bearing capacity thanks to the high strength steel material, high reliability thanks to the uniform material, and the elastic and ductile capacity of steel, making it easy to transport and assemble [2]. Due to the high cost of steel, many structural engineers have worked hard to find ways to reduce costs for steel structures [3,4]. erefore, several solutions have been proposed to increase the rigidity or load capacity of the structure without increasing the weight of steel. Castellated steel beams (CSB) with web openings are among the first suggestions of these solutions [5]. is type of beam is made from wide flange I-beams, then cutting the belly plate in a zigzag line, welding the two halves on top of each other, and welding by vertical welding seams along the beam [3,4,6,7]. is increases the section height but does not increase weight compared to the original solid beam; thus, bending resistance characteristics such as the moment of inertia, section modulus, and the radius of inertia are higher, and the beam stiffness and flexural resistance of sections are enhanced [3,7,8]. erefore, the beam bearing capacity increases, the deflection is small, and the beam is able to exceed the large aperture [7].
However, the web openings will change the stress distribution on the bending sections [4]. Near the web openings, the stress distribution is quite complex, and stress concentration occurs. Moreover, due to the bending moment effect, this area is also subjected to the torsion force [9]. erefore, the critical loads of the structures are changed compared with the conventional beam [5]. So far, many experimental investigations, as well as numerical analyses, have been carried out to determine the behavior of the castellated beams under different loads. e beam failure is caused by numerous damage caused by overall bending, Vierendeel mechanism formation, welded joint rupture in the web, web post shear bucking, and web postcompression buckling [10][11][12][13]. e damage is affected by numerous factors, including geometrical dimensions of beams, loading type, and position, beam boundary conditions, material properties, as well as the distribution of residual stresses and geometric imperfections [14,15]. Konstantinos and Mello [16] investigated the CSB behavior with close web openings by studying seven experimental samples and fourteen numerical simulation samples. e aim is to study the effects of different hole shapes and sizes on the bearing capacity and critical load of the CSB. e CSB finite element models with hexagonal and circular web openings are developed and analyzed using ANSYS.
e results are compared with seven experiments test samples, thereby proposing an experimental formula to predict the load capacity of castellated beams. A numerical model studies the behavior of CSB with hexagonal and octagonal web openings up to failure developed by Soltani et al. [17]. e main purpose is to study and determine the instability in the position of web openings. In the work of Jamadar and Kumbhar [18], the authors used finite element models to determine the optimal hole size for CSB. In general, the numerical approach or laboratory experiments can only be applied to a limited number of cases, not enough to apply for general web openings beams. Furthermore, the cost of the experiment is high and requires a considerable amount of time [5]. erefore, it is necessary to develop an efficient and universal model to study the behavior of CSB or estimate the load-carrying capacity of the CSB.
In recent years, with the rapid development of artificial intelligence technology, machine learning algorithms have been popularized in all areas of life [19][20][21]. Among AI algorithms, ANN is currently effective algorithms to simulate complex technical problems [22,23]. ANN model is capable of solving complex, nonlinear problems, especially in problems where the relationship between the inputs and outputs cannot be established explicitly. An outstanding advantage of the neural network algorithm is the ability to self-study and adjust the weights. us, the calculation results are consistent without depending on mechanical equations, physical chemistry, or subjective opinion. Many complex problems related to structural engineering [24][25][26], geotechnical engineering [26][27][28], and materials science [29][30][31] have been successfully solved. In detail, Abdalla et al. [27] successfully predicted the minimum factor of safety against slope failure in clayey soils using the ANN model. e mechanical properties of FRP concrete are also predicted by the ANN model with high accuracy [32,33]. In the field of calculating steel beams using neural networks, a number of studies have been published, such as the study of Guzelbey et al. [34] and Fonseca et al. [35]. In these studies, a backpropagation (BP) neural network is used to predict the load-carrying capacity of steel beams. e results show that the BP network is more accurate than the numerical result, practical and fast, compared to the FE model. Recently, Amayreh and Saka [36] and Gholizadeh et al. [37] used ANN to predict load failure of the CSB. In the study of Amayreh and Saka, 47 experimental data are collected, the ANN model is built with 8 input parameters, the predicted results are compared with the Blodgett method, and BS Code shows the neural network provides an effective alternative to predicting the failure loads of CSB. In the investigation of Gholizadeh et al., 140 finite element models of the web post are analyzed with 7 input parameters related to geometry size. e BP networks and ANFIS are used to predict the load-carrying capacity of the CSB. e results show that the machine learning method provides better accuracy than the equations proposed in the document. Besides, methods such as genetic algorithm (GP) and an integrated search algorithm of genetic programming and simulated annealing (GSA) are also used to predict the load-carrying capacity of the CSB. e efficiency of the ANN model shows that it is an excellent choice to develop a numerical tool for engineers to predict the loading carrying capacity of CSB, which could help to reduce experimental time consumption and cost.
erefore, the main purpose of this investigation is to propose an efficient ANN model with a more general number of input parameters and to increase the accuracy in predicting the load-carrying capacity of the CSB.
In this work, the ANN model's performance will be studied to predict the load-carrying capacity of the CSB. One of the factors affecting the model performance is to finely determine the ANN architecture. erefore, the first goal of this work is to identify and optimize the ANN architecture to predict the load-carrying capacity of CSB. To achieve this goal, 500 simulations taking into account a data random sampling effect are performed for each model to verify the convergence and feasibility of the proposed model by Monte Carlo simulation (MSC). en, with the optimal ANN architecture, the performance evaluation of the model is performed based on three statistical criteria, including the Coefficient of Determination (R 2 ), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE).

Significance of the Research Study
Accurately predicting the load-carrying capacity of the CSB is of crucial importance because of many possible advantages and contributions to construction design. Available numerical or experimental approaches in the literature still face several limitations, for instance, the limitation of data (Amayreh and Saka [36] with 47 samples; Gholizadeh et al. [37] with 140 samples; Gandomi et al. [38] with 47 samples; and Aminian et al. [39] with 142 samples), accuracy evaluation of the ANN model, or comparison with different prediction results in the literature. ereby, the contribution of the present investigation could be highlighted via the following ideas:

Database Construction
To construct a model to predict the load capacity of the CSB, a database of 150 experimental data is collected, in which 140 beam samples are simulated from the validated finite element model introduced in document [37], and 10 beam samples are tested directly for bearing capacity published in document [13]. e beams are simulated and tested under concentrated loads placed in the center of the beam until failure. e basic parameters determining the beam failure load equivalent to the nine input variables and one output variable used in this study are as follows.
Seven parameters related to the size of the CSB are used: the overall depth of castellated beam (I 1 , mm); the vertical projection of inclined side of opening (I 2 , mm); the web thickness (I 3 , mm); the flange width (I 4 , mm); the flange thickness (I 5 , mm); the width of web post at mi depth (I 6 , mm); the horizontal projection of inclined side of the opening (I 7 , mm), where I 1 varies from 180.00 mm to 550.00 mm (mean of 335.92 mm and standard deviation of 99.60 mm), I 2 ranges from 50.00 mm to 250.00 mm (mean of 103.36 mm and standard deviation of 40.15 mm), I 3 has values between 2.00 mm and 5.00 mm (mean value is 3.65 mm and standard deviation 0.88 mm), I 4 ranges from 58.42 mm to 78.49 mm (the mean value is 68.39 mm and standard deviation of 3.54 mm), the value of I 5 is between 3.99 mm and 6.90 mm (the mean is 5.34 mm and the standard deviation is 1.03 mm), I 6 ranges from 30.00 mm to 95.00 mm (mean 53.75 mm and standard deviation 19.26 mm), and I 7 ranges from 30.00 mm to 149.35 mm (mean value is 59.23 mm and the standard deviation is 22.75 mm). e remaining two parameters are related to the mechanical properties of the material, including the minimum web yield stress (I 8 , MPa) and the minimum flange yield stress (I 9 , MPa). For I 8 , the value varies from 311.65 MPa to 374.40 MPa (mean is 351.36 mm and the standard deviation is 7.12 mm), and I 9 ranges from 307.52 MPa to 383.54 Mpa (mean is 350.97 mm and the standard deviation is 8.25 mm). e detailed statistical information of these variables is shown in Table 1. e geometry and dimensions of the CSB are shown in Figure 1. e dataset is randomly divided into two subsets using a uniform distribution, in which 70% of the data (corresponding to 105 data) is used to train ANN models, and 30% of the remaining data (corresponding to 45 data) is used for model verification. is means that the control data (30%) is entirely unknown to the ANN model. erefore, the forecasting capacity of the ANN model can be assessed objectively and most accurately through the verification section. All data are normalized to the range of [0, 1] for reducing the number of errors in processing by ANN, according to the recommendations of [40].
is process ensures that the training phase of ANN models can be carried out with functional generalization capabilities. Such proportions are expressed using the following equation: where λ and μ are the minimum and maximum values of given variables and χ is the value of the variable to be scaled. In addition, a correlation analysis between the input and output parameters is performed and shown in Figure 2. Figure 2 is created to find a linear statistical correlation between parameters in the database. erefore, a 10 × 10 matrix is established, where the upper triangle represents the values of the correlation coefficient, while the lower triangle shows a scatter plot between the two related variables. e diagonal of the matrix indicates the name of the parameter. e maximum value of the correlation coefficient (R) compared to Y is calculated by 0.52 (for variable I 3 ), followed by 0.39 (for variable I 2 ), 0.28 (for variable I 4 ), 0.24 (for variable I 7 ), 0.23 (for variable I 8 ), 0.21 (for variable I 9 ), 0.12 (for variable I 6 ), 0.09 (for variable I 5 ), and 0.04 (for variable I 1 ).

Artificial Neural Network.
e ANN artificial neural network is a mathematical and statistical model based on the working mechanism of the biological nervous system [41]. ANN does not attempt to simulate the delicate workings of the brain, but they try to replicate the logical activity of the brain by gathering a lot of input in the form of neurons to perform computational or cognitive processes. e purpose of ANN is to define the relationship between the input parameters and the output parameters of the model. However, ANN only uses datasets without prespecifying the math functions that determine the relationship between the input and output parameters of the model. is is an effective soft computation method to solve too complex problems compared to classical mathematics and traditional methods [42].
In this study, a feedforward neural network trained by the backpropagation algorithm is used [43].
is neural network is made up of a series of processing elements, which can be called neurons or nodes. ese neurons are partially or wholly connected through weights (w ji ) and are divided into 3 layers: input layer, output layer, and hidden layers.
During the learning process, the backpropagation algorithm uses the gradient descent search method to adjust the connection weight. e learning process starts from the input data (the input parameter vectors are entered into the Complexity 3   4 Complexity input layer's neurons). At the j hidden layer neuron, the signal value received from the input layer will be composed of a total input value according to the following formula: in which x i are the input parameters and weights (w ji ) and bias (θ j ) will be randomly generated. e pass function will then be used to calculate the output value using the following formula: (3) is output value again serves as the input to the next layer neuron. As it continues, this value is passed to the neuron in the output layer. For a hidden single-layer network, this step will move to reverse propagation. e difference between the output value (y j ) and the actual value (t j ) is called the cost function, calculated as follows: From the cost function, compute the weight derivative of the entered and hidden classes. From there, adjust the weights and bias variables to make the predicted output of the network closer to expected: where w ji (new), θ j (new)are weight value and bias value after adjustment; w ji (old), θ j (old) are weight value and previous bias values; and η is the learning rate. Learning speed is the optimal algorithm parameter (gradient descent). If this parameter is small, it will take many iterations for the function to reach its minimum. Conversely, if this parameter is large, the algorithm will need fewer iterations, but then it is possible that the function will ignore the minima and cannot converge.
To overcome the weights (w ji ) and bias values (θ j ) of the next iteration step that do not fall into a local minimum point, the momentum algorithm is used [44]. is algorithm calculates the amount of change of the variables at time t (v t ) to update the new value: where c is the momentum term.
With multiple hidden layers, the algorithm formulas perform the same steps. After the learning process, the model will be verified by an independent testing database.

Performance Criteria.
During the training of ANN models, it is necessary to quantify the performance of the model to be able to repeat the hyperparameter adjustment to choose the best model possible. Standard quantitative performance measures for a regression model include the Coefficient of Determination (R 2 ), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE) [45,46]. e R 2 criterion is widely used in regression problems to estimate the correlation between the actual value and the predicted results [47]. e value of R 2 is in the range [0; 1]. In addition, RMSE and MAE measure the mean error between actual and predicted outputs [48]. Quantitatively, RMSE and MAE values are closer to 0, and the closer the value of R 2 is to 1, the more accurate the machine learning model is. e following equations represent these values: in which P j is the actual j th output, P j is the predicted j th output, P is the average of the P j , and N is the number of the samples.

Methodology Flow Chart
e methodology of developing artificial neural networks to predict the load-carrying capacity of the CSB includes four primary steps as follows (as shown in Figure 3): Step 1. Database preparation: in this step, a database of 140 finite element simulation beam samples and 10 direct test beam samples is collected to build the ANN model. e basic parameters to predict the load-carrying capacity of the CSB include 9 input variables divided into two groups of variables: the geometric size group and the physical properties of the material. e dataset is randomly divided into two parts, of which 70% of the data is used to train the ANN model and the remaining 30% is used to validate the built model.
Step 2. Determination of the optimum of ANN architecture: in this step, the construction of the optimum of ANN architecture based on the training dataset is carried out. e criteria used to validate the optimal ANN model include R 2 , RMSE, and MAE.
Step 3. Training the optimal model: in this step, the ANN model with the optimum architecture is trained using the training dataset.
Step 4. Validating the model: in this step, the testing dataset is used to test and confirm the trained ANN model. e performance of the ANN model is evaluated by statistical criteria: R 2 , RMSE, and MAE.

Investigation of Model Convergence.
In this section, the determination of the ANN architecture and optimization of the ANN parameters are performed using the gradient descent algorithm. Parameters of the ANN model used in this study are given in Table 2, in which there are eight fixed parameters, and two parameters are subjected to a parametric study, namely, the number of hidden layers and the number of neurons in each hidden layer. In general, choosing the number of hidden layers and neurons in each layer is a trialand-error test, needed to find the best configuration of the network [23]. In this study, ANN models containing one and two hidden layers are analyzed and tested. e number of neurons in each hidden layer is varied from 1 to 15. Regarding each network topology, the network training process is performed. In essence, network training is the process of adjusting the link weights and biases. ese weight values are randomly taken initially, and then the network algorithm adjusts the values during the training phase. To build the network with the highest accuracy, optimization is performed with 1000 epochs to adjust the weights for each given ANN structure. In addition, in order to generalize the ANN model, 500 simulations are performed for each of the analyzed ANN structures. A total of 120,000 simulations are performed, corresponding to 15 architectures for one hidden layer and 225 architectures for two hidden layers. e optimization process for the training and testing parts is evaluated by three statistical criteria, namely, R 2 , RMSE and MAE, presented above and shown in Figure 4. Specifically, Figures 4(a), 4(c), and 4(e) represent the values of the three criteria for the training part, while Figures 4(b), 4(d), and 4(f) represent the testing part. anks to MCS, Figure 4 shows that, with 100 simulations, the values of the three criteria converge in about 10% of the corresponding average values. When the number of simulations increases to 400, the convergence of the ANN model improves (i.e., lower than 5% of the corresponding average values). erefore, it could be stated that the simulations performed with 500 runs give reliable results. Overall, all investigations and results in the next sections are given by averaging the results of 500 simulations for each ANN structure.

Prediction Performance of Different ANN Architectures.
In this section, the performance of different ANN architectures is presented to find the best ANN architecture. e performance evaluation is calculated for both the training and the testing parts by the mean and standard deviation values (StD) of the three statistical criteria (i.e., R 2 , RMSE, and MAE). Figures 5(a), 5(b), and 5(c) depict the mean   Figure 5, it worth noting that the first edge of R 2 , RMSE, and MAE mean value curve is the performance of the ANN model containing 1 hidden layer, with the neuron number varying from 1 to 15. e second edge corresponds to the ANN architecture with 2 hidden layers. e first point of the second edge starts with the performance of the ANN model containing 1 neuron for the first hidden layer and 1 neuron for the second hidden layer or ANN architecture [9-1-1-1]. Each point of the remaining edges corresponds to the performance of ANN architecture containing 1 neuron in the first hidden layer, and the following points are results with 1 to 15 neurons in the second hidden layer. It means that the last point of the second edge corresponds to the performance of ANN architecture [9-1-15-1]. Overall, one edge is shown for ANN architecture containing 1 hidden layer, and 15 edges are shown for ANN architecture containing 2 hidden layers ( Figure 5). Precisely, the 225 performance values of different ANN architectures can also be shown by color-map in Figures 6 and 7 for the training part and testing part, respectively. Besides, Figures 6 and 7 also show the StD values of R 2 , RMSE, and MAE. Figures 6(a) and  7(a) show that the value of R 2 is relatively greater than 0.9 and 0.8 for the training and testing dataset, respectively, for ANN architectures with more than 3 neurons in the first hidden layer. Similar observations are remarked with a specific zone with low values of RMSE and MAE for both the training and testing parts (Figures 6(c), 6(e), 7(c), and 7(e)).
e performance values presented in Figure 5 show that the highest mean value of R 2 � 0.923 and the lowest mean values of RMSE � 7.225, MAE � 5.047 for the testing part, correspond to the ANN architecture containing 1 hidden layer [9-1-1]. erefore, this ANN architecture has the best performance.    e lowest performance is also shown by the highest StD values of R 2 , RMSE, and MAE, which means that the ANN architecture [9-1-15-1] is not stable to estimate a reliable result. Besides, with a higher neuron number of the second hidden layer, the performance of the ANN model decreases with decreasing R 2 values and increasing RMSE, MAE, and StD values. e best architecture of 2 hidden layers corresponds to the case [9-11-1-1] with performance through the mean value of R 2 , RMSE, and MAE of 0.914, 7.883, and 5.233, respectively. However, these values are lower than those of the ANN architecture [9-1-1]. erefore, such an architecture is used to predict the load-carrying capacity of CSB in the next section.

Load-Carrying Capacity Prediction of Best ANN
Architecture.
e optimal architecture of the ANN model is [9-1-1] applied for this section. In this section, the predictive capacity of the best performance [9-1-1] of the ANN model is presented. Specifically, the ANN architecture's prediction results [9-1-1] with the highest predictive capacity among 500 simulations are presented. e comparison between experimental critical load and predicted value by the ANN model is shown in Figure 8 for the training and testing parts. e comparison shows that the predicted values are very close to the experimental ones. e model errors are plotted between the predicted and the experimental values for the training (Figure 9(a)) and the testing parts (Figure 9 Finally, a regression model in Figures 10(a) and 10(b) shows the correlation between the actual and predicted values for the training and testing datasets, respectively. A linear fit is also applied and plotted in each case. It is observed that the linear regression lines are very close to the diagonal lines, which confirms the close correlation between the actual and predicted load-carrying capacity. e calculated values of R 2 for the training dataset and the testing dataset are 0. 959    For the sake of comparison, Table 3 shows the results of this investigation compared with different results available in the literature. For the database, the number of samples in     this investigation is the highest with 150 samples. e number of inputs for the ANN model is 9 inputs. Overall, the best ANN architecture in the present study is able to predict the load-carrying capacity of CSB with the highest accuracy, comparing with the other investigations.

Conclusion
is investigation aims to develop a simple but effective ANN model to predict the load-carrying capacity of CSB. To achieve this purpose, the determination of optimal ANN architecture is carried out, with two cases of hidden layers number varying from 1 to 2. Regarding each case, the neuron number in each hidden layer is varied from 1 to 15. Overall, 240 cases of ANN architectures consisting of 15 cases of 1 hidden layer and 225 cases of 2 hidden layers are proposed. Based on 150 data collected from published studies, 70% of the data are randomly selected and used for the training dataset, whereas the remaining 30% are selected for the testing dataset. A number of 500 simulations are performed for each ANN architecture. e performance of each ANN architecture is evaluated by commonly used statistical criteria, such as Determination Coefficient (R 2 ), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE). e ANN architecture containing 1 hidden layer and 1 neuron is found as the best structure for predicting the load-carrying capacity of the CSB, with excellent agreement between model and experimental results (i.e., values of R 2 , RMSE, and MAE are 0.989, 3.328, and 2.622, resp., for the testing dataset). e results of this investigation can help build a reliable soft computation tool to accurately and quickly predict the load-carrying capacity of CSB. It is important noticing that a parametric design-oriented study of CSB could be conducted in future works, thanks to the excellent accuracy of the proposed ANN model. In this case, Partial Dependence Plots analysis or parametric studies on the geometry parameters could be used to design CSB with targeted load-carrying capacity.
Data Availability e data supporting this manuscript are from previously reported studies and datasets, which have been cited. e processed data are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.