Performance Prediction of Fine-Grained Asphalt Concretes with Different Quarry Fillers by Machine Learning Approaches

In general terms, an artificial neural network is a distributed processor that consists of elementary computational units interconnected. Such structure is inspired by the functioning principles of the biological nervous system and has proven to be effective in identifying complex relationships between an assigned input features vector and an experimental- investigated target vector for any scientific problem. The current paper represents a forward feasibility study on predicting the mechanical response of asphalt concretes prepared with different quarry fillers used as alternatives for traditional limestone filler or portland cement by Machine Learning approaches which consider the chemical properties of the selected fillers and the quarry aggregate types as input variables. In fact, the case study involved several fillers and stone aggregates that were used to produce Marshall specimens of a specific fine-grained asphalt concretes designed originally for the assessment of filler suitability in terms of adhesion phenomenon. The asphalt concrete variants had the same material composition and mix design: all specimens were compacted by 2x50 blows using impact compactor, filler content was fixed at 10% by mass of the mix, the grading curve is roughly the same, the employed bitumen has a 160/220 penetration grade and is about 6% by mass of the mix. The mineralogical composition was investigated by X-ray fluorescence spectrophotometry tests. It represents a non-destructive laboratory analysis that allowed to specify and compare the main oxides composition associated with the employed natural fillers to be identified. Based on the results thus obtained and applying a categorical variable that distinguishes the stone aggregate type, a neural model has been developed that can predict the stiffness modulus of asphalt mixtures: therefore, this study presents a possible procedure for the development of predictive models that can help or improve the mix design process, when different fillers and stone aggregates are available.


Introduction
The laboratory tests that are typically performed to evaluate physical and mechanical properties of mixtures are often both economically and time-consuming. It is also necessary to have laboratory technicians skilled in performing the tests in order to minimize uncertainties and/or errors in the measurements. In addition, if any of the several components of the asphalt mixture needed to be varied, additional experiments would need to be conducted to identify the different mechanical response. For this reason, a mathematical model that takes into account the different features would IOP Publishing doi:10.1088/1757-899X/1203/2/022113 2 save both time and money by going directly to the variation of the single parameter to be investigated. Over the years, several mathematical approaches have been followed to pursue these goals: above all, the use of constitutive equations [1] often implemented on computational platforms within finite element software [2].
Recently, however, some soft-computing techniques such as Artificial Neural Networks (ANNs) have been increasingly employed to solve this kind of problems. By their nature, in fact, ANNs try to find a reliable analytical relationship between input and output variables, regardless the problem of the physical nature. The main purpose of this work is to provide the analytical expressions for the prediction of a mechanical parameter, such as the Stiffness Modulus (stiffness), starting from the modeling by neural networks of selected fillers chemical properties and the different origin quarries of the aggregate used in the mix design process. A detailed description of the operating algorithms of such mathematical models will be provided to allow a full understanding of the neural modeling performed.

Materials
Aggregates from the Zbečno, Brant, and Chlum quarries were chosen for assessment of adhesion because of their availability in the Czech Republic and regular use for HMA production, but also because of their surface properties and petrographic composition of the parent rock. This selection includes aggregates exhibiting diverse adhesion to bituminous binders. The host rock in Zbečno quarry is igneous. From the petrographic point of view, it is a spilite which contains plagioclase strips (andesine) and isometric grains of pyroxenes. Secondary veins with quartz, calcite, chlorite or pumpellyite are abundant. Some spilites contain up to 3 mm of feldspar growth (spilite porphyrites). Zbečno aggregate usually exhibits good bitumen-aggregate adhesion. The aggregate from the Brant quarry contains granite porphyry as the key mineral. Despite its porous surface due to weathering, it is also hydrophilic and therefore susceptible to stripping. The aggregate from the Chlum quarry can be classified as acid leachate (phonolite). Feldspars cannot be detected macroscopically, biotite may be present in small amounts. In general, the aggregates produced from this rock are classified as hydrophilic, exhibiting poor bitumen aggregate adhesion; therefore, suitable adhesion promoters are usually required in the mix design or, if possible, this aggregate is avoided. Soft paving grade bitumen 160/220 with a penetration of 187 dmm and a softening point of 38 °C was used in this study. The use of this type of binder is required by the test protocol given in EN 1744-4, Annex A. This annex also exactly defines the composition of the grading curve which has to be represented by 25 % of 5/8 mm, 25 % of 2/5 mm, 40 % of 0,125/2 mm and 10 % of filler. The bitumen content is optimized in the way to reach for a reference mixture voids content of 5,5±0,5 %-vol. It was necessary to define this for each used aggregate type and use of referential limestone filler (quarry Velké Hydčice). The bitumen content which was fixed for the reference mix was later used for all alternatives where different fillers were applied. For alternative fillers which were used as substitutes to typical limestone filler, several options were selected representing different quarries or in two cases representing asphalt mixing plants where during aggregate heating the released dust is captured. This material is known in asphalt mix production as backhouse filler. More detailed data about the fillers can be found in Valentin et al. [3].
2.1. Spectrometry test X-ray fluorescence (XRF) spectrometry was used to determine percentages of most important oxides in filler samples with focus mainly on SiO2 and CaO. It is established that if the sample contains more than 65% of SiO2, the rock is of acidic origin and usually hydrophilic. On the other hand, presence of CaO indicates that the material is hydrophobic. An ARL QUANT'X EDXRF Spectrometer (Thermo Scientific, USA) equipped with a Rh X-Ray tube and Si(Li) detector crystal was used. The XRF spectrometry data were collected and evaluated using the UniQuant ED 6.32 software (Thermo Scientific, USA). Using this equipment, a relative accuracy ranges between 0.5% and 5.0%, depending on the quantity and concentration of analyzed substances. The studied filler samples were IOP Publishing doi:10.1088/1757-899X/1203/2/022113 3 automatically assessed in helium atmosphere at 25 °C in the whole spectral range. The Table 1 provides the XRF results showing the most significant oxides found in the samples of quarry dusts or backhouse fillers. Based on these oxides, ratios between these oxides (always related to SiO2) were calculated and used later for the machine learning task and modelling.

Asphalt mix and test specimen preparation
For the laboratory asphalt mix production, the bitumen 160/220 was heated to 140°C, the aggregates and fillers to 150°C. The mixing was done by using an Infratest 30 l laboratory mixer. Each asphalt mix was in total mixed for 360s according to the predefined sequence. Afterwards the hot mix was used to produce six Marshall test specimens compacted at 140°C by 2x25 blows (used for ITSR determination), six Marshall test specimens compacted at 140°C by 2x50 blows (used for Marshall test and stiffness determination) and about 1 kg was used for determining maximum density according to EN 12697-5. The six test specimens with lower compaction energy were later used for the water susceptibility test according to EN 12697-12. This test protocol is based on determining a ratio between indirect tensile strength (ITS) values of dry test specimens and those which were water immersed (conditioned in water bath at 40±1°C for 70±2h). The indirect tensile strength was determined according to EN 12697-23 at 15 °C, whereas the test specimens were conditioned at the set temperature for 4 hours prior to testing. Stiffness was tested according to EN 12697-26, using the test method IT-CY (non-destructive test by applying repeated indirect tensile stress) at test temperature of 15 °C which was selected because it is the determining temperature for the strain characteristic used in the Czech pavement design method. After testing stiffness three of the test specimens for each mixture were water immersed. The conditioning according to EN 1744-4, annex A shall be in water bath at 40±1°C for 48h. This standard further defines the possibility to determine volume changes of the water conditioned specimens as well as the loss of Marshall stability (SMA), which is given by the following formula: where SM is the Marshall stability of dry (unconditioned) test specimens (kN) and SMQ is the Marshall stability of water immersed (conditioned) test specimens (kN). The unconditioned as well as water immersed test specimens were then conditioned for 45-50 minutes in a water bath at 60 °C and after this conditioning time immediately tested for Marshall stability according to EN 12697-34. Differently form EN 1744-4, annex A only loss of Marshall stability was determined, the volume changes of immersed test specimens were not measured. On the other hand, additionally to loss of Marshall stability the Marshall stiffness ratio (MTR) was determined according to the following formula: where MTdry is the Marshall stiffness of dry (unconditioned) test specimens (kN/mm) and MTw is the Marshall stiffness of water immersed (conditioned) test specimens (kN/mm). Table 2 reports the mechanical properties of asphalt mix series with different quarry aggregates. It is obvious that the selected alternative fillers can have impact on voids content. The reference mixture containing limestone filler with Zbečno aggregates had a void content of 5.18% by vol. The bitumen content was 6.0% by mass. The statement about potential impact of tested fillers on voids content is based on the results true mainly for backhouse filler PKB and quarry dust from Chornice. For the fixed aggregate mix design this would mean that the bitumen content needs to be slightly increased. On the other hand, for backhouse filler Brant and quarry dusts Chrtníky or Litice the bitumen content could be slightly reduced. The reference mixture containing limestone filler with Brant aggregates had a void content of 5.01% by vol. The bitumen content was 6.4% by mass. The statement about potential impact of tested fillers on voids content is based on the results true for quarry dusts from Plešovice, Litice, Chornice and Chrtníky, whereas the most significant impact was found for first three of these alternative fillers. For the fixed aggregate mix design this would mean that if filler Litice is used the bitumen content could by slightly reduced and on the other hand with filler Plešovice or Chornice the bitumen content needs to be slightly increased. The reference mixture containing limestone filler with Chlum aggregates had a void content of 5.33% by vol. The bitumen content was 6.3% by mass. Only quarry dust form Plešovice resulted in similar voids content. Backhouse fillers Brant and PKB showed higher voids content which would result in the probably demand for slightly increased bitumen content to reach same air voids like for the asphalt mix variant with limestone filler. On the other hand, fillers from Litice, Chrtníky and Chornice resulted in

Artificial Neural Model
An Artificial Neural Network (ANN) is an interconnected set of processing units whose operation is inspired by the human biological nervous system. The network's capability to perform a specific task is related to the neural interconnections' number and strength; the latter is expressed by numerical values, called weights, which are obtained using a learning process. The artificial neurons are inputted with a weighted combination of signals (input features) which is processed according to a nonlinear function, called activation function. A neural network allows the existing (or postulated) relationship between an assigned input features vector and an experimental-investigated output to be regressed. In functions approximation problems such as the current paper object, the input features vector components are the independent variables scores and each corresponding output is the ground-truth dependent variable. The "supervised" learning is based on a training set of input-output pairs, randomly taken from the original data set. A well-trained network is capable of generalizing, i.e., to give the proper output from input features values not considered in the training set.

Spearman Correlation
In order to optimize the data provided as input to the model, correlation analyses were implemented to determine which classes of data were most related. The relationships were investigated using Spearman's Rank Order Correlation. The only assumptions for this kind of non-parametric technique are random samples and independent observations. It has been observed that there was an overall medium positive correlation [4] between the ITSM and the most important oxides ratio in filler samples. For example, between ITSM and SiO2/CaO the relationship was [rho = + .38, n = 126, p<0.0005] with high levels of ITSM associated with higher levels of the SiO2/CaO ratio. A comprehensive description of the rho and p values for each ITSM/oxide ratio pair is shown in Table 4.  The neural network proposed in the current paper is structured as follows. An input layer is composed of 5 neurons, one for each oxides' ratio (SiO2/CaO, SiO2/Al2O3, SiO2/Fe2O3, SiO2/MgO) plus the categorical variable identifying the quarry/filler pair. A hidden layer is equipped with n neurons, with n being an integer in the range {1, …, 25}. The activation units considered in the current study setup are the 4 main-used (in regression problems) functions (Figure 1) namely the exponential linear (ELU), the rectified linear (ReLU), the hyperbolic tangent (TanH) and the logistic sigmoid (LogS) [5]. An output layer with a single neuron and linear activation function produces the Stiffness Modulus outputs corresponding to each input features vector. Such NN structure is often called Shallow Neural Network (SNN) in the relevant literature.

Training Algorithm
Firstly, the data have to be standardized (subtracted from their mean and divided by their standard deviation) for a better functioning of the training algorithm. The latter consists of an initial forward and a following backward passes [6]. First the input feature vector is used to compute the output ̂, by means of an initialized weights and biases matrix W. Subsequently, the target vector is compared with the network output ̂ by a loss function (̂, ) that allows the matrix W to be updated. The loss function, employed in the current study, is the Mean Squared Error (MSE) i.e., the average squared difference between the estimated and the target values. The optimization method applied to update the weights and biases matrix is the Levenberg-Marquardt (LM) algorithm [7] which is one of the most efficient algorithms to train SNNs. After randomly initializing the weights and biases, the training phase can start and it can be expressed as follows: where ∈ {1, ⋯ , }, is the Jacobian matrix of the training loss (⋅) with respect to , is the identity matrix and =̂− is the vector of network errors. The learning rate and every other parameter of the training algorithm were set by the MATLAB®Toolbox, i.e., equal to 0.001, to 10, to 0.1, to 1e+10 and the maximum number of training epochs E equal to 1000. An early-stopping procedure has been implemented in order to avoid overfitting. Such a situation happens when a neural model fits too much the training data and then it is incapable of well-generalizing. For IOP Publishing doi:10.1088/1757-899X/1203/2/022113 8 this reason, the dataset has been divided into 3 macro groups: the training set, the validation set and the test set. The training set allows the matrix W to be properly set, so that the network can perform a specific task, whereas the validation set is used for monitoring the model generalization capability during training. As the training iterations progress, if the validation error increases with respect to the training error, it means that an overfitting situation is occurring. For this reason, a parameter δ is implemented so that, if the above condition occurs δ-times subsequently, the learning phase stops. δ has been set equal to 6. Finally, the test set is used to evaluate the model's predictive capabilities.

Data Augmentation and k-fold cross-validation
Due to the small size of the available dataset (126 input/output pairs), a data augmentation technique and a data partitioning algorithm (known as k-fold cross-validation) have been employed. The first one consists in increasing the amount of data in the learning phase, adding synthetic data that do not change the intrinsic meaning of the original data set from experimental investigations. For this reason, for each quarry and filler types, the average values of the stiffness modulus have been calculated. These values, 21 in total, form the validation set: this process is really important because in this way a representative dataset of all the mixtures considered in the study has been identified. Such a validation set allows the Machine Learning engineer to stop the training phase when the SNN stops correctly regressing the average mechanical response of the mixes. For this reason, the implemented data augmentation process is called "Mean Validation" and is a useful reference for all those applications where multiple replicates of different asphalt mixtures are available. Differently, the k-fold crossvalidation has been implemented for the definition of the training and test sets. This pre-processing step is based on the integer parameter k, that sets the number of partitions of the experimental data set. Each of the k-groups contains the same number of elements: in this way, k-1 groups can be used to train the model, whereas the left-out one to test it. Such procedure is iterative and is repeated k-times so that each of the k-folds is used in a different testing phase [8]. For each iteration, it is necessary to keep a record of the individual test score to obtain the model's overall performance as the arithmetic mean of the k-scores [9]. The value of k was chosen equal to 6 and the method was of a stratified type, i.e., it was ensured that each test set contains one input-target data for each quarry and filler combination. In this way, it was possible to generate 3 sets (for 6 different data partitions) that were all well-representative of the 21 quarry/filler pairs: the 105-data training set, the 21-data validation set (from data augmentation) and the 21-data test set. In order to obtain the best fitted network for each combination of activation function/number of hidden neurons as a result of a k-fold cross-validation partitioning, the parameter was defined: it is the number of re-trains performed for each considered NN structure. A standard literature value = 10 has been employed. Table 5 shows the satisfactory results achieved by the different proposed SNNs expressed through the Ordinary Coefficient of Determination (R 2 ) and the Mean Squared Error (MSE) values. The former represents a statistical measure of how the data are well approximated by the regression line. In general, the value of R 2 always lies between 0 and 1, and the higher it is, the better the model fits the data. The latter represents, instead, an estimate of the generalizing capabilities of the model IOP Publishing doi:10.1088/1757-899X/1203/2/022113 9 through an average of the differences between the experimental investigated modules and those predicted by the SNN squared. SSE is the sum of squared error while SST is the sum of squared total, is the average of the stiffness modules investigated and n represents the number of observations. The most satisfactory result is achieved for the 5-4-1 model using the logistic sigmoid activation function, hereafter referred as LogS-SNN, with a R 2 value of 0.9473 and a MSE value of 0.0445. The graphical trend of the training process is presented in Figure 2 (a). It is possible to notice how the initial values of R 2 (MSE) grow (decrease) until they reach a maximum (minimum) beyond which improvements are no longer appreciable. In this case it can be deduced that using more than 4 neurons in the hidden layer would only make the model more complex without providing any predictive benefit. The Data-Augmentation strategy of averaging the modulus values for each quarry/filler pair to constitute the validation set was successful. In fact, as it is possible to observe in Figure 2 (b), the model is well capable of predicting the average mechanical behavior of each type of mixture. As can be seen in Figure 2 (c), the LogS-IOP Publishing doi:10.1088/1757-899X/1203/2/022113 10 SNN model was then finally trained on the entire dataset so that it could be employed in the future and returned very high values of Pearson Correlation Coefficient R. In fact, the result obtained (R = 0.9743) indicates a very good linear correlation between the target moduli and those predicted by the neural model, emphasizing its high level of accuracy.

Conclusions
In the present study, a SNN approach has been used to numerically model the stiffness modulus of several asphalt concretes prepared with different quarry fillers used as alternatives for traditional limestone filler or portland cement. To produce the specimens, aggregates from 3 different quarries were employed together with 7 different fillers thus generating 21 quarry/filler pairs. For each pair, 6 different samples were produced that had essentially the same compositional features: they were all compacted by 2x50 blows, filler content was fixed at 10% by mass, the grading curve was roughly the same and the employed bitumen had 160/220 penetration grade and was about 6% by mass of the mix. XRF analyses were performed thus identifying the content of the 5 main oxides associated with the employed natural fillers. The proposed neural model, starting from the ratios between these main oxides (always related to SiO2) and from a categorical variable associated to the quarry/filler pair, is able to predict with high accuracy (guaranteed by the very high R-value) the average mechanical behavior of the mixture in terms of stiffness modulus. It is capable, in fact, of approximating an analytical relationship between the oxide ratios and the stiffness modulus thanks to its 5-4-1 structure characterized by a logistic sigmoid activation function which the Hidden Layer is equipped with. This work is clearly a preliminary study since it implements the MATLAB®Toolbox and uses its standard hyperparameters. For the future, it would be interesting to implement hyperparameters optimization algorithms in order to obtain even more reliable results and to extend the range of compositional features. In fact, by investigating mixtures characterized by different grading curves and different filler/bitumen/voids ratios, it would be possible to create a powerful tool for predicting mechanical performance which would be of great help during the mix design processes.