Improved deep learning based multi‐index comprehensive evaluation system for the research on accuracy and real‐time performance of coal gangue particles recognition

The problems of insufficient recognition accuracy, poor real‐time performance and lack of consideration of actual working conditions in the process of intelligent construction of coal mines make this technology still in the research stage and not applied in practical engineering. The purpose of this paper is to establish an accurate and real‐time recognition model, which can quickly distinguish the vibration acceleration signals of coal and gangue under the influence of external factors such as impact position, velocity, and direction by using the different physical properties of coal and gangue particles. Therefore, the accuracy and real‐time of coal gangue recognition model established by different convolutional neural networks (CNN) structures and different position signal input are studied. First, to meet the real‐time requirements, an original CNN recognition model composed of single convolution layer and single pooling layer is established, and the data collected by seven sensors are input in the form of two‐dimensional matrix. However, the stability of the training and test results is insufficient. To solve this problem, once improved CNN (OI‐CNN) recognition model with multiconvolution layers and multipooling layers is built by deepening the network. The experimental results show that the stability and accuracy are improved, but the real‐time performance is poor. Furthermore, through parameter adjustment, the OI‐CNN is changed to the twice improved CNN (TI‐CNN), and the sensor data at different positions are input in the form of one‐dimensional vectors. The results show that the accuracy and real‐time performance of the TI‐CNN coal gangue recognition model are further improved. Finally, according to the research purpose of this paper, the weights of CNN indexes are given, and a multi‐index comprehensive evaluation system (MICES) is established. With the original CNN recognition model as the control, the OI‐CNN recognition model and the TI‐CNN recognition model at different positions are quantitatively compared to obtain the comprehensive evaluation scores of each model. The results show that the MICES of the coal gangue recognition model established based on the TI‐CNN structure and the data input of a single position sensor is the highest, while the sensor position has little effect on the recognition results.

evaluation scores of each model.The results show that the MICES of the coal gangue recognition model established based on the TI-CNN structure and the data input of a single position sensor is the highest, while the sensor position has little effect on the recognition results.

| INTRODUCTION
China's resource endowment of rich coal, short of oil, and little gas determines the dominant position of coal in the energy structure.Top coal caving is one of the commonly used coal mining methods in China, which is suitable for the mining of thick and extra-thick coal seams.It has the advantages of high yield, low cost, and low tunneling rate.In recent years, with the development of intelligent mining industry, the automatic recognition of coal gangue interface in top coal caving has become one of the urgent problems in this field.At present, this work is still in the stage of artificial discrimination.The environment of heading face is harsh and the voice is noisy.The workers are often misjudged by the influence of subjective consciousness, resulting in coal seam under discharge or over discharge.When the coal seam is under discharged and the recovery rate is insufficient, it will cause waste of resources and a large number of broken coal remains in the goaf.Broken coal is prone to spontaneous combustion, which leads to fire and even methane explosion. 1If the coal seam is over discharged and the gangue content increases, the purity of coal will decreases, resulting in insufficient combustion and prone to harmful gases.The mined-out area also increases, easy to collapse.Therefore, an effective coal gangue recognition technology is urgently needed to truly realize the intelligence of top coal caving mining, and accuracy and real-time have been the bottlenecks that the technology cannot be applied to practice.
In recent years, domestic and foreign scholars have done a lot of research on automatic detection methods of coal and gangue in the process of coal mining.][10] The latest research on coal gangue interface recognition are mostly based on these characteristics, which are combined and complementary, or some methods are adopted to make the feature comparison of coal and gangue more obvious.Wang et al. 11,12 proposed a coal gangue interface perception recognition method based on multisensor information fusion of vibration signal, current signal, acoustic emission signal, and infrared flash temperature signal.The recognition accuracy of cutting coal rock specimens with different wear degrees was studied, and the fusion recognition accuracy of coal rock interface was improved.Zhang et al. [13][14][15][16] studied coal gangue recognition by using the difference of infrared thermal images of coal and gangue with liquid intervention, effectively overcoming the problem of low accuracy of traditional coal gangue recognition using infrared images.Wang et al. 17 studied the accurate recognition of gangue that may occur in the process of coal caving, and formed a "trinity" intelligent recognition technology of gangue.Zhang et al. 18 integrated radar and high definition camera for real-time control of accurate coal discharge, improving the accuracy of coal and gangue monitoring.
However, the mine environment is extremely complex.The effect of image recognition is mainly limited by light, the noise rate of acoustic signal is high, the γ-ray are affected by dust, and the near infrared spectroscopy has certain requirements for temperature.The composition of the vibration signal is simple, easy to extract, and less affected by environmental factors, so it has certain advantages in practicability, economy and environmental protection.According to the actual working condition of top coal caving, the vibration acceleration signal acquisition simulation test bed of tail beam is designed, and the acceleration sensor is reasonably arranged at the bottom of the tail beam, so as to obtain the vibration acceleration information of multiple positions at the same time.Some related studies 19,20 were also based on vibration signals, and good recognition results were achieved through different data processing methods and recognition algorithms.Yang et al. [21][22][23] analyzed the dynamic response caused by elastic impact of coal gangue particles on metal plate, and verified the effectiveness of vibration signal for coal gangue recognition.The recognition of coal gangue mixtures with different gangue ratios has achieved high accuracy under different recognition algorithms.Based on these studies, this paper adopts the method of random impact test, and introduces the influencing factors such as impact velocity, direction and position into the experiment, which is more in line with the actual situation.In the previous studies, 24 we conducted multipoint acceleration recognition of coal and gangue based on SVM and serial splicing data, and the highest accuracy reached 92.5%.To further improve the recognition accuracy and take into account the real-time requirements, research on coal gangue recognition was carried out in this paper.
Convolutional neural networks (CNN) is a feedforward neural network that uses convolutional operations for deep learning.In recent years, it has been widely applied in biomedics, [25][26][27][28] engineering technology, [29][30][31] agriculture, [32][33][34] and other related fields.This technology has made breakthrough achievements in image recognition, [35][36][37] target monitoring, [38][39][40] natural speech processing, 41 and other aspects.Some studies have introduced CNN into coal gangue recognition.Jiang et al. 42 takes the mel frequency cepstral coefficients feature matrix of tail beam vibration signal as the training parameter of CNN, and improves the operation speed by optimizing the model structure, but the recognition accuracy is insufficient.Li et al. 43 recognized coal gangue images based on DCN-YOLOv3, and compared the evaluation index values of different models to verify the recognition effect of the proposed method.However, the experimental conditions are far from the actual underground production environment, and the practicability needs to be considered.Obviously, the application of CNN in the field of coal gangue recognition has not reached the ideal state.The gaps and limitations of previous studies are shown in Table 1.
To sum up, there are still three key issues in the related research of coal gangue recognition technology. (

References Limitations
Images [2, 3]  The actual production environment of the mine is dark and dusty, which is far from the experimental conditions, resulting in great challenges in the practical application of coal gangue image recognition.[13-18]   Acoustic signals [4, 5]  The composition of acoustic signals is complex, and it is difficult to perform multiple blind source separation of noise.
Near infrared spectroscopy [8-10]  The corresponding technology is not yet mature.
Vibration signals [19-24]  The ability of effective filtering, feature extraction and classification is insufficient.
CNN [42, 43]  The recognition accuracy is low and the practicability is insufficient.
(3) The training speed of the model is fast, which can quickly train the coal gangue recognition model under different mineral conditions and has good generalization ability.(4) A multi-index comprehensive evaluation system (MICES) for coal gangue recognition based on CNN is proposed.According to the research purpose, each index is given weight to evaluate the recognition model more scientifically and comprehensively.
Based on the above four innovations, this study has made progress in coal gangue recognition technology.This breakthrough has a positive impact on the economy, energy and environment, which can be summarized as follows.(a) In terms of economy, this study reduces the additional cost of gangue production and avoids the cost of environmental restoration caused by gangue treatment.(b) The recovery rate and quality of coal are improved.(c) The hazard of gangue production to the environment is reduced, and the risk of a large amount of harmful gas produced by spontaneous combustion in the goaf decreases.
The specific work route of this paper is as follows.Section 1 introduces the research status, existing problems of coal gangue recognition, and the contribution of this paper.Section 2 introduces the design of random impact test bed, the development of the test, and the establishment of vibration acceleration signal sample set of coal and gangue particles.Section 3 establishes the original CNN structure and inputs the data collected by each sensor in the form of matrix to obtain the recognition results.Section 4 improves the network structure, and then adjusts parameters to explore the real-time performance and accuracy of each constructed model, as well as the influence of sensor positions.In Section 5, the MICES was established.Taking the original CNN recognition model as the control, and the CES of the OI-CNN recognition model and the TI-CNN recognition model at different positions were compared.Section 6 shows the conclusions of this article.

| Design of random impact test bed
The random impact test bed is designed based on the principle that the vibration acceleration produced by coal and gangue particles impacting the tail beam is different.As shown in Figure 1, the impact test bed consists of portal, guide rail, vibration table, feeding device, pulley, and other structures.The vibrating table is fixed at a certain height from the ground, and the wire rope bypassing the pulley can drive the feeding device to move up and down along the guide rail to realize the coal discharge from different heights.According to the actual needs, the coal drawing height can be adjusted in the range of 0.5-4.5 m.In addition, the feeding device is equipped with electromagnetic lock and controller, which can realize the remote control of coal gangue particles discharging process.

| Experiment development and data extraction
According to the testing requirements and the random impact test bed model established in Figure 1, the physical prototype of the test bed as shown in Figure 2C was built.To establish an accurate and comprehensive coal gangue recognition model and explore the sensitivity of sensor positions to the impact difference of coal and gangue, sensors were arranged according to the size of the vibration table.As shown in Figure 2A, three acceleration sensors are arranged in the length direction and two in the width direction.In addition, one acceleration sensor is arranged in the center position.The type of the seven sensors is 1A102E universal piezoelectric acceleration sensor, and they are all arranged on the bottom of the vibration worktable through the magnetic base to prevent damage caused by the impact of coal and gangue falling directly.
After placing randomly selected single particle coal or gangue in the feeding device, the winch is started to adjust the device to a certain height (0.2-1.0 m).Open the feeding exit, single coal, or gangue particle impacts different positions of the vibration table at different speeds and directions.The seven groups of vibration acceleration signals are collected synchronously by the seven channels of the DH8302 dynamic signal acquisition system, and then controlled by the donghua dynamic analysis system and stored to the PC in time.Set the sampling frequency to 4000 Hz and sampling duration to 1 s.Therefore, each vibration acceleration signal is composed of 4000 sampling points, and each coal gangue recognition sample is composed of a matrix with a size of 4000 × 7.
First, according to the above test process, 1400 groups of coal particle impact test were carried out.For each test, a data samples composed of seven vibration acceleration signals as shown in the test system in Figure 2D was obtained.From 1400 groups of coal particle impact test data, 1200 groups of data samples were randomly extracted to form the sample set of coal particle vibration acceleration signal.Similarly, 1400 groups of gangue particle impact tests were carried out, and 1200 groups of data samples were randomly extracted to obtain the sample set of gangue particle vibration acceleration signal.Due to the differences in various physical properties such as surface hardness and density of coal and gangue, there are great differences in the waveform of vibration acceleration signal of coal and gangue.(

1) Convolution layer
The function of the convolution layer is to obtain local information by scanning the whole data structure through a set of convolution kernels for the input of twodimensional (2D) or multidimensional data.The convolution layer has the advantages of local connection and weight sharing, and can extract effective features, simplify the network, and increase the correlation between features. (

2) Pooling layer
The pooling layer can downsample each depth slice.Inserting a pooling layer after each convolutional layer can reduce the number of parameters in the network.The pooling layer acts to accelerate calculation and prevent overfitting.This article selects maximum pooling.
(3) Full connection layer The role of fully connected layer is to use activation function to perform some linear transformations on high-dimensional data of convolution layer, and then output the results for classification problems.Common activation functions are "Relu" and "Sigmoid."In this neural network, "Relu" is used in the middle layer of neural network as the activation function, and "Sigmoid" is used in the last layer to output the results.
(a) "Relu" is a piecewise linear function that outputs zero when input is less than or equal to 0, and returns an input value when input is greater than 0. When using Relu function, only some neurons are activated, so the network is more sparse.The activation function can be used to better mine the relevant features and fit the data, and its expression is as follows.
(b) The "Sigmoid" function, which is aimed at dichotomies, maps the output between (0, 1) as shown below.

| Establishment of coal gangue recognition model based on CNN
CNN can be used for both filtering and classification tasks at the end of the network.In previous studies, CNN has best effect on image recognition.The image is usually input in the form of 2D matrix or 3D tensor.Therefore, we input vibration acceleration sensor data in the form of 2D matrix through reasonable data structure.Seven signals of each impact are extracted through seven acceleration sensors, and each signal has 4000 sampling points.Therefore, each coal and gangue sample is composed of a 2D matrix with a size of 4000 × 7, as shown in Figure 4.
The training speed of CNN is not only related to the amount of data, but also affected by network structure, depth and other factors.To reduce computation and meet real-time requirements as much as possible, the original CNN coal gangue recognition model is established as shown in Table 2, which consists of one convolutional one pooling layer, and one fully connected layer (three layers in total).The convolution layer uses a 3 × 3 We performed a variety of tuning parameters under this CNN structure, and Table 1 shows three of the parameter combinations under the network structure.
Since the size of the signal matrix consists of only seven columns, to reduce the loss of effective information in the process of convolution and keep the original size of the signal digital matrix after convolution, zeroes are added around the digital matrix, as shown in Figure 5.

| Analysis of recognition results
Through the above analysis, 400 groups of coal and gangue vibration acceleration signal samples were selected as the training set and 200 groups as the test set respectively, and the data were input into the original CNN model in the form of 2D matrix.As shown in Figure 6A-C The accuracy of the first training data is too low and the loss is too large, so there is not much reference significance.In addition, from the first training to the second training, the results showed great mutations, so it is considered unreasonable, and the data of the first training will be completely ignored in the following study.As shown in Figure 6A defects caused by poor network structure cannot be compensated by parameter adjustment.
Single sample recognition time is the time taken for each training and test divided by the total number of samples used in the training and test.In the process of top coal caving, only a single signal sample collected by coal and gangue impact is recognized, so as to decide to continue coal caving or shut down.Single sample recognition time refers to the time allocated to a single sample for each training from data processing to recognition after the system collects vibration acceleration signal.
Average single sample recognition time (ASSRT) refers to the average single sample recognition time of each training within the effective training times.ASSRT can effectively reflect the real-time performance of the recognition model, so this paper uses it as an indicator to measure the time.The ASSRT of the original model is 0.0198 s.

| IMPROVEMENT AND APPLICATION OF COAL GANGUE RECOGNITION MODEL BASED ON CNN
According to the conclusion of the previous chapter, the accuracy of the original CNN coal gangue recognition model is insufficient and the network structure is extremely unstable.In this section, the accuracy and stability of coal gangue recognition model are improved by deepening the network structure.Then, the 2D CNN is changed to 1D CNN through parameters adjustment, so as to obtain the TI-CNN coal gangue recognition model, thus solving the real-time problem.The improvement of the model will take the first parameter combination as an example.

| Improvement of network structure
When the original CNN structure is applied to the signal data matrix, the training accuracy and test accuracy are extremely unstable.To solve this problem, we have improved the network structure.The OI-CNN structure consists of four convolutional layers, two pooling layers, and one fully connected layer (a total of seven layers).Each two convolutional layers are connected by one pooling layer.Its structure distribution and parameters are shown in Table 3.
The same coal and gangue samples were input into the above CNN structure with 400 groups as training set and 200 groups as test set.Data are input in the form of matrix for 150 times of training, and the variation rules of training set accuracy, test set accuracy, training set loss and test set loss are obtained, as shown in Figure 7.
As shown in Figure 7, after the small amplitude oscillation of the first few training sessions, the four index parameters of OI-CNN tend to be relatively stable, with improved accuracy and good stability.However, due to the deepening of the network structure, the ASSRT of this model increases to | 4085 0.0358 s, and the real-time performance of coal gangue recognition model decreases.

| Adjustment of input data and parameters
The OI-CNN recognition model solves the problem of insufficient stability, but lacks real-time performance.Therefore, we choose to reduce the amount of data, and input the signal collected by a single position acceleration sensor in the form of 1D vector for recognition.Some network parameters, such as convolution kernel and pooling size, are adjusted to adapt to 1D signal data input, while the TI-CNN recognition model is similar to the OI-CNN network model in overall structure.
The CNN structure shown in Table 4 above is applied to train and test signal sample sets at seven positions, respectively, and the first 150 training and test results are shown in Figure 8A-G.
Figure 8A-G corresponds to the recognition results of data collected by sensors No. 1-7 on the bottom surface of vibration worktable in Figure 2.After several rounds of training, the accuracy of training and test can reach above 97%, or even maintain at 100%.The curves of training set loss rate and test set loss rate finally coincide and decrease to zero.The performance of TI-CNN applied to single position sensor data is better than that applied to all data.Moreover, the ASSRT will decrease due to the decrease of data volume.Table 5 shows the corresponding ASSRT of the seven sensor positions.
Inputting the data of different single position sensors into the TI-CNN recognition model, the ASSRT of single sample is between 5.24 × 10 −3 -5.86 × 10 −3 s, which can meet the real-time requirements by referring to the actual coal caving speed.

| Evaluation method
The MICES has five evaluation indexes, including training accuracy (A 1 ), test accuracy (A 2 ), training set loss rate (L 1 ), test set loss rate (L 2 ) and ASSRT (t m ).In previous studies, there was no scientific weight allocation for each evaluation index of CNN, while this paper assigned weights to each index according to the specific research purpose and the importance of each index to the recognition model.Accuracy and time are the most important indexes to evaluate the model, and in practical application, test accuracy is more important than training accuracy.Compared with other indexes, loss rate has the least influence on this research problem.Overall, the importance of each index can be expressed as 2 ; and their weight distribution is shown in Table 6.
Then the model evaluation score formula of this sampling is as follows.7.
Table 6 shows that from the original CNN recognition model to the OI-CNN recognition model, although its accuracy is improved, CES is not significantly improved due to the reduction of real-time performance.The TI-CNN recognition model has achieved high CES due to its improved accuracy and real-time performance.As can be seen from Figure 9, the CES obtained by the input of different single position sensor data into the TI-CNN recognition model is significantly higher than that of the original CNN recognition model and the OI-CNN recognition model, which proves the effectiveness of the TI-CNN structure in coal gangue recognition.There is little difference in the CES obtained from three Indexes Index calculation formula Weight (ω i ) Index evaluation score (F i ) Loss rate of training set (L 1 ) bottom plate, but it has no obvious advantage compared with other sensors.Therefore, the sensor position has little influence on the recognition accuracy in this study.

| Model performance evaluation
Section 5.2 has proved the improvement of comprehensive performance of coal gangue recognition model under the first parameter combination.In this section, the CES of the original CNN and TI-CNN under the second parameter combination and the third parameter combination are calculated, as shown in Tables 8 and 9.
It can be seen from Tables 8 and 9 that the comprehensive evaluation scores of TI-CNN under the two parameter combinations are greatly improved compared with the original CNN.Therefore, it is proved that the improved CNN based coal gangue recognition model has high precision, real-time and generalization ability.

| CONCLUSION
In view of the problems of insufficient accuracy, poor real-time performance and weak generalization ability of coal gangue recognition, this paper builds a random impact simulation test bed to carry out experiments to obtain coal gangue recognition sample set.First, the CNN structure model of single convolution layer and single pooling layer is trained and recognized, and the stability of the structure is insufficient.Then, OI-CNN recognition model with good stability and high accuracy is obtained by increasing the network layer, but the realtime performance of the model is poor.Through parameters adjustment, the OI-CNN is upgraded to TI-CNN coal gangue recognition model, with improved accuracy and real-time performance.Finally, the three coal gangue recognition models are comprehensively evaluated with multiple indexes.This paper mainly draws the following conclusions.In the design of the MICES, it is only considered that the weighting coefficients of various indicators are set manually.More scientific automatic weight parameter setting will be the future research direction.This study has proved the effectiveness of 1D CNN structure and single position sensor data input for coal gangue recognition, which has been further improved in accuracy and real-time performance, providing a theoretical basis and technical basis for the realization of intelligent mining technology of top coal caving.

F
I G U R E 2 Testing system.GANGUE RECOGNITION TECHNOLOGY BASED ON CNN 3.1 | Principle of CNN CNN is a typical feedforward neural network, as shown in Figure 3, which consists of input layer, convolutional layer, pooling layer, full connection layer, and output layer.It is a deep learning method developed for image recognition problems.Compared with other machine learning models, CNN has the advantage of avoiding feature extraction and selection.CNN is a deep machine learning model under supervised learning, which has strong adaptability.It is good at mining local features of data and extracting global features for classification.
, in the first 150 training times of the network under three parameter combinations, the variation rules of the four indexes including training set accuracy (train_acc), test set accuracy (test_acc), training set loss rate (train_loss), and test set loss rate (test_loss) were obtained.
, from the second training to the 110th training, although the loss rate is very low, the floating range of training set accuracy and test set accuracy is large.Moreover, when the training times reached 110, the accuracy of training set and test set decreased rapidly to about 50%.The original network structure is extremely unstable.Figure 6B,C also face the problems of low recognition accuracy and model instability.Therefore, the F I G U R E 5 Zero padding diagram.F I G U R E 6 Results of original convolutional neural network recognition model.(A) The first parameter combination.(B) The second parameter combination.(C) The third parameter combination.

T A B L E 3 2 F
CNN structure and related parameters.I G U R E 7 Results of once improved convolutional neural networks recognition model.ZHANG ET AL.
model training process diagram, it can be seen that the improved CNN recognition model has better stability than the original CNN recognition model.When using a single position sensor data vector to train and test the improved CNN structural model, the accuracy and real-time performance reach the best.Below, we quantitatively evaluated three CNN coal gangue recognition models by integrating different indicators.Since the training results are easily interfered by uncertain factors (bad values, various factors in the training process, etc.), the 2400 sets of data in the sample set of vibration acceleration signals of coal and gangue particles are sampled for three times for evaluation, and the evaluation scores obtained from the three samplings are averaged to obtain the final evaluation score.Each time, 400 groups of coal and gangue were taken separately as the training set and 200 groups as the test set.And the data of the three samplings are partly different.Then, the performance of three CNN recognition models and the sensitivity of seven sensor positions are compared.(All calculations ignore the data from the first round of training).

T 2 F 5 . 2 |
A B L E 4 1D CNN2 structure and related parameters.I G U R E 8 Results of twice improved convolutional neural networks recognition model.ZHANG ET AL. | 4087 Evaluation result According to the above formula, the model evaluation scores of each sampling of the original CNN recognition model, the OI-CNN recognition model and the TI-CNN recognition model are calculated.The comprehensive evaluation score (CES) of each model is obtained by taking average values of sampling for three times, and the results are shown in Table

− 3 T A B L E 6
samplings of any model, which indicates that the performance of coal and gangue recognition model established in this paper is relatively stable.On the other hand, it indicates that the model has strong generalization ability.No. 7 sensor is located in the center of the T A B L E 5 ASSRT for sensors at seven positions.Multi-index comprehensive evaluation system.

( 1 )
In this paper, 2800 groups of tests (1400 groups of coal and 1400 groups of gangue) were carried out by building a random impact simulation test bed, from which 1200 groups of coal acceleration signal data and 1200 groups of gangue acceleration signal data were randomly extracted to form a coal gangue recognition sample set.Each group of coal gangue samples are vibration acceleration signals extracted from seven positions of the vibration worktable in the form of signal data matrix.(2) To meet the real-time requirements, the original CNN structure model of single convolution layer and single pooling layer is constructed.The ASSRT of the model is 0.0198 s, which can meet the real-time requirements, but the model stability is insufficient.(3) The OI-CNN recognition model was obtained by deepening the network structure, which could solve the problem of insufficient stability of the original CNN recognition model and further improve the accuracy.However, the ASSRT increased to 0.0358 s, and the real-time performance decreased.(4) To solve the problem of insufficient real-time performance of the OI-CNN recognition model, the 2D CNN structure is changed to 1D CNN structure by adjusting parameters, and the data of each position sensor are respectively input into the TI-CNN recognition model in the form of 1D vector.After several training sessions, the accuracy of training set and test set at each single position is improved to 100%, and the ASSRT is reduced to 5.24 × 10 −3 -5.86 × 10 −3 s.The accuracy and real-time performance of data input from different single position sensors are compared, and there is no significant difference.(5) A MICES based on CNN coal gangue recognition model is proposed to evaluate the three models, and the TI-CNN coal gangue recognition model has the highest CES.
In the table, a i is the training set accuracy of the ith training.b i is the test set accuracy of the ith training; l i is the training set loss of the ith training; n i is the test set loss of the ith training; T is the total time of model training; s tr is the number of training set samples; s te is the number of test set samples F 1 , F 2 , F 3 , F 4 , F 5 is the evaluation scores of A 1 , A 2 , L 1 , L 2 , and t m , respectively.Model evaluation score.
F I G U R E 9 Model evaluation score.
Model evaluation score under the second parameter combination.Model evaluation score under the third parameter combination.
T A B L E 8