Machine Learning Models for COVID-19 Confirmed Cases Prediction: A Meta-Analysis Approach

COVID-19, CoronaVirus Disease – 2019, belongs to the genus of Coronaviridae. COVID-19 is no longer pandemic but rather endemic with the number of deaths around the world of more than 3,166,516 cases. This reality has placed a massive burden on limited healthcare systems. Thus, many researchers try to develop a prediction model to further understand this phenomenon. One of the recent methods used is machine learning models that learn from the historical data and make predictions about the events. These data mining techniques have been used to predict the number of confirmed cases of COVID-19. This paper investigated the variability of the effect size on the correlation performance of machine learning models in predicting confirmed cases of COVID-19 using meta-analysis. It explored the correlation between actual and predicted COVID-19 cases from different Neural Network machine learning models by means of estimated variance, chi-square heterogeneity (Q), heterogeneity index (I2) and random effect model. The results gave a good summary effect of 95% confidence interval. Based on chi-square heterogeneity (Q) and heterogeneity index (I2), it was found that the correlations were heterogeneous among the studies. The 95% confidence interval of effect summary also supported the difference in correlation between actual and predicted number of confirmed COVID-19 cases among the studies. There was no evidence of publication bias based on funnel plot and Egger and Begg’s test. Hence, findings from this study provide evidence of good prediction performance from the Neural Network model based on a combination of studies that can later serve in the prediction of COVID-19 confirmed cases.


1.
Introduction Novel coronavirus pneumonia named as COVID-19 by WHO has been spreading all over the world. WHO has announced that COVID-19 outbreak is pandemic as more and more countries are experiencing the high increment of virus transmission cases and death which amounted to 2,627,630 cases and can offer consistent performance, the present systematic review is designed with a more focused research question.

3.
Methodology This paper utilized meta-analysis to analyse several machine learning models in several studies of COVID-19 cases. Meta-analysis is a statistical analysis that combines the results from multiple scientific studies. It can be performed when there are multiple studies reporting the same measurements that are expected to have some degrees of error. Meta-analysis is widely used in research to evaluate the evidence in areas such as finance, pharmaceuticals, economics, genetics, and others [20]. It is often being used for the purpose of integrating the findings from combined studies. The summary statistics derived from the primary analysis of previous studies are used as "data" in meta-analysis. The "data" refers to the effect size considered under study in terms of statistical analysis. Meta-analysis can help to assess the amount of variability that exists among the previous studies, despite combining the findings or results across the studies. Therefore, the purpose of meta-analysis in this study is to investigate the variability of the effect size on the correlation performance of machine learning modes in predicting confirmed cases of COVID-19. The meta-analysis involved several steps as follows:

Search Strategy and Selection Criteria
Literature search for articles in several scientific journals or preprint platforms was made through IEEE Xplore, Google Scholar, UiTM online database, PrubMed, Scopus, PLOS Neglected Tropical Diseases, Web of Sciences, medRxiv, Science Direct and others. The keywords used for our searching were "number of confirmed COVID-19", "Machine Learning Algorithm", "data mining", "correlation" and "prediction". This study involved all previous studies that were related with the number of confirmed COVID-19 cases using various algorithms in machine learning while the parameter value was Pearson Correlation Coefficient (r). Pearson correlation coefficient, in this case, measures the relationship between the actual and predicted values of confirmed COVID-19 cases from the machine learning algorithm. If correlation r is close to 1, there is a close relationship, and the model is performing well. Twelve studies which fulfilled the criteria of selection were collected. The exclusion criteria of these studies were the number of death cases and recovery cases since some journals did not report the value of death or recovery cases.

Selection of Articles and Data Extraction
There are several performance parameters values that can be used in meta-analysis, among which are Root Mean Square Error (RMSE), Mean Square Error (MSE), Mean Absolute Error (MAE) and Correlation performance. In this study, the selection of articles was based on Pearson correlation. This study considered the articles that reported the Pearson correlation performance values which represented the correlation between the actual and the predicted values of the model. The inclusion criteria for this study was all papers that used machine learning methods as data analysis. There are a variety of machine learning methods being used by researchers in predicting COVID-19 cases such as Decision Tree, Regression, Support Vector Machine, Neural Network. However, the most commonly used machine learning algorithm is Neural Network (NN) [14,15]. Thus, only papers that used the Neural Network algorithm and reported correlation as a measure of performance in the analysis were selected in this study. Another inclusion criterion was papers that only analysed the number of confirmed cases of COVID-19 data as the target variable. There was no language restriction during the journal articles searching. The information was selected based on the following characteristics: authors, year, publication, statistical method, algorithm, and studied variables (time and number of confirmed COVID-19 cases). Based on the first step of document identification, this study managed to identify about 53 related works from the search by using specified keywords as mentioned previously. After going through document screening, about 12 studies out of 53 studies were shortlisted. However, only 5 papers were eligible and chosen in this meta-analysis study based on inclusion and exclusion criteria as mentioned

Effect Size
The effect size is an important measure in meta-analysis which reflects the magnitude of the treatment effect or the strength of relationship between two variables [21][22]. The effect size does not only measure the relationship but also represents the impact of an intervention such as the impact of teaching method on test scores, medical treatment risk of infection and others. In general, effect size could refer to the estimate of a single value. The availability of the effect size is important as the analysis relies on the effect size in meta-analysis. If the effect size is not available or the type of effect size used is different, the study is excluded. There are many types of effect size such as odds ratio (for clinical trials), Pearson correlation and Fisher's Z score (for observational studies) and Cohen'd and Hedges'g (experimental studies) [23]. Pearson correlation coefficient (r) was chosen as the effect size in this study to measure the performance and relationship between the actual and predicted values of confirmed COVID-19 cases. The Pearson correlation coefficient value in this study was extracted from the selected studies which fulfilled all the inclusion criteria.

Statistical Analysis
In the meta-analysis, the statistical analyses computed were variability estimation, heterogeneity statistics and assessment of correlation between studies. The computation involved is explained in the following section.

Variability Estimation.
The first step in this study was to obtain the estimate of overall correlation value by combining all Pearson correlation coefficients value (r) from each study. Then, the calculation for weight estimate of observed variance (̂( ) ) and estimated variance due to sampling error (̂) was done by using the Hunter and Schmidt approach [21]. This is due to the use of correlation value as the effect size [20,25]. Here, the aim was to estimate variability in the effect size which was correlation used in this study. The weight estimate of observed variance, ̂( ) was computed in equation (1). The higher the value of ̂( ) , the greater the variability of the correlation values across studies. The estimated variance due to the sampling error, ̂ was computed as shown in equation (2). The estimated variance was taken into consideration in this study since it can be used to estimate the distribution of the effect size. The value was calculated as follow: where n = Mean of sample size across studies 3.4.2. Heterogeneity Statistics. The next step was to compute both Chi-Squared of heterogeneity and Heterogeneity index, 2 as in equation (3) and equation (4) respectively. The calculation of heterogeneity index was done to assess the existence of variation among the studies as the population was different for each study. where; and df is the degree of freedom, W is the weight of each study, is the effect size (correlation coefficient, r) and SE is the standard error. 2 index is measured as the percentage of variability in point estimates which is due to heterogeneity across study. The value for chi-squared heterogeneity if there is no heterogeneity is equal to its degrees of freedom. Hence, 2 is the percentage of the chi-squared statistic which is not explained by the variation within the studies. 2 without the 100 is essentially an intra class correlation coefficient. It represents the percentage of the total variation which is due to variation between studies. Higgin et al. [25] suggested that 2 = 0% (no heterogeneity), 2 = 25% (low heterogeneity), 2 = 50% (moderate heterogeneity) and 2 = 75% (high heterogeneity). 2 The value of Q is the measure of heterogeneity among the studies. In heterogeneity hypothesis testing, Q value is used in making the decision of rejecting or failing to reject the null hypothesis. If heterogeneity is present among the studies which reject the null hypothesis, the random effect model is appropriate. If there is no heterogeneity, it should consider the fixed effect.

Random Weight Effect.
If the heterogeneity index exists, we proceed with the analysis of the random effect model. To reassess the consistency of the result on the presence or absence of heterogeneity among the studies, the adjusted index 2 and adjusted Chi-Squared of heterogeneity ′ were calculated. Adjusted weight of each individual study v, was calculated as a constant value. The constant value v, and adjusted weight of each study wv were computed as in equation (5)  Then, based on equation (5), the adjusted Chi-Square heterogeneity, ′ and adjusted heterogeneity index 2 , were obtained as in equation (6) and equation (7).

Assessing the Effect Summary and Correlation among studies.
Next, the effect summary of the overall effect size, overall standard error and overall variance was computed. The overall correlation among the previous studies was obtained by calculating the 95% of confidence interval. Confidence interval is calculated to indicate whether there is a difference in the effect of correlation among the studies. The effect summary (ES) for the effect size, standard error and the variance was calculated as in equation (8) to equation (10). is the effect summary for variance. Thus, the 95% of confidence interval was obtained and calculated as below.

1.96( )
The absence of zero value (0) that lies within the range bound will indicate the existence of the difference and variability of the correlation among the studies and vice versa.

Forest Plot.
Forest plots are typically used to summarize previous published findings in a subject area review. It is used to display the results of meta-analysis and acts as a tool to indicate where formal meta-analytic evaluation can be useful. A forest plot gives a visual suggestion on the amount of study heterogeneity which shows the estimated common effect of all in one figure. A forest plot displays confidence interval and point estimate (correlation) represented by whisker for multiple findings within a study in a horizontal orientation. Explanatory text displayed next to these points and whisker diagrams provides details regarding the data presented [26].

Publication Bias.
The funnel plot is one of the tools used in meta-analyses to visually detect the presence of publication bias. It is a scatter plot that plots the standard error against the effect size. Common forest plot is used to plot the effect estimates on the horizontal scale and the measure of study size on the vertical axis. Scatter plot will spread wider at the bottom of the graph if effect estimates are taken from small studies while bigger studies will narrow the spread. Trials are likely to converge around the true underlying effect size as the size of the trial increases. An ideal funnel plot is where the included studies are shown to be scattered either side of the overall effect line in a symmetrical manner. Severe asymmetry to either side is an indication that publication bias may be present [27]. However, the decision visualized from the funnel plot whether it is symmetric or not is subjective. Hence, this study used Egger test [28] to assess the absence of publication bias. There exists publication bias if the probability value for Egger test is less than 0.05.

4.
Result and Discussion Table 1 below shows the variability of the correlation values among the studies using Hunter and Schmidt computed for each study. Based on table 1, the correlation values were treated as the effect size. It can be seen that there was a strong relationship between the actual and predicted values on the number of confirmed COVID-19 cases among the Neural Network machine learning algorithms across the studies. The higher value correlation, the better the model is represented. The variability among the studies was assessed by looking at the value of the observed variance and estimated variance due to the sampling error. The result of observed variance showed small variability of the correlation values across the studies. Hunter and Schmidt [24] stated that since estimated variance due to sampling error is small and can be used to estimate the distribution of the effect size, the various algorithms used in Neural Network are found to have a high correlation with the number of confirmed COVID-19 cases.

4.2.
Heterogeneity Result in Various Algorithm Two important measurements were computed in order to get the heterogeneity analysis which were Chisquared Heterogeneity (Q) and Heterogeneity Index ( 2 ). The findings are tabulated in table 2. The absence or presence of heterogeneity among the studies can be observed by looking at Q value. Table 2 presents the Q value (18.9051) which was greater than Q-tabulated (9.488). This can be concluded that there was enough evidence that heterogeneity existed between the correlation values across the studies as the null hypothesis was rejected due to heterogeneity. Heterogeneity index was further calculated to measure the percentage of variation of heterogeneity. The value of 2 I = 78.84% confirmed there was high heterogeneity across the studies. Random effect model was calculated since the heterogeneity was present in this study which showed the effect size varied among the studies. The result of random effect heterogeneity is shown in table 3. In the random effect model, the adjusted chi-squared heterogeneity and adjusted heterogeneity index were calculated by adding the constant value. Chi-squared heterogeneity index and heterogeneity index were adjusted to reassess the consistency of the presence of heterogeneity among studies. Table 3 shows that ′ was (9.7477) greater than (9.488) which provided enough evidence to reject the null hypothesis. It can be concluded that there was heterogeneity in various algorithms in the Neural Network among the studies. The adjusted value of the heterogeneity index was approximately 58.96% which suggested moderate heterogeneity for various algorithms to predict the number of COVID-19 confirmed cases in random effect models across the studies. Thus, we can conclude that the findings in table 2 and table 3 are consistent.  Table 4 summarizes the meta-analysis correlation between actual and predicted values on the number of confirmed COVID-19 cases among various Neural Network machine learning algorithms across the selected studies including the 95% confidence interval. The results are shown as below: To assess a significant difference in the correlation of actual and predicted values on the number of confirmed COVID-19 cases among the Neural Network machine learning algorithms across the studies, a 95% confidence interval of the effect size was calculated based on correlation, standard error and variance value. Table 4 shows that there were no zero values within the lower boundary and upper boundary for all studies. In addition, for overall studies, there was no presence of zero values within the interval. It can be concluded that there was a significant difference in correlation between actual and predicted values of confirmed COVID-19 cases among the machine learning algorithms across the studies.

4.4.
Forecast Plot Figure 1 shows the forest plot of the studies. The width of the line is the confidence interval on the effect estimates of individual studies. From the confidence interval line, we can conclude that there was a statistically significant difference in the correlation between actual and predicted values on the number of confirmed COVID-19 cases among the Neural Network machine learning algorithms across the studies since there was no zero value. The area of the box represents the weight given on the study. Most of the studies were equally weighted. The diamond below the studies represents the overall effect.

4.5.
Funnel Plot The final inspection in meta-analysis was to examine any publication bias. Figure 2 shows the funnel plot of all studies. The plot displays the effect size (x-axis) and the standard error (y-axis). Based on figure 2, the standard error of the correlation is plotted against correlation (r) values that represent the precision. Each dot represents a separate study. The two dotted lines on either side represent the pseudo 95% confidence interval. The middle solid line indicates the overall effect from the meta-analysis. From the funnel plot, the studies showed the presence of heterogeneity and no publication bias. The points were shown to be scattered at either side of the overall effect line in a symmetrical manner. To further confirm this, the Egger test was conducted to identify the association between the effect estimate and standard error. If p-value of Egger test is less than 0.05, there exists publication bias. Table 5 shows that the p-value for the Egger test was 0.8731 which was greater than 0.05. Therefore, we can conclude that there was no publication bias in this study.

5.
Conclusion This study analysed the model performance of machine learning algorithms in predicting the number of confirmed COVID-19 cases. A meta-analysis approach was used on previous studies which mainly focused on Pearson correlation between actual and predicted values on the number of confirmed COVID-19 among Neural Network machine learning algorithms. It was found that the Neural Network model displays a good performance since it has a high mean correlation, low observed variance and estimated variance due to sampling error. The higher the value of the correlation, the better the model performance based on that algorithm. The correlation between actual and predicted values of confirmed Covid-19 cases was found to be heterogeneous across the studies based on the reported Chi-square heterogeneity test of Q. This study also confirmed that there is moderate heterogeneity between these studies as the value of adjusted 2 58.96% adj I = .
Based on 95% confidence interval for effect summary, there is a significant difference in correlation between actual and predicted values on the number of confirmed COVID-19 cases among the Neural Network machine learning algorithms across the studies. There is no publication bias among the selected studies. Thus, we can conclude that Neural Network prediction models are robust enough to be recommended in the future prediction and may be helpful to understand the upcoming situation since all the studies give high value of correlation coefficient. The prediction methodology using the updated dataset must be explored so that this type of study is continuously enhanced. The use of the most accurate and appropriate machine learning method is also a must. Combination of different algorithms and activation functions can be designed to assess the improvement of the model in terms of performance.