Applying Neural Networks to Recover Values of Monitoring Parameters for COVID-19 Patients in the ICU

Abstract

From the moment a patient is admitted to the hospital, monitoring begins, and specific information is collected. The continuous flow of parameters, including clinical and analytical data, serves as a significant source of information. However, there are situations in which not all values from medical tests can be obtained. This paper aims to predict the medical test values of COVID-19 patients in the intensive care unit (ICU). By retrieving the missing medical test values, the model provides healthcare professionals with an additional tool and more information with which to combat COVID-19. The proposed approach utilizes a customizable deep learning model. Three types of neural networks, namely Multilayer Perceptron (MLP), Long/Short-Term Memory (LSTM), and Gated Recurrent Units (GRU), are employed. The parameters of these neural networks are configured to determine the model that delivers the optimal performance. Evaluation of the model’s performance is conducted using metrics such as Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Mean Absolute Error (MAE). The application of the proposed model achieves predictions of the retrieved medical test values, resulting in RMSE = 7.237, MAPE = 5.572, and MAE = 4.791. Moreover, the article explores various scenarios in which the model exhibits higher accuracy. This model can be adapted and utilized in the diagnosis of future infectious diseases that share characteristics with Coronavirus Disease 2019 (COVID-19).

1. Introduction

On 31 December 2019, a novel coronavirus outbreak occurred in Wuhan, China, and was later named 2019-nCoV or Coronavirus Disease (COVID-19); it had a significant impact worldwide and alarmed the medical community [1]. Recognizing the severity of the spread and the lack of action, the World Health Organization (WHO) declared COVID-19 a pandemic [2].

The initial clinical symptoms of COVID-19 include fever, a dry cough, and fatigue. However, some patients develop acute respiratory distress syndrome (ARDS), requiring admission to the intensive care unit (ICU) [3]. As the number of COVID-19 cases surged, hospitals faced unprecedented pressure, overwhelming healthcare systems in many countries [4].

Upon a patient’s hospital admission, various data, such as their socio-demographic characteristics and COVID-19-related medical history, are collected and monitored. Additionally, a continuous stream of parameters, including clinical and laboratory measurements, provides a vast amount of data.

However, the surge in COVID-19 cases has led to an overwhelming hospital situation, preventing the performance of all the medical tests required in order to obtain crucial clinical parameters (data) from COVID-19 patients. This lack of data has made it challenging to assess patient progress accurately.

Given these circumstances, exploring novel tools that can enhance the speed and accuracy of patient assessment through technology is crucial. This research paper aims to develop a machine-learning-driven predictive model capable of forecasting multiple medical factors of COVID-19 patients in the ICU. Such a model could provide healthcare professionals with supplementary information to make well-informed decisions regarding patient management and treatment. Ultimately, this advancement could improve healthcare provision and lead to better patient outcomes, optimizing the allocation of limited medical resources.

This study has the potential to make significant contributions to the field by enhancing patient diagnosis and follow-up, complementing existing clinical approaches, and being applicable to similar diseases in the future. By delivering accurate predictions of multiple medical factors, the study can improve clinical decision making and enable more personalized and effective treatment. Moreover, it offers an additional perspective that complements the traditional clinical techniques used in ICU care for COVID-19 patients. Integrating machine-learning-based data and analytics enables the attainment of a more comprehensive and unbiased understanding of the patients’ condition. The global impact of COVID-19 as an infectious disease has emphasized the importance of efficient predictive and diagnostic tools. The developed model serves as a foundation for future research and applications in similar disease domains.

This paper aims to predict the medical test values of COVID-19 ICU patients using a neural network model. By recovering the missing values of medical tests, the model provides healthcare professionals with an additional tool and more information with which to combat COVID-19. The model incorporates three types of neural network algorithms: Multilayer Perceptron (MLP), Long/Short-Term Memory (LSTM), and Gated Recurrent Units (GRU). Various parameters of the networks are configured to obtain the most accurate model.

The document’s structure is as follows: Section 2 discusses previous related work. Section 3 provides details about the database and the medical requirements needed to form it. Section 4 describes the techniques, tools, and possible configurations used in the research. Section 5 explains the research design and execution, followed by the model evaluation in Section 5.1. The results of the utilized model are presented in Section 6.1, followed by a discussion of these results and an explanation.

2. Related Work

Machine Learning (ML) is a potent field within artificial intelligence that has demonstrated its significant potential to impact medical diagnostics. ML has been successfully employed in diagnosing various types of cancers, including liver cancer [5], skin cancer [6], breast cancer [7], and cervical cancer in women [8]. Additionally, it has been utilized for diagnosing and predicting cardiovascular diseases [9], as well as the diagnosis and treatment of diabetes [10,11]. ML has also shown promise in predicting the progression of Alzheimer’s disease [12,13] and assisting patients with chronic kidney disease [14], among numerous other applications.

With the emergence of the COVID-19 crisis and the favorable outcomes observed when applying ML algorithms to the prediction and prognosis of other diseases, studies have been initiated to aid medical professionals in combating the COVID-19 disease. The objective of this research [15] is to explore the relevance of employing artificial intelligence in order to resolve the health crisis caused by the COVID-19 pandemic. This study investigates the potential of algorithms and machine learning techniques to effectively combat the virus and mitigate its societal impact. ML was also used in [16] to estimate the demand for COVID-19 masks in densely populated cities such as Wuhan, Guangzhou, and Hangzhou in China.

The prediction of COVID-19 has been achieved using alternative non-clinical methods, such as supervised machine learning models, to identify at-risk patients without relying solely on PCR testing [17]. Models have been developed for the pre-hospital prediction of adverse outcomes in patients with suspected COVID-19 [18], for predicting the number of hospitalizations caused by COVID-19 [19], anticipating the admission of COVID-19 patients to the ICU [20,21], and predicting mortality in COVID-19 pneumonia patients [22]. Imaging techniques, particularly computational tomography scans, have proven to be particularly useful for the machine-learning-based recognition of COVID-19 [23,24,25].

Recent advances in ML research and development have significantly enhanced the prediction, diagnosis, and detection of COVID-19. One review paper [26] provides valuable insights into how ML can effectively manage the COVID-19 virus and mitigate the impact of the pandemic.

In the study conducted in [27], five different time series forecasting models were analyzed: autoregression (AR), exponential smoothing (ES), multilayer perceptron (MLP), short-term memory (LSTM), and autoregressive integrated moving average (ARIMA), along with their hybrid models. The objective was to identify the best machine learning model for forecasting COVID-19 cases.

The effective management of COVID-19 cases is crucial, particularly when resources are limited. The proper classification of patients is essential to achieving this goal. This article [28] presents a machine-assisted diagnostic system that can predict disease severity in COVID-19 patients. By doing so, it enables the better anticipation and planning of the required treatment for each patient.

Patients with COVID-19 exhibit various clinical characteristics. By leveraging these characteristics, machine learning approaches have been developed to automate the prediction and prognosis of COVID-19 severity [29,30,31,32,33].

Some studies have employed machine learning algorithms to predict the values of specific medical variables, such as glucose levels [34,35,36] or body temperature [37]. However, these works are limited as they only predict the value of a single variable. This paper introduces a significant innovation by proposing a method based on the prediction of multiple medical variables. It presents a system that predicts several medical variables that directly impact a pathology such as COVID-19.

3. Materials

The data were generated by the medical staff of the Complejo Hospitalario Universitario Insular—Materno Infantil (CHUIMI) in Las Palmas de Gran Canaria, in Spain. The Ethical Committee belonging to CHUIMI approved the use of data for this research work with the reference “2020-231-1 COVID-19”, with an extension of this research taking place in 2022. It is a private database and will not be shared with the public as it contains the confidential data of the patients. This database contains the actual measurements of various tests performed on patients diagnosed with SARS-CoV-2 disease between March 2020 and January 2021. The following criteria were taken into account for the formation of the database:

• Inclusion Criteria:
  • COVID-PCR-positive patients in need of admission to the ICU.
• Exclusion Criteria:
  • COVID-PCR-negative patients.
  • COVID-PCR-positive patients who did not require admission to the ICU.
  • Refusal of the patient (or legal representative) to participate in the study.

Considering the above criteria, the database consists of 72,075 medical data for a total of 214 patients. All of the data taken from the patients are composed of the parameters listed in

Table 1. Database parameter.

Parameter Name	Description
NHC	Patient identification number to maintain patient privacy
DATE_MARK	Date and time at which this data were recorded
MEDICALRECORD	Number of the medical staff recording data from the medical tests
GENDER	Patient’s gender
DATE_BIRTH	Patient’s date of birth
NAME_INDICATOR	Name of the medical variable obtained in the test by the medical staff
VALUE	Numerical value of the medical variable
UNIT	Unit of the numerical value of the medical variable
DATE_TAKEN	Date and time of the medical test
SITUATION	Patient’s situation (WARD/ICU)

.

Within the parameters described above in Table 1, it can be observed that there is a parameter called “NAME_INDICATOR”. This parameter refers to the names of the medical variables for which different values were obtained when performing medical tests on the patients. These medical variables are detailed in

Table 2. Medical variables in the database.

Medical Variables
1. CK–MB mass            2. Hemoglobin            3. Procalcitonin            4. Serum Urea            5. Lactate            6. PCR            7. Capillary blood glucose            8. Leucocytes

.

Although the article does not mention any other specific alternative datasets, there are different data sources available that can be used to study the relationships among the monitoring parameters in COVID-19 patients in the ICU. However, it is important to bear in mind that the choice of dataset may influence the results and the generalisability of the findings. The representativeness of the data, the quality of the collection, and the balance between the quantity and relevance of the variables included may affect the accuracy and applicability of the results obtained. Therefore, the careful assessment and consideration of multiple data sources are essential for more robust conclusions.

In our case, we had previously collaborated with the Complejo Hospitalario Universitario Insular—Materno Infantil, and were thus aware of the value and quality of the data provided. Consequently, we decided to employ them with confidence.

4. Methods

4.1. Research Framework

In this paper, a customized deep learning model is developed. The model is used to retrieve missing medical data from different medical tests using the values obtained from different patients. Up to 3 types of artificial neural network algorithms are used to obtain the best performing one. In addition, the model allows the parameters of the neural networks used to be configured in order to obtain the best result. These configurable parameters are the number of layers, the size of the training sample, the batch size, the number of epochs and the optimisation algorithm. Figure 1 shows the research framework used.

4.2. Preprocesing

The first step to perform before using the model is to preprocess the database. The ideal situation described by the hospital’s own medical staff would be to have a database in which 3 values for each medical variable, listed in Table 2, are obtained for each patient each day.

These 3 values of the medical variable would be obtained throughout the day and would be distributed in 3 slots, depending on the moment they are obtained, as follows:

• Slot 1: value of the medical variable obtained between 00:00 and 08:00 h.
• Slot 2: value of the medical variable obtained between 08:00 and 16:00 h.
• Slot 3: value of the medical variable obtained between 16:00 and 00:00 h.

Figure 2 shows what the ideal situation would look like in which, for one patient, 3 values of a medical variable have been obtained. This ideal situation would be repeated every day, for each medical variable, for each patient.

Given that the database provided by the Complejo Hospitalario Universitario Insular—Materno Infantil de Las Palmas de Gran Canaria does not have this ideal situation, the first step to perform is adapting it to follow the pattern of the ideal scenario. To achieve this, the data are divided into blocks. For each patient, 8 blocks will be obtained (1 for each medical variable), and there will be a total of 1712 blocks, as there are 214 patients. Each block will be divided into days (as many days as the patient is admitted to the ICU), and each day will be divided into 3 slots (following the scheme in Figure 2). Once this slot structure is available, the value of the medical variable will be assigned to the corresponding slot. If no medical variable value is available in a slot, this slot will remain empty.

Figure 3 shows a possible scenario of the database after adaptation.

Once the data in the different blocks and time slots have been pre-processed, it is possible to determine in an orderly way which values of the medical variables are already present and which values are missing. Once we have each block, it is necessary to standardize them. Data standardisation is a technique commonly used in ML, including the use of neural networks, to normalize data before training a model. In this way, model validation metrics can then be applied to search for the best possible configuration. A common way to normalize data is to calculate the standard deviation and mean for each test and then subtract the mean and divide the number by the standard deviation. This is known as z-score standardisation. In this way, each test value will be on a similar scale, regardless of the units of the original medical test. The following equation shows how this z-score standardisation is performed.

$$Z=\frac{x-\mu }{\mathsf{\sigma }}$$

(1)

where $Z$ is the standard score, $x$ is the block values, $\mu$ is the sample mean and $\mathsf{\sigma }$ is the sample standard deviation.

Subsequently, by applying the proposed model to each block, the missing values for each patient and type of medical test are obtained.

4.3. Neural Networks Architecture

In this process, the selection of the algorithm type and the number of networks in the ensemble is crucial, as illustrated in Figure 1. The proposed model utilizes three types of algorithms: MLP, GRU, and LSTM. However, it is essential to explain why an ensemble of I networks for each algorithm type is used (i being an integer representing the number of networks in the architecture). Ensembles of neural networks have been shown to minimize generalization errors, provide higher accuracy, and enhance learning [38,39,40,41].

The operation of this ensemble of neural networks is straightforward. In our model, i networks operate in parallel and independently. The missing predicted values obtained from these i networks are subsequently averaged. This approach allows us to obtain the prediction of missing values as if only a single neural network were used, while benefiting from the advantages of utilizing a set of i neural networks.

Having described the ensemble of neural networks, let us explain the algorithms used in each network of the ensemble. It is important to note that although three types of algorithms are employed for each ensemble, only networks with the same type of algorithm can be used within an ensemble. For instance, if an ensemble with the LSTM algorithm is used, all i networks in the ensemble will exclusively utilize the LSTM algorithm, without mixing different algorithm types within a single ensemble.

The first algorithm employed is MLP. The MLP neural network learns through the process of the forward propagation and backpropagation of error. In forward propagation, the input is passed through the network, producing an output. The error between the expected output and the actual output is then used to update the network’s weights and biases via the error backpropagation algorithm [42].

The LSTM recurrent unit is a type of recurrent neural network that allows information to flow through time and utilizes gates to control the information flow. LSTM units consist of three gates: the input gate, the output gate, and the forgetting gate. The input gate determines how much information from the current input should be stored in the short-term memory. The forgetting gate determines how much information from the short-term memory should be discarded. The output gate determines how much information from the short-term memory should be sent to the output [43].

The GRU recurrent neural network also enables information to flow through time and utilizes gates to control the information flow. These gates are used to determine which information should be retained and which information should be discarded. The GRU unit comprises two gates: the update gate and the reset gate. The update gate decides which information from the previous input should be retained, and which information from the new input should be incorporated. The reset gate determines how much information from the previous input should be forgotten [44].

Figure 4 illustrates the diagrams and connections of the three types of algorithms used.

There exist various viable alternatives to MLP, LSTM, and GRU for establishing correlations among monitoring parameters in ICU COVID-19 patients. Among these alternatives, notable ones comprise Support Vector Machines (SVM), Decision Trees, Random Forests, Gradient Boosting Methods, and Convolutional Neural Networks (CNN). The adoption of MLP, LSTM, and GRU offers distinct advantages, primarily due to their ability to comprehend intricate associations and handle sequential data. These techniques excel at capturing temporal dependencies and long-term relationships within the data, which is crucial in the context of monitoring COVID-19 patients in the ICU. The aforementioned soft computing techniques display remarkable flexibility and adaptability across diverse data types and problem domains. They can be finely tuned and optimized to specifically address the challenge of establishing parameter relationships in COVID-19 patients within the ICU setting, thereby enabling more precise modelling and the enhanced interpretation of results.

4.4. Model Parameter Settings

In addition to selecting the algorithm type and the set of neural networks, it is possible to modify various parameters of the neural networks to optimize the prediction model. The following paragraphs explain the different parameters in the order of their configuration.

The first configurable parameter is the number of layers used in the model. A neural network with two layers can learn more complex features and perform more sophisticated tasks. This is because the first layer can learn increasingly abstract and complex feature representations as the network goes deeper. Two layers can be used in a similar sense, where one layer sequentially receives input and the other layer receives the output from the first layer. The output of the second layer is utilized for the final task, such as sequence classification. This architecture enables hierarchical representations of the data to be learned and enhances the network’s ability to capture long-term dependencies. In contrast, a bidirectional architecture employs two layers of neural networks, with one layer processing the input in order and the other layer processing the input in reverse order. The outputs of both layers are concatenated and used for the final task. This architecture is employed to capture dependencies in both directions of the sequence.

The second parameter that can be modified is the amount of data used for training the model, which consequently affects the number of data points used for testing the model. The test set must be distinct from the training set to ensure that the model has not overfit or memorized the training data. During training, the model utilizes a training dataset to learn how to make predictions. The larger the training dataset, the more information the model has to learn regarding patterns and relationships in the data, thereby improving its ability to make accurate predictions. The number of data points used in the test set should be large enough to provide an accurate assessment of the model’s performance on new and independent data.

Another parameter that can be modified in our model is the number of epochs. The number of epochs represents the number of times the complete training dataset is processed during the training process of a neural network model. The number of epochs should be sufficient for the model to learn complex patterns and relationships in the data.

The next configurable parameter is the number of hidden units. Hidden units are neurons that exist between the input and output layers of the network, without direct connections to the input or output. The number of hidden units influences the network’s ability to learn the relevant characteristics of the data and generalize them well to new data.

The optimization algorithm is another configurable parameter in our model. In the context of a neural network model, the optimization algorithm refers to the method used to adjust the weights of the connections between neurons during the training process. The goal of the optimization algorithm is to minimize the loss function, which measures the difference between the model’s output and the expected output. The proposed model will utilize two types of optimization algorithms: Adam and RMSprop. Adam is a popular optimization algorithm that combines gradient and quadratic gradient moments to adjust the neural network weights. It is efficient and generally produces satisfactory results across a wide range of problems. RMSprop is another optimization algorithm that adapts the learning rate based on the gradient magnitude. RMSprop tends to perform well in non-stationary optimization problems.

The final parameter that can be configured in our model is the batch size. The batch size is an important parameter in the training of a neural network model, referring to the number of training examples processed in a single iteration of the training algorithm.

There are additional parameters that can be adjusted, including the initial learning rate, learning rate scheduling, or learning rate decay factor, among others. However, the selection of these parameters is based on prior research in the field of medical variable prediction [34]. This research has demonstrated that these parameters yield promising and reliable outcomes in terms of prediction and accuracy.

Furthermore, the parameter choices take into consideration technical constraints, such as limitations in computational resources or runtime requirements. The chosen parameters strike a favourable balance between model performance and computational efficiency, ensuring that the model remains practical and applicable within clinical or healthcare contexts.

5. Experimental Methodology

This section will explain the methodology used in this study to recover the values of the monitoring parameters for COVID-19 patients. The objective of the application of this methodology was to modify the proposed model in order to achieve the best results. Figure 5 shows the flow chart followed.

The methodology employed can be described as follows. Once the data has undergone pre-processing and sorting, the neural network architecture is chosen. This involves selecting the algorithm type and the number of networks in the ensemble. The three types of algorithms (MLP, GRU, and LSTM) are applied independently to each block, with the number of networks in the ensemble varied to obtain the predicted values. The optimal architecture is determined based on the lowest RMSE (Root Mean Square Error) and computational cost. This selected architecture is then used in the model to retrieve the missing data.

After selecting the architecture type, the next step is to modify the different configurable parameters in the model to achieve the highest accuracy. Various scenarios are designed by adjusting these parameters to find the best model. The selection of the best model parameters involves independently re-selecting each block and determining the predicted values.

First, the number of layers used is modified. In our model, we consider three possible options: one layer in the processing direction, two layers in the processing direction, or one bidirectional layer.

Subsequently, the ratio between the training and test data is adjusted, followed by the selection of the number of epochs.

Finally, the number of hidden units, the optimization algorithm, and the batch size are varied. To compare the best model configurations, the model is validated using the metrics explained in Section 5.1 each time a new parameter is adjusted.

There are multiple alternatives that should be considered within the proposed methodology of the study. These alternatives encompass a range of possibilities, including different data pre-processing techniques, cross-validation strategies, additional evaluation metrics, and the exploration of more advanced techniques such as semi-supervised learning or transfer learning.

The employed methodology in this study offers several notable advantages. It involves the careful selection of the most suitable neural network architecture and the meticulous optimization of model parameters. The impact of this methodology on the obtained results is significant. By judiciously choosing the appropriate neural network architecture and optimizing the model parameters, the accuracy and generalizability of the model are enhanced. Consequently, more precise predictions of the medical parameters of COVID-19 patients in the ICU can be achieved.

5.1. Validation Metrics

In order to evaluate the results obtained in the predictions of the values of the medical variables, three indicators are used. The first is the Root Mean Squared Error (RMSE), the second is the Mean Absolute Percentage Error (MAPE), and the third is the Mean Absolute Error (MAE), as shown in the following equations:

$$RMSE=\sqrt{\frac{{\sum }_{j=1}^J{\left({ModelPredictiveValue}_j-{RealValue}_j\right)}^2}{J}}$$

(2)

$$MAPE=\frac{1}{J}\sum _{j=1}^J\left|\frac{{ModelPredictiveValue}_l-{RealValue}_j}{{RealValue}_j}\right|$$

(3)

$$MAE=\frac{1}{J}\sum _{j=1}^J\left|{ModelPredictiveValue}_j-{RealValue}_j\right|$$

(4)

Besides RMSE, MAPE, and MAE, there are additional alternatives available for assessing the prediction results of medical variables, such as the coefficient of determination (R²), Lin’s concordance index (CCC), relative mean error (MER), the correlation coefficient of concordance (CCC), and the cross-validation index (CVI).

The utilization of RMSE, MAPE, and MAE as metrics for evaluating the prediction results of medical variables offers significant advantages. These metrics are widely used in the field, enabling informed decision making and facilitating comparisons with other studies or models. Their simplicity, interpretability, and ability to provide a quantitative assessment of model performance make them highly advantageous.

By employing the selection of these three metrics (RMSE, MAPE, and MAE), various aspects of the prediction results of medical variables can be evaluated. RMSE, by considering the square root of squared errors, penalizes larger errors and provides a measure of the overall dispersion of errors. MAPE, by measuring the average percentage error, enables an assessment of the model’s relative performance and its ability to predict values accurately in percentage terms. MAE, by calculating the average absolute error, directly measures the average magnitude of errors.

6. Results and Discussion

6.1. Results

The objective of this study was to obtain and establish the model with which the best results could be obtained. For the comparison of results, the following initial configuration of the model was established:

• Number of layers: One-layer sequence processing.
• Training data/testing data: 60%/40%.
• Optimization algorithm: Adam.
• Number of hidden units: 200.
• Number of epochs: 50.
• Batch size: 128.

6.1.1. Neural Network Architecture Comparison

We tested the three algorithms described above. MLP is a direct feed-forward neural network with no memory capability, while GRU and LSTM are recurrent neural networks with recurrent connections and gates that allow them to remember previous sequence information. In addition, we can compare the results of the three algorithms as a function of the number of neural networks forming the ensemble in order to find the optimal architecture.

Table 3. Results of the model.

Number of Networks in the Ensemble	MLP	LSTM	GRU
1	32.125	16.113	15.717
2	32.005	16.100	15.727
3	31.821	15.932	15.327
5	31.754	15.726	15.102
7	31.652	15.592	14.906
8	31.594	15.452	14.887
10	31.459	15.326	14.776
12	31.458	15.322	14.752
14	31.456	15.320	14.756
15	31.456	15.320	14.753

shows the RMSE results of the three algorithms as a function of the number of networks in the ensemble. Furthermore, Figure 6 displays the outcomes obtained from assessing the assembly’s performance with varying numbers of nets. The aim is to identify, based on the obtained results, the optimal number of nets required to form the assembly for each of the three architectures.

Based on the results, the optimal architecture of the model can be established. In this case, the algorithm that offers the best results is GRU. As for the number of neural networks in the ensemble, it can be set at 10. From 10 networks onwards, the model hardly improves, and when deciding whether to compromise between computational cost and the best results, it can be determined that 10 networks is the best solution. Using the previous data, the next step is to modify the parameters to determine which is the best model. In addition to the initial parameters described in Section 6.1, we will assume that the model uses the GRU algorithm and 10 neural networks in the ensemble. In the following sections, the different parameters will be modified.

6.1.2. Number of Layer Comparison

In this configuration, we test whether it is more effective to use neural networks with one layer in the processing direction, to use two consecutive layers in the processing direction, or to use a bidirectional layer that reads data in both directions. The results indicate that, for our model, it is most effective to use a single layer in the processing direction, as shown in

Table 4. Results of the model.

Layers	RMSE	MAPE	MAE
One-layer sequence processing	14.776	2.733	12.137
Two-layer sequence processing	19.453	3.891	16.914
Bi-directional	21.412	4.211	17.165

.

6.1.3. Training Data Size Comparison

To choose the training and test data, a random cross-validation approach is implemented, which entails dividing the dataset into multiple randomized subsets. Given that the model comprises a collection of 10 networks, as previously mentioned, each network within the ensemble will be assigned a distinct random subset for training and testing purposes. This cross-validation technique serves to address overfitting concerns by striking a balance between the model’s performance using the training data and its ability to generalize to unseen data. The proportion of data used for training and testing is also varied, the results of which are shown in

Table 5. Results of the model.

(%) Data Train: (%) Data Test	RMSE	MAPE	MAE
50:50	15.245	1.271	12.526
60:40	14.776	2.733	12.137
70:30	14.949	3.204	12.339
80:20	15.017	8.970	12.455
90:10	15.530	8.723	13.154

. The best results are obtained when we use 60% of the data for training and 40% for testing.

6.1.4. Number of Epochs Comparison

Each epoch involves a pass through the entire data set and an adjustment of the model weights to reduce error. In general, a larger number of epochs can improve model performance by allowing the learning algorithm to adjust the model weights more accurately. However, too many epochs can lead to over-fitting, which means that the model fits the training data too well and does not generalize well to new data.

Table 6. Results of the model.

Number of Epochs	RMSE	MAPE	MAE
25	16.774	4.125	14.284
50	14.776	2.733	12.137
100	12.379	3.416	7.724
200	10.237	3.876	5.342
300	10.233	3.561	5.122

shows the results obtained. Considering the above, and looking at the results in Table 6, the best solution would be to use 200 epochs, as the model would not improve much more but still consume a lot of computational resources.

6.1.5. Comparison of Number of Hidden Units

A common approach is to start with a small number of hidden units and gradually increase the number until the model’s performance on a validation set stops improving noticeably. In general, a neural network with more hidden units can learn more complex features and represent more complex functions, but it also requires more computational resources and can lead to over-fitting. Considering the above, and looking at the results in

Table 7. Results of the model.

Number of Hidden	RMSE	MAPE	MAE
100	11.834	4.261	6.423
200	10.237	3.876	5.342
300	10.007	3.963	6.344
400	9.973	4.972	6.949
500	9.616	6.077	7.004
600	9.317	6.572	7.714
650	9.311	6.569	7.911
700	9.307	6.422	7.111

, the best solution would be to use 600 hidden units, as the model would not improve much further but still consume a lot of computational resources.

6.1.6. Optimization Algorithm Comparison

The matching algorithms used are Adam and RMSProp. Adam uses both the moving average and the moving variance of the gradients, has an adaptive learning rate and uses one moment in the parameter update. On the other hand, RMSProp only uses the moving average of the gradients and has an adaptive learning rate per parameter dimension.

Table 8. Results of the model.

Optimization Algorithm	RMSE	MAPE	MAE
Adam	9.317	6.572	7.714
RMSProp	7.237	5.572	4.791

shows that the best model is the one that uses the RMSProp algorithm.

6.1.7. Batch Size Comparison

A small batch size may be more efficient in terms of memory and training time, but may also lead to greater variability in weight updates. A large batch size may provide more stable updates but may require more memory and may be slower in training. Considering the above and the results shown in

Table 9. Results of the model.

Batch Size	RMSE	MAPE	MAE
16	7.277	5.751	4.961
32	7.256	5.665	4.813
64	7.241	5.582	4.853
128	7.237	5.572	4.791
256	7.331	6.071	4.882

, the best configuration is the one that uses a batch size of 128.

6.1.8. Best Model

Considering the best results obtained when the different parameters have been modified, the model with the following characteristics is selected:

• Algorithm: GRU;
• Number of networks in the ensemble: 10;
• Number of layers: one-layer sequence processing;
• Training data/testing data: 60%/40%;
• Optimization algorithm: RMSProp;
• Number of hidden units: 600;
• Number of epochs: 200;
• Batch size: 128.

Table 10. Results of the model.

RMSE	MAPE	MAE
7.237	5.572	4.791

shows the results of the final model used in this paper.

For a better understanding of the results obtained,

Table 11. Results of grouping data according to medical variables.

Medical Variable	RMSE	MAPE	MAE
1. CK–MB mass	2.765	6.226	2.042
2. Hemoglobin	1.512	0.615	0.909
3. Procalcitonin	1.065	7.123	0.411
4. Serum Urea	11.144	1.312	6.654
5. Lactate	0.154	0.523	0.095
6. PCR	1.776	21.593	1.101
7. Capillary blood glucose	36.547	0.795	26.213
8. Leucocytes	2.469	3.876	1.284

evaluates the results of the different medical tests using the three indicators RMSE, MAPE and MAE.

6.2. Discussion

One of the key strengths of our model resides in its performance relative to previous studies. With an RMSE value of 7.237, our model surpasses the results reported in [34], with an RMSE of 21.5 (data collected every 30 min), those reported in [35], with an RMSE of 33.27 (data collected every 60 min), and those reported in [36], with an RMSE of 37.7 (data collected every 60 min). Despite our data being collected at intervals of 8 h, our model demonstrates a superior performance, underscoring its robustness and capacity to generate accurate predictions. This suggests that our model holds promise as a valuable tool in clinical settings where frequent data collection may pose challenges, offering reliable predictions based on less frequent measurements.

It is important to assess the performance of the model. To do so, we can check how the model behaves as a function of the number of days the patient is in hospital. Figure 7 shows the histogram that enables us to determine the distribution of the data of the different blocks according to the number of days the patient is in hospital, in order to detect patterns or trends.

Figure 7 shows that the highest frequency with which different medical test values appear in the dataset is during the first 20 days that the patient is in hospital. It may be interesting to demonstrate the behaviour of the model used if we only consider medical tests performed on patients who are in hospital for 20 days or less, and if we only consider medical tests performed on patients who are in hospital for more than 20 days.

Table 12. Results of the model with patients admitted for 20 days or less.

	RMSE	MAPE	MAE
(≤20 days in hospital)	5.757	1.265	3.765
(>20 days in hospital)	8.967	8.145	5.402

shows the results of the model in these cases.

Table 12 showcases the findings of applying the model to patients with a hospital stay of 20 days or less. It is clear that the model exhibits superior performance in the initial scenario, in which patients are hospitalized for a shorter time. In this context, the RMSE, MAPE, and MAE values demonstrate a lower magnitude compared to the subsequent scenario. However, it is crucial to acknowledge that the increase in RMSE in the second scenario may be attributed to outliers or exceptional cases. Such occurrences can significantly influence the evaluation metrics, resulting in a higher RMSE. Nevertheless, the model sustains its overall effectiveness. Furthermore, it is essential to consider the average duration of hospitalization in the utilized database, which amounts to 12 days. This indicates that the majority of patients align with the first scenario, encompassing 81% of the dataset. Consequently, the model performs admirably in scenarios involving the vast majority of patients. Despite the potentially higher RMSE values in the second scenario, it is worth noting that most patients belong to the first scenario, in which the model proves effective. This justification stems from the clinical relevance of the acquired results for the majority of patients, along with the recognition that outliers can impact the evaluation metrics.

By examining the individual medical variables presented in Table 11, we can gain deeper insights into the model’s performance. The varying levels of accuracy across specific variables indicate that our model exhibits notable effectiveness in predicting certain parameters, while it could benefit from further fine-tuning in others. This observation unveils avenues for future research and enhancements to the model by delving into the underlying factors that influence distinct prediction performances across different medical variables. This understanding can inform targeted improvements and potentially facilitate tailored predictions tailored to the unique characteristics of each variable.

Suggestions can be proposed to enhance the obtained outcomes. One prospective recommendation is the implementation of external validation. In order to thoroughly evaluate the model’s robustness, the option of conducting external validation by utilizing diverse data sources or medical centres can be explored. This approach would effectively gauge the model’s capacity to extrapolate beyond the original study data and corroborate its utility in diverse clinical scenarios. Moreover, the integration of supplementary temporal information holds potential. Alongside the data’s sampling frequency, contemplating the inclusion of additional temporal indicators within the model could notably augment the prediction accuracy. For instance, incorporating trends or patterns of temporal variation into medical variables could significantly improve the capturing of disease progression, thereby enabling more precise predictions.

Our predictive model also provides an opportunity to advance our comprehension of disease progression and treatment response. Through an analysis of the associations between the predicted medical variables and the patient outcomes, we can enhance our understanding of the fundamental mechanisms and factors that contribute to the severity and advancement of diseases. This understanding, in turn, can inform the development of targeted interventions and personalized treatment strategies. Furthermore, the capability of our model to generate precise predictions using less frequent measurements emphasizes the significance of identifying pivotal moments and critical factors that exert a substantial influence on patient outcomes. This knowledge can serve as a guiding principle for future investigations, facilitating the refinement of our model and ultimately leading to enhanced patient care and the effective management of intricate ailments.

7. Conclusions

There exists a wide range of clinical and analytical values present in COVID-19 patients during their stay in the Intensive Care Unit (ICU). In the field of medicine, the clinical utility of this mathematical model is crucial in determining which analytical and clinical variables contribute to the better prediction of outcomes. From a medical perspective, creating predictive models based on parameters that have a strong relationship with favourable results is vital for selecting appropriate medical and analytical values to include in the study of these patients during their hospital admission.

It is important to acknowledge that the medical variable data used in this paper are highly dynamic and therefore pose significant challenges in prediction. While there are several articles utilizing medical variable data from COVID-19 patients in the ICU [20,45,46], these studies focus more on predicting the patient’s overall condition rather than estimating the specific medical variables themselves.

As discussed throughout the document, no articles have been found that employ machine learning algorithms to simultaneously predict the multiple medical variables of ICU patients. Some papers do predict individual medical variables, such as glucose [34,35,36] or body temperature [37]. Therefore, the content of this paper, with its ability to predict multiple medical variable values simultaneously, can be considered an innovative approach.

One limitation of this paper could be the relatively long time intervals between medical variable measurements. If medical variable values were captured at shorter intervals, such as every 4 or 2 h, the RMSE value would likely decrease as the data would be more dynamic.

Regarding future directions, one avenue of research is the selection and appropriate combination of multiple medical parameters to create more robust mathematical models. Additionally, further testing of the neural network model could be conducted on different patient groups, such as individuals of various ages or with different overall health states, in order to assess the model’s accuracy in these specific circumstances. It would also be beneficial to incorporate more data when training the neural network model, potentially improving the accuracy of the predictions. Furthermore, comparing the performance of the neural network model with other machine learning models, such as decision tree models or different types of neural networks, could help determine the most suitable approach to predicting medical test values in COVID-19 ICU patients. It would also be valuable to conduct further analyses to understand why the model achieves higher accuracy for certain medical variables and explore methods that can be used to enhance accuracy for these specific variables.

Another potential area for future research involves predicting COVID-19 disease risk distributions in diverse geographic locations. This can be achieved by utilizing demographic, epidemiological, and mobility data within a spatio-temporal model that captures the complex dynamics of disease transmission. Notably, the adoption of the KL–MLP–LSTM model, previously developed and shown to be effective in estimating risk distributions during disease outbreaks [47], could be a promising approach.

Furthermore, developing a neural-network-based classification model presents an intriguing opportunity to leverage COVID-19 data and extract relevant features such as age, gender, and other risk factors. Such a model can play a crucial role in predicting patients’ classification into specific risk groups, thereby enhancing risk assessment and stratification efforts. Drawing inspiration from the model proposed in [48], the utilization of a convolutional neural network (CNN) can facilitate the construction of a hierarchical structure of learned features, enabling the accurate identification of risk groups.

Lastly, the model applied in this paper could find utility in clinical practice by providing additional information to medical professionals and improving the management and diagnosis of patients displaying COVID-19-like symptoms who may be at risk of future diseases.