Development and validation of a nomogram to predict postoperative pulmonary complications following thoracoscopic surgery

Study population

This retrospective study was approved by the Ethics Committee of the Second Affiliated Hospital of Anhui Medical University (No. YX2020-038) and registered in the Chinese Clinical Trial Registry (No. ChiCTR2000033329), in accordance with the TRIPOD checklist. From August 2018 to March 2020, 905 patients who underwent thoracoscopic surgery at the Second Affiliated Hospital of Anhui Medical University were enrolled and analyzed. The following were exclusion criteria: emergency and trauma patients; age <18 years old; patients with pulmonary infection and pleural effusion before the operation; patients who underwent thoracotomy or if the operation was cancelled; patients who underwent re-operations; and patients whose relevant data were missing or incomplete.

According to the requirements of the ethics committee, informed consent was carried out according to the following methods: A. If the patient has passed away or cannot be contacted, apply for exemption of informed consent according to the process of ethics committee; B. If patients agree to include their medical records in this study, they should sign the informed consent; C. If the patient agrees to provide his/her data for this study while does not want to sign the informed consent form, a witness must be present. The data can only be included after the witness signed in the informed consent; D. If patients explicitly refuse to include their medical records in this study, their data will be excluded.

Data collection

Based on clinical importance, scientific knowledge, previous research (Agostini et al., 2010; Licker et al., 2006; Stephan et al., 2000) and the actual data that could be collected for a retrospective study, 27 demographic and clinical variables of patients undergoing thoracoscopic surgery were collected. General information included age, sex, American Society of Anesthesiologists (ASA) grade, preoperative mean arterial pressure (MAP), body mass index (BMI), and history of hypertension, diabetes mellitus, stroke, heart disease, COPD, smoking (ever or current), and alcohol use. Laboratory examination data before the operation included leukocyte, red blood cell, and platelet counts and aspartate aminotransferase, alanine aminotransferase, blood glucose, serum creatinine, blood urea nitrogen, serum sodium, serum potassium, and serum albumin levels. Information on the operation and anesthesia management included the duration of the operation, airway management, the duration of one-lung ventilation, and tpyes of operation.

The primary end-point of this study was PPCs, which were defined according to the European Perioperative Clinical Outcome (EPCO), including respiratory infection, respiratory failure, pleural effusion, atelectasis, pneumothorax, bronchospasm, and aspiration pneumonia (Jammer et al., 2015; Miskovic & Lumb, 2017) (Table S1). An event was considered a PPC if one of these complications was detected within the first 7 postoperative days. The diagnosis of PPCs was made by two experienced researchers based on the patient’s clinical data. In the case of a dispute, the diagnosis was discussed by the research group.

Statistical analysis

All patient demographic and clinical data were randomly divided into a training cohort and validation cohort according to an 80%:20% ratio by setting the random seed “1234567” in the “caTools” package of R software (version 3.6.3; http://www.r-project.org). The training and validation cohorts were used to derive and validate the model, respectively. Continuous variables such as age and BMI were categorized after being assessed based on clinical reference values and scientific knowledge. The data were compared using the χ² test or Fisher’s exact test, and the results are expressed as whole numbers and proportions.

Univariate logistic regression analysis was performed to identify risk factors for PPCs following thoracoscopic surgery in the training cohort. All variables associated with PPCs with a p-value less than 0.1 were candidates for backward stepwise multivariate analysis with the Akaike information criterion (AIC) to investigate independent risk factors. The nomogram was formulated based on the results of AIC multivariate logistic regression analysis in the training cohort, and selected variables were incorporated in the nomogram to predict the probability of PPCs.

The total scores of the nomogram were calculated for each patient in the training cohort. A receiver operating characteristic (ROC) curve was analyzed to calculate the optimal cut-off points based on the Youden index in the training cohort, after which the cut-off points in the training cohort were used in both cohorts to verify the generalizability of the model. The accuracy, precision, sensitivity, specificity, and the concordance index (C-index) were calculated, and ROC curves were constructed for the training and validation cohorts. The predictive performance of the nomogram was assessed by the C-index with 1000 bootstrap replicates and calibrated with 100 bootstrap replicates to decrease overfitting in the training and validation cohorts, respectively. A C-index value of 0.5 indicates no discrimination; a value of 1.0 indicates perfect discrimination ability with different outcomes. In an excellently calibrated model, the predictions should fall on a diagonal line of 45 degrees, which represents the ideal relationship between the observed outcome frequencies and the predicted probabilities.

All tests were 2-tailed, and p-values less than 0.05 were considered statistically significant. Statistical analysis was carried out using SPSS 22.0 software (Chicago, IL, USA) and R software. The R software packages included rms, foreign, caTools, tableone and pROC.

Patient characteristics

During the study period, 1084 consecutive patients underwent thoracoscopic surgery. Among the cases, 48 involved emergency and trauma surgery, 13 patients were younger than 18 years old, 21 cases were converted to open surgery, and there were missing or incomplete data for 97 patients. No patient explicitly refused to include his/her data for this research. Thus, a total of 905 patients who met the inclusion criteria were included in the present study. All patient demographic and clinical data were randomly divided into a training cohort (724 cases) and a validation cohort (181 cases) at a ratio of 80%:20% (Fig. S1). Of the included cases, 207 patients exhibited PPCs, with an incidence of 22.87%. The details of the PPCs are shown in Table S2. Among those who experienced PPCs, 166 patients (22.93%) were allocated to the training cohort, and 41 patients (22.65%) were allotted to the validation cohort. The demographic and clinical data of the training and validation cohorts are shown in Table 1. The p-values and standardized mean differences (SMDs) suggested that the partition of training cohort data and validation cohort data was balanced.

Independent risk factors for PPCs

Backward stepwise selection using the AIC in logistic regression modelling identified the following 7 variables as having the strongest associations with PPC risk: ASA grade III/IV, operation time longer than 180 min, one-lung ventilation time longer than 60 min, and history of stroke, heart disease, COPD, and smoking. According to the multivariable analysis, ASA grade III/IV [odds ratio (OR), 10.171; 95% confidence interval (CI) [5.598–18.926]; p < 0.001], operation time longer than 180 min (OR, 2.480; 95% CI [1.363–4.563]; p = 0.003) or longer than 300 min (OR, 12.516; 95% CI [5.232–17.766]; p < 0.001), one-lung ventilation time longer than 60 min (OR, 2.947; 95% CI [1.257–7.648]; p = 0.018), history of stroke (OR, 4.209; 95% CI [1.518–11.683]; p = 0.006), history of COPD (OR, 2.888; 95% CI [1.230–6.980]; p = 0.016), and history of smoking (OR, 4.101; 95% CI [2.544–6.654]; p < 0.001) were each independently associated with PPCs (Table 2).

Development and validation of a PPC-predicting nomogram

The nomogram acquired from the training cohort to predict PPCs after thoracoscopic surgery is shown in Fig. 1. The nomogram was created based on the following 7 factors: ASA grade (I/II or III/IV), operation time (<180, 180–300, or ≥ 300 min), one-lung ventilation time (<60 or ≥ 60 min), history of stroke (no or yes), history of heart disease (no or yes), history of COPD (no or yes), and history of smoking (no or yes). A higher point total based on the sum of the assigned number of points for each factor in the nomogram was associated with a higher risk of PPCs.

The total scores of the nomogram of each patient in the training cohort were calculated. ROC curves were analyzed to calculate the optimal cut-off point based on the Youden index in the training cohort, which yielded a value of 134 points. The accuracy, precision, sensitivity, specificity, and C-index are presented in Table 3. This cut-off point was used in both cohorts. The areas under the ROC curve (AUCs) were 0.894 (95% CI [0.866–0.922]) and 0.867 (95% CI [0.810–0.925]) in the training and validation cohorts, respectively (Fig. 2). In addition, the C-indices with 1000 bootstrap replicates were 0.894 (95% CI [0.866–0.921]) and 0.868 (0.811−0.925) in the training and validation cohorts, respectively (Table 2), indicating the good performance of the nomogram in estimating the risk of PPCs. The calibration plots with 100 bootstrap results of the training and validation cohorts are shown in Fig. 3, which vividly demonstrates good agreement regarding the presence of PPCs between the risk estimation by the nomogram acquired from the training cohort and the actual PPC occurrence in the training and validation cohorts.