Translation and Validation of the Boston Technical Performance Score in a Developing Country

Introduction The Technical Performance Score (TPS) was developed and subsequently refined at the Boston Children's Hospital. Our objective was to translate and validate its application in a developing country. Methods The score was translated into the Portuguese language and approved by the TPS authors. Subsequently, we studied 1,030 surgeries from June 2018 to October 2020. TPS could not be assigned in 58 surgeries, and these were excluded. Surgical risk score was evaluated using Risk Adjustment in Congenital Heart Surgery (or RACHS-1). The impact of TPS on outcomes was studied using multivariable linear and logistic regression adjusting for important perioperative covariates. Results Median age and weight were 2.2 (interquartile range [IQR] = 0.5-13) years and 10.8 (IQR = 5.6-40) kilograms, respectively. In-hospital mortality was 6.58% (n=64), and postoperative complications occurred in 19.7% (n=192) of the cases. TPS was categorized as 1 in 359 cases (37%), 2 in 464 (47.7%), and 3 in 149 (15.3%). Multivariable analysis identified TPS class 3 as a predictor of longer hospital stay (coefficient: 6.6; standard error: 2.2; P=0.003), higher number of complications (odds ratio [OR]: 1.84; 95% confidence interval [CI]: 1.1-3; P=0.01), and higher mortality (OR: 3.2; 95% CI: 1.4-7; P=0.004). Conclusion TPS translated into the Portuguese language was validated and showed to be able to predict higher mortality, complication rate, and prolonged postoperative hospital stay in a high-volume Latin-American congenital heart surgery program. TPS is generalizable and can be used as an outcome assessment tool in resource diverse settings.


INTRODUCTION
Congenital heart disease (CHD) remains a major healthcare issue, and mitigating CHD mortality in developing countries could have a significant impact on the elevated childhood mortality rates [1][2][3] . As the vast majority of CHD require surgical treatment, improvement in surgical outcomes is highly desirable. Several factors affect the final outcome of CHD repair, and predictors of unfavorable events can often be identified before surgery [4][5][6] .
It is known that an optimal repair with trivial or no residual lesions results in better outcomes to the patient. However, it is not uncommon to find some degree of residual lesions after surgical repair, particularly after complex surgeries with multicomponent repairs [7,8] .
The big question is the acceptable level of residual lesion. Or yet, what is the real impact of these echocardiographic findings on the patient's outcomes? The Boston Children's Hospital developed a quality assessment tool, the Technical Performance Score (TPS) [9][10][11][12][13] , that quantified the degree of residual lesions based on echocardiographic and clinical findings.
Our objective was to translate TPS into the Portuguese language and to validate its utility and significance in a developing country with less predictable outcomes for CHD surgery [14][15][16][17] .

Translation
We performed the translation using international protocols for the validation of scales and scores [18] . Two cardiac surgeons with a degree in English language made two separate versions of the TPS translated to Portuguese (LAM, DFT) [19] . Then, a panel of experts selected a final version. The third step consisted of a back-translation to the English language performed by a professional translator.
Finally, consensus Portuguese and back-translation versions were sent to the author (MN) for approval [19] . The final Portuguese TPS version is available as a supplement file.

Validation
We included consecutive surgeries performed at a single institution between June 2018 and October 2020. Institutional approval with waiver of consent was obtained for this quality improvement initiative under the registration number 95807418.1.0000.0068. We collected demographic data and calculated the complexity risk score using the Risk Adjustment in Congenital Heart Surgery (RACHS-1) mortality categories [20,21] . Cardiopulmonary bypass (CPB) and aortic cross-clamping times, serum lactate levels at the end of surgery, vasoactive inotropic Abstract Introduction: The Technical Performance Score (TPS) was developed and subsequently refined at the Boston Children's Hospital. Our objective was to translate and validate its application in a developing country.
Methods: The score was translated into the Portuguese language and approved by the TPS authors. Subsequently, we studied 1,030 surgeries from June 2018 to October 2020. TPS could not be assigned in 58 surgeries, and these were excluded. Surgical risk score was evaluated using Risk Adjustment in Congenital Heart Surgery (or RACHS-1). The impact of TPS on outcomes was studied using multivariable linear and logistic regression adjusting for important perioperative covariates.
Results: Median age and weight were 2.2 (interquartile range [IQR] = 0.5-13) years and 10.8 (IQR = 5.6-40) kilograms, respectively. Inhospital mortality was 6.58% (n=64), and postoperative complications occurred in 19 score (VIS) in the immediate postoperative period and in the first 24 hours following surgery, and TPS were assigned as outlined below. Measured outcomes were the incidence of postoperative complications, postoperative length of hospital stay (LOS), and mortality.

Technical Performance Score
TPS was calculated based on the last postoperative echocardiogram available before discharge or death and other clinical and laboratory data as published in previous articles from the Boston group [8,[9][10][11][12][13]22] . The TPS was calculated for the index operation of each hospitalization. Index operation is defined as the first congenital heart surgery performed either on CPB or off bypass.

Vasoactive Inotropic Score
We calculated VIS when the patient arrived at the intensive care unit and then 24 hours later [23,24] .
Reoperations or surgical and catheter based reintervention on the anatomic areas of repair were not considered as adverse events because they are components of TPS.

Operative Mortality
Operative mortality was considered as death in the hospital or within 30 days of the operation if the patient was discharged before this period.

Exclusion Criteria
Operations where TPS could not be calculated either due to lack of availability of postoperative echocardiogram or because the surgery performed does not have a TPS developed yet were excluded.

Statistical Analysis
Data were described using medians and interquartile ranges (IQR) of 25 and 75% for continuous variables and absolute numbers and percentages for categorical variables.
We considered individual index surgeries rather than individual patients in our analyses because the same patient may have had more than one index surgery during the study' s time span. For example, a patient who underwent Norwood procedure as a neonate, was discharged home, and four months later came back for a programed bidirectional Glenn operation was computed as two surgeries. However, if the patient had more than one surgical procedure in the same hospital stay, only the index surgery was considered.
The primary outcome was operative mortality. Secondary outcomes included postoperative complications and postoperative LOS. Other covariates included in adjusted analyses consisted of perioperative variables such as age, weight, case complexity as defined by RACHS-1 mortality categories, VIS and serum lactate at the end of surgery, and VIS 24 hours after surgery.
To study the associations between categorical outcome variables (mortality and postoperative complications) and explanatory variables, simple and multivariable logistic regression models were used. To evaluate the multivariable models, C statistic and receiver operating characteristic curve were used.
To analyze the associations between the continuous outcome (postoperative LOS) and the explanatory variables, simple and multivariable linear regression models were used.
The level of significance adopted in the tests was 0.05. Twotailed hypotheses were considered. The software R (version 3.6.0) was used to carry out all the analyzes.

RESULTS
A total of 1,030 consecutive eligible index surgeries between June 2018 and October 2020 were analyzed, 58 surgeries were subsequently excluded because TPS could not be assigned. Median age of the 972 patients was 2.2 (IQR = 0.5-13) years, and median weight was 10.8 (IQR = 5.6-40) kilograms (kg). Overall operative mortality was 6.58% (n=64), and postoperative complications occurred in 192 (19.7%) cases. Demographic distribution and overall outcomes are described in Table 1. Mortality in patients with TPS class 3 was statistically higher than in those with TPS classes 1 and 2 (TPS class 3 = 18.1%, TPS class 2 = 5.4%, and TPS class 1 = 3.3%; P<0.001). The distribution of mortality according to TPS is shown in Figure 1A and Table 2.
Aditionally, patients with higher RACHS-1 mortality risk had a higher chance of death, complications, and LOS. Likewise, a higher incidence of inadequate performance (TPS class 3) was observed in patients with higher RACHS-1 mortality category (Tables 2 and 3).
Multivariable analysis to determine the association between TPS and outcomes adjusted for important perioperative variables including case complexity determined that TPS class 3 was associated with a 1.8 times greater chance of postoperative complications, and 3.2 times greater chance of death compared to TPS class 1. There was an average of 6.6 more days of hospitalization in the postoperative period in surgeries classified as TPS class 3. On the other hand, outcomes of surgeries assigned as TPS class 2 were not statistically different from those assigned as TPS 1 (Tables 4, 5, and 6).

DISCUSSION
Outcomes after congenital heart surgery are consequences of multiple variables. Therefore, it becomes challenging to predict the impact of surgical correction itself on the final result [27] .
Since 2018, we have been using TPS as part of our congenital cardiac surgical team's performance analysis. Our findings in 972 surgeries strengthens the role of TPS as a predictor of adverse outcomes after adjusting for other covariates. Patients assigned as an inadequate surgical result had 3.2 times more chance of death and 1.8 more chances of complication.
Our multivariable models presented a significant AUC (0.86 for mortality and 0.74 for complications), even including perioperative variables, some of which collinear with TPS (CPB Since 2007, researchers at the Boston Children's Hospital have been demonstrating that TPS is an important predictor of short and medium-term outcomes [6,[9][10][11][12][13]22,28] . Researchers from England and Japan also evaluated their surgical performance and postoperative outcomes using TPS with similar findings [29,30] . Therefore, the importance of translating TPS into Portuguese and its routine use and validation in our environment was time, VIS, and lactate levels). Patients with TPS class 3 who were discharged home stayed in hospital an average of six more days when compared to patients whose surgery was assigned as TPS classes 1 and 2. This demonstrates that pursuing a more technically adequate surgery, in addition to saving lives, may decrease resource utilization, leading to cost savings and the possibility of operating on more patients.

Limitations
This is a retrospective single-center study with its inherent limitation related to missing data, inaccuracy, and incompleteness in documentation of events. Generalizability of this study's findings may be limited to centers with a similar case mix and resource distribution. We also included factors such as CPB times, key to increase the awareness of this tool among our surgical community. Our expectation is that this will also allow comparison and evaluation of surgical performance in smaller programs with erratic outcomes, helping them identify opportunities for improvement.
Nevertheless, we observed considerably higher mortality and complication rates in surgeries with TPS assigned as classes 1 and 2 than have been previously reported in developed countries, suggesting that TPS explains some, but not all of the differences in outcomes across the globe [22,31] .
Jacques et al. [32] previously demonstrated that, in their experience, postoperative errors had more impact on mortality  Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved; final approval of the version to be published Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved; final approval of the version to be published Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved; final approval of the version to be published Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved; final approval of the version to be published Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved; final approval of the version to be published Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved; final approval of the version to be published aortic cross-clamping times, VIS, and lactate levels that are known to be collinear with TPS and thus may have diluted the effects of the covariates in the model. However, despite these limitations, this study provides outcome analyses on a large sample from the biggest Latin-American cardiology center.

CONCLUSION
TPS class 3 was a predictor of higher mortality and complications with consequent longer postoperative LOS in a developing country high-volume congenital cardiovascular surgery program. The present study further corroborates the validity of TPS as a predictor of postoperative outcomes. The translation of TPS associated with its guidelines may help expand adoption in our environment and assist more programs in targeting improvements in outcomes.