Were Traditional Chinese Medicine Injections Ecacious for Angina Pectoris? A Frequentist Network Meta-Analysis of Randomized Controlled Trials

Background: The ecacy of traditional Chinese medicine injections (TCMIs) for angina pectoris has never been well investigated for lacking quality assessment of evidence. This study aimed to conduct a comprehensive and rigorous network meta-analysis and assess the quality of evidence according to the Grading of Recommendations and Assessment, Development, and Evaluation (GRADE) approach to compare the ecacy of all TCMIs in treating angina pectoris. Methods: Following the protocol (reference: randomized controlled trials (RCTs) which compared one TCMI with another TCMI or conventional treatments on anginal outcome measures (i.e. symptomatic improvement, electrocardiography improvement, symptomatic recovery, and electrocardiography recovery) were included. The risk of bias among included RCTs was assessed with the revised Cochrane’s risk of bias tool 2. Frequentist statistical analyses including subgroup analysis, sensitivity analysis, meta-regression and publication bias analysis were performed. The certainty of evidence was assessed with the GRADE approach. Results: Totally, 475 RCTs including all 24 TCMIs were identied, while the quality of all but two included RCTs was poor. According to the network meta-analysis, Honghua (Saower) injection were preferable both in improving symptoms and electrocardiography. However, signicant inconsistency showed the intransitivity among indirect comparisons, results in network meta-analysis seemed thus not trustworthy. The quality of evidence was assessed as low or very low. Conclusions: The low-quality evidence reduced the condence in the ecacious results. Current evidence hardly supports the benecial effects of TCMIs in treating angina pectoris. other TCMIs, identied the preferable treatments in improving electrocardiography. RCTs precious the ndings reproduce and misled healthcare decision-making. RCTs with unclear randomization, allocation concealment, or blinding methods tended to exaggerate the estimate of treatment effect [56, 57]. Of included RCTs in this only 2 RCTs [42, 43] reported detailed methods, such as random allocation, blind design, withdraw of participants and statistical analysis plan. Other included RCTs lacked adequate information on implementation. Eighty-four RCTs reported vague random sequence generation; 26 RCTs reported blind methods but lacked the information on specic process; RCTs reported the number of drop-out cases (Supplement 5). The overall quality of the included RCTs RCTs with inappropriate randomization or blinding methods pose a risk of 7%-23% exaggeration on the treatment effect [58, of TCMIs be than our ecacy The ecacy of TCMIs for angina direct


Introduction
Angina pectoris represents the clinical manifestation of temporary myocardial ischemia resulting from the insu cient coronary blood supply [1]. Patients suffer from episodes of pain or discomfort in the chest which could spread to the back, shoulder, the lower jaw or ngers lasting for several minutes [2]. It was estimated that angina pectoris affected 3.6% of population in China [3]. Guidelines recommended rst-line agents including beta-blockers (e.g., metoprolol), calcium channel blockers (e.g., verapamil), and nitrates (e.g., nitroglycerin) as conventional treatments for ischemia relief [4,5]. In China, traditional Chinese medicines (TCM) are commonly used as complementary or alternative therapy in treating angina pectoris [6,7].
Traditional Chinese medicine injections (TCMIs) are prepared by extracting active substances from the single herb or the herbal medicine compounds [8]. With a rapid onset and high bioavailability, TCMIs have an advantage for the management of circulatory diseases [9]. The sales of TCMIs for cardio-cerebrovascular diseases have reached over 10 billion dollars in 2016, accounting for two thirds whole market of TCMIs in Network meta-analysis evaluates comparative e cacy by synthesizing direct and indirect evidence of multiple treatments [12]. Without the restriction for head-to-head comparisons, network meta-analysis has become increasingly employed to recommend the optimal treatments in healthcare decision-making. Nevertheless, clinicians not only balanced the bene ts and risks of the intervention, but also considered the certainty We searched studies using TCMIs for angina pectoris to end of November 2019 in PubMed, the Cochrane Library, Embase, Web of Science, ScienceDirect, China National Knowledge Infrastructure Library (CNKI), Wanfang Data, ClinicTrials.gov. Chinese databases (i.e. CNKI and Wanfang data) were searched with the combination of the TCMIs and angina pectoris in title or abstract. The full-text search with the TCMI's synonyms was performed in English databases. We included RCTs in adults which were head-to-head comparisons of selected TCMIs, or comparing with conventional treatments, and included RCTs were required to report symptomatic improvement (SYM-I), symptomatic recovery (SYM-R) or electrocardiography improvement (ECG-I), electrocardiography recovery (ECG-R). De nitions of outcome measures were in accordance with guidelines for clinical research of TCMs or cardiovascular drugs [26,27].

Data extraction and quality assessment
Two authors (G Gao and Y Jia) independently extracted data and compared results. Data items were extracted as follows: (a) the rst author, (b) number of authors, (c) years of publication, (d) type of angina, (e) drugs and their dosages, (f) outcome data, (g) sample sizes, (h) follow-up periods, (i) gender proportions, (j) average ages, (k) methods of random sequence generation, (l) allocation concealment, (m) blinding methods.
Two authors (G Gao and Y Jia) independently assessed the quality of eligible RCTs according to the revised Cochrane's risk of bias tool for randomized trials (RoB 2) [28] with discrepancies resolved by consulting the third author (SW Leung). The RoB 2 was the structural approach to set a series of signalling questions for ve domains (randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome, the selection of the reported result). Following the algorithm based on answers to signalling questions, we judged each domain as low risk, some concerns or high risk. The overall risk of bias was determined by the lowest judgment in any of the domains [28].

Data analysis
We selected a random-effects model [29] to perform the pairwise meta-analysis and network meta-analysis with the R software (version 3.3.3) [30]. For each outcome measure, the treatment effect was presented as odds ratios (ORs) and 95% con dence intervals (CIs). Frequentist pairwise meta-analysis was conducted with the R package "metafor" [31]. Study heterogeneity was assessed by the statistic of I-square (I 2 ) [32]. For groups including at least 5 RCTs, we performed subgroup analysis, sensitivity analysis and meta-regression based on study characteristics and RCT quality. Publication bias was investigated with the funnel plot [33], Begg's rank correlation test [34], Egger's regression test [35], the trim-ll methods [36] and the Copas methods [37]. A p-value < 0.05 was considered statistically signi cant.
Frequentist network meta-analysis on each outcome measure was conducted using the R package "netmeta" [38]. The ranking of treatments was determined by the P-score, and P-score from 1 to 0 indicated the treatment effects decreased [39]. Inconsistency in network meta-analysis was assessed by total Q statistics which was composed of heterogeneity Q of within designs and inconsistency Q of between designs. Local inconsistency was explored by the node-splitting approach [40], and highlighted in the heat graph [41]. Besides, subgroup and sensitivity analyses based on study characteristics and RCT quality were performed to determine the preferable treatments on each outcome measure (Supplement 10).

Evaluation of evidence quality
According to the GRADE approach [15][16][17], the quality of evidence on each outcome measure was assessed as high, moderate, low, or very low.
The initial evidence from RCTs was "high", then would rate down in ve reasons (high risk of bias, imprecision, inconsistency, indirectness, or publication bias) [15].We downgraded the quality of evidence by high risk of bias (i.e. low quality of RCTs), imprecision (i.e. the 95% CI covering the threshold 1, the number of included studies less 5 ), inconsistency (i.e. I 2 > 30%, signi cant difference in subgroup analysis), or publication bias (i.e. statistically signi cant results of Begg's or Egger's test). The initial quality of indirect evidence was determined by the lower quality between pieces of direct evidence having the common comparator, then we downgraded the indirect evidence by imprecision whose 95% CI covered the threshold 1. The certainty of evidence from network meta-analysis depended on the higher quality between the direct and indirect e cacy evidence [16,17].

475 RCTs for inclusion
A total of 12654 records were identi ed through database searching. After removing duplicates and ineligible records, 475 RCTs (n = 48511) with all 24 TCMIs were nally included in this study (Supplement 3), of which 102 RCTs were head-to-head comparisons of TCMIs, 373 RCTs compared the TCMIs with convention treatments. All included studies were published in Chinese between 2000 and 2019. The sample size ranged from 30 to 789, the average sample size was 102.13, and the median sample size was 86. The follow-up periods ranged from 7 days to 60 days with a median of 14 days (Supplement 4). 46] had some concerns about the overall risk of bias considering the potential bias in selection of the reported result. Other 470 RCTs failed to describe detailed randomization, blinding methods, the change of the number of participants from initial to completing trials, the pre-speci c trial protocol or statistical analysis plan, leading to the overall high risk of bias (Supplement 5). The proportion of the high risk of bias in ve domains was 99.5% (randomization process), 87.79% (deviations from intended interventions), 87.79% (missing outcome data), 98.95% (measurement of the outcome), and 53.68% (the selection of the reported result).

E cacious overall effect sizes
Twenty-seven pairwise meta-analyses including 19 TCMIs compared with conventional treatments and 8 TCMIs of head-to-head comparisons were conducted on symptomatic outcome data (Table 1). Twenty-eight pairwise meta-analyses including 17 comparisons of conventional treatments and 11 head-to-head comparisons were conducted on electrocardiography outcome data ( Table 2). Results showed the interventions of treatment groups always more e cacious than comparators.    Table 3. We performed sensitivity analysis by respectively removing RCTs with high risk of bias or some concerns in each domain of RoB 2 [28].
Sensitivity analysis adjusting for included studies did not nd signi cant difference compared with the overall results.

Sample sizes, the average ages and follow-up periods as potential moderators in meta-regression
We conducted the meta-regression on included RCTs that showed the heterogeneity (i.e. I 2 > 30% or signi cant difference among subgroups) based on sample sizes, dosages, averages ages, the proportion of female participants and follow-up periods. Sample sizes, the average ages and follow-up periods seemed to be associated with the e cacy estimates (Table 4), while the impact of average ages on ECG-I of Huangqi (Astragalus) injection (β=-0.0988, p = 0.0131) should be carefully interpreted as the average age was a study characteristic rather than an individual characteristic. The result in the bracket of the coe cient estimate was the number of included studies with the available covariates.

Substantial publication bias among included RCTs
Seven pairwise meta-analyses including 157 RCTs on symptomatic outcome data and 10 pairwise meta-analyses including 114 RCTs on electrocardiography outcome data showed a signi cant publication bias. All but four adjusted ORs with the trim-ll methods [36] and the Copas methods [37] were no more than the overall results (Supplement 6).

Results of network meta-analysis
Symptomatic outcome data were reported in 317 RCTs and electrocardiography outcome data were reported in 252 RCTs (Supplement 13).
Evidence networks on symptomatic and electrocardiography outcomes were presented in Fig. 2.

Best treatments on primary outcomes
The network meta-analysis for symptomatic outcomes included 23 TCMIs with 31593 participants. Comparative e cacy among treatments on symptomatic outcomes was displayed in Fig. 3. Except for Xinmailong, Xiangdan and Mailuoning injection, other TCMIs showed better e cacy than conventional treatments on SYM-I ( Fig. 2(a)). Ciwujia injection had the rst rank on SYM-I according to the P-score (OR = 7.59, 95% CI: 2.85-20.21). Ciwujia injection was superior to other treatments in improving anginal symptoms, although only 3 RCTs on Ciwujia injection were incorporated into this network meta-analysis. We found substantial inconsistency in the overall network (Q = 2379.9, p < 0.0001). According to the heat plot (Supplement 8), the inconsistency was mainly contributed by the comparison of Danshen (Salvia miltiorrhiza) injections and conventional treatments, in which there was a statistically signi cant inconsistency between the direct and indirect comparison (z=-4.1, p < 0.0001).
The network meta-analysis for electrocardiography outcomes included 21 TCMIs with 26243 participants. Comparative e cacy among treatments on electrocardiographic outcomes was displayed in Fig. 4. All but Danshen (Salvia miltiorrhiza) injection showed a better performance than conventional treatments on ECG-I (Fig. 2(c)). Gualoupi (Pericarpium Trichosanthis) injection had the rst rank on ECG-I (OR = 7.48, 95% CI: 4.68-11.95). Gualoupi (Pericarpium Trichosanthis) injection was superior to other TCMIs in improving electrocardiography, while only 4 RCTs on overall network (Q = 1062.58, p < 0.0001). The heat plot (Supplement 8) indicated the inconsistency was mainly contributed by the comparison of Honghua huangsesu (Sa ower yellow) injection and conventional treatments, in which there was a statistically signi cant inconsistency between the direct and indirect comparison (z = 3.34, p = 0.0008).

Best treatments of secondary outcomes
Comparative e cacy among treatments on SYM-R was displayed in Fig. 3. Eighteen TCMIs for angina pectoris had better performance than conventional treatments on SYM-R (Supplement 9). Honghua (Sa ower) injection was ranked as the best treatment in terms of SYM-R (OR = 2.98, 95% CI: 2.98-4.38). Signi cant inconsistency was identi ed with the total Q statistic of 2379.9 (p < 0.0001) in the overall network of SYM-R.
Comparative e cacy among treatments on ECG-R was displayed in Fig. 4

Preferable TCMIs with robust superiority in ranking of treatments
As a statistical difference was observed in subgroup analysis based on conventional treatments (Table 3), we removed the RCTs comparing Dengzhanxixin (Erigeron breviscapus) or Danhong injection with conventional treatments (monotherapy or combinational therapy) to perform the sensitivity analysis. According to subgroup and sensitivity analysis (Supplement 10), Hongjingtian (Rhodiola), Honghua huangsesu (Sa ower yellow) injection on SYM-I as well as Honghua (Sa ower) injection on SYM-R always showed the superiority to other TCMIs, they were identi ed as the preferable treatments in improving anginal symptoms; Gualoupi (Pericarpium Trichosanthis) and Honghua (Sa ower) injection both on ECG-I and ECG-R always showed the superiority to other TCMIs, they were identi ed as the preferable treatments in improving electrocardiography.

Low or very low quality of the evidence
The quality of direct evidence was assessed as low or very low with the GRADE approach [15]. Risk of bias was deemed as very serious for the generally poor quality of included RCTs, all evidence was thus downgraded to low. Then heterogeneity (I 2 > 30% or statistical difference in subgroup analysis), imprecision (the number of included studies less 5 or the 95% CI covering the threshold 1) and publication bias ( According to GRADE-NMA approach [16,17], very low quality of direct evidence on preferable treatments caused indirect evidence very low. Due to little head-to-head evidence of TCMIs, indirect evidence contributed to the very low quality of the comparative evidence on the preferable treatments. Hongjingtian (Rhodiola) injection on SYM-I included 8.7% low quality and 91.3% very low quality of comparative evidence, and the quality of the comparative evidence on other preferable treatments was very low.

Discussion
A total of 475 RCTs with 48511 participants were available in this network meta-analysis. Our study found that: (1)  TCMIs in improving electrocardiography; (4) the overall quality of included RCTs was poor due to the integrity issues, furthermore, the quality of evidence was rated low or very low.
The protocol of systematic reviews and meta-analyses that reported the designs and methodologies before conduct would enhance the study designs and transparency, hence the reliability and reproducibility of evidence [21]. Our study was impartial, valid and reliable in compliant with the pre-speci ed protocol [24,25]. Previous network meta-analyses [19,20] without protocols did not identify the limitations which included PRISMAdiscrepancy, inappropriate model selection based on heterogeneity test, inadequate statistical analysis, and lack of evidence assessment (Supplement 12). Therefore, their results and conclusions [19,20] were biased and untrustworthy. Our study conducted a rigorous and comprehensive network meta-analysis in accordance with the PRISMA statements [21][22][23] and the pre-speci ed protocol [24,25]. We employed the random-effects model for data synthesis with an assumption of different populations among eligible RCTs [29]. In our study, the robustness of overall results was tested by subgroup analysis, sensitivity analysis, meta-regression and publication bias analysis. Quality assessment of the evidence with the GRADE approach [15][16][17] would help inform the strength of clinical recommendations. This study provided the best evidence to evaluate the e cacy of TCMIs for angina pectoris.
meta-analysis were from published RCTs. There was signi cant publication bias in 157 RCTs on symptomatic outcomes and 114 RCTs on electrocardiography outcomes. Publication bias explained why small studies could observe larger effects, which was also known as the smallstudy effects [48]. Since the median sample size in this study was merely 86, the effect sizes tended to be overestimated on the account of the small-study effects. The greater e cacy of Kudiezi injection ECG-R (Table 3) and the e cacy of Huangqi (Astragalus) injection on ECG-I (Table 4) in smaller studies showed the potential publication bias among their included studies, while meta-regression analysis suggested a positive association between the e cacy of Shuxuening injection on SYM-I and sample sizes (Table 4, β = 0.0037, p = 0.0243). According to subgroup analysis (Table 3, Supplement 10), Shuxuening injection has more e cacious on symptomatic outcomes in treating unstable angina pectoris.
Studies [49,50] found that Ginkgo biloba extracts of Shuxuening injection improved the imbalance between nitric oxide and endothelin-1 in patients with coronary artery disease, resulting in exerting vasodilating effects and promoting the coronary blood ow. Antiplatelet therapyaspirin seemed the optimal treatment for unstable angina pectoris [51]. Ginkgolide from Shuxuening injection was demonstrated as the antagonist for platelet activating factor (PAF) that induced platelet aggregation [52]. Shuxuening injection could reduce the PAF level to inhibit thrombosis [53]. In addition, statistically insigni cant results by Begg's test [34] or Egger's test [35] cannot con rm the nonentity of publication bias, because the power of two tests was limited for meta-analyses with fewer than 10 included studies [54], thereby results that did not found signi cant publication bias should be cautiously interpreted.
The quality of the included studies in meta-analyses has been always emphasized [55]. Poor RCTs not only wasted precious resources, but also made the ndings hardly reproduce and even misled healthcare decision-making. RCTs with unclear randomization, allocation concealment, or blinding methods tended to exaggerate the estimate of treatment effect [56,57]. Of included RCTs in this study, only 2 RCTs [42,43] reported detailed methods, such as random allocation, blind design, withdraw of participants and statistical analysis plan. Other included RCTs lacked adequate information on implementation. Eighty-four RCTs reported vague random sequence generation; 26 RCTs reported blind methods but lacked the information on speci c process; 16 RCTs reported the number of drop-out cases (Supplement 5). The overall quality of the included RCTs was poor. RCTs with inappropriate randomization or blinding methods would pose a risk of 7%-23% exaggeration on the treatment effect [58,59], the true effects of TCMIs might be smaller than our e cacy estimates.
The quality of RCTs on TCMs has always been problematic [60][61][62], while its impact on the certainty of evidence has rarely been taken into consideration. According to the GRADE approach [15][16][17], "very serious" risk of bias from poor quality of included RCTs rated down the certainty of all direct evidence, further restricting the indirect evidence that initiated from the quality of direct evidence. As a result, the low quality of included RCTs would degrade the certainty of evidence both from pairwise and network meta-analysis.
Quality of network evidence summarized the certainty of each inter-comparison. In this study with 24 TCMIs included, it was a great challenge to undertake the evidence assessment 288 times on each outcome measure. We were more concerned about the quality of the evidence of preferable treatments, because it could be inappropriate to recommend a higher ranked treatment with low quality of evidence [63]. Very low quality of network evidence indicated the comparative e cacies of preferable TCMIs were substantially different from the true effect.
The consistency is the statistical manifestation of transitivity in the network meta-analysis [40]. We identi ed signi cant inconsistencies in network meta-analysis on all outcome measures. Thus, it seems not plausible to hold the transitivity assumption. We did not rate down the quality of indirect e cacy estimates by intransitivity because the evidence had downgraded to very low by imprecision. The e cacy of TCMIs for angina pectoris was evaluated with direct evidence because direct evidence does not rely on the transitivity assumption [64]. Direct evidence was mainly assessed as very low, there was a substantial difference between the true effect and the e cacy estimate. Although meta-analysis suggested a better performance of TCMIs in treating angina pectoris, we have very little con dence in these e cacy estimates.
Making healthcare decisions was complicated, not only depending on the e cacy of treatments. It was insu cient to recommend optimal treatments according to the ranking of comparative e cacy in previous network meta-analyses [19,20]. Decision-makers not only considered the overall trade-off between the e cacy and safety of interventions but also were in uenced by the certainty of evidence. A lower-ranked intervention with high quality evidence might be more preferable than a higher-ranked intervention with low quality evidence for patients and clinicians [63]. less than 5 included RCTs hardly assessed the robustness of overall results. It seemed that the uncertainty of effect size had little impact on the ranking of treatments based on P-scores. Overall network meta-analysis on SYM-R showed the Xinmailong injection was as effective as conventional treatments (OR = 7.91, 95% CI: 0.79-79.1) despite its fth ranking (Supplement 9). We performed subgroup and sensitivity analysis of network meta-analysis to determine the preferable treatments with robust effect sizes.

Conclusion
The overall e cacy estimates showed TCMIs were associated with bene cial effects for angina pectoris, however, low quality of current evidence cannot support the superiority of TCMIs in treating angina pectoris. Risk of bias across the included RCTs using the revised Cochrane's tool.