Direct Comparison of Cardiovascular Magnetic Resonance and Single-Photon Emission Computed Tomography for Detection of Coronary Artery Disease: A Meta-Analysis

Objective To use direct comparative studies or randomised controlled trials to compare the accuracy of cardiac magnetic resonance (CMR) and single-photon emission computed tomography (SPECT) for the detection of obstructive coronary artery disease (CAD). Materials and Methods Various databases were searched for original articles published prior to June 2013. Studies were selected that performed both CMR and SPECT in the same or randomised patients to detect CAD and that presented sufficient data to allow construction of contingency tables. For each study, the true-positive, false-positive, true-negative, and false-negative values were extracted or derived, and 2×2 contingency tables were constructed. To reduce heterogeneity, the meta-analysis was carried out in two parts: (1) coronary territory-based analysis and (2) patient-based analysis. Results 10 studies (5 studies based on patient, 4 studies based on coronary territory, and 1 study based on both) were included in the meta-analysis with a total of 1727 patients. The methodological quality was moderate. For part (1), the summary estimates were as follows: for CMR based on patient–a sensitivity of 0.79 (95% confidence interval: 0.72–0.84) and a specificity of 0.75 (0.65–0.83); for SPECT based on patient–a sensitivity of 0.70 (0.59–0.79) and a specificity of 0.76 (0.66–0.83). For part (2), the summary estimates for CMR based on coronary territory were a sensitivity of 0.80 (0.73–0.85) and a specificity of 0.87 (0.81–0.91), and the summary estimates for SPECT based on coronary territory were a sensitivity of 0.67 (0.60–0.72) and a specificity of 0.80 (0.75–0.84). Conclusions Compared with SPECT, CMR is more sensitive to detect CAD on a per-patient basis. Nonetheless, large scale, well-designed trials are necessary to assess its clinical value on a per-coronary territory basis.


Introduction
Coronary artery disease (CAD) is the leading cause of death in industrialized countries, and the prevalence is expected to increase worldwide [1]. The management of patients with known or suspected CAD is ideally guided by documentation of myocardial ischaemia for optimal medical therapy [2,3].
Compared with invasive coronary angiography (CA), various noninvasive functional imaging techniques with a low rate of cardiac events such as single-photon emission tomography (SPECT), cardiac magnetic resonance (CMR), or positron emission tomography (PET) perfusion imaging are used to diagnose coronary heart disease and assess the need for revascularisation [4,5]. In these techniques, SPECT has been widely used to evaluate myocardial ischemia. However, it exposes patients to ionising radiation, and estimates of its accuracy vary widely [6]. An increasing number of cardiac magnetic resonance (CMR) studies documented a high diagnostic performance of obstructive CAD and showed its prognostic value. Compared with SPECT, it has some advantages, such as the lack of ionising radiation, high spatial resolution, and its ability to assess multiple aspects of heart [7].
Both small or large scale, single or multi centre studies [8][9][10][11][12][13][14][15][16][17] have tested the accuracy of CMR compared with SPECT directly for the detection of coronary heart disease, against a reference standard of invasive coronary angiography (CA). However, the findings of these studies have been largely inconsistent. Therefore, we conducted a meta-analysis of the literature to estimate the accuracy of CMR compared with SPECT for the detection CAD. To obtain the best evidence of the diagnostic accuracy of these two methods, we restricted the scope of the meta-analysis to direct comparative studies or randomised controlled trials.

Criteria for Inclusion in the Study
We aimed to include studies published in any language. However, we eliminated non-English and non-Chinese articles for which a full-text translation or evaluation could not be obtained. For the detection of CAD, studies were eligible if the following criteria were met: 1. Adult patients were CAD or suspected of having CAD. 2. Both CMR and SPECT were evaluated in the same patient population (direct comparison) against an acceptable reference standard (as defined later), or patients were randomised within a study to receive either CMR or SPECT. 3. Invasive coronary angiography was the reference standard. 4. The data reported in the primary studies were sufficient for the calculation of true-positive, false-positive, true-negative, or false-negative values.
We included both prospective and retrospective studies. We excluded studies if there were fewer than 20 patients and if multiple reports were published for the same study population. In the latter case, the most detailed or recent publication was chosen.

Data Sources
PUBMED, EMBASE, Web of Science, and the Cochrane Library were searched independently by two observers. The search strategy included both subject headings (MeSH terms) and keywords for the target condition (coronary artery disease) and the imaging techniques under investigation (CMR and SPECT). We also included a methodological filter for studies of diagnostic accuracy. We limited our search to publications with the search term in the title or abstract of the article and a publication date no later than June 2013. Review articles, letters, comments, case reports, and unpublished articles were excluded. Extensive crosschecking of the reference lists of all retrieved articles was performed.

Selection of Articles
Two authors initially screened the titles and abstracts of the search results and retrieved all potentially relevant reports in full. Next, they independently reviewed all relevant reports according to the predefined inclusion criteria. Disagreements were resolved by consensus or arbitration by a third author who assessed all of the involved items. The majority opinion was used for the analysis.

Quality Assessment and Data Extraction
The same three authors extracted data from the selected reports. The methodological quality of the included studies was assessed independently by two observers using the quality assessment of diagnostic studies (QUADAS-2) tool, which was specifically developed for systematic reviews of diagnostic accuracy studies [18][19][20]. Meanwhile, the relevant data were also extracted from each study, including the author, journal name, study nation, year of publication, description of the study population, study design characteristics, magnetic field strength, type of pulse sequences, type of SPECT, and descriptions of the interpretations of the diagnostic tests.

Meta-analysis
For each study, the true-positive (TP), false-positive (FP), truenegative (TN), false-negative (FN), sensitivity (SEN), specificity (SPE), positive likelihood ratio (PLR), negative likelihood ratio (NLR) and diagnostic odds ratio (DOR) values for the detection of lesions were extracted or derived, and 262 contingency tables were constructed. We calculated the sensitivity and specificity with 95% confidence intervals (CI) for each imaging test in each study. We tabulated results for studies based on per-patient separately from those for studies based on per-coronary territory. We drew forest plots to show the variation of SEN and SPE estimates together with their 95% CI. We constructed hierarchical summary receiver operating characteristic (HSROC) curves to assess SEN and SPE [21].
Exploring heterogeneity is critical for understanding the factors that influence accuracy estimates and for evaluating the appropriateness of the statistical pooling of accuracy estimates from various studies. Visual inspection of the forest plots, standard x2testing, and the inconsistency index (I-squared, I 2 ) were used to estimate the heterogeneity of the individual studies using Stata software (Stata Corporation, College Station, TX, USA). P,0.1 or I 2 .50% suggested notable heterogeneity [22]. If notable heterogeneities were detected, the test performance was summarised using a random-effects coefficient binary regression model; otherwise, a fixed-effects coefficient binary regression model was used [23].
There are several study-level covariates that might have contributed to heterogeneity (such as study design, study characteristics, magnetic field strength) in a review. However, we did not assess factors by subgroup analyses that as in small metaanalyses this is likely to produce unreliable estimates.
With the Stata software, the presence of publication bias was assessed by producing a Deeks funnel plot and an asymmetry test. Publication bias [24,25] was considered to be present if there was a nonzero slope coefficient (P,0.05), which suggested that only small studies reporting high accuracy had been published, and small studies reporting lower accuracy had likely not been published (P.0.1), which suggested that there was no evidence of notable publication bias.
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement [26] was used to improve the reporting of our research (Figure 1 and Checklist S1).

Results
The database search initially yielded 235 potential literature citations, and one additional record was identified through a grey literature search (Figure 1). After review of the titles and abstracts, 198 of these studies were excluded because they were not relevant studies. After reading the full texts, we excluded 28 of the remaining 38 articles for the following reasons: the article lacked sufficient information to enable completion of a 262 contingency table, the article was not available, the article with fewer than 20 patients, or the article was not published in English. After this final screening, 10 published studies met our inclusion criteria, with 5 studies assessed base on patient, 4 studies assessed base on coronary territory (CT) and 1 study assessed base on both. Neither cost-effectiveness nor practicality was taken into consideration.
The data abstracted from these individual studies are summarised in Table 1. According to QUADAS-2, the quality assessment for the 10 studies was moderate. The results of the distribution of the study design are shown in Figure 2.
To reduce heterogeneity, the analysis was carried out in two parts: (a) comparison of CMR with SPECT based on coronary territory (b) comparison of CMR with SPECT based on patient. For part (1), Figure 3 shows the forest plots of the sensitivity and specificity estimates for CMR and SPECT for five studies. The   Table 2. The pairs of observed sensitivity and specificity values for parts (1) and (2) are presented in HSROC curves in Figures 5 and 6, respectively.
The results of Deeks funnel plot asymmetry test (P = 0.084) showed no evidence of notable publication bias (Figure 7).

Discussion
Although CMR has been already included in international guidelines for the non-invasive detection of coronary heart disease,it does not have the consistent findings from the previous meta-analysis and systematic review [5,27,28] which assessed different imaging modalities directly or indirectly, including perfusion-CMR and SPECT, for the diagnosis of CAD (Table 3). Recently, two large scale diagnostic accuracy and clinical outcome datas, MR-IMPACT II [8] and CE-MARC [9], on that have been published and also have an inconsistent finding. As direct approach is known to provide better measurements of the diagnostic accuracy of two different methods [29], we focused exclusively on direct comparative studies that evaluated both CMR and SPECT in randomised controlled or the same patients. Comparison with these previous meta-analysis and systematic review, there may be also another strength in our research: with more careful selection of articles, two direct comparative studies which missed in their researches were included in ours.
In this limited cohort of studies, the main results can be summarized as follows: 1). When assessed base on coronary territory, the diagnostic performance of perfusion-CMR assessed as SEN, SPE, and the area under the ROC curve (AUC) were superior over SPECT in detecting CAD. This findings of our study were in accord with the meta-analysis of Iwata et al. [27], which only included studies assessed base on coronary territory. 2). When assessed base on patient, the diagnostic performance of perfusion-CMR assessed as SEN and AUC values were superior over SPECT in detecting CAD, but inferior in SPE. Two previous indirect comparative meta-analyses demonstrated significantly higher SEN and DOR than SPECT but failed to show significant superiority in SPE ( Table 2). As indirect comparative metaanalysis, we believe that might be caused by the different patients included in various studies. One may also speculate that may be caused by threshold effect with different diagnostic cut-off values, various data obtain and analysis methods. Such as, CE-MARC used a multiparametric CMR protocol, unlike other studies in our research, which used only the perfusion CMR components.
Compared with the result based on coronary territory, the diagnostic performace of CMR based on patient is relative low. We speculate that is related to the fact that perfusion CMR was more compared with the macroscopic coronary artery anatomy, but not assess, for example, collateral flow on the microvascular level [8].
As the numbers of comparative studies available in our research are relatively small, we did not assess factors such as different techniques or higher field strengths by subgroup analyses. New techniques and higher field strengths may make a further improvement in diagnostic accuracy. Jogiya et al. [30] reported that CMR showed a sensitivity, specificity, and diagnostic accuracy of 91%, 90%, and 91%, respectively, on patient basis and 79%, 92%, and 88% on coronary territory basis, with three-    dimensional perfusion technique at 3 Tesla. It may offer CMR a more diagnostic performance over SPECT in future. Some inherent limitations exist in our study and should be considered when interpreting our results. First, the number of comparative studies available in the literature and the sample size of these studies are relatively small, which is a particular problem in diagnostic studies [31]. These shortcomings may result in an overestimation of diagnostic accuracy, particularly in studies including non-representative samples of patients and invalid reference standards [29]. However, a systematic review [32], focused on meta-analysis studies from the Cochrane Database, showed that the number of studies eligible for meta-analysis is typically small in all medical areas and for all outcomes and interventions covered by the Cochrane Reviews. Second, six of ten studies did not enroll a consecutive or random sample of patients, which tended to be a risk of bias in patient selection. Third, publication bias may be of any meta-analysis. Our meta-analysis was based only on published studies, which are prone to report positive or significant results; the studies whose results are not significant or negative are often rejected or are not even submitted. Although it is suggested that the quality of the data reported in articles accepted for publication in peer-reviewed journals is superior to the quality of unpublished data [33], only including published studies may lead to reporting bias.
In conclusion, a limited number of studies demonstrated that CMR is more sensitive to detect CAD than SPECT in both on a per-patient basis and per-coronary territory, but inferior in specificity on a per-patient. In future, with new techniques and higher field strengths, large-scale, well-designed trials, are necessary to compare the diagnostic value of these two imaging techniques.