Types of studies

We include studies that meet the following inclusion criteria:

1. A minimum sample size of 10 patients with histologically‐proven primary carcinoma of the stomach;

2. Evaluation of endoscopic ultrasonography (EUS) compared with histopathology of primary tumor (T‐stage) and regional lymph nodes (N‐stage);

3. Sufficient data to construct a two‐by‐two contingency table such that the cells in the table could be labeled as true positive, false positive, true negative, and false negative (see the Target conditions for more details).

This type of study typically include both retrospective and prospective series of patients. As long as the above information is available,we did not exclude any specific type of study design.

We excluded studies that had possible overlap with the selected studies (i.e. studies from the same study group, institution, and period of inclusion). We excluded studies reporting on EUS performed before preoperative chemotherapy and or radiotherapy (neoadjuvant therapy) in order to avoid the confounding effect of disease downstaging by neoadjuvant treatments.

Participants

For this review, patients were people with gastric carcinoma undergoing preoperative locoregional disease staging (T‐stage and N‐stage) by means of EUS and postoperative pathology evaluation of the surgical specimen, including those having early gastric cancer (EGC) or advanced gastric cancer (AGC). We imposed no restrictions by age, gender or any other category.

Index tests

The index test is EUS. We compared the results of EUS to those of pathology evaluation (reference test) in terms of both T‐stage and N‐stage (see Target conditions for more details).

We did not consider any comparator test.

Target conditions

The target condition was gastric cancer locoregional staging, for both primary tumor depth and regional lymph node status.

For lymph node status (N‐stage), we considered a patient either negative if no lymph node was metastatic (N0) or positive if one or more lymph nodes were metastatic (N+), as assessed by pathology evaluation.

For the primary tumor invasion of the gastric wall (T‐stage), we considered two main conditions according to the clinical questions that EUS aims to answer:

1. In order to identify patients who would best benefit from surgery without preoperative radio‐chemotherapy, EUS was to be investigated for its ability to distinguish superficial tumors (T1 ‐ T2) versus advanced tumors (AGC, T3 ‐ T4, which are likely to benefit from neoadjuvant preoperative chemotherapy); in this case, a patient was considered either positive if his/her gastric cancer was classified as T1 ‐ T2 by pathology examination, or negative if his/her gastric cancer is classified as T3 ‐ T4.

2. Within the frame of superficial cancers (T1 ‐ T2), in order to identify patients with superficial tumors amenable to endoscopic resection (T1 tumors), EUS was investigated for its ability to distinguish T1 tumors (EGC) versus T2 tumors; in this case, a patient was considered either positive if his/her gastric cancer is classified as T1 by pathology evaluation, or negative if his/her gastric cancer is classified as T2.

Finally, where the data permitted and within the frame of EGC (T1 tumors), EUS was also tested for its ability to further discriminate between T1a and T1b tumors, since it is believed that the former type of cancers benefit the most from endoscopic mucosal resection (EMR). To this end, a patient was considered either positive if his/her gastric cancer was classified as T1a by pathology evaluation, or negative if his/her gastric cancer was classified as T1b.

Reference standards

The reference standard was routine histopathology evaluation (i.e., microscopic examination of hematoxylin‐eosin stained samples) of primary tumor and regional lymph nodes. Since pathological examination of the surgical specimen is the only way to know precisely the depth of invasion through the gastric wall as well as the status of regional lymph nodes, all eligible patients must have undergone surgery and all tumors must have undergone routine pathology evaluation. According to the pathology report, four T categories (T1 to T4) indicate the extent of gastric wall invasion by the primary tumor; the status of the regional lymph nodes (positive versus negative) was also taken into consideration.

Electronic searches

We grouped key words to combine four 'concepts' that must be included in a paper reporting on the subject under investigation in this review:

1. malignant neoplasm (cancer, carcinoma)

2. body site (gastric, stomach)

3. diagnostic method (endoscopic ultrasound, EUS)

4. disease staging

We systematically searched the following databases.

1. The Cochrane Library (the Cochrane Central Register of Controlled Trials (CENTRAL)) (2015, Issue 1) (Appendix 2)

2. MEDLINE (from 1988 to January 2015) (Appendix 3)

3. EMBASE (from 1988 to January 2015) (Appendix 4)

4. NIHR Prospero Register

5. MEDION (http://www.mediondatabase.nl/)

6. ARIF (www.arif.bham.ac.uk/databases.shtml)

7. ClinicalTrials.gov (clinicaltrials.gov/)

8. Current Controlled Trials MetaRegister (www.controlled‐trials.com/mrct/)

9. WHO ICTRP (www.who.int/ictrp/en/)

Searching other resources

We searched for additional references by cross‐checking bibliographies of retrieved full‐text papers.

Selection of studies

Both review authors (SM and SP) independently selected the studies, resolving discrepancies by iteration, discussion and consensus. Where we retrieved articles in languages other than English, we were able to assess those in Italian, French and Spanish for eligibility.

Data extraction and management

We extracted relevant data from the articles selected for inclusion in the meta‐analysis. In addition to the accuracy data, we also recorded the following information for each study:

1. Overall study characteristics, including the first author, country, language, and date of publication;

2. Study patient characteristics;

3. Features of the index test, e.g. type of echoendoscope, ultrasound frequency, and EUS criteria for tumor depth and lymph node status.

In case of missing data, we contacted the authors of the study to obtain the missing information. None of the three authors we contacted (Caletti 1993; Dittler 1993; Murata 1988) was able to provide data.

When raw data were presented in three‐by‐three or four‐by‐four tables (e.g., when the tumor depth or lymph node stage are defined by more than two categories), we constructed two‐by‐two contingency tables by considering a given T, or any N‐positive category, as the 'positive' state to be distinguished from the other T categories, or from the N‐negative cases. For instance, if an article presented data in a table reporting the number of N0, N1, N2 and N3 cases, we collapsed the data into a table with N0 and N+ (sum of N1, N2 and N3) cases.

We extracted data separately on primary tumor depth (T‐stage) and regional lymph node status (N‐stage).

We assembled all data in a dedicated database built within an Excel spreadsheet, where each row corresponded to a single study and variables of interest were recorded in the columns.

Assessment of methodological quality

We assessed data quality using a standard procedure according to the Quality Assessment of Diagnostic Accuracy Studies (QUADAS‐2) criteria (Whiting 2011). When there was at least one 'no' or 'unclear' response to a signaling question for a given domain, we scored the risk of bias as high or unclear, respectively. See Appendix 5 for details of the findings.

Statistical analysis and data synthesis

We performed statistical analysis according to Cochrane guidelines for diagnostic test accuracy (DTA) reviews (Macaskill 2010).

We used coupled forest plots to display the number of true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN), as well as sensitivity and specificity, with their 95% confidence intervals (CI), for all included studies. Visual inspection of forest plots can provide a clue to heterogeneity within single studies. We also used summary receiver operating characteristic (SROC) plots to display the results of individual studies in a ROC space, each study being plotted as a single sensitivity‐specificity point.

As currently recommended for meta‐analysis of diagnostic accuracy studies (Harbord 2008; Leeflang 2008; Macaskill 2010; Reitsma 2005; Rutter 2001), we used hierarchical models to obtain summary estimates of EUS performance in terms of ability to discriminate primary gastric cancer depth of invasion (T‐stage) and regional lymph node status (N‐stage).

According to the bivariate method (Reitsma 2005), we calculated overall sensitivity and specificity and their 95% confidence intervals (CIs) and predictive intervals, based on the binomial distributions of the true positives and true negatives. Besides accounting for study size and between‐study heterogeneity, the bivariate model adjusts for the frequently observed negative correlation between the sensitivity and the specificity of the index test (threshold effect). An additional advantage of using the bivariate model is that the bivariate nature of the original data can be maintained throughout the analysis, allowing the generation of reliable summary estimates of sensitivity and specificity. For the bivariate model, the summary estimates of sensitivity and specificity represent an 'average' operating point across studies. We analyzed primary tumor depth (T‐stage) and regional lymph node status (N‐stage) separately.

We evaluated the clinical (or patient‐relevant) utility of EUS using likelihood ratios, which we computed directly from the summary estimates of sensitivity and specificity, to enable the calculation of post‐test probability (based on the Bayes' theorem) by means of the Fagan's nomogram (Deeks 2004). The Fagan’s nomogram is a graphical tool which in routine clinical practice allows one to use the results of a diagnostic test to estimate a patient’s probability of having a disease (post‐test probability) based on two pieces of information: the pre‐test probability (usually the incidence of the disease/condition) and the test result. In this nomogram, a straight line drawn from a patient’s pre‐test probability of disease (left axis) through the likelihood ratio of the test (middle axis) intersects with the post‐test probability of disease (right axis).

We conducted statistical analyses using both Review Manager 5 software (RevMan 2014) as well as the Metandi and Midas programs for the STATA software (Stata 2009).

Since currently available imaging tools are associated with accuracy sensitivity and specificity values around 80% (see Background section), we considered this as a 'desirable' value with which the EUS diagnostic performance can be compared.

Investigations of heterogeneity

As it is common in diagnostic accuracy studies, we anticipated that there would be substantial between‐study variation in reported pairs of sensitivity and specificity values.

Coupled forest plots, which display both sensitivity and specificity of all included studies, provide a visual clue to heterogeneity of the results on a single‐study basis.

In order to formally investigate potential sources of heterogeneity other than the threshold effect, we used subgroup analysis and meta‐regression by including covariates (study sample size, publication year, type of EUS array, study quality, country, stomach site) in the bivariate model, which enabled us to assess the effect of various factors on the diagnostic accuracy of EUS.

Sensitivity analyses

We conducted sensitivity analyses to assess the impact on the summary effects of low‐quality studies, as defined by the identification of a high risk of bias for one or more QUADAS‐2 items, as well as the presence of specific types of primary gastric cancer morphology (e.g., ulcerated tumors) or location (e.g., cardia region of the stomach).

We also used the 'leave‐one‐out' procedure to assess the impact of each study on the meta‐analysis results (leading study effect).

Assessment of reporting bias

We conducted formal testing for small‐study effects (which include publication bias) by a regression of diagnostic odds ratio (DOR), which describes the odds of positive test results in patients with disease compared with the odds of positive test results in those without disease, on a natural logarithm scale against 1/sqrt ESS (effective sample size), weighting by ESS. P < 0.10 for the slope coefficient indicates significant asymmetry of the funnel plot (Deeks 2005).