Negative predictive value of preoperative computed tomography in determining pathologic local invasion, nodal disease, and abdominal metastases in gastric cancer

Background Before undergoing curative-intent resection of gastric adenocarcinoma ( ga ), most patients undergo abdominal computed tomography ( ct ) imaging to determine contraindications to resection (local invasion, distant metastases). However, the ability to detect contraindications is variable, and the literature is limited to single-institution studies. We sought to assess, on a population level, the clinical relevance of preoperative ct in evaluating the resectability of ga tumours in patients undergoing surgery. Methods In a provincial cancer registry, 2414 patients with ga diagnosed during 2005–2008 at 116 institutions were identified, and a primary chart review of radiology, operative, and pathology reports was performed for all patients. Preoperative abdominal ct reports were compared with intraoperative findings and final pathology reports (reference standard) to determine the negative predictive value ( npv ) of ct in assessing local invasion, nodal involvement, and intra-abdominal metastases. metastases was 52.3% ( n = 407). Conclusions Preoperative abdominal ct imaging reported as negative is most accurate in determining local invasion and least accurate in nodal assessment. The poor npv of ct should be taken into account when selecting patients for staging laparoscopy.


INTRODUCTION
Gastric adenocarcinoma (ga) is the 2nd-leading cause of cancer death worldwide, and the most common malignancy in many countries 1 . Staging of ga includes computed tomography (ct) imaging of the abdomen to assess for local invasion, regional lymph node involvement, or intra-abdominal metastases before surgical resection of the tumour 2 . The presence of local invasion or nodal disease can influence the decision for neoadjuvant treatment, and the presence of intra-abdominal metastases is considered to be a contraindication to resection in the absence of symptoms 2 . The ability of preoperative abdominal imaging read as negative to correctly rule out such findings-that is, the negative predictive value (npv)therefore determines whether patients requiring preoperative chemotherapy are identified and appropriately treated before surgery. Several studies have documented the npv of ct imaging in the preoperative staging of ga; however, the reported npv varies 3,4 . The existing literature is limited mainly to single-institution studies, which might not reflect results at a population level. The bulk of the literature on pre operative radiologic staging in ga comes from East Asian countries with a higher incidence of ga; few studies from North America or Europe about the preoperative radiologic staging of ga exist 5,6 . However, abdominal ct remains central to the preoperative evaluation and staging algorithm 2 . We therefore performed a retrospective populationlevel cohort analysis to assess the clinical relevance of preoperative abdominal ct imaging in detecting local invasion by tumour of adjacent structures, regional lymph node metastasis, and intra-abdominal metastases among patients undergoing surgery for ga, and we report the corresponding npvs.

METHODS
Using the Ontario Cancer Registry, patients diagnosed with ga in the province of Ontario between 1 April 2005 and 31 March 2008 were identified. The Ontario Cancer Registry is a passive registry of all incident cancer cases in Ontario, the largest province in Canada (population 13 million). In total, 2414 patients treated at 116 institutions were identified. A primary chart review was performed for all patients, and the results were abstracted. Those records were linked via an encrypted patient identifier to the provincial universal health insurance program (Ontario Health Insurance Plan) database, Cancer Care Ontario's Activity Level Reporting database, the Canadian Institute for Health Information database, and the Registered Persons Database.
The Ontario Health Insurance Plan database was used to identify claims submitted for reimbursement by physicians to the provincial universal health insurance program. Each claim contains a service code and a date of service, which were used to identify patients receiving neoadjuvant therapy. The Activity Level Reporting database contains records of all radiation and systemic therapy delivered at oncology clinics in the province; it was also used to identify receipt of neoadjuvant therapy. The Canadian Institute for Health Information database contains dates and details of operative procedures. The Registered Persons Database is a registry of all Ontario Health Insurance Plan-eligible persons in the province and contains demographic information such as age, sex, date of death, rural residence, and income quintile.
Research Ethics Board approval was obtained from the appropriate institutions, and the study was conducted in accordance with all confidentiality and privacy policies and regulations.

Cohort Creation
Patients were excluded from analysis ( Figure 1) if they were less than 18 or more than 100 years of age; had received preoperative chemotherapy or radiation therapy; had undergone endoscopic mucosal resection as their sole operative treatment modality; did not have a preoperative abdominal ct report, pathology report, or operative report available; or had undergone ct imaging reported as positive for the outcome of interest (local invasion, lymph node involvement, abdominal metastases). All ct, intraoperative, and pathology reports were abstracted by a single abstractor (JVR) using a standardized abstraction form. In cases in which multiple preoperative abdominal ct reports were available for a single patient, the ct report closest chronologically to the date of surgery was selected for analysis.
Three cohorts were then created ( Figure 1). The first two cohorts (curative resection) included patients who underwent operative gastric resection with no preoperative or intraoperative evidence of intra-abdominal metastases [M0 according to the American Joint Committee on Cancer staging system, 6th edition (ajcc6)] 7 . For those patients, the preoperative ct imaging reports were compared with the final pathology reports for their resection specimens. The 1st cohort (n = 536) was used to assess radiologic detection of local tumour invasion of adjacent structures, and the 2nd cohort (n = 450), to assess detection of regional lymph node metastasis. The 3rd cohort included patients who underwent operative abdominal exploration, which included resection, bypass, or biopsy. For those patients, preoperative ct imaging was compared with intraoperative findings and the pathology reports for specimens, which included both biopsies and resections. Hence, that cohort was used to assess radiologic detection of intra-abdominal metastases (n = 407).
For each outcome of interest, only patients whose records (ct, operative, and pathology reports) contained the necessary information regarding that outcome were analyzed. The ajcc6 staging was used in the analysis because that version was the one in use during the study period.

Variable Definitions
Local invasion of adjacent structures was defined as direct tumour extension and invasion into the following structures: spleen, colon, liver, diaphragm, pancreas, abdominal wall, adrenal gland, kidney, small intestine, and retroperitoneum 7 . In the ajcc6 staging manual, the presence of local invasion denotes a T4 tumour 7 . That staging was captured in ct reports indicating "invasion" and in pathology reports indicating "invasion" or "T4" status or similar. Regional lymph nodes measuring more than 1 cm, as well as those termed "enlarged," "suspicious," "pathologic," or "involved," in ct reports were recorded as positive for nodal disease. Pathology reports indicating regional lymph node positivity, or "N1-3" status were recorded as positive. Intra-abdominal metastases were identified in ct reports and operative notes as "metastasis," as a specified site of metastatic disease (ovary, liver, omentum), or as "carcinomatosis," "diffuse peritoneal involvement," "nodularity," or similar. Pathology reports indicating "M1" status or positive biopsies of distant lesions were recorded as positive for intra-abdominal metastases.
Pathology reports were used as the reference standard for confirmation of local invasion, lymph node metastasis, and abdominal metastases. For abdominal metastases, intraoperative findings were combined with pathologic findings and were compared with preoperative ct imaging to estimate the npv of radiologic evaluation before the operative evaluation.

Statistical Analysis
For patients not undergoing surgical exploration, it was not possible to determine a reference standard with which to compare ct diagnosis, and those patients were therefore excluded from analysis. Because the results of a positive ct report potentially influenced the decision to proceed with surgery and to obtain pathology (reference standard), patients with a preoperative ct reported as positive for local invasion, nodal metastasis, or intra-abdominal metastases remained at risk of incorporation bias. Therefore, in accordance with the Standards for the Reporting of Diagnostic Accuracy Studies guidelines, patients with a ct reported as positive for the outcome of interest and the corresponding values calculated using true positives or false positives (sensitivity, specificity, accuracy, positive predictive value) are not reported 8 .
Baseline demographic and clinicopathologic characteristics were recorded: age, ajcc6 stage, T stage, N stage, and tumour location 7 . In accordance with institutional protocols, all categories containing fewer than 6 individuals were suppressed to ensure patient privacy. Preoperative abdominal ct reports were distinguished depending on whether they indicated uncertainty or no uncertainty about the presence of local invasion or abdominal metastases. Terms suggestive of radiologic uncertainty coded as "probably yes" or "probably no" included "probably," "maybe," "possibly," "cannot exclude," "may," and similar uncertainty phrases. The npvs were calculated for subgroups (uncertainty vs. certainty) and for the overall cohort, wherein uncertain reports were grouped with definite reports. Because any ct report describing suspicious regional lymph node irregularity was coded as indicating nodal disease, subgroup analysis with respect to uncertainty for nodal disease was not performed. The ct reports that did not specifically comment on the presence of a particular finding (local invasion, nodal disease, distant metastases) were coded as negative for that finding, reflecting how such a report would be interpreted by the treating physician or physicians. A sensitivity analysis comparing rates of npv over time was conducted to assess for changes over time and for the influence of new equipment, technologies, and secular trends. All statistical analyses were conducted using the SAS software application (version 9.

RESULTS
Table i presents baseline demographic and clinicopathologic attributes of the included cohorts. Median time from preoperative ct imaging to surgical intervention was 20 days for patients undergoing resection (interquartile range: 10-35 days; n = 570) and 19 days for those undergoing exploration (interquartile range: 8-34 days; n = 1127) An analysis of the overall patient cohort indicated that, compared with patients having a ct report negative for local invasion, nodal metastasis, or distant metastases, patients with a ct report positive for those findings were less likely to undergo surgery (11% vs. 49%, 26% vs. 67%, and 20% vs. 55% respectively).

Local Invasion
Of the 570 patients who received gastrectomy, 536 had preoperative ct imaging that was read as negative for local invasion and postoperative pathology reports that described the presence or absence of local invasion of adjacent structures (ajcc6 T4). Only in very few patients (13.1%) was local invasion demonstrated on final pathology ( Table ii). The npv of ct in detecting local invasion was 86.9% (Table ii). In the few patients whose ct imaging was read as negative for local invasion, but indicating uncertainty (n = 13), all were found to be negative for local invasion on final pathology (npv: 100%; Table ii).

Regional Lymph Node Metastasis
Of the 570 patients who received curative resection, 450 had preoperative ct imaging that was read as negative for lymph node involvement and histopathologic details of the presence or absence of nodal disease. The npv of ct in detecting regional lymph node metastasis was 43.3% (Table iii).
A sensitivity analysis by year was performed to assess for changes in npv over time and the influence of new technologies or techniques. The results for both local invasion (range: 86%-88%) and nodal metastasis (range: 37%-52%) indicate that overall npv was stable over time.

Intra-abdominal Metastases
Of the 1127 patients who underwent abdominal exploration for assessment of intra-abdominal metastatic disease, 407 had preoperative abdominal ct imaging that was read as negative for distant metastases and the necessary information relating to intraoperative and pathologic findings. The overall npv of ct in detecting abdominal metastases was 52.3% (Table iv). The ct reports expressing uncertainty about metastases had a npv of 49.3%; reports indicating a definite absence of metastases had a npv of 53.3%.

DISCUSSION
The present study compared, at the population level, findings from radiologic, intraoperative, and pathologic reports to determine the npv of preoperative abdominal ct in assessing the resectability of ga among patients undergoing surgery. It sought to answer an important question for physicians treating gastric cancer: what is the likelihood that a preoperative ct read as negative (that is, resectable) will be found to be correct at the time of surgery?
The patient cohort analyzed here is representative of the burden of ga in Western countries, with the preponderance of patients being older and presenting with more advanced disease than is common in cohorts from East Asian nations 9 . Because local invasion of adjacent structures constitutes an indication for multivisceral resection (likely with neoadjuvant therapy) or nonoperative management 10 , and because the present analysis excluded  patients receiving neoadjuvant therapy or nonoperative management, it is appropriate that only a small proportion of patients had locally invasive disease. Because npv depends on the prevalence of a given finding within the population under investigation, the North American cohort analyzed, which typically presents with a higher incidence of local invasion, nodal metastasis, and distant metastases than is found in cohorts from Eastern countries 9 , will demonstrate a correspondingly different npv. The advantage in reporting our results is that they represent the predictive value of ct in assessing resectability in a different population than has been reported in the bulk of the literature, and they are, therefore, of use to physicians treating gastric cancer in North America 3 .
The npv of preoperative ct in detecting lymph node involvement in our study (43.3%) was poor, and ranked among the lowest when compared with other contemporaneous publications, in which detection ranged from 47% to 94% 11,[13][14][15][16][18][19][20][21] . Most of those studies were performed at single institutions with high volumes of gastric cancer and with lower incidences of advanced disease 9 . The inferior results reported in the present study could be secondary to significant heterogeneity, including radiologist experience, model of ct scanner, and ct protocol in use. As well, the optimal definition of an "enlarged" lymph node based on size criteria is under debate, with 6 mm, 8 mm, and 10 mm all having been suggested in the literature 19 ; the results reported in our study could therefore have changed had a cut-off of 6 mm or 8 mm been applied. Importantly, this study having been population-based, the results represent the decision-influencing results obtained across a population of patients undergoing surgery for ga.
The observed npv of preoperative ct in detecting intra-abdominal metastases among patients undergoing surgical exploration was 52.3%. That result must be interpreted with caution, because on account of missing operative and pathology report data, only a small proportion of patients undergoing exploration were included in the final analysis. Although it is difficult to speculate on the direction of the potential bias introduced by the large number of missing data points, given the critical clinical implications of a false-negative finding, such findings might have been documented more carefully than true negative findings. The available data might therefore be enriched with false negatives, artificially lowering the observed npv. Rates as low as 66% have been reported in the contemporary literature 17 , although most series report npvs above 80% 14,22,23 . An alternative explanation for our finding relates to the relative rarity of ga in the region investigated, the often subtle presentation of abdominal metastases on ct imaging, and the heterogeneity in ct protocol and radiologist experience with ga characteristic of a population-based study. Given the management implications of the finding of metastatic disease, the results of the present study underscore the importance of laparoscopy in the staging of ga 22 .
To our knowledge, this publication is the first to examine the effects of uncertainty in radiologic reporting in ga. It is notable that although reports expressing uncertainty were only slightly less accurate in assessing intra-abdominal metastases, they were more accurate in assessing local invasion-although the latter finding is most likely attributable to the low number of patients whose reports expressed uncertainty regarding local invasion. Publications addressing the phenomenon of uncertainty in radiologic reporting have emphasized the importance of the clinician-radiologist relationship and dialogue in communicating uncertain findings 24,25 . The use of structured radiologic reporting tools has also been posited as a means of mitigating uncertainty 26 .
The present study has several limitations. The myriad ct scanners and protocols used in this patient cohort make comparisons with single-institution studies using standardized equipment difficult. Complementary imaging studies such as ultrasonography, endoscopy, magnetic resonance imaging, and positron-emission tomography could have been used to compensate for the deficiencies in ct accuracy, but any such reports were not incorporated into the analysis.
Importantly, the analyzed cohort was limited to patients who underwent surgery; the cohort of patients in whom preoperatively detected local invasion, extensive lymph node metastasis, or abdominal metastases precluded surgery were therefore excluded from the analysis. The limitations of the study design thus make it impossible to know to what degree the preoperative ct imaging results influenced the decision to proceed with surgery; however, it is likely that ct imaging reported as "unresectable" would be less likely to undergo surgery 2 . To minimize incorporation bias, wherein the results of the test being evaluated (ct) influence the decision to obtain the reference standard (pathology) 27 , it was decided that calculations incorporating positive ct findings (true positives, false positives) be excluded from analysis 28 . Patients with positive ct findings are at risk of not undergoing surgery (and not being evaluated by the reference standard) based on the results of the ct itself, and are therefore likely biased. Thus, in accordance with the Standards for the Reporting of Diagnostic Accuracy Studies guidelines and the resources upon which those guidelines are based, values incorporating positive ct findings are not reported (accuracy, sensitivity, specificity, positive predictive value) 8,28 . Negative ct findings would not have prevented patients from being evaluated by the reference standard, and therefore the npvs reported should be unbiased.

CONCLUSIONS
Our report presents the first population-based analysis of npvs in preoperative ct staging of ga in a low-incidence country. The poor predictive value of ct in evaluating nodal involvement and intra-abdominal metastases underscores the importance of diagnostic or staging laparoscopy and appropriate lymph node harvest. Future studies might examine the utilization and performance characteristics of complementary staging methods at the population level. Additionally, further investigation into the phenomenon of uncertainty in radiologic reporting and its effect on accuracy should be undertaken.