Appropriateness of CT pulmonary angiograms according to current diagnostic guidelines based on risk stratification: A retrospective single-center study

Background and Aim. Assessment of appropriateness of CT pulmonary angiograms (CTPA) in patients with suspected pulmonary embolism (PE) is based on risk stratification algorithms such as simplified the Geneva Score (sGS) in combination with D-dimer blood tests. The aim of this study was to validate the diagnostic yield and appropriateness of CTPA examinations in accordance with 2014 European Society of Cardiology (ESC) guidelines. Materials and Methods. Data from 155 outpatients who underwent CTPA for clinical suspicion of PE were gathered from the radiology information system (RIS) and the clinical information system (CIS). We assessed the presence of sGS items and D-dimer blood test results in RIS from CTPA request forms and from clinical documentation in CIS. Results. Based on the RIS, there were 2.6% patients with high (sGS≥3) and 97.4% patients with low pre-test PE probability (sGS<3), and CTPA indication was formally comprehendible in 75.5% using sGS and D-dimer blood tests. Based on RIS and CIS data in combination, there were 41.3% patients with high and 58.7% patients with low pre-test PE probability, and CTPA indication was formally comprehendible in 88.4%. Using RIS and CIS in combination, PE probability was upgraded from low to high probability in 39.7% compared with RIS alone. In 12.9%, there was a lack of data in RIS for CTPA justification. Conclusion. There is a high diagnostic yield when applying current diagnostic guidelines to our data. There was however a notable discrepancy between the data transferred to the CTPA request forms from the full clinical documentation, therefore not readily available for clinical decision making.


INTRODUCTION
Acute pulmonary embolism (PE) is a serious clinical condition that most commonly arises from deep vein thrombosis (DVT). The incidence of PE is estimated to be approximately 60-70 per 100.000, and that of DVT approximately 124 per 100.000 of the general population 1 .
The clinical data indicate that PE occurs mostly in the age group of 60 to 70 years, though autopsy data show that there is an even higher incidence in the 70 to 80 age group. Even in patients with thrombolytic treatment, the mortality can be as high as 30% (ref. 2 ). The mortality rate of diagnosed and treated PE is 8%, and up to 10% of acute PE patients die suddenly. Two of three patients succumbing to PE die within 2 h after presentation 3,4 .
The clinical diagnosis of PE is non-specific. Many symptoms of PE are common to other pulmonary and cardiovascular diseases which is why laboratory results and imaging methods are vital to confirm or exclude the diagnosis. CT pulmonary angiography (CTPA) is the gold standard imaging technique for suspected PE. However, it carries the burden of radiation exposure and the need for using an iodine-based intravenous contrast agent. CTPA is being overused in many departments resulting in clinically non-significant incidental findings, unnecessary follow-ups or contrast-induced nephropathy [5][6][7][8][9][10][11][12][13] . This is why several clinical scoring systems have been developed to identify clinically stable outpatients who have an increased PE risk and may benefit from CTPA, such as the Geneva Score, the Wells Score, and the YEARS Algorithm 14 . As PE risk is different in inpatients as well as in all patients with evidence of shock or hypotension, there are specific diagnostic algorithms available for these individuals.
For assessing the need and appropriateness of CTPA in clinically stable outpatients with suspected PE, the 2014 European Society of Cardiology (ESC) guidelines 15 on pulmonary embolism suggest to use the simplified Geneva Score (sGS) and the D-dimer blood test result. The sGS is calculated as the sum of points that are allocated for the presence of specific clinical parameters (Table 1). Based on the sGS, the PE risk of patients with suspected PE can be categorized into different risk stratification groups. The two-level scheme differentiates between low (sGS<3) and high pre-test probability for PE (sGS≥3). The three-level scheme differentiates between low (sGS<2), intermediate (2≥sGS<5), and high (sGS≥5) pre-test probability for PE, respectively. According to Ceriani et al., the prevalence of PE is reported as follows: in the high-risk group up to 65%; in the intermediate-risk group up to 30%, and in the low-risk group up to 10% of cases 16 . According to the ESC guidelines 15 , a high pre-test probability for PE using the sGS and the two-level risk stratification scheme is needed to make appropriate use of CTPA. In patients with low pre-test probability of PE using the two-level risk stratification scheme, a positive D-dimer blood test result surpassing the threshold of 0.5 mg/L is necessary to appropriately justify the use of CTPA.
In daily practice, it is generally the task of the radiologist to formally justify the use of ionizing radiation by reviewing if the requested type of imaging examination is necessary and appropriate for answering the clinical question. In Germany, for example, this responsibility is bindingly specified in the Radiation Protection Ordinance. To adhere to these legal obligations, the radiologist is dependent on the availability of proper clinical information, that still is generally retained in clinical information systems (CIS) in a non-standardized and unstructured manner and that is not fully available in and accessible from separate radiology information systems (RIS). Moreover, the amount and selection of clinical information included in the request form used for ordering imaging exams basically depend on manual insertion by the referring physician and, thus, are subject to the referring physicians' motivation to provide extensive data to the radiology department.
The aim of our study was to analyze the availability of clinical information to the radiologist using CIS and RIS in regards of formal justification of CTPA according to the ESC guidelines 15 . Therefore, we performed a retrospective assessment of electronic health records of clinically stable outpatients referred to CTPA for suspected PE.

MATERIALS AND METHODS
This was a retrospective exploratory single-center analysis of CTPA examinations requested for clinical suspicion of PE between January 2017 and April 2018. The study was approved by the local ethics committee with a waiver of informed consent. CTPA examinations were per-    (1) This represents all cases which had one or more clinically significant CT findings. Clinically significant CT findings were defined as either of the following: Pneumonia, pneumothorax, pleural effusion, bronchitis or bronchiolitis, malignancy, signs of pulmonary hypertension, signs of pulmonary edema. formed using multidetector CT scanners (iCT Brilliance and iQon, Philips Healthcare, Best, The Netherlands). All dedicated CTPA examinations acquired during the study period to exclude or affirm PE in patients of at least 18 years of age were primarily included. CTPA examinations that were acquired in inpatients were excluded.
The primary outcome measure of this study was the pre-test probability for presence of PE according to the sGS based two-level risk stratification scheme compared between data extraction from the RIS alone and data extraction from RIS and CIS in combination.
Secondary outcome measures were: -the proportion of formally justifiable CTPA examinations by the radiologist using sGS and D-dimer blood test results compared between data extraction from the RIS alone and data extraction from RIS and CIS in combination, -the prevalence of PE as a function of two-level and three-level risk stratification groups using sGS and D-dimer blood test results, the prevalence of additional findings aside from PE in CTPA. In our hospital, RIS and CIS are separate software solutions communicating via the health level 7 standard, and electronic request forms have to be completed manually by the referring physician. Data relevant for justification of CTPA examinations according to diagnostic guidelines 15 including sGS data items and D-dimer blood test results were extracted from the RIS on the one hand and from the hospital CIS and RIS in combination on the other hand.
In CIS and RIS, data are stored in an unstructured non-standardized manner so that information has to be inferred from free-text documents including CTPA request forms, radiology reports, and health care reports. sGS data items not mentioned in these source documents were considered absent. Additional clinical information not relevant for sGS assessment as well as other radiological imaging findings documented in the CTPA reports were recorded. Clinically significant CT findings were defined as either of the following: Pneumonia, pneumothorax, pleural effusion, bronchitis or bronchiolitis, malignancy, signs of pulmonary hypertension, signs of pulmonary edema.
Numeric data on availability of sGS data items and D-dimer blood test results were analyzed in regards to the source of the data using descriptive statistics. Interval data are expressed as means ± standard deviation. Ordinal data are expressed as medians and range.

Study population
During the study period, CTPA examinations were performed in 285 patients. 130 of these were performed in inpatients and were excluded. Thus, in total, 155 patients were enrolled (84 women with a mean age of 64.2±19.1 years; 71 men with a mean age of 61.1±16.0 years).

PE prevalence and risk stratification
In total, 25.9% (40/155) of the CTPA examinations were positive for PE. According to the two-level PE risk stratification based on the sGS, there were 40.6% (63/155) patients with high pre-test probability for PE (sGS≥3) and 59.3% (92/155) with low pre-test probability (sGS<3) (see Fig. 1). Prevalence of PE was 36.5% (23/63) and 18.5% (17/92) in these groups, respectively. In 72.9% (113/155) patients, the D-dimer blood test result was greater than the cutoff. Of these, 65.5% (74/113) had low pre-test probability for PE according to the two-level scheme, so that the D-dimer blood test result alone justified the CTPA examination. In this group, there were 20.3% (15/74) patients with PE. PE prevalence according to the three-level risk stratification score are given in Table  2. In patients with low pre-test probability according to the three-level scheme, 22.5% (9/40) were diagnosed with PE.

Availability of data in RIS
Based on data derived from the RIS alone, there were 2.6% (4/155) patients with sGS ≥ 3 and 97.4% (151/155) patients with sGS<3. In 72.9% (113/155) patients, D-dimer blood test results were available in RIS. In 100% (113/113) of these, the D-dimer blood test result was above the threshold. CTPA examinations could be formally justified in 75.5% (117/155) patients using sGS and D-dimer blood test results by the radiologist using RIS alone. The median sGS derived from RIS alone was 1 (range, 0-4).

Availability of data in RIS and CIS in combination
Based on data derived from the RIS and CIS in combination, there were 41.3% (64/155) patients with sGS≥3 and 58.7% (91/155) patients with sGS<3. Thus, using RIS and CIS in combination, risk for presence of PE according to the two-level scheme was upgraded from the low pre-test probability group to the high pre-test probability group in 39.7% (60/151) of patients. As the data source for D-dimer blood test results were the same for RIS and CIS, availability of D-dimer blood test results was similar to RIS alone. CTPA examinations could be formally justified in 88.4% (137/155) patients using RIS and CIS data in combination. In 12.9% (20/155) of patients, there was a discrepancy relevant for justification of CTPA indication between using RIS only and using RIS and CIS in combination. The median sGS derived from RIS and CIS in combination was 2 (range, 0-4). sGS data items that were considered positive using the CIS but were most frequently missing in the RIS are summarized in Table 2.

DISCUSSION
The results of this study show that there is frequently a remarkable loss of data considered necessary for formal justification of CTPA examinations by the radiologist when using an information technology environment characterized by manual ordering of imaging studies and unstructured non-standardized data storage. The deficient availability of clinical information concerning PE risk stratification in the radiology department can be attributed mainly to insufficient transfer of data from free-text based electronic health records (EHR) in the CIS to the RIS.
There is much discussion and research regarding the advantages of structured EHR data. Evaluating CTPA appropriateness in the question of suspected PE is an expedient use case, since the parameters used for decision making are readily available to the clinician through physical examination and laboratory results. Results of implementing such evidence-based clinical decision support (CDS) has already been published in several articles. Mills et al. reported on an immediate and sustained increase of the diagnostic yield of CTPA (ref. 21 ). Geeting et al. found that the appropriateness of CTPA increased whilst lowering its overuse, without showing any significant impact on diagnostic yield 22 . Both studies showed no statistically significant change in overall CTPA utilization.
In our study, the great majority of indications of CTPAs were retrospectively comprehensible. However, availability of clinical information in the RIS concerning assessment of the sGS was mostly insufficient and had to be combined with laboratory results or manual data extraction from full clinical documentation from the CIS to formally justify CTPA. On average, more than one Geneva score parameter was missing in RIS compared with CIS.
In 18 cases (12%) CTPA indication could not have been sufficiently justified according to available clinical and laboratory data. Out of these, 2 cases (11%) had confirmed PE, which correlates with 12% cases reported in previous studies 16 . Aside from the imperfect sensitivity of the sGS for identification of patients at risk for PE, failure of documenting sGS data parameters actually present in a given patient in free-text based EHR may be a reason for these false negatives.
In clinically stable outpatients there are fewer false positive D-dimer values, which is an important factor in the diagnostic guidelines 15 . Previous studies showed that there is limited reliability of the D-dimer value as a diagnostic cut-off in hospitalized patients [18][19][20] . Some studies also suggest age-adjusted value of D-dimers would work better as a diagnostic cutoff for both hospitalized patients and outpatients 23-25 . Current guidelines, however, and the sGS do not make general use of age-adjusted D-dimer values. In our study, in 73% the D-dimers were greater than the cutoff, and in 66% of those D-dimers alone justified the examination regardless of a negative clinical probability score.
When comparing the actual prevalence of PE in the risk groups incorporated in the three-level scheme, there was evidence of PE in 27% of the patients in the intermediate-risk group, which is in accordance with Ceriani et al. (30%) (ref. 16 ). In the low risk group, there was evidence of PE in 23% of the patients, which is higher than expected taking previous studies into account 16 . Of note, in 85% of these cases, the CTPA indication was formally comprehensible, because the D-dimer values were greater than the cutoff. PE prevalence in the high-risk group can hardly be compared as there was only one patient in this group in our study. Using the two-level scheme to identify patients unlikely of having PE, there were 18% positive PE findings in this group, compared to 12% findings in previous studies 16 .
In total, there were 71% clinically significant CT findings in the intermediate-risk group and 55% in the low-risk group suggesting that acquisition of the CTPA examination may have influenced the management of the patient even in the absence of PE. The usability of the diagnostic guidelines and the performance of risk stratification algorithms may be hindered by the fact that ordering a chest CT serves not only as a means to rule out PE but also to exclude other pathological findings as well.
Our study has important limitations. Firstly, the number of cases is small. However, a representative time period of 16 months was analyzed. Secondly, the study was retrospective and there was no intervention in regard of changing the way clinical data is stored and imaging is requested to assess if data availability can be increased. Thirdly, the generalizability of our data is hampered by the fact that the information technology environment concerning RIS and CIS and their interaction differs from hospital to hospital. Nonetheless, a separated RIS and CIS is present in many hospitals across the world.

CONCLUSION
In total, 25.9% CTPA examinations were positive for pulmonary embolism, of which 88.4% were correctly ordered according to ESC guidelines when all the clinical and laboratory data are combined. However, the low risk group still included 22.5% cases of PE -almost twice as much as previous authors reported, which we attribute to inconsistent input data relevant for the examination justification.