Interpretation of results of PCR and B-D-glucan for the diagnosis of Pneumocystis Jirovecii Pneumonia in immunocompromised adults with acute respiratory failure

Background The accuracy of a diagnostic test depends on its intrinsic characteristics and the disease incidence. This study aims to depict post-test probability of Pneumocystis pneumonia (PJP), according to results of PCR and Beta-D-Glucan (BDG) tests in patients with acute respiratory failure (ARF). Materials and methods Diagnostic performance of PCR and BDG was extracted from literature. Incidence of Pneumocystis pneumonia was assessed in a dataset of 2243 non-HIV immunocompromised patients with ARF. Incidence of Pneumocystis pneumonia was simulated assuming a normal distribution in 5000 random incidence samples. Post-test probability was assessed using Bayes theorem. Results Incidence of PJP in non-HIV ARF patients was 4.1% (95%CI 3.3-5). Supervised classification identified 4 subgroups of interest with incidence ranging from 2.0% (No ground glass opacities; 95%CI 1.4–2.8) to 20.2% (hematopoietic cell transplantation, ground glass opacities and no PJP prophylaxis; 95%CI 14.1–27.7). In the overall population, positive post-test probability was 32.9% (95%CI 31.1–34.8) and 22.8% (95%CI 21.5–24.3) for PCR and BDG, respectively. Negative post-test probability of being infected was 0.10% (95%CI 0.09–0.11) and 0.23% (95%CI 0.21–0.25) for PCR and BDG, respectively. In the highest risk subgroup, positive predictive value was 74.5% (95%CI 72.0-76.7) and 63.8% (95%CI 60.8–65.8) for PCR and BDG, respectively. Conclusion Although both tests yield a high intrinsic performance, the low incidence of PJP in this cohort resulted in a low positive post-test probability. We propose a method to illustrate pre and post-test probability relationship that may improve clinician perception of diagnostic test performance according to disease incidence in predefined clinical settings. Supplementary Information The online version contains supplementary material available at 10.1186/s13613-024-01337-8.


Introduction
Biomarkers are used in various fields of medicine for screening and diagnosis, including prognostic or risk stratification [1][2][3][4][5].Their usefulness and accuracy is dependent of intrinsic characteristics of the test (sensitivity, specificity) but also of the context in which the test is employed, as disease incidence influences extrinsic performance (positive and negative predictive value) [6].Standards for Reporting of Diagnostic Accuracy Studies (STARD) guidelines, aiming to homogenize reporting of diagnostic performance assessment, underline the need to adequately assess studied population and to report extrinsic performance of tests [7].Hence, impact of changes in pre-test probability over diagnostic test extrinsic performance is a well described mathematic correlation [8][9][10].Nevertheless, pre-test probability may be difficult to assess [11], changes in post-test probability according to this later difficult to estimate, basic concepts of diagnostic test performance are poorly understood by medical students [10,12], and usual indices of intrinsic performance may be misleadingly reassuring [13].As a consequence, pre-test probability has been found to be taken into account infrequently by physicians when interpreting diagnostic test results [14,15].
Alternative presentation of diagnostic test performance, including visual aids according to frequency has been advocated by some authors [10,13].Pneumocystis pneumonia in non-HIV patients might be suitable to reassess influence of pre-test probability on diagnostic test performance for several reasons: The disease is severe, and associated with a high morbidity and mortality [16][17][18].The incidence of the disease is limited although context, clinical presentation, and radiological patterns may significantly changes pre-test probability [17,19].Last, available diagnostic tests, namely quantitative polymerase chain reaction (PCR) and β-D Glucan (BDG), have been described to have a good to very good intrinsic performance [20][21][22][23].These tests are however quantitative, suggesting that intrinsic test performance may vary according to degree of positivity and adding further complexity in test interpretation [20][21][22][23].These conditions not only underline needs for visual description for extrinsic test performance and potential clinical implications of findings, but also suggest that expert statements may be needed to help physician on daily basis.
The objective of this study was to assess incidence of Pneumocystis pneumonia in the general population of critically ill immunocompromised patients with acute respiratory failure, to detect subgroups of specific risk and to depict post-test probability of Pneumocystis pneumonia according to PCR and BDG tests in this setting.

Study population
In way to assess Pneumocystis pneumonia incidence, two distinct prospectively collected datasets were used.
The first set was the TRIALOH study dataset [24].Patients were prospectively included from 2010 to 2012.The study was carried out in 17 university or universityaffiliated centers in France and Belgium that belonged to a research network instituted in 2005.In all 17 centers, a senior intensivist and a senior hematologist were available around the clock and make ICU-admission decisions together.The appropriate ethics committees approved this study [24].In this set, the attending physician assessed occurrence of acute respiratory failure prospectively and three independent experts reviewed all etiological diagnoses of acute respiratory failure.Since this study focuses on hematological patients with various reason for ICU admission, and to avoid artificially decreasing incidence of Pneumocystis pneumonia, only patients with Acute Respiratory Failure as the main reason for ICU admission were included in the current study.
The EFRAIM study was a multinational, observational prospective cohort study performed from Nov 2015 to July 2016 [25].Investigators were critical care physicians from 16 countries with extensive experience in the management of various cohorts of critically ill immunocompromised patients.Participating providers obtained institutional review board (IRB) approval from their institutions in accordance with local ethics regulations.Only adult patients with acute respiratory failure, based upon predefined criterion were included in this study.All etiological diagnoses were reviewed by two study investigators for coherence and for alignment with established definitions [25].

Definitions
Pneumocystis pneumonia In both studies, etiologies of pulmonary involvement were diagnosed based on predefined criteria [26].These criteria included type of immune defect, time between onset of the disease and ICU admission, radiologic presentation, and microbiological tests including direct search for Pneumocystis (direct examination with Gomorri-Grocott stain, immunostain, or PCR pneumocystis), and clinical course of patients.Beta-D-Glucan was uncommonly used in participating centers during both studies periods.For both included studies, study investigators reviewed a posteriori every diagnoses for coherence and alignment with established definitions (EFRAIM, TRIALOH).Ground glass opacities were defined as any degree of ground glass opacities on CT-scan.
Anti-Pneumocystis prophylaxis was based on patients' prescription before ICU admission without any regard to adherence.
Lymphoid hematological malignancy was defined as any acute or chronic underlying hematological malignancy including acute lymphoid leukemia, non-Hodgkin's lymphoma, and chronic lymphoid leukemia.
Hematopoietic Stem Cell Transplantation (HSCT) was defined by any allogeneic or autologous stem cell transplantation independently of the conditioning protocol, origin or compatibility of donor cells, and without regard for delay since HSCT transplantation.

Estimation of intrinsic performances of PCR and BD glucan by systematic literature review
A systematic review was performed on MEDLINE database using "Pneumocystis pneumonia (MeSH)" AND "sensitivity and specificity (MeSH)" AND/OR "(1-3)-β-D-Glucan" AND/OR "PCR" NOT "HIV (MeSH)".Estimation of diagnostic test performance was validated by three authors (LC, VL, MD).

Experts' priors
Physicians' priors were assessed before and after study results presentation.To do so, a standardized questionnaire assessing perception of priors with regard to incidence and post-test probability were obtained using visual analogic scale ranging from 0 to 100.Some data regarding experts' characteristics were concomitantly obtained.Priors were searched for during a meeting of our research group (Groupe de Recherche en Réanimation Respiratoire et Onco-Hematologique).These meeting are held three time a year, contain both didactic presentations and presentation of study results focused on critically-ill cancer patients, with an attendance ranging from 50 to 100 intensivists.Responders were defined as being an expert if they presented on symposia or published in this field.

Statistical analysis
Incidence and 95% confidence interval were computed by normal approximation in the general immunocompromised population with ARF.
To assess incidence and relevant clusters, a supervised tree partitioning was performed.Incidence and 95% confidence interval were computed in subgroup of interest identified by tree partitioning.Variables included in the supervised tree partitioning were age, gender, underlying immune defect, SOFA score, presence of ground glass opacities and preexisting prophylaxis.
Incidence and pre-test probability were then simulated in 5000 samples for the total cohort and subgroups identified by tree partitioning.These populations were simulated using a normal distribution centered on observed incidence and its confidence interval.
Diagnostic performance was modelized according to observed intrinsic performance, assuming binary results (positive vs. negative) and disregarding change in performance which may arise from quantitative analysis of either test or sample in which PCR is performed.Sensitivity and specificity were computed assuming a 0.5% uncertainty, following a continuous distribution and according to three hypotheses being, respectively: Intermediate diagnostic performance: both sensitivity and specificity centered around the median observed performance for both PCR and BDG; High sensitivity (weakly positive test): sensitivity was centered to the highest range of CI and specificity to the lowest range of the observed 95% confidence interval.
High specificity (highly positive test): sensitivity was centered to the lowest range of the observed CI and specificity to the lowest range of CI.
Post-test probability was computed using Bayes theorem and according for incidence and computed diagnostic test performance for the total cohort and the identified subgroups.Results are reported as median (IQR) and binary plot comparing pretest probability and post-test probability.In these plots, at a given pre-test probability, variability post-test probability reflect uncertainty regarding diagnostic test performance, and range of post-test probability uncertainty regarding true disease incidence.Post-test probability of successive concordant or discordant PCR and BDG was assessed assuming complete conditional independence of tests.
Last, in way to assess the influence of findings on physicians' priors, densities of priors and difference between priors and findings were plotted before and after presentation of the results.
Analyses were performed using R software version 4.3.4(R Project for Statistical Computing, Wien, Austria), rpart and infer packages.
The observed Pneumocystis pneumonia incidences varied from 2.0% (95%CI 1.8-2.2) in patients without ground glass opacities to 20.3% (95% 18.3-22.2) in patients without prophylaxis, with underlying lymphoid leukemia or previous stem cell transplantation and with presence of ground glass opacities.

Simulated incidence and post-test probability
The simulated incidence of of Pneumocystis pneumonia is reported in Tables 1 and 2.
Performances according to risks clusters and test performance are depicted in Tables 1 and 2. In the highest risk group, post-test probability if test found positive ranged from 60.0 to 82.2% for PCR (Table 1).Similarly, post-test probability if test found positive ranged from 58.0 to 68.6% for BDG (Table 2).
Figures S1 and S2 report post-test probability according PCR results and subgroups.Figures S3 and S4 report post-test probability according PCR results and subgroups.
Figure S5 reports post-test probability after successive PCR and BDG testing assuming complete conditional independence of both tests.

Intensivists' priors
Overall, 25 physicians were interviewed before and after presentation of our results.Of them 5 had published a manuscript on Pneumocystis pneumonia or had been invited as speaker to our research group meeting and were considered experts.Median age of responders was

Discussion
Our study is the first to the best of our knowledge to assess incidence of Pneumocystis in way to delineate post-test probability of disease in identified subgroups.According to our results, despite assumption of a high sensitivity and specificity, positive post-test probability of Pneumocystis pneumonia is limited after positive PCR or BDG test as consequences of the limited incidence of the disease in immunocompromised patients.We propose a visual representation (Figs. 2 and 3) that may help physician to appreciate interaction between observed incidence and intrinsic diagnostic test performance at bedside while taking into account incidence and diagnostic test performance uncertainties.
In line with previous studies, the incidence of Pneumocystis pneumonia is low in the population of critically ill immunocompromised patients admitted for an acute respiratory failure [17,25,28].In line with previous studies, identified risk factors of Pneumocystis pneumonia were preexisting lymphoid disease or stem cell transplantation, ground glass opacities at CT scan and lack of Pneumocystis prophylaxis [17,29].In this large cohort of patients suggest incidence ranging from 2% (1.4-2.8%) in   lowest risk subgroup to 20.2% (17.8-27.8%) in the high risk subgroup.Not surprisingly, even assuming excellent intrinsic performance of Pneumocystis PCR and BDG, our results underline that the poor positive post-test probability resulting from the low incidence translates into high risk of false positive diagnosis and, more importantly, failure to identify culprit of the ARF with potentially negative consequences [19,25,30].
Interestingly, although arising from simulation data, our attempt to depict graphically relationship between incidence in risk subgroup and post-test probability translated into dramatic change in perception of disease incidence, diagnostic performance of the test in this setting and ultimately perception of potential significance of diagnostic test [12,31].Previous studies underlined frequent physicians' misinterpretation of disease incidence [32,33], frequent misperception and overestimation of diagnostic test performance [10,34,35] and most importantly a limited comprehension of pre-test/post-test relationship and its implication at bedside [10,34,35].Thus, when facing various hypotheses of disease incidence, previous studies suggested a lack of perception of changes in post-test probability by physicians [14].In addition, previous studies suggested misperception to persist even when diagnostic test performance were described as likelihood ratios rather than as sensitivity and specificities, these later being independent of disease incidence 13 .Thus, previous studies suggested use of natural frequency of disease to allow better perception of diagnostic test performance [10], short course of Bayesian reasoning to partly improved this perception [12] and, best of all, graphical representation to allow improved understanding of diagnostic test performance (32).Although these results may deserve to be confirmed, our depiction of the pre-test / post-test relationship in various subgroups may help in depicting accurately a known and mathematical relationship, may help physician in apprehending input of positive or negative test in various subgroup or clinical scenarios, while accounting for uncertainty both in term of diagnostic test performance and in disease incidence.
This study has several limitations.First, we aimed in assessing incidence and depicting incidence/post-test probability in various predefined subgroups.Although this approach may make sense at a population level, it disregards fact that for a given patient, pre-test probability ranges from 0 to 1 and cannot be limited to the incidence of the disease.Hence, several subgroup of interest were not tested including stratification according to duration of symptoms before ICU admission, type of anti-Pneumocystis prophylaxis, or specific symptoms of ARF.In this line, solid organ recipients were underrepresented which may have influenced our results and results of the high-risk group may have been modified by emerging targeted therapies.Our results may however help in identifying subgroup of patients in whom tests may be useless or at least should be interpreted only if negative.This limit suggests additional studies in specific subgroups may be required to refine our results and improve overall view of pre-test probability.Last, although we tried to depict impact of successive test, these results are probably misleading.Hence, both test are likely to be highly correlated, and covariance of both test has never been assessed to the best of our knowledge.As consequences, performance of concordant PCR and BDG in our study probably overestimate post-test probability and dedicated studies are needed.
Moreover, diagnoses of Pneumocystis pneumonia in the initial dataset were confirmed in most cases using either PCR and/or BDG test, in specific setting and after expert validation.Although this could have impaired assessment of diagnostic test performance, we only used these data to set range of incidence and not to validate intrinsic test performance of the test, this limit being unlikely to have influenced our findings.Furthermore, we lack validated gold standard in confirming Pneumocystis pneumonia.Therefore, incidence in the study population may have been overestimated.This bias however strengthens our findings, lower incidence translating into lower positive post-test probability.Last, although we validated our results when compared to experts' priors, no validation against existing standards was performed.Thus, whether our visual representation may perform better than classical Fagan's nomogram with usual pre-test probability range underlined (example given as figure S9) may deserve to be assessed in future studies.

Conclusion
In this study we hypothesized and validated that despite excellent intrinsic performance, both Pneumocystis PCR and BDG displayed a limited positive predictive value in critically ill immunocompromised patients with acute respiratory failure.This analysis underlines need for adequate pre-test estimation of probability of Pneumocystis pneumonia to allow interpretation of laboratory tests results.We display a visual representation that may help physician to understand influence of observed incidence on post-test probability of a disease and be a first step implement clinical vignette to underline case-scenario in which these tests might be relevant.

Fig. 1
Fig. 1 Tree reporting main clusters as regard to Pneumocystis pneumonia in the analyzed datasets (n = 2243)

Fig. 3 Fig. 2
Fig. 3 Relationship between incidence and post-test probability in the high-risk population of immunocompromised patients with ARF and according to PCR result (positive = red, negative = blue).Diagnostic test performance are ranged from highly sensitive (light color) to sensitive test (dark color)

Table 1
Post-test probability of positive and negative PCR in non HIV critically ill patients with respiratory failure for the overall population and for the different subgroups of incidence

Table 2
Post-test probability of positive and negative BDG in non HIV critically ill patients with respiratory failure for the overall population and for the different subgroups of incidence