Probiotics are defined as “live microorganisms, which when administered in adequate amounts, confer a health benefit on the host.”1 Mechanisms by which probiotics offer potential health benefits are not yet fully elucidated. They may include enhanced gut barrier function, reduced gastrointestinal pathogenic bacterial load through competitive inhibition, modification of the gut microbiome, and modulation of the host immune system. These effects may reduce the incidence of healthcare-associated infections.2,3

Probiotics have been studied in randomized-controlled trials (RCT) in a variety of conditions in the hospital setting with evidence suggesting benefits, including the reduction of healthcare-associated infections.4,5 In the intensive care unit (ICU), probiotics have been studied for the prevention of ventilator-associated pneumonia (VAP).3,6 Multiple probiotic strains (i.e., Lactobacillus, Bifidobacterium, Saccharomyces) and doses have been systematically reviewed; a meta-analysis revealed a risk reduction of 0.74 for VAP (95% confidence interval [CI], 0.61 to 0.90; P = 0.002), showing a potential effect across species.6 As the most common healthcare-associated infection in ICU, VAP is associated with a two-fold attributable risk of death, and an attributable cost of 10,000–13,000 USD/patient.7

Further evidence suggests that probiotics can reduce the incidence of diarrhea, specifically Clostridioides difficile-associated diarrhea (CDAD), which can cause pseudomembranous colitis, toxic megacolon, and death.8 A Cochrane systematic review and meta-analysis of 31 RCTs including 8,672 patients receiving concurrent administration of probiotics (any dose, any strain) and antibiotics showed that probiotics prevented CDAD compared with placebo (based on moderate certainty evidence), with heterogeneous evidence for a specific species or dose effect.8 Treatment for CDAD is expensive (8,911–30,049 USD/patient).9

Among critically ill patients, the clinical effectiveness of probiotics in preventing VAP, CDAD, and other infectious outcomes was evaluated in a recently completed but as yet unpublished multicentre RCT (Probiotics: Prevention of Severe Pneumonia and Endotracheal Colonization Trial - PROSPECT; NCT01782755), with additional RCTs ongoing (PRINCESS: Probiotics to reduce infections in care home residents; ISRCTN16392920).

Health economic evaluations produce important evidence to inform clinical decisions and health policy creation. The objective of this systematic review is to summarize cost or cost-effectiveness evidence of a broad spectrum of strategies involving probiotics (different doses and strains) in hospitalized adult patients. The research question was: in hospitalized adult patients (population), do probiotics (intervention: any strain, any dose) vs placebo/no treatment (comparator: usual care) show cost-effectiveness in preventing healthcare-associated infections (VAP, CDAD, and antibiotic-associated diarrhea [AAD])?

Methods

Data sources and searches

Our search strategy is outlined in eAppendix 1 (available as Electronic Supplementary Material [ESM]). Searches were performed by a clinical librarian (A.I.) with experience in conducting electronic literature searches. Searches underwent peer-review of electronic search strategies by a professional librarian and our authors. No publication type or language restrictions were applied.

To identify additional potentially relevant studies, we also checked reference lists of identified articles within our systematic review search, to examine what source inputs were utilized in their economic evaluations.

Study selection and quality assessment

Two reviewers independently assessed each citation and applied inclusion/exclusion criteria (Figure). Two reviewers (V.L./J.B.) independently screened abstracts in the first stage, and the full-text in the second stage. Disagreements were resolved by a third reviewer (B.R./F.X.). We listed the characteristics of the included studies (Table 1). Quality of studies was critically appraised (Table 2) using the Joanna Briggs Institute for Critical Appraisal of Economic Evaluations tool10 and the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.11 Our systematic review has been registered in PROSPERO (international prospective register of systematic reviews): CRD42019129929 (www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=129929). Our literature search commenced before registration, and data extraction was underway (but not completed) when registered (started January 31, 2019, registered April 25, 2019).

Figure
figure 1

E-PROSPECT PRISMA flow diagram. Selection criteria: (1) full economic evaluation (cost-minimization, cost-benefit, cost-effectiveness, cost-utility) conducted alongside clinical studies or via economic modeling, (2) the study described hospitalized patients, (3) the study included probiotics as a treatment, (4) the study described drug acquisition costs, the costs of providing prophylaxis, costs of complications, (5) the study described the effect of prophylaxis with respect to one of our clinical outcomes of interest including VAP, CDAD and antibiotic-associated diarrhea (AAD)

Full size image
Table 1 Summary of health economic studies of probiotics
Full size table
Table 2 Critical appraisal of study articles
Full size table

Data extraction

Independently and in duplicate, our reviewers (V.L./J.B.) extracted data using pre-developed abstraction forms (eAppendix 2 available as ESM). We attempted to contact study authors for all study-related data, if not previously published. All currencies were converted to Canadian dollars (CAD) for the year 2018 utilizing the World Bank Official Exchange Rate.12 Incremental costs, effectiveness outcomes, or cost-effectiveness ratios are presented in Table 3.

Table 3 Incremental costs, effects, and cost efficacy ratios for the probiotics vs comparator (placebo/no treatment/usual care)
Full size table

Risk of bias assessment

Randomized-controlled trials used as data sources for the health economic evaluation were assessed using the Cochrane Collaboration Risk of Bias (ROB) tool.13 Non-randomized trials were assessed using the Newcastle-Ottawa Scale.14 Surveys were assessed using the ROB tool from the McMaster University Clinical Advances Through Research and Information Translation (CLARITY).15 The assessment schemas are found in eAppendix 3 (available as ESM) or in the footnotes of eAppendix 4A–D (available as ESM).

For model-based economic designs, we assessed ROB in the contributing inputs from multiple source studies for the models. We decided a priori that, if each source input in a particular economic model had low ROB, the overall model would likely have a low ROB (even for varied types of studies—from RCTs to surveys). If any source study had an unknown/high ROB (identified as the weakest link), the entire economic evaluation would be assessed an unknown/high ROB. For source articles drawn from systematic reviews, guidelines documents, or economic evaluations, we did not assess ROB unless that source was not previously assessed in eAppendix 4A–D (available as ESM). We did not assess ROB when data were derived from an externally established public database (i.e., Consumer Price Index).

Data synthesis and analysis

We summarized the economic evaluation data (e.g., resource utilization, costs, cost-effectiveness ratios) in terms of point estimates and 95% CIs or ranges, if available. Categorical variables were reported as counts/proportions. Given the heterogeneity among the included studies, we could not conduct a meta-analysis. This review is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) guidelines.16

Grading of recommendations assessment, development, and evaluation (GRADE)

We used the GRADE approach (Table 4) to assess the following domains: ROB, indirectness, imprecision, inconsistency, and other considerations. Certainty in evidence from RCTs started as high, while observational studies started as low. Final quality was rated high, moderate, low, or very low.17

Table 4 Grading of Recommendations Assessment, Development and Evaluation (GRADE) of Probiotics Systematic Review Outcomes: VAP, CDAD, AAD
Full size table

Results

Study comparisons, populations, and format

Out of 721 records identified through database searches, 147 duplicates were removed and 526 excluded based on an irrelevant title/abstract. The full-text of 48 papers was retrieved for comprehensive evaluation, of which 41 were excluded (Figure).

Of seven studies included in this systematic review (Table 1), one study was a RCT-based cost-effectiveness analysis (CEA).18 Six studies were model-based economic evaluations using CEA or incremental cost.19,20,21,22,23,24 Two also reported cost-utility analysis.19,23 One evaluation investigated VAP,18 six investigated CDAD,19,20,21,22,23,24 and three investigated AAD.19,22,24

Study perspectives, time horizon, and funding

Three studies were conducted in the United States,18,20,23 two in the United Kingdom,19,22 one in Canada,25 and one in Belgium.26 Four studies were conducted from the societal perspective18,19,24 (aggregation of all perspectives, taking into account time costs, opportunity costs, and community preferences, i.e., patient, payer, hospital)27 and seven from the perspective of a specific payer (four public and three private payers).18,19,20,21,22,23,24 The time horizon (duration of time for follow-up, over which health outcomes and costs are calculated) ranged from three to 52 weeks. A probiotic manufacturer supported three of seven (43%) studies.20,22,24

Study quality and risk of bias

Study quality is summarized in Table 2. Two studies obtained effectiveness data from meta-analysis,21,23 while seven studies obtained data from RCTs or observational trials.18,19,20,21,22,23,24 All performed sensitivity analyses.18,19,20,21,22,23,24

For assessing ROB in RCTs (eAppendix 4A [available as ESM]), three studies19,20,21 had a low ROB, while four studies28,29,30,31 had unclear/high ROB. Common ROB issues were selection, performance, detection, attrition, and reporting bias. For observational study ROB (eAppendix 4B [available as ESM]), there were six high-quality cohort observational studies,25,28,29,32,33,34 and ten low-quality cohort studies.26,30,31,35,36,37,38,39,40 Common ROB were selection (only selected group of patients representing the intervention cohort, no description of non-exposed non-intervention cohort, and no demonstration that outcome of interest was absent at the start of the study), comparability (study did not control for age, antibiotic/probiotic exposure, or additional factors), and outcome (short follow-up).

For ROB in case-control studies (eAppendix 4C [available as ESM]), there was one high quality study41 and one low quality study.42 Common ROB were selection (no description of case definition, representativeness shows potential for selection bias, no description of control—case-study only, no description of source), comparability (study did not control for age, antibiotic/probiotic exposure, or additional factors), and outcome (no method of ascertainment for controls, non-response rate, and different with no designation).

For ROB in surveys (eAppendix 4D [available as ESM]), there were two high-quality studies,43,44 and two studies with a mix of low/high ROB.18,45 Common ROB issues were low response rates (< 50%), missing data (> 15% within questionnaires), and no evidence of reliability/validity of the survey instrument.

Cost and effect estimates

The cost and effect estimates are shown in Table 3. Individual natural units and unit cost per resource are presented in eAppendix 5 (available as ESM).

Ventilator-associated pneumonia

One evaluation investigated VAP (Table 3). Using a Markov model for a cost-benefit analysis, prophylactic probiotics (with subglottic endotracheal tubes) showed cost benefit for preventing VAP, with a willingness-to-pay (WTP) of 50,000–100,000 USD (70,807–141,614 CAD) per case of VAP averted (median [range] cost estimate of 15,958 USD [7,000–35,000] or 22,623 CAD [9,913–49,566] per VAP case). The incremental cost-effectiveness ratio (ICER) between probiotics and no probiotics showed dominance of probiotics over placebo (with usual care). Sensitivity analysis showed continued dominance in a multiple scenarios (reducing cost of VAP, and increasing hourly nursing wages). There was a substantial increase in cost-savings with probiotics when VAP risk reduction was increased vs placebo.18

Clostridioides difficile-associated diarrhea

Among six studies examining the cost-effectiveness of probiotics in CDAD (Table 3), four studies found probiotics to be cost-effective/incremental cost-saving,20,22,23,24 one study showed no difference,19 and one study showed cost-effectiveness in certain scenarios.23

Kamdeu Fansi et al. found a cost-savings dose response for probiotics vs placebo. There was a cost-saving of 1,968 USD (2,152 CAD) for a single dose of probiotics (per CDAD case prevented) compared with placebo. For a double dose of probiotics per day, there was a cost-saving of 2,661 USD (2,910 CAD) compared with placebo.20 Leal et al. showed cost-savings of 538 CAD per patient (340 CAD for probiotics vs 878 CAD for usual care) for CDAD.21

Shen et al. showed a cost-saving of 840 USD (1,150 CAD) per case of CDAD averted, with dominance of probiotics (lower cost and higher effectiveness) in the base case. Nevertheless, there were scenarios (i.e., young patients) in which the ICER was not cost-effective (age, 18–44 yr; CDAD risk, 0.6%: ICER, 884,100 USD/quality-adjusted life-year [QALY] and 1,196,609 CAD/QALY).23 Furthermore, Allen et al. showed there was no difference in total healthcare costs between probiotics (£8020; 95% CI, 7620 to 8420 and (15,629 CAD; 95% CI, 14,850 to 16,409) and placebo (£8010; 95% CI, 7600 to 8420 and 15,601 CAD; 95% CI, 14,811 to 16,409).19

Antibiotic-associated diarrhea

Among three studies examining the cost-effectiveness of probiotics for AAD (Table 3), two studies found probiotics to be cost-effective,22,24 with one study showing no difference between probiotics and placebo.19

Lenoir-Wijnkoop et al. showed a mean cost-saving of £339 (642.94 CAD) per hospitalized patient for probiotics vs no treatment for prevention of AAD.22 Vermeersch et al. found cost-savings of €50.30 (75.74 CAD) using a bottom-up approach and €28.10 (42.31 CAD) using a top-down approach per AAD patient treated with antibiotics from a payer’s perspective. From a hospital/societal perspective, there was a cost-saving of €95.20 (143.35 CAD) (bottom-up) and €14.70 (22.13 CAD) (top-down) per AAD patient treated with probiotics.19

Conversely, Allen et al. found that probiotics were not cost-effective, with an ICER for AAD prevention of £4531.36 (£3,439.80–£5,622.92; 8,830.58 CAD [6,703.39–10,957.79]), and a base-case cost-utility of £189,662 (369,608 CAD) per QALY, for a WTP threshold of < £20,000 (38,975 CAD)/QALY.19

Sponsorship, economic perspective, trial vs. model-based, and placebo vs. no probiotic subgroup comparisons

Overall, of the seven studies included, six (86%) economic evaluations favoured probiotics as cost-effective/cost-saving in the base case. Three studies (43%) were sponsored by the manufacturer (Lactobacillus acidophilus/casei/paracasei). All three reported favourable findings towards probiotics. Three of four studies without manufacturer sponsorship favoured probiotics. Publication bias cannot be excluded.

The one trial-based economic evaluation did not show cost-effectiveness for its outcome,19 while all six model-based evaluations showed cost-effectiveness in their base cases and certain sensitivity analyses.18,20,21,22,23,24 For economic perspective subgroups, six of seven (86%) payer perspectives were cost-effective, while two of three (66%) of societal perspectives were cost-effective. For comparators, control arms (placebo vs no probiotic subgroups), two of three (66%) with placebo control arms were cost-effective, while four of four (100%) with no treatment/usual care control arms were cost-effective.

Grading of recommendations assessment, development, and evaluation assessment

The GRADE assessment46 (Table 4) found very low certainty of evidence for probiotic use for VAP, CDAD, and AAD.

The outcome of VAP included one model-based economic evaluation. We downgraded for ROB (serious ROB from multiple model inputs with unclear/high ROB) and imprecision (serious for only one study in analysis).18

The outcome of CDAD included six health economic evaluations (one RCT-based and five model-based). We downgraded for ROB (serious: multiple model inputs with unclear/high ROB), inconsistency (serious: one not cost-effective and five cost-effective) and imprecision (CI crosses zero for one RCT economic evaluation, with many small studies included).19,20,21,22,23,24

The outcome of AAD included thee health economic evaluations (one RCT-based and two model-based). We downgraded for ROB (serious: multiple model inputs with unclear/high ROB), inconsistency (serious: one study not cost-effective, and two studies cost-effective) and imprecision (serious: CIs crossing zero in the largest RCT to date, with many small studies included).19,22,24

Discussion

In this systematic review of economic evaluations of probiotics in hospitalized adult patients, we found that most of the studies suggest probiotics are cost-saving/cost-effective in preventing VAP, CDAD, or AAD.18,19,20,21,22,23,24 Nevertheless, the largest trial-based RCT paired with a health economic evaluation to date found no difference in clinical outcomes, and no cost-effectiveness/cost-utility.19 The conclusions drawn from the collective studies in this systematic review are based on very low certainty evidence from the ROB and GRADE assessments, precluding strong inferences or definitive recommendations regarding probiotics.

We found no prior systematic reviews that focused on economic evaluations of probiotic prophylaxis in hospitalized patients, hence we conducted our own. Among economic evaluations included in this review, incremental costs/ICERs were expressed in costs per healthcare-associated infection event prevented, but heterogeneity in reporting prevented meta-analysis conduction. Further, variable time horizons make comparisons of economic evaluations problematic (specifically ICERs) as costs and resource utilization may change over different time horizons. Changes in time horizons or perspectives can lead to differing parameters (costs [direct vs indirect], or outcomes [patient vs payer]). Many studies only reported incremental costs rather than true ICERs. Results from different perspectives, time horizons, and variable incremental cost reporting all represent disparate cost outcomes, which need to be interpreted carefully within context.

Moreover, there are large ranges in WTP, which are difficult to interpret with no conventional WTP benchmarks for prevention of VAP, CDAD, and AAD. Different countries may differ on values quality of life and WTP, making benchmarks difficult to establish across jurisdictions. Cost-utility parameters (like cost per life-year or QALY gained) were less commonly reported. If cost per QALYs were available, it would help to inform economic comparisons with other healthcare interventions.

Compared with other infection-prevention strategies, probiotics appear to be similarly cost-effective. A study examining concomitantly administered central-line associated bloodstream infection (CLABSIs) and VAP programs combined documented ICERs of 14,250.74 USD (20,533.24 CAD)/life-year gained and 23,277.86 USD (33,540.02 CAD)/QALY.47 Multifaceted quality improvement programs for reducing CLABSIs in ICUs have shown dominance (lower cost and higher effectiveness) in 80% of model scenarios using probabilistic sensitivity analysis.48 A proactive model infection-control program for multi-drug resistant (MDR) organisms in general-surgical ICUs showed an ICER of 3,804 USD (5,320.01 CAD) per case averted of transmission of MDR organisms in one year compared with standard infection control. For a WTP threshold of 14,000 USD (19,579.43 CAD) per transmission averted, there is a 42% probability of being cost-effective, and 100% probability when WTP thresholds were 22,000 USD (30,767.68 CAD).49 These similarities suggest that adoption of probiotics for prevention of healthcare-associated infections could be cost-effective.

New interventions studied in economic evaluations are occasionally sponsored by drug manufacturers. This potentially introduces bias in model construction and interpretation of results. In a retrospective analysis of 107 studies in five leading medical journals with regard to outcome and sources of funding, trials sponsored by pharmaceutical companies were more likely to favour the new drug over traditional therapy.50,51

In our systematic review, three studies were funded by manufacturers and all found the sponsored intervention to be more economically attractive, which could suggest potential publication bias (although this was not proven). This is tempered by three of four peer-review funded studies that also showed cost-effectiveness. Hence, methodologically rigorous trials with concomitant economic evaluations from peer-review funded studies are needed to ensure proper interpretation of results.

Strengths of our review include adherence to rigorous methodology, consisting of a comprehensive search strategy, broad eligibility criteria, and study selection by two independent adjudicators to minimize selection bias.17 We conducted data abstraction and appraisal in duplicate, using established criteria for assessing economic evaluations.11 We performed assessments of study quality employing ROB assessments, including assessment of source studies utilized in model-based economic evaluations.13,14,15 We performed assessment of level of certainty using GRADE.17 We also addressed the relationship of for-profit industry sponsorship potentially influencing the reporting of economic evaluations.

This review also has limitations. The inclusion of only seven studies influences precision. Rare product-specific complications such as probiotic-induced complications (i.e., bacteremia) are unclear, underscoring the need for additional safety data. Overall GRADE certainty of evidence was very low for all outcomes, rendering conclusions non-definitive. Our review included only adult patients and may not be applicable to pediatric populations. Evaluated reports varied widely with respect to patient population, time horizon of therapy, and payer perspective, which challenges the generalizability and interpretation of these findings.

Conclusion

This systematic review found that probiotics may be an economically attractive strategy for the prevention of healthcare-associated infections in most studies. Nevertheless, our GRADE summary indicates a very low quality/certainty of evidence, such that inferences are weak regarding the health economic evaluation of probiotics in adult hospitalized patients. Future RCTs should include concomitant economic evaluations, including clinical outcomes and costs associated with probiotics, to inform bedside practice, clinical guidelines, and healthcare policy. To this end, an economic evaluation of PROSPECT (E-PROSPECT) is planned.