Probiotics are live microorganisms that may confer a potential health benefit on the host.1 They are reported to enhance gut barrier function, reduce host pathogenic bacterial load, modify gut microbiota, and modulate the immune system.2,3,4,5 Randomized trials of probiotics suggest benefits including reduced healthcare-associated infections, including ventilator-associated pneumonia (VAP)6,7,8,9,10 and Clostridioides difficile-associated diarrhea (CDAD).11 Nevertheless, probiotics have modest additional drug-acquisition costs associated with their use. Whether probiotics are used in critical care practice will depend on the ability of probiotics to prevent healthcare-associated infections and reduce healthcare resource consumption associated with infection. Prior health economic evaluations of other interventions for critically ill patients have shown important cost-effectiveness differences12,13,14 despite the clinical effectiveness of these interventions derived from randomized trials being uncertain.13,14,15 Cost-effective analyses in the intensive care unit (ICU) setting are important, given that critical care is expensive16,17,18,19,20 and even minimal additional drug acquisition costs can still lead to large incremental differences in healthcare costs. Therefore, practice decisions should be guided by rigorous comparative economic and clinical effectiveness research to inform bedside care, clinical guidelines, and policy.21,22,23,24

A recent multicentre blinded, randomized trial—the Probiotics to Prevent Severe Pneumonia and Endotracheal Colonization Trial (PROSPECT, www.ClinicalTrials.gov: NCT01782755)—compared the efficacy of probiotics plus usual care (probiotics group) vs placebo plus usual care (usual care group).25,26,27 The trial found no difference between probiotics and usual care regarding VAP, CDAD, antibiotic-associated diarrhea (AAD), or death.28 We therefore conducted this economic evaluation alongside the PROSPECT trial (E-PROSPECT) following an a priori protocol.29 We measured healthcare resource use and costs, within the context of clinical outcomes, to determine the incremental cost-effectiveness of probiotics in addition to usual care vs usual care alone in critically ill patients requiring invasive mechanical ventilation.

Methods

Design

The primary objective of E-PROSPECT was to estimate the incremental costs per VAP prevented associated with the use of probiotics and usual care (probiotics group) vs usual care and placebo (usual care group) during hospitalization. Secondary outcomes also assessed cost-effectiveness of CDAD, AAD, and mortality.25,26,27,29 We performed the economic evaluation from the public healthcare payer’s perspective, over the time horizon of the ICU randomization to in-hospital discharge or death (Table 1). We developed the economic evaluation according to established economic evaluation guidelines, including cost-effectiveness analysis recommendations23 and Consolidated Health Economic Evaluation Reporting Standards30 (Electronic Supplementary Material [ESM] eAppendix 1) with a checklist (ESM eTable 1).

Table 1 Summary of health economic evaluation framework (E-PROSPECT)
Full size table

We prespecified the statistical analysis plan as part of the E-PROSPECT protocol before trial completion and unblinding.29 A priori informed consent was obtained from each trial participant or their substitute decision-maker. This economic evaluation was approved by the Hamilton Integrated Research Ethics Board (REB) of McMaster University (project identifier: REB#: 15-322) to include clinical outcome and unit costing data at all participating centers.

Patients

The Probiotics to Prevent of Severe Pneumonia and Endotracheal Colonization Trial was an international randomized trial in which clinicians, adjudicators, and patients were blinded. Critically ill patients received either probiotics (1 × 1010 colony forming units of Lactobacillus rhamnosus GG [i-Health, Inc., Cromwell, CT, USA]) or identical placebo suspended in tap water administered enterally twice daily while in the ICU. Detailed eligibility criteria are described elsewhere.27

From October 2013 to March 2019, we randomized 2,653 critically ill mechanically ventilated patients. We recorded unit costs after the last patient was recruited, and prior to PROSPECT analysis and publication. We excluded from all analyses three patients who received no study product and had no data collection. In the final analysis, 2,650 patients were included; 1,332 in the placebo and 1,318 in the probiotics group.28 The economic analyses were based on the intention-to-treat principle.

Clinical outcomes

We collected the clinical effects, frequencies, or proportions, per-patient event rates for all enrolled patients. The primary clinical outcome unpinning this economic evaluation was the difference in VAP infections. Secondary clinical outcomes included differences in CDAD, AAD, and mortality. Given the in-hospital time horizon and emphasis on infection, we did not measure health-related quality of life (quality-adjusted life-years) or extrapolate lifetime outcomes.

Unit costs and health resource use

The E-PROSPECT steering committee reviewed the relative importance of cost variables.29 If a unit cost for a particular line-item was considered to be small and/or infrequent, and similar12 between the groups (and therefore unlikely to influence the incremental difference in total costs), then that line-item was not incorporated and was removed from the final analysis. We report individual unit costs per line-item per jurisdiction (ESM eAppendix 2).

Healthcare resource use was collected for 2,650 patients enrolled in 44 hospitals in three countries (41 hospitals in Canada, two in the USA, and one in Saudi Arabia).

We developed a line-item list of unit costs/healthcare resource use (by category: medications, physician/personnel, diagnostic radiology/laboratory testing, operative/nonoperative procedures and per-day hospital [e.g., hoteling] costs not otherwise encompassed) with total costing (resource use multiplied by unit cost) methodology described elsewhere (ESM eAppendix 1).29 We defined hospital hoteling unit costs as direct nonmedical costs (general services/procedures which benefit more than one patient at a time, [e.g., utilities like electricity and hydro]).26,27 We collected ICU or ward per diem costs (disaggregated where possible) based on length of stay as the hoteling costs. Duplicate disaggregated unit costs reported at a site level were removed.

We preferentially recorded unit costs published by public healthcare payers (e.g., schedule of benefits within a regional health system). For unit costs not available through the public sources, we performed a pilot study at nine participating centers (representing each jurisdiction).29 A jurisdiction was defined as a territorial area (e.g., province, state, or territory) that is responsible for the costing and delivery of healthcare in that region.29 We collected data from hospital’s accounting, human resources, pharmacy, and radiology or laboratory departments, where available.12,31

If a specific line-item unit cost was not attainable for a specific jurisdiction, we: (1) asked another site within the same jurisdiction for missing unit costs; (2) used a mean unit cost approach for the country’s jurisdictions, which did report unit costs (with estimated standard errors);12,29,31 and, (3) when no data were available in a certain jurisdiction, we used multiple imputation or derived a cost ratio from previously acquired line-items to derive the missing unit costs.29 We recorded professional consultation or procedural/surgical costing (performance, interpretation, or both), and technical costs for procedures, where applicable.

Costing, primary cost-effectiveness analysis, and subgroup/sensitivity analyses

We used descriptive analyses, including means with standard deviations (SDs), medians with interquartile ranges [IQRs], and counts with proportions to describe baseline characteristics, effects, and cost estimates where appropriate. We adjusted all costs to 2019 USD, accounting for differential inflation and currency exchange rates.32,33,34,35 We used international currency conversion instead of purchase power parity (PPP)-based conversions, as health-specific PPPs are not available for all countries.29

For our base-case/primary analysis, individual resource use was multiplied by jurisdiction unit costs to calculate individual patient total costs.29 We calculated total costs for the probiotic and usual care groups by summing each of the individual patient costs, and then dividing by the number of patients to calculate the mean cost per patient in each group. Incremental costs were taken as the difference in mean per-patient costs between groups. We defined incremental effects as the difference in proportions of clinical outcomes between groups (given differing sample sizes between groups).

For missing data, we chose imputation methods as outlined in our statistical analysis plan.29,36,37 In brief, we estimated an appropriate “standard dose” for nontitrated medications (e.g., chlorhexidine) and a clinically appropriate “medium dose” for various titratable medications (e.g., vasopressors, inotropes). Electronic Supplementary Material eTable 2 outlines assumptions for estimating other resource use. The incremental cost-effectiveness ratio (ICER) measured the ratio of incremental costs per incremental clinical outcome of probiotics vs usual care for each of the clinical outcomes (VAP, CDAD, AAD, and mortality).29

We conducted prespecified subgroup analyses, including diagnostic category (medical, surgical, trauma)38; age < 65 yr, 65–75 yr, and > 75 yr39,40; frailty status (baseline Clinical Frailty Score > 5 vs < 5)41; patients who received vs did not receive antibiotics within two days of randomization;27 prevalent (present at the time of enrolment) vs non-prevalent pneumonia.27

To assess the uncertainty associated with cost and effects estimation, we used nonparametric bootstrapping with replacement techniques to generate 1,000 simulated pairs of costs and effects for probiotics and usual care groups for all outcomes (VAP, CDAD, AAD, mortality). We used cost-effectiveness acceptability curves (CEAC) to present the probability of probiotics being cost effective over a wide range of willingness-to-pay (WTP) thresholds. We performed sensitivity analyses with variations of estimates of pairs of potentially influential variables (e.g., per day cost of care in ICU, a time horizon of 60 days, and Canadian jurisdictions) across plausible ranges to determine if different estimates change the overall results.

We performed all analysis using Excel version 14.0.6 (Microsoft Corporation, Redmond, WA, USA), and SAS version 9.4 (SAS Institute Inc., Cary, NC, US).

Results

Characteristics of study population

Patient characteristics of the E-PROSPECT trial are as published in the trial report.28 The mean (SD) age of enrolled patients was 59.8 (16.5) yr, 40.1% were female, and 76.5% were medical admissions.

Clinical outcomes and incremental effects

The main findings and clinical outcomes (event rates) of PROSPECT are described in Table 2. The difference in proportions of VAP events between probiotic vs placebo groups was 0.6% (21.9% vs. 21.3%; 95% confidence interval [CI], −2.5 to 3.7). The difference in proportions of CDAD events in the ICU between groups was -0.3% (1.4% vs 1.7%; 95% CI, −0.8 to 0.9). The difference in proportions of AAD events between groups was 0.5% (59.6% vs 59.1%; 95% CI, −3.2 to 4.2). The difference in proportions of hospital mortality events between groups was -1.1% (27.5% vs 28.6%; 95% CI, −4.5 to 2.3). There were no important differences in effects between groups for any of the primary or secondary outcomes.28

Table 2 Incremental cost-effectiveness ratios or dominance for primary outcome of VAP and secondary outcomes of CDAD, AAD, mortality (mean cost and effects, per patient) in E-PROSPECT
Full size table

Healthcare resource use and costs

Resource use and mean unit cost are outlined in ESM eTable 3. Healthcare resource use varied in key areas between probiotics and usual care groups: personnel (ICU physician, ICU nurse, pharmacist, respiratory therapist, physiotherapist, dietician, social worker, and unit clerk) and ICU hoteling (22,824 vs 21,103 days; 17.3 vs 15.8 days/patient; mean difference, 1.5 days/patient; 95% CI, −0.2 to 3.1; P = 0.08), invasive ventilator days (13,853 vs 13,496 days; 10.5 vs 10.1 days/patient; mean difference, 0.38 days/patient; 95% CI, −0.5 to 1.2; P = 0.37).

The mean (SD) cost per patient was USD 66,914 (91,098) for the probiotic group (median [IQR], USD 42,947 [22,239 to 76,205]) and compared with USD 62,701 (78,676) for usual care (median [IQR], USD 41,102 [23,170 to 75,140]). The incremental cost per patient between groups was USD 4,213 (95% CI, −2,269 to 10,708; P = 0.20) (Table 2).

Primary cost-effectiveness analysis with subgroup and sensitivity analyses

The E-PROSPECT CEA is presented in ESM eAppendix 3. For the primary, base-case analysis, probiotics were dominated (more expensive, less or similar in effectiveness) by the usual care strategy for VAP events (Table 2) on the cost-effectiveness plane (Fig. 1). Therefore, an ICER was not calculated for VAP.

Fig. 1
figure 1

Incremental cost-effectiveness plane for VAP (probiotics vs usual care): point-estimate (red) and nonparametric bootstrapping simulations (blue). Point-estimate indicates that overall probiotics were more expensive and more harmful compared with usual care (probiotics are dominated by usual care). CI = confidence interval; ICER = incremental cost-effectiveness ratio; ICU = intensive care unit; PROSPECT = Probiotics to Prevent Severe Pneumonia and Endotracheal Colonization Trial; USD = United States dollar; VAP = ventilator-associated pneumonia

Full size image

All ICERs and cost-effectiveness plots for CDAD, AAD, and mortality for secondary outcomes are presented in Table 2 and ESM eFigs 1–3, respectively. The ICER for CDAD was USD 1,473,400 per CDAD event (95% CI, undefined). Probiotics for AAD were dominated by usual care. For mortality, the ICER was USD 396,764 per death (95% CI, undefined).

The CEACs are presented in Fig. 2 for VAP. Probiotics were again dominated by usual care for VAP. Across a WTP threshold of USD 0 to USD 50,000 per VAP event, probiotics were only cost effective in ~31% of simulations (Fig. 2). Other CEACs for CDAD, AAD, and mortality are shown in ESM eFigs 4–6. Usual care remained the most economically attractive strategy for all reasonable WTPs.

Fig. 2
figure 2

Cost-effectiveness acceptability curve for VAP (probiotics vs usual care) for varying WTP thresholds. Probiotics were only cost-effective compared with usual care in 31% of scenarios, which only increased to 39% at a willingness-to-pay threshold of 500,000 USD. CI = confidence interval; ICER = incremental cost-effectiveness ratio; ICU = intensive care unit; PROSPECT = Probiotics to Prevent Severe Pneumonia and Endotracheal Colonization Trial; USD = United States dollar; VAP = ventilator-associated pneumonia; WTP = willingness-to-pay

Full size image

Our prespecified subgroup analyses (age, diagnostic category, frailty status, antibiotics within two days, prevalent vs non-prevalent pneumonia, and jurisdiction) revealed no differences between subgroups for cost-effectiveness (ESM eTable 4).

In sensitivity analyses, the usual care strategy remained the most cost-effective strategy when hoteling costs varied from USD 2,000 to 4,000 per ICU day. When using a time horizon of 60 days, probiotics remained dominated by usual care (USD 58,404 vs USD 55,932; mean incremental cost difference, USD 2,471; 95% CI, −1,339 to 6,282; P = 0.21), although with lower mean incremental cost difference compared with the base case (ESM eFigs 7, 8). For Canadian jurisdictions only (ESM eFigs 9, 10), probiotics were still dominated by usual care (USD 64,176 vs USD 59,593; mean incremental cost difference, USD 4,583; 95% CI, −1,530 to 10,696; P = 0.14). These parameter variability and sensitivity analyses did not change the outcomes or overall conclusions for VAP.

Our tornado diagram indicates that probiotics were not a major cost driver in E-PROSPECT (Fig. 3), while ICU hoteling, ICU nursing, ward nursing, ward hoteling, and other personnel were the major cost drivers. In one-way sensitivity analysis, varying the cost of probiotics from USD 0.78 to USD 20 resulted in an incremental cost difference of only USD 440 (ESM eFig. 11).

Fig. 3
figure 3

Tornado Diagram of Cost Drivers in E-PROSPECT

Full size image

Discussion

We found that using the probiotics Lactobacillus rhamnosus GG in addition to usual care was more costly than usual care alone and associated with similar rates of VAP,28 implying the probiotics are not a cost-effective treatment strategy for mechanically ventilated critically ill adults (although uncertainty remains given the nonparametric bootstrap findings based on this single study). Subgroup and sensitivity analyses (including shortening to 60-day time horizon, adjustments in per ICU-day hoteling costs, and focusing on Canadian jurisdictions only) did not alter these conclusions.

Our findings from E-PROSPECT supplement the clinical findings from PROSPECT, which showed a lack of clinical benefit with probiotics.28 PROSPECT and E-PROSPECT results differ importantly from prior randomized controlled trials summarized in a systematic review and meta-analysis showing probiotic efficacy10 and a systematic review of health economic evaluations of probiotics showing cost-effectiveness for preventing healthcare-associated infections.42

Despite a lack of important differences in ICU hoteling length of stay (difference of only 0.65 days per patient) in PROSPECT, at the health economic population level, there were an overall additional 1,721 ICU days in the probiotics group compared with the usual care group. This additional time in ICU was the largest incremental cost-driver of comparative resource use in the health economic evaluation, despite no clinical difference seen in PROSPECT.

This highlights an important difference in reporting between health economic evaluations (focusing on cost estimation with means and SDs) vs clinical trial frequentist inferences (focusing on statistical significance with 95% CIs or P values). The E-PROSPECT findings exemplify how a nonsignificant clinical difference in length of stay may still have an important impact on incremental cost estimation in a health economic evaluation. Clinicians and health policymakers may still need to consider these important costs from a broad population perspective and budgetary standpoint. Every dollar spent for a nonbeneficial or not cost-effective intervention is an opportunity cost for other interventions in a finite system, with potential harms to other patients within budgetary constraints.43

Ventilator-associated pneumonia prevention bundles include multiple interventions despite low- to moderate-quality evidence (e.g., chlorhexidine oral decontamination),44,45 which has led clinicians to re-evaluate indiscriminate use of these interventions. Further studies exploring VAP prevention should rigorously evaluate both effectiveness and costs. Some guidelines do not recommend prescribing probiotics in selected populations,46 while other guidelines do not recommend for or against the routine use of probiotics in standard-of-care VAP prevention bundles.47,48,49,50,51,52 In light of the findings from PROSPECT and E-PROSPECT, we do not suggest the routine incorporation of probiotics into VAP prevention bundles.

There are several strengths of this study. The protocol was prospectively designed with collection of predetermined costs and effects, including preplanned subgroup and sensitivity analyses of both the trial and economic evaluation to minimize bias.29 Clinical effects and costs are based on patient-level data from a randomized trial (rather than model-based, hypothetical cohorts with inputs incorporated from multiple sources), increasing the internal validity for both costs and effects. Capturing jurisdictional costs and effects with their own distributions and variance allowed for a more precise estimate of between-group differences, which increases the generalizability of these findings. Finally, this economic analysis was funded by peer-reviewed sources and the funding agency took no role in the study design, conduct, analysis, interpretation, or decision to publish.

This study also has limitations. First, the relatively short time horizon (time to in-hospital discharge/death) may miss additional costs associated with downstream health consequences secondary to VAP (e.g., physiotherapy, rehabilitation, home oxygen, outpatient healthcare use, etc.). Second, patient-reported outcomes such as quality-of-life were not measured in the trial. Clinical outcomes might therefore not fully capture the impact of treatments in ICU on quality-of-life. Third, this health economic evaluation derived data from a randomized trial and our findings may not represent the same treatment effects and costs as in routine clinical practice.12 In addition, although PROSPECT and E-PROSPECT compared the probiotic Lactobacillus rhamnosus GG with placebo and usual care, effectiveness and cost-effectiveness analyses may differ with other probiotics, strains, and doses.

Conclusions

When considering a public healthcare payer’s perspective, administration of Lactobacillus rhamnosus GG for VAP prophylaxis in critically ill patients had similar effects but incurred higher costs than usual care did. This analysis suggests that incorporating probiotics into VAP prevention bundles will unlikely benefit patients or healthcare systems from a clinical or cost-effectiveness standpoint, and even nondiscriminate use of probiotics should be avoided.