Interventions affecting mortality in critically ill and perioperative patients: A systematic review of contemporary trials

doi:10.1016/j.jcrc.2017.05.005

Journal of Critical Care

Volume 41, October 2017, Pages 107-111

https://doi.org/10.1016/j.jcrc.2017.05.005 Get rights and content

Highlights

•
No systematic exploration of confounders in RCTs reporting a significant effect on mortality has been performed.
•
47% of these trials were single centre, 31% were not analysed according to the intention to treat principle.
•
Blinded and/or multicentre design was associated with an increased number needed to treat/harm.
•
Major systematic biases exist and affect trial findings irrespective of the intervention being studied.

Abstract

Purpose

Confounders in randomized controlled trials (RCTs) reporting significant effects on mortality in critically ill patients using non-surgical techniques have not been systematically explored. We aimed to identify factors unrelated to the reported intervention that might have affected the findings and robustness of such trials.

Methods

We searched Pubmed/MEDLINE for all RCTs on any non-surgical interventions reporting an effect on unadjusted mortality in critically ill patients between 1/1/2000 and 1/12/2015. We assessed: the number needed to treat/harm (NNT or NNH), sample size, trial design (blinded/unblinded, single or multinational, single or multicenter (sRCT or mRCT)), intention to treat (ITT) analysis, and countries of origin.

Results

Almost half of RCTs were sRCTs. Median sample size was small, and 1/3 were not analyzed according to ITT principle. Lack of ITT analysis was associated with greater effect size (p = 0.0028). Harm was more likely in mRCTs (p = 0.002) and/or in blinded RCTs (p = 0.003). Blinded RCTs had double sample size (p = 0.007) and an increased NNT/NNH (p = 0.002). Finally, mRCTs had higher NNT (p = 0.005) and NNH (p = 0.02), and harm was only detected in studies from Western countries (p = 0.007).

Conclusions

These observations imply that major systematic biases exist and affect trial findings irrespective of the intervention being studied.

Introduction

Decreasing mortality in critically ill and postoperative patients is a public health goal. Thus, the primary outcome measure of multiple interventional trials [1]. Such patients are at high risk of death [2], [3], [4], [5], [6] and represent one of the main areas of health care expenditure in the western world [7]. Accordingly, any study reporting the effect of an intervention on mortality (either an increase or a decrease) has the potential to significantly change clinical practice worldwide, save thousands of lives, and reduce health-care costs [8].

According to Evidence Base Medicine (EBM) principles, randomized controlled trials (RCT) represent the most robust source of evidence to guide practice [9]. However, in the field of critical care and postoperative medicine, no assessment has been made of what confounding factors may affect the findings of such RCT beyond the intervention itself and whether any systematic biases exist, which may affect trial findings.

Accordingly, we systematically identified all contemporary RCTs of non-surgical intervention in critical care and postoperative medicine (all studies published since 2000) and reported in peer reviewed journals, which showed a statistically significant impact on mortality. The aim of our study was to identify whether there were confounding factors unrelated to the interventions, which might have systematically affected trial findings.

Section snippets

Systematic search and article selection

PubMed/MEDLINE were searched for all RCTs of any non-surgical intervention influencing unadjusted landmark mortality in critically ill and postoperative patients published between January 1st, 2000 and December 1st 2015 (see full PubMed search strategies in Supplementary Appendix). Additional articles were suggested by experts and obtained from a cross-check of references from primary papers.

Articles were then selected for further assessment if they met all the following criteria:

1)
Publication in

General study characteristics

The five search strategies initially returned > 60 thousand RCTs. After excluding overlaps, our search identified 56,554 potential manuscripts published between January 2000 and December 2015. Of these, 139RCTs met the inclusion criteria (Fig. 1). The references and the PubMed links for all 139 abstracts are available in Supplementary Table 1.

Of the 139 papers identified, 119 (85.6%) reported interventions that decreased mortality, and 20 (14.4%) reported interventions that increased mortality.

Key findings

We performed a systematic analysis of all contemporary trials reporting interventions that significantly affected mortality in critically ill or postoperative patients. We found that almost half of such trials were single center in design, that median sample size was small, and that one third were not analyzed according to the ITT principle, a strategy that inflated effect size by a third. Moreover, we found that mRCTs were more likely to show harm; that studies showing harm were twice as

Conclusions

We performed a systematic analysis of all contemporary trials reporting interventions that significantly affected mortality in critically ill or perioperative patients. We found that one-third of trials were not analyzed according to the ITT principle, a strategy that inflated effect size by a third; that harm was more likely to be found in mRCTs and/or blinded RCTs; that blinded or mRCTs increased the NNT/NNH by more than a third, and that harm was only detected in studies from western

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflicts of interest

None.

References (20)

D.K. Heyland et al.
Is it “worthwhile” to continue treating patients with a prolonged stay (> 14 days) in the ICU? An economic evaluation
Chest
(1998)
R. Bender
Calculating confidence intervals for the number needed to treat
Control Clin Trials
(2001)
G.A. Ospina-Tascón et al.
Multicenter, randomized, controlled trials evaluating mortality in intensive care: doomed to fail?
Crit Care Med
(2008)
B. Afessa et al.
Evaluating the performance of an institution using an intensive care unit benchmark
Mayo Clin Proc
(2005)
S. Christensen et al.
Comparison of Charlson comorbidity index with SAPS and APACHE scores
Clin Epidemiol
(2011)
J.M. Eddleston et al.
Survival, morbidity and quality of life after discharge from intensive care
Crit Care Med
(2000)
V.D. Mayr et al.
Causes of death and determinants of outcome in critically ill patients
Crit Care
(2006)
N.A. Halpern et al.
Critical care medicine in the United States, 2000–2005: an analysis of bed numbers, occupancy rates, ayer mix, and costs
Crit Care Med
(2010)
G. Landoni et al.
Mortality in multicenter critical care trials: an analysis of interventions with a significant effect
Crit Care Med
(2015)
Evidence-Based Medicine Working Group
Evidence-based medicine. A new approach to teaching the practice of medicine
JAMA
(1992)

There are more references available in the full text version of this article.

Cited by (13)

Four Decades of Randomized Clinical Trials Influencing Mortality in Critically Ill and Perioperative Patients
2022, Journal of Cardiothoracic and Vascular Anesthesia
Citation Excerpt :
These results are consistent with those previously obtained, in which the probability of finding harm in mRCTs was larger as compared with single-center RCTs. The fact that mRCTs also are published more in medical journals with a high IF than single-center clinical trials previously was reported.29 Several limitations existed in this cross-sectional analysis.
Randomized clinical trials (RCTs) represent the highest level of scientific evidence. The aim of this review was to map and summarize the main characteristics and publication trends of RCTs with a statistically significant effect on mortality in critically ill and perioperative patients.
A mapping review of RCTs published between January 1982 and January 2021. The authors searched PubMed/MEDLINE and EMBASE for RCTs reporting mortality data. A descriptive analysis was conducted, including general and methodologic information of all these RCTs with a statistically significant difference (p < 0.05) in mortality.
The authors identified 340 studies published in 115 journals from 42 countries. The most represented clinical areas were ventilatory support (n = 58, 17%) and hemodynamics (n = 56, 16%). A detrimental effect on survival was described in 47 (14%) RCTs. Denmark had the highest number of published trials per million inhabitants. A total of 40 (12%) RCTs were led by a female author. The intention-to-treat principle was applied overall in 60% of RCTs, though this percentage increased up to 75% when the study was published in journals with high impact factor.
In the largest contemporary RCTs database of interventions significantly influencing mortality, the authors found an increase in scientific production. United States, China, France, Italy, and the United Kingdom contributed with 172 (51%) RCTs over 40 years. Only 20% of the studies were multinational collaborations, though this percentage increased over time. The presence of women as first authors was 1 out of 8 RCTs.
Long-term outcome of perioperative low cardiac output syndrome in cardiac surgery: 1-year results of a multicenter randomized trial
2020, Journal of Critical Care
Citation Excerpt :
Notably, recent meta-analyses on levosimendan use found no beneficial effect on mortality with levosimendan when only low-risk-of-bias studies where analyzed [39,40]. The CHEETAH trial was a pragmatic, international mRCT, thereby designed to carry the highest degree of external validity and level of evidence [41,42]. At the same time, we cannot exclude that a more strict protocol for hemodynamic management and earlier administration of levosimendan (i.e. preoperative or intraoperative) would have yielded different results, although both the LICORN and LEVO-CTS demonstrated that also preoperative administration do not confer additional benefit.
Perioperative myocardial dysfunction occurs frequently in cardiac surgery, and is a risk factor for morbidity and mortality. Levosimendan has been suggested to reduce mortality of patients with perioperative myocardial dysfunction. However, long-term outcome data on its efficacy in cardiac surgery are lacking.
Cardiac surgery patients with perioperative myocardial dysfunction were randomized to levosimendan or placebo, in addition to standard inotropic care. One-year mortality data were collected.
We randomized 506 patients (248 to levosimendan 258 to placebo). At 1-year follow-up, 41 patients (16.5%) died in the levosimendan group, while 47 (18.3%) died in the placebo group (absolute risk difference −1.8; 95% CI −8.4 to 4.9; P = .60). Female sex, history of chronic obstructive pulmonary disease, previous myocardial infarction, serum creatinine, hematocrit, mean arterial pressure, and duration of cardiopulmonary bypass were independently associated with 1-year mortality.
Levosimendan administration does not improve 1-year survival in cardiac surgery patients with perioperative myocardial dysfunction. One-year mortality in these patients is 17%. Six predictive factors for long-term mortality were identified.
NCT00994825 (ClinicalTrials.gov).
Adrenergic Downregulation in Critical Care: Molecular Mechanisms and Therapeutic Evidence
2020, Journal of Cardiothoracic and Vascular Anesthesia
Citation Excerpt :
In addition, patients in the vasopressin cohort also showed a reduction in the incidence of postoperative atrial fibrillation.151 However, given the single-center nature of this trial, these promising results require confirmation in mRCTs.152–154 Overall, vasopressin use was observed to be safe and to effectively increase blood pressure in catecholamine-resistant shock without reducing mortality.
Catecholamines remain the mainstay of therapy for acute cardiovascular dysfunction. However, adrenergic receptors quickly undergo desensitization and downregulation after prolonged stimulation. Moreover, prolonged exposure to high circulating catecholamines levels is associated with several adverse effects on different organ systems.
Unfortunately, in critically ill patients, adrenergic downregulation translates into progressive reduction of cardiovascular response to exogenous catecholamine administration, leading to refractory shock. Accordingly, there has been a growing interest in recent years toward use of noncatecholaminergic inotropes and vasopressors.
Several studies investigating a wide variety of catecholamine-sparing strategies (eg, levosimendan, vasopressin, β-blockers, steroids, and use of mechanical circulatory support) have been published recently. Use of these agents was associated with improvement in hemodynamics and decreased catecholamine use but without a clear beneficial effect on major clinical outcomes. Accordingly, additional research is needed to define the optimal management of catecholamine-resistant shock.
The impact of non-blinding in critical care medicine trials
2018, Journal of Critical Care
Citation Excerpt :
We performed a systematic Pubmed search to identify all the RCTs published in peer-reviewed journals with a statistically significant effect on mortality in critically ill patients. Five search strategies [10] were used and are fully described in the supplemental material (see Appendix). In addition, we employed literature snowballing (i.e. scanning of references of retrieved studies and meta-analyses) and contacted experts in the field in order to identify further studies.
It is uncertain whether, in critical care medicine, non-blinded trials are associated with a bias toward a different effect size. The aim of our study was to assess if conducting non-blinded/open label studies leads to greater effect size than blinded studies, and to provide an estimate of the weight of this difference.
We systematically searched all papers published in peer-reviewed journals between January 2000 and December 2015, dealing with non surgical interventions in critically ill adults and reporting a statistically significant difference in mortality. We assessed the number needed to treat (NNT) of each trial to estimate the treatment effect size and we divided studies into non-blinded, single-blinded and double-blinded. We searched for correlation between the treatment effect size and blinding, and adjusted for the other trial variables.
We identified 119 critically ill randomized controlled trials. Of these, 69 studies were non-blinded and 50 were blinded. The median NNT in non-blinded studies was 5 [IQR 4–7] while it was 7 [IQR 5–7] in the blinded studies (p < .001).
The NNT for blinded studies is 40% higher than for unblinded studies. This should be taken into account when planning and interpreting the findings of non-blinded studies performed in critically ill settings.
Interventions affecting mortality in critically ill and perioperative patients: A systematic review of contemporary trials
2017, Journal of Critical Care
More quality for trials in critically ill patients
2017, Journal of Critical Care

View all citing articles on Scopus

View full text

Clinical PotpourriInterventions affecting mortality in critically ill and perioperative patients: A systematic review of contemporary trials

Highlights

Abstract

Purpose

Methods

Results

Conclusions

Introduction

Section snippets

Systematic search and article selection

General study characteristics

Key findings

Conclusions

Funding

Conflicts of interest

Chest

Control Clin Trials

Multicenter, randomized, controlled trials evaluating mortality in intensive care: doomed to fail?

Crit Care Med

Evaluating the performance of an institution using an intensive care unit benchmark

Mayo Clin Proc

Comparison of Charlson comorbidity index with SAPS and APACHE scores

Clin Epidemiol

Survival, morbidity and quality of life after discharge from intensive care

Crit Care Med

Causes of death and determinants of outcome in critically ill patients

Crit Care

Critical care medicine in the United States, 2000–2005: an analysis of bed numbers, occupancy rates, ayer mix, and costs

Crit Care Med

Mortality in multicenter critical care trials: an analysis of interventions with a significant effect

Crit Care Med

Evidence-based medicine. A new approach to teaching the practice of medicine

JAMA

Clinical Potpourri
Interventions affecting mortality in critically ill and perioperative patients: A systematic review of contemporary trials