Elsevier

Journal of Cardiac Failure

Volume 23, Issue 9, September 2017, Pages 680-687
Journal of Cardiac Failure

Methodology
Risk Prediction Models for Incident Heart Failure: A Systematic Review of Methodology and Model Performance

https://doi.org/10.1016/j.cardfail.2017.03.005Get rights and content

Highlights

  • Many of the published heart failure (HF) risk prediction models have sufficient discriminative ability (C-statistics > 0.70), but few have been externally validated.

  • Methods not recommended for the conduct and reporting of risk prediction modeling were frequently used in existing HF models, including variable selection based on univariate significance, exclusion of missing values, and categorization of continuous predictors.

  • In light of this, application of existing HF models should be undertaken with careful consideration of the population under study and comparability with the cohort used to derive the model, as well as acknowledgment of any limitations conferred by methodology underpinning the model.

Abstract

Background

Numerous models predicting the risk of incident heart failure (HF) have been developed; however, evidence of their methodological rigor and reporting remains unclear. This study critically appraises the methods underpinning incident HF risk prediction models.

Methods and Results

EMBASE and PubMed were searched for articles published between 1990 and June 2016 that reported at least 1 multivariable model for prediction of HF. Model development information, including study design, variable coding, missing data, and predictor selection, was extracted. Nineteen studies reporting 40 risk prediction models were included. Existing models have acceptable discriminative ability (C-statistics > 0.70), although only 6 models were externally validated. Candidate variable selection was based on statistical significance from a univariate screening in 11 models, whereas it was unclear in 12 models. Continuous predictors were retained in 16 models, whereas it was unclear how continuous variables were handled in 16 models. Missing values were excluded in 19 of 23 models that reported missing data, and the number of events per variable was < 10 in 13 models. Only 2 models presented recommended regression equations. There was significant heterogeneity in discriminative ability of models with respect to age (P < .001) and sample size (P = .007).

Conclusions

There is an abundance of HF risk prediction models that had sufficient discriminative ability, although few are externally validated. Methods not recommended for the conduct and reporting of risk prediction modeling were frequently used, and resulting algorithms should be applied with caution.

Introduction

Heart failure (HF) remains a major cause of morbidity, mortality, and economic burden worldwide.1, 2, 3 The clinical syndrome of HF represents the final stage of the continuum of cardiovascular diseases and, although pharmacological treatments could slow the progression of HF, they are not curative.4, 5

Several studies have shown that pharmacologic and lifestyle interventions in populations at risk of HF can significantly reduce the risk of new-onset HF.4, 6, 7, 8 Targeting at-risk groups with preventive therapy requires stratification of asymptomatic individuals according to their risk of developing HF.4 Prognostic models, which mathematically combine multiple predictors to estimate future risk of an outcome, have been increasingly recognized for their potential to aid health care providers to make more accurate predictions of outcome of interest.6, 8, 9

The number of risk prediction models predicting specific disease conditions has increased in recent years.10, 11 Previous studies have reported frequent use of inappropriate methods for model development and reporting among models developed to predict cardiovascular diseases, cancer, or diabetes.10, 11, 12 Inappropriate methodology can compromise the accuracy and reliability of the risk estimates, hence reducing their applicability to clinical practice.12, 13

Although an abundance of HF models has been reported, comprehensive evaluation of the methods of model development and presentation has not been described.14, 15 In light of the growing number of risk prediction models, evaluation of the methods underpinning model development, validation, and performance are important to guide decisions regarding use and choice of prognostic models for risk predictions.

In this study, we sought to review the model development and reporting methodology of multivariable risk prediction models for incident HF. We aimed to identify the most common risk predictors included in models as well as examine study characteristics associated with model performance.

Section snippets

Methods

This review was undertaken according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (Supplementary Table S1).

Results

A total of 2327 abstracts were selected for in-depth review, of which 2284 were excluded based on titles and abstracts because of duplication or not meeting the inclusion criteria. Forty-three full-text articles were reviewed, of which 24 were excluded. This resulted in 19 studies that reported 40 different HF prediction models for the review (Fig. 1). At least 2 different models were reported in 11 of the 19 studies.

Discussion

This review demonstrated that there is an abundance of HF risk prediction models that had sufficiently high discriminative ability, although only a few have been externally validated. In this study, we found widespread use of methods that are not recommended for the conduct and reporting of risk prediction models. Despite the heterogeneity in the methodology of the studies, some predictors were consistently included in the models. Participant age (P < .001), and sample size (P = .007) in the

Conclusions

In conclusion, there is an abundance of HF risk prediction models displaying sufficient discriminative ability, although many have methodological limitations and are not externally validated. There is a substantial body of literature providing methodological guidance on how to conduct, validate, and report prediction models that should be the basis for future HF prediction research.13, 38, 47, 49 While addressing the methodological limitations, future research in the area should also focus on

Disclosure

Dr Reid is supported by a Research Fellowship from the National Health and Medical Research Council of Australia.

References (49)

  • V. Lazzarini et al.

    Heart failure in elderly patients: distinctive features and unresolved issues

    Eur J Heart Fail

    (2013)
  • G. Tocci et al.

    Development of heart failure in recent hypertension trials

    J Hypertens

    (2008)
  • G.S. Bleumink et al.

    Quantifying the heart failure epidemic: prevalence, incidence rate, lifetime risk and prognosis of heart failure The Rotterdam Study

    Eur Heart J

    (2004)
  • P. Ponikowski et al.

    Heart failure: preventing disease and death worldwide

    ESC Heart Fail

    (2014)
  • K.D. Aaronson et al.

    Heart failure prognostic models: why bother?

    Circ Heart Fail

    (2012)
  • The Heart Outcomes Prevention Evaluation Study Investigators

    Effects of an angiotensin-converting–enzyme inhibitor, ramipril, on cardiovascular events in high-risk patients

    N Engl J Med

    (2000)
  • J. Butler

    Primary prevention of heart failure

    ISRN Cardiol

    (2012)
  • A. Vickers

    Prediction models in cancer care

    CA Cancer J Clin

    (2011)
  • J.A. Damen et al.

    Prediction models for cardiovascular disease risk in the general population: systematic review

    BMJ

    (2016)
  • G. Collins et al.

    Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting

    BMC Med

    (2011)
  • T. Anothaisintawee et al.

    Risk prediction models of breast cancer: a systematic review of model performances

    Breast Cancer Res Treat

    (2011)
  • E.W. Steyerberg et al.

    Towards better clinical prediction models: seven steps for development and an ABCD for validation

    Eur Heart J

    (2014)
  • J.B. Echouffo-Tcheugui et al.

    Population risk prediction models for incident heart failure: a systematic review

    Circ Heart Fail

    (2015)
  • J.H. McDonald

    Handbook of biological statistics

    (2009)
  • Cited by (0)

    Funding: This work was supported by a National Health and Medical Research Council of Australia (NHMRC) Program Grant (1092642) awarded to C.M.R. The funding bodies had no role in the design and conduct of the study, and interpretation of the data, or the preparation, review, or approval of the manuscript.

    View full text