Skip to main content
SpringerLink
Log in

Replicating Health Economic Models: Firm Foundations or a House of Cards?

  • Current Opinion
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

A Letter to the Editor to this article was published on 27 September 2017

Abstract

Health economic evaluation is a framework for the comparative analysis of the incremental health gains and costs associated with competing decision alternatives. The process of developing health economic models is usually complex, financially expensive and time-consuming. For these reasons, model development is sometimes based on previous model-based analyses; this endeavour is usually referred to as model replication. Such model replication activity may involve the comprehensive reproduction of an existing model or ‘borrowing’ all or part of a previously developed model structure. Generally speaking, the replication of an existing model may require substantially less effort than developing a new de novo model by bypassing, or undertaking in only a perfunctory manner, certain aspects of model development such as the development of a complete conceptual model and/or comprehensive literature searching for model parameters. A further motivation for model replication may be to draw on the credibility or prestige of previous analyses that have been published and/or used to inform decision making. The acceptability and appropriateness of replicating models depends on the decision-making context: there exists a trade-off between the ‘savings’ afforded by model replication and the potential ‘costs’ associated with reduced model credibility due to the omission of certain stages of model development. This paper provides an overview of the different levels of, and motivations for, replicating health economic models, and discusses the advantages, disadvantages and caveats associated with this type of modelling activity. Irrespective of whether replicated models should be considered appropriate or not, complete replicability is generally accepted as a desirable property of health economic models, as reflected in critical appraisal checklists and good practice guidelines. To this end, the feasibility of comprehensive model replication is explored empirically across a small number of recent case studies. Recommendations are put forward for improving reporting standards to enhance comprehensive model replicability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Briggs A, Claxton K, Sculpher M. Decision modelling for health economic evaluation. 1st ed. New York: Oxford University Press; 2006.

    Google Scholar 

  2. Popper K. The logic of scientific discovery. 1st ed. London: Routledge; 1992.

    Google Scholar 

  3. Weinstein M, O’Brien B, Hornberger J, Jackson J, Johannesson M, McCabe M, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR task force on good research practices—modeling studies. Value Health. 2003;6(1):9–17.

    Article  PubMed  Google Scholar 

  4. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D. Consolidated health economic evaluation reporting standards (CHEERS)—explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value Health. 2013;16(2):231–50.

    Article  PubMed  Google Scholar 

  5. Eddy D. Technology assessment: the role of mathematical modeling assessing medical technology. 1st ed. Washington, DC: National Academy Press; 1985.

    Google Scholar 

  6. Akehurst R, Anderson P, Brazier J, Brennan A, Briggs A, Buxton M, et al. Decision analytic modelling in the economic evaluation of health technologies. Pharmacoeconomics. 2000;17(5):443–4.

    Article  Google Scholar 

  7. Eddy D, Hollingworth W, Caro J, Tsevat J, McDonald K, Wong J. Model transparency and validation: a report of the ISPOR-SMDM modeling good research practices task force-7. Med Decis Marking. 2012;32(5):733–43.

    Article  Google Scholar 

  8. McManus E. How reproducible are economic decision-analytic models? A systematic review and replication study—study outline. 2016. Available at: http://www.clahrcprojects.co.uk/impact/publications/how-reproducible-are-economic-decision-analytic-models-systematic-review-and.

  9. Woods S, Rizzo J. Cost-effectiveness of antidepressant treatment reassessed. Br J Psychiatry. 1997;170:257–63.

    Article  CAS  PubMed  Google Scholar 

  10. Batty A, Hansen R, Bloudek L, Varon S, Hayward E, Pennington B, et al. The cost-effectiveness of onabotulinumtoxinA for the prophylaxis of headache in adults with chronic migraine in the UK. J Med Econ. 2013;16(7):877–87.

    Article  PubMed  Google Scholar 

  11. Smolen L, Klein T, Kelton K. Replication of a published Markov chronic migraine cost-effectiveness analysis model for purposes of early phase adaptation and expansion. Value Health. 2015;83:A19.

    Article  Google Scholar 

  12. Palmer S, Sculpher M, Philips Z, Robinson M, Ginnelly L, Bakhai A, et al. Management of non-ST-elevation acute coronary syndromes: how cost-effective are glycoprotein IIb/IIIA antagonists in the UK National Health Service? Int J Cardiol. 2005;100(2):229–40.

    Article  PubMed  Google Scholar 

  13. AstraZeneca Ltd. Ticagrelor for the treatment of acute coronary syndromes: manufacturer’s submission to the National Institute for Health and Care Excellence. 2011. Available at: https://www.nice.org.uk/guidance/TA236/documents/acute-coronary-syndromes-ticagrelor-astrazeneca4.

  14. The Medicines Company. Bivalirudin (Angiox®) for the treatment of ST-segment elevation myocardial infarction: manufacturer’s submission to the National Institute for Health and Care Excellence. 2011. Available at: https://www.nice.org.uk/guidance/TA230/documents/myocardial-infarction-persistent-stsegment-elevation-bivalirudin-the-medicines-company2.

  15. Chilcott J, McCabe C, Tappenden P, O’Hagan A, Cooper N, Abrams K, et al. Modelling the cost effectiveness of interferon beta and glatiramer acetate in the management of multiple sclerosis. Commentary: evaluating disease modifying treatments in multiple sclerosis. Br Med J. 2003;326(7388):522.

    Article  Google Scholar 

  16. Allen F, Montgomery S, Maruszczak M, Kusel J, Adlard N. Convergence yet continued complexity: a systematic review and critique of health economic models of relapsing-remitting multiple sclerosis in the United Kingdom. Value Health. 2015;18(6):925–38.

    Article  PubMed  Google Scholar 

  17. Biogen Idec Ltd. Natalizumab (Tysabri®) for the treatment of adults with highly active relapsing remitting multiple sclerosis: manufacturer’s submission to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA127/documents/multiple-sclerosis-natalizumab-manufacturer-submissions-biogen-idec-uk-and-elan-pharma-international-ltd-joint-development-agreement-confidential-information-removed2.

  18. Gani R, Giovannoni G, Bates D, Kemball B, Hughes S, Kerrigan J. Cost-effectiveness analyses of natalizumab (Tysabri) compared with other disease-modifying therapies for people with highly active relapsing-remitting multiple sclerosis in the UK. Pharmacoeconomics. 2008;26(7):617–27.

    Article  PubMed  Google Scholar 

  19. Genzyme. Teriflunomide for the treatment of relapsing remitting multiple sclerosis in adults: manufacturer’s submission to the National Institute for Health and Care Excellence. 2013. Available at: https://www.nice.org.uk/guidance/TA303/documents/multiple-sclerosis-relapsing-teriflunomide-evaluation-report4.

  20. Novartis Pharmaceuticals UK Ltd. Fingolimod for the treatment of relapsing remitting multiple sclerosis in adults: manufacturer’s submission to the National Institute for Health and Care Excellence. 2011. Available at: https://www.nice.org.uk/guidance/TA254/documents/manufacturer-submission-from-novartis2.

  21. Biogen Idec Ltd. Dimethyl fumarate for the treatment of adult patients with relapsing remitting multiple sclerosis: manufacturer’s submission to the National Institute for Health and Care Excellence. 2013. Available at: https://www.nice.org.uk/guidance/TA320/documents/multiple-sclerosis-relapsingremitting-dimethyl-fumarate-evaluation-report2.

  22. Woolacott N, Hawkins N, Mason A, Kainth A, Khadjesari Z, Bravo Vergel Y, et al. Efalizumab and etanercept for the treatment of psoriasis: assessment report submitted to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA103/documents/psoriasis-efalizumab-and-etanercept-assessment-report2.

  23. Schering-Plough Ltd. Remicade® in the treatment of psoriasis in England and Wales: manufacturer’s submission to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA134/documents/psoriasis-infliximab-scheringplough-ltd2.

  24. Abbott Laboratories Ltd. Adalimumab (Humira®) for the treatment of moderate to severe plaque psoriasis: manufacturer’s submission to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA146/documents/psoriasis-adalimumab-appraisal-consultation-manufacturer-submission-abbott-laboratories-ltd2.

  25. Janssen-Cilag Ltd. Ustekinumab (Stelara) for the treatment of moderate to severe plaque psoriasis in England and Wales: manufacturer’s submission to the National Institute for Health and Care Excellence. 2009. Available at: https://www.nice.org.uk/guidance/TA180/documents/psoriasis-ustekinumab-manufacturer-submission-janssencilag2.

  26. Eggington S, Tappenden P, Pandor A, Paisley S, Saunders M, Seymour M, et al. The clinical and cost-effectiveness of oxaliplatin and capecitabine for the adjuvant treatment of colon cancer: systematic review and economic evaluation. Br J Cancer. 2006;95(9):1195–201.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  27. Tappenden P, Jones R, Paisley S, Carroll C. The cost-effectiveness of bevacizumab in the first-line treatment of metastatic colorectal cancer in England and Wales. Eur J Cancer. 2007;43(17):2487–94.

    Article  CAS  PubMed  Google Scholar 

  28. Kearns B, Rafia R, Leaviss J, Preston L, Brazier J, Palmer S, et al. The cost-effectiveness of changes to the care pathway used to identify depression and provide treatment amongst people with diabetes in England: a model-based economic evaluation. BMC Health Serv Res. 2017;17(1):78.

    Article  PubMed Central  PubMed  Google Scholar 

  29. Tosh J, Kearns B, Brennan A, Parry G, Ricketts T, Saxon D, et al. Innovation in health economic modelling of service improvements for longer-term depression: demonstration in a local health community. BMC Health Serv Res. 2013;13:150.

    Article  PubMed Central  PubMed  Google Scholar 

  30. Clarke P, Gray A, Legood R, Briggs A, Holman R. The impact of diabetes-related complications on healthcare costs: results from the United Kingdom Prospective Diabetes Study (UKPDS Study No. 65). Diabetic Med. 2003;20(6):442–50.

    Article  CAS  PubMed  Google Scholar 

  31. Alva M, Gray A, Mihaylova B, Clarke P. The effect of diabetes complications on health-related quality of life: the importance of longitudinal data to address patient heterogeneity. Health Econ. 2014;23(4):487–500.

    Article  PubMed  Google Scholar 

  32. Hayes A, Leal J, Gray A, Holman R, Clarke P, et al. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56(9):1925–33.

    Article  CAS  PubMed  Google Scholar 

  33. National Institute for Health and Care Excellence. Bosutinib for previously treated chronic myeloid leukaemia. 2013. Available at: https://www.nice.org.uk/guidance/ta299.

  34. National Institute for Health and Care Excellence. Dasatinib, high-dose imatinib and nilotinib for the treatment of imatinib-resistant chronic myeloid leukaemia (CML) (part review of NICE technology appraisal guidance 70), and dasatinib and nilotinib for people with CML for whom treatment with imatinib has failed because of intolerance. 2012. Available at: https://www.nice.org.uk/guidance/ta299.

  35. Stevenson M, Archer R, Tosh J, Simpson E, Everson-Hock E, Stevens J, et al. Adalimumab, etanercept, infliximab, certolizumab pegol, golimumab, tocilizumab and abatacept for the treatment of rheumatoid arthritis not previously treated with disease-modifying antirheumatic drugs and after the failure of conventional disease-modifying antirheumatic drugs only: systematic review and economic evaluation. Health Technol Assess. 2016;20(35):1–610.

    Article  Google Scholar 

  36. National Institute for Health and Care Excellence. Adalimumab, etanercept, infliximab, rituximab and abatacept for the treatment of rheumatoid arthritis after the failure of a TNF inhibitor. 2013. Available at: https://www.nice.org.uk/guidance/ta195.

  37. National Institute for Health and Care Excellence. Adalimumab, etanercept and infliximab for the treatment of rheumatoid arthritis. 2013. Available at: https://www.nice.org.uk/guidance/ta130.

  38. Norton S, Sacker A, Dixey J, Done J, Williams P, Young A. Trajectories of functional limitation in early rheumatoid arthritis and their association with mortality. Rheumatology. 2013;52(11):2016–24.

    Article  PubMed  Google Scholar 

  39. Tappenden P, Chilcott J. Avoiding and identifying errors and other threats to the credibility of health economic models. Pharmacoeconomics. 2014;32(10):967–79.

    Article  PubMed  Google Scholar 

  40. Elvidge J, Bullement A, Hatswell A. Cost effectiveness of characterised chondrocyte implantation for treatment of cartilage defects of the knee in the UK. Pharmacoeconomics. 2016;34(11):1145–59.

    Article  PubMed  Google Scholar 

  41. Tappenden P, Ren S, Archer R, Harvey R, Martyn St-James M, Basarir H, et al. A model-based economic evaluation of biologic and non-biologic options for the treatment of adults with moderately-to-severely active ulcerative colitis after the failure of conventional therapy. Pharmacoeconomics. 2016;34(10):1023–38.

    Article  PubMed  Google Scholar 

  42. Davies M, Glah D, Chubb B, Konidaris G, McEwan P. Cost effectiveness of IDegLira vs. alternative basal insulin intensification therapies in patients with type 2 diabetes mellitus uncontrolled on basal insulin in a UK setting. Pharmacoeconomics. 2016;34(9):953–66.

    Article  PubMed  Google Scholar 

  43. Oddershede L, Walker S, Stöhr W, Dunn D, Arenas-Pinto A, Paton NI, et al. Cost effectiveness of protease inhibitor monotherapy versus standard triple therapy in the long-term management of HIV patients: analysis using evidence from the PIVOT trial. Pharmacoeconomics. 2016;34(8):795–804.

    Article  PubMed  Google Scholar 

  44. Versteegh M. Impact on the incremental cost-effectiveness ratio of using alternatives to EQ-5D in a Markov model for multiple sclerosis. Pharmacoeconomics. 2016;34(11):1133–44.

    Article  PubMed Central  PubMed  Google Scholar 

  45. McEwan P, Foos V, Palmer J, Lamotte M, Lloyd A, Grant D, et al. Validation of the IMS CORE diabetes model. Value Health. 2014;17(6):714–24.

    Article  PubMed  Google Scholar 

  46. Barber J, Thompson S. Multiple regression of cost data: use of generalised linear models. J Health Serv Res Policy. 2004;9(4):197–204.

    Article  PubMed  Google Scholar 

  47. Dunlop W, Mason N, Kenworthy J, Akehurst R. Benefits, challenges and potential strategies of open source health economic models. Pharmacoeconomics. 2017;35(1):125–8.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Health and Related Research, University of Sheffield, Sheffield, UK

    Inigo Bermejo & Paul Tappenden

  2. Division of Population Health, Health Services Research and Primary Care, University of Manchester, Manchester, UK

    Ji-Hee Youn

Authors
  1. Inigo Bermejo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Tappenden
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ji-Hee Youn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Inigo Bermejo.

Ethics declarations

IB, PT and JY did not receive any funding to support the drafting of this manuscript and have no conflicts of interest to declare. PT attempted to replicate four of the five models included in the pilot study, IB replicated the remaining model, and JY replicated one of the models. All authors were involved in writing the manuscript.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bermejo, I., Tappenden, P. & Youn, JH. Replicating Health Economic Models: Firm Foundations or a House of Cards?. PharmacoEconomics 35, 1113–1121 (2017). https://doi.org/10.1007/s40273-017-0553-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-017-0553-x

Search

Navigation