Abstract
Transparency in decision modelling is an evolving concept. Recently, discussion has moved from reporting standards to open-source implementation of decision analytic models. However, in the debate about the supposed advantages and disadvantages of greater transparency, there is a lack of definition. The purpose of this article is not to present a case for or against transparency, but rather to provide a more nuanced understanding of what transparency means in the context of decision modelling and how it could be addressed. To this end, we review and summarise the discourse to date, drawing on our collective experience. We outline a taxonomy of the different manifestations of transparency, including reporting standards, reference models, collaboration, model registration, peer review and open-source modelling. Further, we map out the role and incentives for the various stakeholders, including industry, research organisations, publishers and decision makers. We outline the anticipated advantages and disadvantages of greater transparency with respect to each manifestation, as well as the perceived barriers and facilitators to greater transparency. These are considered with respect to the different stakeholders and with reference to issues including intellectual property, legality, standards, quality assurance, code integrity, health technology assessment processes, incentives, funding, software, access and deployment options, data protection and stakeholder engagement. For each manifestation of transparency, we discuss the ‘what’, ‘why’, ‘who’ and ‘how’. Specifically, their meaning, why the community might (or might not) wish to embrace them, whose engagement as stakeholders is required and how relevant objectives might be realised. We identify current initiatives aimed to improve transparency to exemplify efforts in current practice and for the future.
Similar content being viewed by others
References
Daniels N, Sabin J. Limits to health care: fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philosophy Public Affairs. 1997;26:303–50.
Daniels N, Sabin JE. Accountability for reasonableness. Setting limits fairly: can we learn to share medical resources?. Oxford: Oxford University Press; 2002.
Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB. Model transparency and validation: a report of the ISPOR-SMDM modeling good research practices task force–7. Med Decis Mak. 2012;32:733–43.
Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess. 2004;iii:iv (ix–xi, 1–158).
Sampson CJ, Wrightson T. Model registration: a call to action. PharmacoEcon Open. 2017;1:73–7.
Freemantle N, Mason J. Publication bias in clinical trials and economic analyses. PharmacoEconomics. 1997;12:10–6.
Sacristán JA, Bolaños E, Hernández JM, Soto J, Galende I. Publication bias in health economic studies. Pharmacoeconomics. 1997;11:289–92.
Mt Hood Database [Internet]. Mt hood diabetes challenge network. [cited 2019 Feb 18]. https://www.mthooddiabeteschallenge.com/registry.
CISNET Model Registry Home [Internet]. [cited 2019 Feb 18]. https://resources.cisnet.cancer.gov/registry.
Global Health CEA—Open-Source Model Clearinghouse [Internet]. [cited 2019 Feb 18]. http://healtheconomics.tuftsmedicalcenter.org/orchard/open-source-model-clearinghouse.
Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS) statement. PharmacoEconomics. 2013;31:361–7.
Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices—overview: a report of the ISPOR-SMDM modeling good research practices task force–1. Med Decis Mak. 2012;32:667–77.
Sanghera S, Frew E, Roberts T. Adapting the CHEERS statement for reporting cost-benefit analysis. PharmacoEconomics. 2015;33:533–4.
Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS)—explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value in Health. 2013;16:231–50.
Haji Ali Afzali H, Karnon J. Addressing the challenge for well informed and consistent reimbursement decisions: the case for reference models. PharmacoEconomics. 2011;29:823–5.
Afzali HHA, Karnon J, Merlin T. Improving the accuracy and comparability of model-based economic evaluations of health technologies for reimbursement decisions: a methodological framework for the development of reference models. Med Decis Mak. 2013;33:325–32.
Frederix GWJ, Haji Ali Afzali H, Dasbach EJ, Ward RL. Development and use of disease-specific (reference) models for economic evaluations of health technologies: an overview of key issues and potential solutions. PharmacoEconomics. 2015;33:777–81.
Lewsey JD, Lawson KD, Ford I, Fox K a. A, Ritchie LD, Tunstall-Pedoe H, et al. A cardiovascular disease policy model that predicts life expectancy taking into account socioeconomic deprivation. Heart. 2015;101:201–8.
Rydzak CE, Cotich KL, Sax PE, Hsu HE, Wang B, Losina E, et al. Assessing the performance of a computer-based policy model of HIV and AIDS. PLOS One. 2010;5:e12647.
Cancer Intervention and Surveillance Modeling Network (CISNET) [Internet]. [cited 2019 Feb 18]. https://cisnet.cancer.gov/.
Hennessy DA, Flanagan WM, Tanuseputro P, Bennett C, Tuna M, Kopec J, et al. The Population Health Model (POHEM): an overview of rationale, methods and applications. Popul Health Metr. 2015;13:24.
Sadatsafavi M, Ghanbarian S, Adibi A, Johnson K, FitzGerald JM, Flanagan W, et al. Development and validation of the evaluation platform in COPD (EPIC): a population-based outcomes model of COPD for Canada. Med Decis Mak. 2019;39:152–67.
Sullivan W, Hirst M, Beard S, Gladwell D, Fagnani F, López Bastida J, et al. Economic evaluation in chronic pain: a systematic review and de novo flexible economic model. Eur J Health Econ. 2016;17:755–70.
Incerti D, Curtis JR, Shafrin J, Lakdawalla DN, Jansen JP. A flexible open-source decision model for value assessment of biologic treatment for rheumatoid arthritis. Pharmacoeconomics. 2019;37(6):829–43.
Global Health CEA—About the Clearinghouse [Internet]. [cited 2019 Feb 14]. http://healtheconomics.tuftsmedicalcenter.org/orchard/about-the-clearinghouse.
Open-Source Value Platform [Internet]. Innovation and value initiative. [cited 2019 Feb 14]. https://www.thevalueinitiative.org/open-source-value-project/.
Priem J, Hemminger BM. Decoupling the scholarly journal. Front Comput Neurosci. 2012;6:19.
Arnold RJG, Ekins S. Time for cooperation in health economics among the modelling community. PharmacoEconomics. 2010;28:609–13.
American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27:2262–5.
Hayes AJ, Leal J, Gray AM, Holman RR, Clarke PM. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56:1925–33.
Palmer AJ, Si L, Tew M, Hua X, Willis MS, Asseburg C, et al. Computer modeling of diabetes and its transparency: a report on the eighth mount hood challenge. Value in Health. 2018;21:724–31.
Project HERCULES [Internet]. Duchenne UK. [cited 2019 Feb 14]. https://www.duchenneuk.org/project-hercules.
Hoogendoorn M, Feenstra TL, Asukai Y, Briggs AH, Hansen RN, Leidl R, et al. External validation of health economic decision models for chronic obstructive pulmonary disease (COPD): report of the third COPD modeling meeting. Value Health. 2017;20:397–403.
Mount Hood 4 Modeling Group. Computer modeling of diabetes and its complications: a report on the fourth mount hood challenge meeting. Diabetes Care. 2007;30:1638–46.
Palmer AJ. Computer modeling of diabetes and its complications: a report on the fifth mount hood challenge meeting. Value in Health. 2013;16:670–85.
About Cochrane | Cochrane Library [Internet]. [cited 2019 Feb 14]. https://www.cochranelibrary.com/about/about-cochrane.
Goeree R, Levin L. Building bridges between academic research and policy formulation: the PRUFE framework—an integral part of ontario’s evidence-based HTPA process. PharmacoEconomics. 2006;24:1143–56.
Drummond M, Sorenson C. Use of pharmacoeconomics in drug reimbursement in Australia, Canada and the UK: what can we learn from international experience? In: Arnold RJG, editor. Pharmacoeconomics: from theory to practice. Boca Raton: CRC Press; 2009. p. 175–96.
Cooper NJ, Sutton AJ, Ades AE, Paisley S, Jones DR, on behalf of the working group on the ‘use of evidence in economic decision models’. Use of evidence in economic decision models: practical issues and methodological challenges. Health Economics. 2007;16:1277–86.
Barton P. Development of the Birmingham rheumatoid arthritis model: past, present and future plans. Rheumatology (Oxford). 2011;50:iv32–8.
Localio AR, Goodman SN, Meibohm A, Cornell JE, Stack CB, Ross EA, et al. Statistical code to support the scientific story. Ann Internal Med. 2018;168:828.
Christensen G, Miguel E. Transparency, reproducibility, and the credibility of economics research. J Econ Lit. 2018;56:920–80.
Merlo G, Page K, Ratcliffe J, Halton K, Graves N. Bridging the gap: exploring the barriers to using economic evidence in healthcare decision making and strategies for improving uptake. Appl Health Econ Health Policy. 2015;13:303–9.
Vemer P, Corro Ramos I, van Voorn GAK, Al MJ, Feenstra TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. PharmacoEconomics. 2016;34:349–61.
Hua X, Lung TW-C, Palmer A, Si L, Herman WH, Clarke P. How consistent is the relationship between improved glucose control and modelled health outcomes for people with type 2 diabetes mellitus? A systematic review. PharmacoEconomics. 2017;35:319–29.
Goodacre S. Being economical with the truth: how to make your idea appear cost effective. Emerg Med J. 2002;19:301–4.
Dunlop WCN, Mason N, Kenworthy J, Akehurst RL. Benefits, challenges and potential strategies of open source health economic models. PharmacoEconomics. 2017;35:125–8.
Laine C, Goodman SN, Griswold ME, Sox HC. Reproducible research: moving toward research the public can really trust. Ann Internal Med. 2007;146:450.
Lampe K, Mäkelä M, Garrido MV, Anttila H, Autti-Rämö I, Hicks NJ, et al. The HTA core model: a novel method for producing and reporting health technology assessments. Int J Technol Assess Health Care. 2009;25:9–20.
EUnetHTA. HTA Core ModelVersion 3.0 [Internet]. 2016 Jan. Report No.: JA2 WP8. [cited 2019 Feb 14]. https://www.eunethta.eu/wp-content/uploads/2018/03/HTACoreModel3.0-1.pdf.
Academy of Managed Care Pharmacy. The AMCP Format for Formulary Submissions [Internet]. 2012. Report No.: Version 3.1. [cited 2019 Feb 14]. http://amcp.org/practice-resources/amcp-format-formulary-submisions.pdf.
Hatswell AJ, Chandler F. Sharing is caring: the case for company-level collaboration in pharmacoeconomic modelling. PharmacoEconomics. 2017;35:755–7.
Cole A, Towse A. Legal Barriers to the Better Use of Health Data to Deliver Pharmaceutical Innovation. OHE Consulting Report, London: Office of Health Economics; 2018. https://ideas.repec.org/p/ohe/conrep/002096.html.
R Core Team. R: a language and environment for statistical computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing; 2018 [cited 2019 Feb 14]. https://www.R-project.org/.
Python Software Foundation. Python Language Reference [Internet]. 2008. [cited 2019 Feb 14] http://www.python.org.
Frederix GWJ. Check your checklist: the danger of over- and underestimating the quality of economic evaluations. PharmacoEconomics Open. 2019;. https://doi.org/10.1007/s41669-019-0118-3.
Jalal H, Pechlivanoglou P, Krijkamp E, Alarid-Escudero F, Enns E, Hunink MGM. An overview of R in health decision sciences. Med Decis Mak. 2017;37:735–46.
Chang W, Cheng J, Allaire JJ, Xie Y, McPherson J. Shiny: web application framework for R [Internet]. 2018 [cited 2019 Feb 14]. https://CRAN.R-project.org/package=shiny.
Dabbish L, Stuart C, Tsay J, Herbsleb J. Social coding in github: transparency and collaboration in an open software repository. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work [Internet]. New York, NY, USA: ACM; 2012 [cited 2019 Feb 19]. p. 1277–1286. http://doi.acm.org/10.1145/2145204.2145396.
Bryan S, Mitton C, Donaldson C. Breaking the addiction to technology adoption. Health Econ. 2014;23:379–83.
Scotland G, Bryan S. Why do health economists promote technology adoption rather than the search for efficiency? A proposal for a change in our approach to economic evaluation in health care. Med Decis Mak. 2017;37:139–47.
Bertagnolli MM, Sartor O, Chabner BA, Rothenberg ML, Khozin S, Hugh-Jones C, et al. Advantages of a truly open-access data-sharing model. N Engl J Med. 2017;376:1178–81.
Sotos JG, Huyen Y, Borrelli A. Correspondence: data-sharing models. N Engl J Med. 2017;376:2305–6.
Ekins S, Arnold RJG. From machine learning in drug discovery to pharmacoeconomics. In: Arnold RJG, editor. Pharmacoeconomics: from theory to practice. 2nd ed. Boca Raton: CRC Press; 2019.
Arnold RJG, Ekins S. Ahead of our time: collaboration in modeling then and now. PharmacoEconomics. 2017;35:975–6.
Government of Canada CI of HR. Strategy for patient-oriented research [Internet]. 2018 [cited 2019 Feb 18]. http://www.cihr-irsc.gc.ca/e/41204.html.
PCORI [Internet]. [cited 2019 Feb 18]. https://www.pcori.org/.
INVOLVE | INVOLVE Supporting public involvement in NHS, public health and social care research [Internet]. [cited 2019 Feb 18]. https://www.invo.org.uk/.
van Voorn GAK, Vemer P, Hamerlijnck D, Ramos IC, Teunissen GJ, Al M, et al. The missing stakeholder group: why patients should be involved in health economic modelling. Appl Health Econ Health Policy. 2016;14:129–33.
Acknowledgements
SE kindly acknowledges his colleagues at Collaborations Pharmaceuticals.
Author information
Authors and Affiliations
Contributions
All authors were involved in the conception and planning of the manuscript, wrote sections, edited and commented on drafts and approved the submitted version. CS organised and led the write-up and contributed to the writing of all sections. RA wrote parts of Sects. 1–5. SB wrote parts of Sects. 4 and 5. PC wrote parts of Sects. 2. SE wrote parts of Sects. 2, 4 and 5. AH wrote parts of Sects. 2–5. NH wrote parts of Sects. 1, 3 and 4. SL wrote parts of Sects. 2–5. DM wrote parts of Sects. 1, 3 and 5. MS wrote parts of Sects. 2–5. WS wrote parts of Sects. 1–5. EW wrote parts of Sects. 1–4. TW wrote parts of Sects. 2–5.
Corresponding author
Ethics declarations
No specific funding was received to support the preparation of this manuscript. CS is an employee of the Office of Health Economics, a registered charity, research organisation and consultancy, which receives funding from a variety of sources including the Association of the British Pharmaceutical Industry and the National Institute for Health Research (NIHR). CS has received honoraria from the NIHR in relation to peer review activities. CS has received travel support from Duke University for attendance at a meeting related to the content of this manuscript. SB receives salary support from the University of British Columbia, Vancouver Coastal Health and the British Columbia Academic Health Science Network. He previously chaired CADTH’s Health Technology Expert Review Panel and provides consultancy advice to CADTH, for which he received honoraria and travel support. All of his decision model project work is supported through public sources, primarily the Canadian Institutes for Health Research (CIHR) and the BC Ministry of Health. PC has received payment for workshops run alongside Mt Hood conferences. SE is CEO and Owner of Collaborations Pharmaceuticals, Inc. SE has received a grant from the National Institutes for Health to build machine learning models and software. DM is employed by the University of Calgary, with a salary funded by Arthur J.E. Child Chair in Rheumatology and Canada Research Chair in Health Systems and Services Research. DM’s research is funded through multiple funding organisations including the CIHR, Arthritis Society, CRA and AAC in peer-reviewed funding competitions, none of which relate to this work. DM has received reimbursement for travel from Illumina and Janssen for attendance and presentation at scientific meetings not related to this work. RA, AH, NH, SL, MS, WS, and EW declare that they have no conflicts of interest. TW is a Co-Editor of PharmacoEconomics, but was not involved in the peer review nor any editorial decisions for this paper.
Funding
SE acknowledges funding from NIH/ NIGMS R43GM122196, R44GM122196-02A1.
Rights and permissions
About this article
Cite this article
Sampson, C.J., Arnold, R., Bryan, S. et al. Transparency in Decision Modelling: What, Why, Who and How?. PharmacoEconomics 37, 1355–1369 (2019). https://doi.org/10.1007/s40273-019-00819-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40273-019-00819-z