Abstract
Rigorous evidence about the broad range of harms that might be experienced by a patient in the course of testing and treatment is sparse. We aimed to generate recommendations for how researchers might more comprehensively evaluate potential harms of healthcare interventions, to allow clinicians and patients to better include this evidence in clinical decision-making. We propose seven domains of harms of tests and treatments that are relevant to patients: (1) physical impairment, (2) psychological distress, (3) social disruption, (4) disruption in connection to healthcare, (5) labeling, (6) financial impact, and (7) treatment burden. These domains will include a range of severity of harms and variation in timing after testing or treatment, attributable to the service itself or a resulting care cascade. Although some new measures may be needed, diverse data and tools are available to allow the assessment of harms comprehensively across these domains. We encourage researchers to evaluate harms in sub-populations, since the harms experienced may differ importantly by demographics, social determinants, presence of comorbid illness, psychological state, and other characteristics. Regulators, funders, and editors might require either assessment or reporting of harms in each domain or require justification for inclusion and exclusion of different domains.
INTRODUCTION
When choosing a test or treatment, clinicians and patients consider its potential benefits and harms.1 Ideally, these are known from studies in similar patients in comparable care settings with sufficient follow-up. The clinician assesses this evidence, incorporates patient values and preferences, and decides with the patient about proceeding with testing or treatment.2 This process of patient-centered evidence-based care relies on evidence availability, preferably at the point of care, to inform decisions.
While evidence about an intervention’s efficacy or effectiveness is frequently available, rigorous evidence about the broad range of harms that a patient might experience is scant. Fewer than half of randomized trial reports include all captured harm data3 and captured data may be limited. Further, data from trials are often insufficient as evidence of rarer harms often emerge over time.4 This one-sided evidence impedes fully informed, shared decision-making. In this paper, we explore the problem of the limited scope of considered harms in assessments of tests and treatments. Our goal was to generate recommendations for how researchers might more comprehensively evaluate potential harms of healthcare interventions.
CURRENT UNDERSTANDING OF HARMS OF HEALTH SERVICES
While patient safety is addressed by many groups including Departments of Health and the Joint Commission, the assessment of harms of drugs and devices is generally the domain of medical product regulators. The US Food and Drug Administration (FDA) refers to harms of interventions as “risks,” which are product-associated adverse events and unfavorable effects. The FDA also recognizes that treatment burden from an intervention can impact patients’ health, functioning, or well-being.5 Similarly, health technology assessment principles adopted by many countries include safety as a prime consideration for supporting coverage, alongside effectiveness, economic impact, and ethical and social considerations.6 However, current framing of safety issues is insufficient to capture the full breadth of potential harms.
The harms that should be considered when designing research have not been comprehensively described. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines include discussion about appropriate methods for reporting both benefits and harms, but does not specifically categorize harms,7 noting that harms information is under-reported in systematic reviews, even when available in the original studies.8 The US Agency for Healthcare Research and Quality (AHRQ) has described best practices for collecting harms in systematic reviews, including comparative effectiveness reviews,9, 10 emphasizing the importance of identifying, selecting and prioritizing harms based on severity and frequency, and focusing on harms that are important to patients. While the AHRQ recommendations allude to non-physical harms, such as overdiagnosis and labeling, its framework focuses primarily on physical effects despite increasing emphasis on non-physical harms within the patient safety movement.11 Academic groups have created frameworks focusing on harms of screening, a taxonomy of harms from diagnostic testing, and a conceptual model of documented harms of overuse;12,13,14 we sought to build on this work.
In August 2019, 32 diverse experts gathered for a one-day meeting to discuss harms specifically related to healthcare overuse. During the meeting, a subgroup of health services researchers with experience working to reduce low-value care discussed the broader topic of harms of interventions; the authors continued working together afterward to develop recommendations. We proceeded with iterative literature review, conducted follow-up virtual meetings, and obtained input from four patient representatives on the broader topic of harms of interventions. We compiled information and developed this paper based on our findings, with disagreements resolved through discussion. Here we propose categories of harms to consider when evaluating any healthcare service and provide examples of data sources and measurement tools to enable evidence generation about harms in each domain.
DOMAINS OF HARMS
We recognize seven domains of potential harms to a patient receiving a test or treatment. We include harms that may be experienced beyond the immediate period of receipt of the service. These are (1) physical impairment; (2) psychological distress; (3) social disruption, defined as interference with relationships or altered social identity or status; (4) disruption in connection to healthcare; (5) labeling—the impact of being assigned a diagnosis; (6) financial impact; and (7) treatment burden—the workload from managing healthcare conditions (Table 1).
These domains should be considered by clinicians reviewing evidence about harms and by researchers designing studies and establishing clinical registries to evaluate the impact of a test or treatment on patients. The intensity of harms should be measured across domains according to their frequency, timing, duration, and severity. Within each domain, intensity may range from largely inconsequential (bruising from phlebotomy or small prescription copay) to devastating (a life-threatening complication or bankruptcy) and harms may be short-lived or long-lasting. Researchers should recognize that harms may arise as a direct consequence of the intervention or from the cascade of care following the original service.15 Clinical cascades can be critically important. For example, computed tomography (CT) of the lung with intravenous contrast might result in physical harms like an allergic reaction to the contrast agent. That test might also incidentally reveal a nodule that is later biopsied as part of a cascade of care, resulting in a pneumothorax (an additional physical harm), social disruption, and psychological distress. This variability in the relationship of harms to the culprit intervention, in terms of both timing and directness, suggests the importance of deliberate specification of the timing of measurement of harms and the challenges of appropriately attributing harms to the original healthcare service. The question of the appropriateness of the culprit intervention adds further nuance to the discussion of harms.
SOURCES OF DATA FOR UNDERSTANDING HARMS ACROSS DOMAINS
Researchers evaluating tests and treatments have choices regarding data collection for detecting and quantifying harms in these seven domains. While all domains should be considered during study design, researchers may prioritize some domains over others in a single study. Data sources for generating evidence about harms of healthcare services can be primary (collected actively by the investigator) or secondary (existing data leveraged for research, including registries); both may be needed to capture the full range of potential harms (Table 2). Evidence captured during usual care, possibly within a pragmatic trial, may be the best source of data for patients receiving the intervention outside of a tightly controlled efficacy trial or cohort study. This type of evidence may be critical, as harms experienced in usual care may differ substantially from those experienced by patients enrolled in studies. Patients outside of studies are often more diverse and medically complex, the population of clinicians providing care is broader, there is greater likelihood of treatment interactions, and monitoring for early predictors of harm may be less complete. Even for interventions evaluated in clinical trials, in which many harms are directly measured, the use of registries or other longitudinal data collection may be necessary for understanding the delayed effects of interventions or those resulting from care cascades. Understanding these late-occurring harms is critical.
The seven harm domains may vary in importance at different points along the translational pathway from phase I drug studies to evaluations of implementation efforts. For example, financial and labeling harms are typically unimportant or unmeasurable in early-phase clinical trials, but may be essential for understanding harmful effects of interventions used in practice. Different research designs may be needed to adequately capture the harms of an intervention as compared with its benefits; appropriate designs should be informed by the availability of outcomes in the data.
MEASUREMENT TOOLS FOR PRIMARY DATA COLLECTION
Many measurement tools are available for primary data collection about the harms of interventions. Learning directly from patients is essential. Recent emphasis on patient-reported outcomes has led to international efforts to define important disease outcomes and validate measurement tools. Examples include the Patient-Reported Outcomes Measurement Information System (PROMIS) from the US National Institutes of Health16 and the Patient-Reported Indicators Surveys (PaRIS), developed by the Organisation for Economic Co-operation and Development (OECD) for the collection of patient-reported health indicators.17 Other potentially valuable measures are those from the International Consortium for Health Outcomes Measurement (ICHOM), which has created tools to measure standard sets of outcomes associated with particular diseases.18 Many of these tools capture outcomes in several, although not all, of the seven harm domains. For many outcomes in these proposed domains, there are disease-specific measurement tools and disease-agnostic tools that are widely applicable (Supplemental Table). Qualitative approaches may also be helpful for exploring outcomes in some domains, particularly if no validated tools exist.
HETEROGENEITY CONSIDERATIONS AND REPORTING
Just as there is heterogeneity in treatment benefit among patient subgroups, there is likely to be heterogeneity in experiences of harms. Researchers should anticipate the need to evaluate harms in sub-populations, since they may differ importantly by age, race, sex, social determinants, preferred language, comorbid illness, medication usage, psychological state, or other characteristics. Further, harms reporting and quantification should parallel the reporting of benefits, generally using both absolute (e.g., absolute risk increase, number-needed-to-harm) and relative numbers (e.g., relative risk, hazard ratio). In addition, harms are sometimes best understood as composite outcomes (e.g., the number of patients with any serious adverse event) and must be compared with those from other treatment options.19 Notably, a more robust literature on harms may uncover important biases that will require novel criteria to quantify.8
STAKEHOLDER INVOLVEMENT
Patients must be central to prioritizing harms outcomes across domains and, with other stakeholders, to fine-tuning the domain names for optimal acceptance. Informed decision-making about health services requires an understanding of how patients value potential benefits and harms; research design should incorporate that understanding. Better understanding of harms across domains benefits multiple stakeholders. Knowledge of the breadth of potential harms will enable policy makers to make better, fully informed decisions about priority setting, funding, and availability of services. For example, harms information might inform decisions to restrict some services to centers of excellence, to sub-specialists who can deliver the intervention more safely, or to select patient sub-populations. At the organizational level, awareness of harms across domains enables more complete understanding of patient safety through measurement of specific harms-related outcomes. Finally, at the level of the clinical encounter, knowledge of harms enables clinicians to appropriately counsel patients about benefits and harms to optimize and individualize care.
APPLICATIONS OF THE SEVEN DOMAINS OF HARMS
Implementing and operationalizing these recommendations is possible through several approaches. First, researchers should consider the broad range of harms when designing studies, including early clinical trials, comparative effectiveness studies, and systematic reviews, and when establishing registries. Attention to the range of domains could be encouraged through several mechanisms. Medical journals might require investigators publishing primary research to submit a checklist indicating which domains, and which outcomes within those domains, were included in the study. The CONSORT Harms Extension (for clinical trials) and the PRISMA Harms Checklist (for systematic reviews) already recommend best reporting practices, although without describing which types of harms should be included.7, 20 Harms domains could be incorporated into such checklists or applied similarly to the requirement of some journals for transparency about patient partners.21 Being asked about the scope of considered harms may prompt investigators to consider the seven domains when defining outcomes of interest. Regulators and funders of all types can play a role by requiring either reporting of harms in each domain or a justification for exclusion of domains. These groups can provide explicit a priori guidance for researchers designing research or registries.
The domains have several applications outside of research. Implementation experts could use them to design data collection instruments that are attentive to late-occurring harms and inform efforts to design interventions that minimize harms. Patient support organizations and funders could leverage awareness of these domains to encourage patients to report harms, especially those related to drugs, and target limited resources. Health professionals reading the literature should consider the domains when evaluating the evidence about the breadth of harms across domains. Perhaps most importantly, clinicians and patients making decisions about health services should be mindful of all domains of harms. Patient-physician conversations that acknowledge the wide range of harms, and the wide range of their severity within domains, will normalize a broad view of the impact of health services and ultimately allow for more fully informed decision-making.
CONCLUSIONS
Physicians are biased toward action and tend to focus on physical issues, and overestimate benefits and underestimate harms of health services.22, 23 Improving the measurement and reporting of the full range of patient harms from tests and treatments is vital to counter these biases and enable more informed decision-making and more patient-centered care.
References
Morgan DJ, Scherer LD, Korenstein D. Improving physician communication about treatment decisions: Reconsideration of “Risks vs. Benefits”. JAMA 2020 Sept 8; 324(10): 937-8.
Puhan MA, Singh S, Weiss CO, et al. A framework for organizing and selecting quantitative approaches for benefit-harm assessment. BMC Med Res Methodol 2012;12:173. doi: https://doi.org/10.1186/1471-2288-12-173 [=
Pitrou I, Boutron I, Ahmad N, et al. Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med 2009;169(19):1756-61. doi: https://doi.org/10.1001/archinternmed.2009.306 =
Downing NS, Shah ND, Aminawung JA, et al. Postmarket Safety Events Among Novel Therapeutics Approved by the US Food and Drug Administration Between 2001 and 2010. JAMA 2017;317(18):1854-63. doi: https://doi.org/10.1001/jama.2017.5150
United States Food and Drug Adminstration. Patient-Focused Drug Development Glossary, 2018.
Mytton OT, Velazquez A, Banken R, et al. Introducing new technology safely. Qual Saf Health Care 2010;19 Suppl 2:i9-14. doi: https://doi.org/10.1136/qshc.2009.038554 [published Online First: 2010/08/20]
Ioannidis JP, Evans SJ, Gotzsche PC, et al. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med 2004;141(10):781-8. doi: https://doi.org/10.7326/0003-4819-141-10-200411160-00009
Papanikolaou PN, Ioannidis JP. Availability of large-scale evidence on specific harms from systematic reviews of randomized trials. Am J Med 2004;117(8):582-9. doi: https://doi.org/10.1016/j.amjmed.2004.04.026
Chou R, Aronson N, Atkins D, et al. AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. J Clin Epidemiol 2010;63(5):502-12. doi: https://doi.org/10.1016/j.jclinepi.2008.06.007
Chou R, Baker WL, Bañez L, Iyer S, Myers ER, Newberry S, Pincock L, Robinson KA, Sardenga L, Sathe N, Springs S, Wilt TJ. Prioritization and Selection of Harms for Inclusion in Systematic Reviews. Methods Guide for Comparative Effectiveness Reviews. (Prepared by the Scientific Resource Center under Contract No. 290-2012-0004-C). AHRQ Publication No. AHRQ Pub No. 17(18)-EHC-034-EF. Rockville, MD: Agency for Healthcare Research and Quality; February 2018. www.effectivehealthcare.ahrq.gov/reports/final.cfmhttps://doi.org/10.23970/AHRQEPCMETHGUIDE1.
Cooper J, Williams H, Hibbert P et al. Classification of patient-safety incidents in primary care. Bull World Health Org; 2018; 96(7):498-505.
Harris RP, Sheridan SL, Lewis CL, et al. The Harms of Screening. A Proposed Taxonomy and Application to Lung Cancer Screening. JAMA Intern Med 2014; 174(2):281-5.
Korenstein D, Chimonas S, Barrow B, et al. Development of a Conceptual Map of Negative Consequences for Patients of Overuse of Medical Tests and Treatments. JAMA Intern Med 2018;178(10):1401-07. doi: https://doi.org/10.1001/jamainternmed.2018.3573
Schiff GD, Martin SA, Eidelman DH, et al. Ten Principles for More Conservative, Care-Full Diagnosis. Annals Intern Med 2018; 169(9): 643-5.
Deyo RA. Cascade effects of medical technology. Annu Rev Public Health 2002;23:23-44. doi: https://doi.org/10.1146/annurev.publhealth.23.092101.134534
NIH. Patient-Reported Outcomes Measurement Information System, 2004. Available at: https://www.nia.nih.gov/research/resource/patient-reported-outcomes-measurement-information-system-promis. Accessed 30 January, 2020.
Organisation for Economic Co-operation and Development. Available from: https://www.oecd.org/health/paris.htm. Accessed 27 January, 2020.
International Consortium for Health Outcomes Measurement. Available at: https://www.ichom.org/standard-sets/. Accessed 27 January, 2020.
Sedrakyan A, Shih C. Improving depiction of benefits and harms: analyses of studies of well-known therapeutics and review of high-impact medical journals. Med Care 2007;45(10 Supl 2):S23-8. doi: https://doi.org/10.1097/MLR.0b013e3180642f69
Zorzela L, Loke YK, Ioannidis JP, et al. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ (Clinical research ed) 2016;352:i157. doi: https://doi.org/10.1136/bmj.i157
Liabo K, Boddy K, Burchmore H, et al. Clarifying the roles of patients in research. BMJ (Clinical research ed) 2018;361:k1463. doi: https://doi.org/10.1136/bmj.k1463 [published Online First: 2018/04/11]
Hoffmann TC, Del Mar C. Clinicians' Expectations of the Benefits and Harms of Treatments, Screening, and Tests: A Systematic Review. JAMA Intern Med 2017;177(3):407-19. doi: https://doi.org/10.1001/jamainternmed.2016.8254
Scott IA, Soon J, Elshaug AG, et al. Countering cognitive biases in minimising low value care. Med J Austral 2017;206(9):407-11. doi: https://doi.org/10.5694/mja16.00999
World Health Organization. The Conceptual Framework for the International Classification for Patient Safety v.11 Technical Annex 2 - Glossary of Patient Safety Concepts and References, 2009.
Brodersen J, Hansson A, Johansson M, et al. Consequences of screening in abdominal aortic aneurysm: development and dimensionality of a questionnaire. J Patient Rep Outcomes 2017;2:37. doi: https://doi.org/10.1186/s41687-018-0066-1
DeFrank JT, Barclay C, Sheridan S, et al. The psychological harms of screening: the evidence we have versus the evidence we need. J Gen Intern Med 2015;30(2):242-8. doi: https://doi.org/10.1007/s11606-014-2996-5
Wu GX, Raz DJ, Brown L, et al. Psychological Burden Associated With Lung Cancer Screening: A Systematic Review. Clin Lung Cancer 2016;17(5):315-24. doi: https://doi.org/10.1016/j.cllc.2016.03.007
Sturgeon JA, Carriere JS, Kao MJ, et al. Social Disruption Mediates the Relationship Between Perceived Injustice and Anger in Chronic Pain: a Collaborative Health Outcomes Information Registry Study. Ann Behav Med 2016;50(6):802-12. doi: https://doi.org/10.1007/s12160-016-9808-6
Chipidza FE, Wallwork RS, Stern TA. Impact of the Doctor-Patient Relationship. Prim Care Comanion CNS Disord 2015;17(5) doi: https://doi.org/10.4088/PCC.15f01840
Eveleigh RM, Muskens E, van Ravesteijn H, et al. An overview of 19 instruments assessing the doctor-patient relationship: different models or concepts are used. J Clin Epidemiol 2012;65(1):10-5. doi: https://doi.org/10.1016/j.jclinepi.2011.05.011
Ridd M, Shaw A, Lewis G, et al. The patient-doctor relationship: a synthesis of the qualitative literature on patients' perspectives. Brit J Gen Pract 2009;59(561):e116-33. doi: https://doi.org/10.3399/bjgp09X420248 [published Online First: 2009/04/04]
Petrie K, Weinman J. Patients’ Perceptions of Their Illness: The Dynamo of Volition in Health Care. Curr Dir Psychol Sci 2012;21(1):60-65. doi: https://doi.org/10.1177/0963721411429456
Petrie KJ, MacKrill K, Derksen C, et al. An illness by any other name: The effect of renaming gout on illness and treatment perceptions. Health Psychol 2018;37(1):37-41. doi: https://doi.org/10.1037/hea0000548
Martino L. 2007 Section 5. Stigma and how to tackle it. HealthKnowledge. https://www.healthknowledge.org.uk/public-health-textbook/medical-sociology-policy-economics/4a-concepts-health-illness/section3. Accessed June 20, 2020
de Souza JA, Yap BJ, Wroblewski K, et al. Measuring financial toxicity as a clinically relevant patient-reported outcome: The validation of the COmprehensive Score for financial Toxicity (COST). Cancer 2017;123(3):476-84. doi: https://doi.org/10.1002/cncr.30369 =
Gallacher KI, Montori VM, May CR, et al. Treatment Burden and Multimorbidity. Basicmedical Key, 2016. https://basicmedicalkey.com/treatment-burden-and-multimorbidity/ Accessed June 20, 2020
Acknowledgments
The authors would like to thank the Lown Institute, which sponsored the meeting in 2019 on harms of overuse.
Funding
This project was not directly funded. DK’s work was supported in part by a grant from the National Cancer Institute to Memorial Sloan Kettering Cancer Center (P30 CA008748-50). JBS reports support from K24 AG049036-01A1 from the National Institute on Aging.
Author information
Authors and Affiliations
Contributions
All authors work in quality improvement and low-value healthcare policy-making and research. All authors provided substantial contributions to the conception and design of the work and to the acquisition, analysis, or interpretation of data. They contributed to the draft and approved the final version. DK is the guarantor.
Corresponding author
Ethics declarations
Conflict of Interest
DK reports that her spouse serves on the Scientific Advisory Boards of Vedanta Biosciences and Opentrons and does consulting work for Takeda. AGE holds the HCF Research Foundation Professorial Fellowship and has received direct income from providing low-value care-related data analytics and advice to the following: Australian state government health departments—Victoria, Queensland; Australian Dept of Veterans Affairs; Medibank Ltd.; Private Healthcare Australia (PHA); Australian Defence Force Joint Health Command. AGE receives direct income as a Ministerial appointee to the Australian Government Department of Health’s Medicare Benefits Schedule (MBS) Review Taskforce, as a Board Member of the New South Wales Bureau of Health Information, and as advisor to Cancer Australia. He is a member of the Expert Advisory Committee for Evidence-based Interventions (NICE, NHSCC, NHS Improvement and Academy of Medical Royal Colleges). In the past 36 months, JSR received research support through Yale University from Medtronic, Inc. and the Food and Drug Administration (FDA) to develop methods for postmarket surveillance of medical devices (U01FD004585), from the Centers of Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting (HHSM-500-2013-13018I), and from the Blue Cross Blue Shield Association to better understand medical technology evaluation; Dr. Ross currently receives research support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the Food and Drug Administration to establish Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program (U01FD005938), from the Medical Device Innovation Consortium as part of the National Evaluation System for Health Technology (NEST), from the Agency for Healthcare Research and Quality (R01HS022882), from the National Heart, Lung and Blood Institute of the National Institutes of Health (NIH) (R01HS025164, R01HL144644), and from the Laura and John Arnold Foundation to establish the Good Pharma Scorecard at Bioethics International and to establish the Collaboration for Research Integrity and Transparency (CRIT) at Yale. DJM reports funding from the National Institutes of Health, the Centers for Disease Control and Prevention, the Agency for Healthcare Research and Quality (AHRQ) and US Department of Veterans’ affairs (VA). RJC receives grant support from PCORI and from the American College of Emergency Physicians. Other authors report no potential conflicts of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Prior Presentations
None
Supplementary Information
ESM 1
(DOCX 121 kb)
Rights and permissions
About this article
Cite this article
Korenstein, D., Harris, R., Elshaug, A.G. et al. To Expand the Evidence Base About Harms from Tests and Treatments. J GEN INTERN MED 36, 2105–2110 (2021). https://doi.org/10.1007/s11606-021-06597-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-021-06597-9