Narrative reviewCommentary: Statistical significance and clinical significance - A call to consider patient reported outcome measures, effect size, confidence interval and minimal clinically important difference (MCID)
Introduction
When reviewing the statistics of research and its interpreted outcomes, a clinician must consider three questions: 1. How reliable are the results? 2. Are the results due to chance? and, 3. Do the results matter to a patient? To answer these questions, an understanding of patient reported outcome measures (PROMs), statistical and clinical significance is needed.
Patient-report outcome measures (PROMs) are used in clinical practice to provide a somewhat objective measurement of patient progress with respect to their management (Cella et al., 2010; Fleischmann and Vaughan, 2018). These measures are particularly valuable for demonstrating improvements in pain levels (Turk et al., 2006), activities of daily living (Cella et al., 2010) and functional activities, to both the patient and third-party payers (e.g. worker's compensation) (Blyth et al., 2003). However, Chiarotto et al. (2016) argue that PROMs lack content validity when measuring physical functioning in low back pain, and lack structural validity as this concept has received limited attention in the literature (Chiarotto et al., 2016, 2018). Notwithstanding, data obtained from PROMs can be used to demonstrate how healthcare interventions may affect various aspects of a person's quality of life and serve as a mechanism to monitor treatment effectiveness (Roach, 2006). Developing an understanding of how best to evaluate PROMs requires an understanding of the concepts of statistical significance and clinical significance to inform clinical decision-making. The purpose of the current commentary is first to explore the basic statistical concepts that can be applied to demonstrate these changes in PROM scores, and then highlights the importance of clinical significance in clinical practice.
Section snippets
Statistical significance
In most quantitative experiments, researchers investigate if there is a difference between intervention groups by performing statistical tests and reporting an associated p-value. Box 1 provides an example of how this approach may be applied in a research design.
Once researchers have performed statistical analysis (from Box 1 example) they compare the between group results (p-value) to an a-priori alpha level, a parameter designed to determine how likely the result was due to random chance. At
Effect size
Absolute effect size provides an indication of the magnitude of the difference between the averages or mean, between the two interventions in the example described previously (Ferguson, 2009; Nakagawa and Cuthill, 2007). Used to quantify the difference between two groups, it has an advantage over the use of tests of statistical significance alone as it places the emphasis on the size of the difference (Nakagawa and Cuthill, 2007). Multiple authors posit that effect sizes should be reported
Conclusion
This commentary has argued that reporting p-values is insufficient in both the research and practice contexts as it provides clinicians with little clinically useful information to measure change in their patients. It is most clinically relevant if data are collected through the use of patient reported outcome measures (PROM), then effect size and MCID are reported so that clinicians can make evidence informed decisions for patient centred care. Clinicians need to consider the statistical
Statement of competing interests
The authors state that there are no competing interests.
References (44)
- et al.
Chronic pain, work performance and litigation
Pain
(2003) - et al.
The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008
J. Clin. Epidemiol.
(2010) - et al.
A systematic review highlights the need to investigate the content validity of patient-reported outcome measures for physical functioning in patients with low back pain
J. Clin. Epidemiol.
(2018) - et al.
Choosing the right outcome measurement instruments for patients with low back pain
Best Pract. Res. Clin. Rheumatol.
(2016) - et al.
Four methods of estimating the minimal important difference score were compared to establish a clinically significant change in Headache Impact Test
J. Clin. Epidemiol.
(2006) - et al.
Understanding the minimum clinically important difference: a review of concepts and methods
Spine J.
(2007) - et al.
Defining clinically meaningful change in health-related quality of life
J. Clin. Epidemiol.
(2003) - et al.
Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale
Pain
(2001) - et al.
The challenges and opportunities of using patient reported outcome measures (PROMs) in clinical practice
Int. J. Osteopath. Med.
(2018) - et al.
Methods to explain the clinical significance of health status measures
Mayo Clin. Proc.
(2002)
Measurement of health status: ascertaining the minimal clinically important difference
Contr. Clin. Trials
Assessing the minimal important difference in symptoms: a comparison of two techniques
J. Clin. Epidemiol.
The SF-36 and SGRQ: validity and first look at minimum important differences in IPF
Respir. Med.
Developing patient-reported outcome measures for pain clinical trials: impact recommendations
Pain
Minimally important differences were estimated for six Patient-Reported Outcomes Measurement Information System-Cancer scales in advanced-stage cancer patients
J. Clin. Epidemiol.
Significance testing: why does it prevail?
Eur. J. Epidemiol.
Avoiding pitfalls when using information-theoretic methods
J. Wildl. Manag.
Sufficiently important difference: expanding the framework of clinical significance
Med. Decis. Making
Many faces of the minimal clinically important difference (MCID): a literature review and directions for future research
Curr. Opin. Rheumatol.
Confidence intervals assess both clinical significance and statistical significance
Ann. Intern. Med.
Neck pain: clinical practice guidelines linked to the international classification of functioning, disability, and health from the orthopaedic section of the American physical therapy association
J. Orthop. Sports Phys. Ther.
Clinimetrics corner: the minimal clinically important change score (MCID): a necessary pretense
J. Man. Manip. Ther.
Cited by (22)
Osteopathic management of non-specific neck pain: Preliminary findings from a cross sectional study of Australian osteopaths
2024, International Journal of Osteopathic MedicineNeurocognitive functioning following lung cancer treatment: The PRO-Long Study
2022, Technical Innovations and Patient Support in Radiation OncologyCitation Excerpt :However, research on standards of meaningful clinical important cognitive impairment, developed cooperatively with patients, is currently lacking.[10] Whereas statistical significance refers to the reliability of the data, MCID indicates the smallest change in outcome that is meaningful to the patient [34]. MCIDs facilitate the interpretation of clinical relevance of score changes over time.
Using anchor-based methods to determine the smallest effect size of interest
2021, Journal of Experimental Social PsychologyCitation Excerpt :However, psychologists are more commonly interested in the subjective experiences of people directly, and not in observers' perceptions. A more patient-centered anchor-based approach to estimate the minimal important difference also uses a global judgment about whether patients have improved or worsened, but asks the patients themselves to provide a subjective global rating of change, which is then used as an anchor (Cuijpers, Turner, Koole, Van Dijke, & Smit, 2014; Devji et al., 2020; Dworkin et al., 2008; Ebrahim et al., 2017; Fleischmann & Vaughan, 2019; Guyatt et al., 2002; Jaeschke, Singer, & Guyatt, 1989; King, 2011; Lydick & Epstein, 1993). In this approach, the construct of interest is also measured at two time-points (T1 and T2), for example before and after an intervention or manipulation.
The ICD-11 and opportunities for the osteopathy profession
2020, International Journal of Osteopathic MedicineA pilot study of virtual reality for inpatients with opioid use disorder
2024, American Journal on AddictionsAn Overview of Statistical and Clinical Concepts: The Use and Interpretation of Confidence Interval
2023, Journal of Isfahan Medical School