Review
Determining the extent of cognitive change after coronary surgery: a review of statistical procedures

https://doi.org/10.1016/S0003-4975(01)03375-6Get rights and content

Abstract

Currently, cognitive decline after coronary surgery is said to be significant if the individual’s postoperative test score is at least 1 standard deviation (SD) worse than their preoperative score. This “1-SD” technique fails to account for factors that may confound interpretation of serially acquired cognitive test scores, including regression to the mean, measurement error caused by poor test-retest reliability, and practice effects. We review the many alternative and potentially superior statistical techniques that have been described in the neuropsychologic and psychiatric literature for differentiating “true” changes in cognitive test score from changes caused by these factors.

Section snippets

Issues associated with serial cognitive assessment

Most conventional cognitive tests are designed for the identification of brain dysfunction, rather than the assessment of change in brain functions over time [14]. Although such tests are suitable for the investigation of brain-behavior relationships in cognitively impaired individuals, they possess two psychometric properties that restrict their applicability for serial assessment. First, many cognitive tests have limited or nonequivalent alternate forms that may result in performance changes

Statistical techniques for determining the significance of change in cognitive test score

Table 1 lists some common statistical procedures for assessing change in cognitive test score, defines them statistically, and provides a reference where they have been applied to cognitive data. In this section, we discuss those techniques that provide the best correction for sources of measurement error and other systematic influences on test performance.

Examples from published reports

A number of recent studies have investigated the ability of these different statistical techniques to differentiate between “true” changes in cognition and changes attributable to measurement error. These studies aimed to determine the practical ability of these models to predict a follow-up score from a baseline score. For example, Temkin and colleagues [25] compared standard and practice effect–corrected RCIs with linear and multiple regression techniques as predictors of follow-up

Summary and conclusions

As noted above, the statistical techniques described here have arisen partly because of the psychometric limitations of many conventional cognitive and neuropsychological tests (eg, ceiling effects, poor reliability, limited range of possible scores). These techniques may be broken into three broad subtypes according to whether they attempt to account for none, some, or all possible sources of measurement error and the influence of other systematic influences on test performance. The first

References (30)

  • N.S. Jacobson et al.

    Psychotherapy outcome researchmethods for reporting variability and evaluating clinical significance

    Behav Therapy

    (1984)
  • S. Newman et al.

    Acute neuropsychological consequences of coronary artery bypass surgery

    Curr Psychol Res Rev

    (1987)
  • M.F. Newman et al.

    Longitudinal assessment of neurocognitive function after coronary-artery bypass surgery

    N Engl J Med

    (2001)
  • D.A. Stump

    Selection and clinical significance of neuropsychologic tests

    Ann Thorac Surg

    (1995)
  • E. Symes et al.

    Issues associated with the identification of cognitive change following coronary artery bypass grafting

    ANZ J Psychiatr

    (2000)
  • Cited by (108)

    • Neuropsychological assessment

      2023, Encyclopedia of Mental Health, Third Edition: Volume 1-3
    • Measuring and monitoring cognition in the postoperative period

      2020, Best Practice and Research: Clinical Anaesthesiology
      Citation Excerpt :

      Multiple-regression methods are used to build statistical models that can predict postoperative performance based on preoperative scores plus any other relevant variables known to influence change to different degrees (e.g. age, education, retest intervals, RTM, practice effects) [50]. Although large samples are needed to formulate accurate estimates of change using these methods, they do enable the development of models that account for a large number of the myriad confounding variables identified in POCD research, allowing closer inspection of many potential causes of variability within the data [75]. In recent attempts to align POCD with DSM-V criteria for neurocognitive disorders, it has been suggested that caseness should be based on corrected z-scores, with a decline of 1–2 SDs below either normative or control data characterised as mild disorder and >2SDs as major disorder [18], leaving open the question of whether modified RCIs or regression-based change scores should be used.

    • Incidence and Clinical Significance of Cerebral Embolism During Atrial Fibrillation Ablation With Duty-Cycled Phased-Radiofrequency Versus Cooled-Radiofrequency: A Randomized Controlled Trial

      2019, JACC: Clinical Electrophysiology
      Citation Excerpt :

      There are limited data about cognitive functioning after AF ablation (22). In several studies on other procedures (e.g., after coronary artery bypass grafting), statistical techniques have been implemented to determine “true” (i.e., statistically significant) cognitive decline at the individual level (23). In addition, for major neurocognitive disorder as defined by using international diagnostic guidelines for mental disorders (24), a meaningful decrease in test performance is typically ≥2 SDs below appropriate norms or reference groups (third percentile or below).

    • Postoperative Cognitive Dysfunction

      2016, Complications in Neuroanesthesia
    • Longitudinal changes in the dopamine transporter and cognition in suicide attempters with charcoal burning

      2015, Psychiatry Research - Neuroimaging
      Citation Excerpt :

      Several methodological limitations of our study need to be considered. First, the influences of practice effects should be considered for cognitive evaluations given repeatedly to the same subject (Heaton et al., 2001; Kesler et al., 2001; Collie et al., 2002; Pietrzak et al., 2010). Currently, only one previous CO-poisoning follow-up study has reported that correcting for practice effects decreased apparent improvements in verbal memory performance or performance in the Trail Making Test over time (Kesler et al., 2001).

    View all citing articles on Scopus
    View full text