Suicide Risk Assessment

In clinical populations, suicidal behaviors are future events that should be the focus of prevention. A recent systematic review estimated the frequency of suicide after hospital-treated self-harm at 1.6% after 12 months and 3.9% at 5 years, with repetition of self-harm at 16.3% after 12 months (Carroll, Metcalfe, & Gunnell, 2014b). Another systematic review focusing on psychiatric inpatients estimated that suicide after discharge was around 0.5% per-person-year; the highest rate was 1.1% per-person-year 3 months after discharge (Chung et al., 2017). In the UK, 6.5% of psychiatric inpatients were admitted for self-harm 12 months after discharge (Gunnell et al., 2008).

Risk assessments are done to classify individuals as high risk or low risk for future suicidal behaviors. This classification (stratification) is used to determine the allocation of after-care aimed at preventing these behaviors. The high-risk stratum are offered specific interventions (e.g., psychiatric hospitalization, close nursing observation [for inpatients], face-to-face or telephone follow-up and identified community support) or more intense intervention (e.g., greater frequency of reviews in inpatient and community settings). Risk stratification is widely practiced (Quinlivan et al., 2014) and endorsed (Suicide Prevention Resource Center, 2015).

However, the inaccuracy of suicide prediction has been known for 60 years (Rosen, 1954). In a seminal paper, Pokorny used the suicide rate for his patients (500 per 100,000) and invoked an almost perfect predictive instrument (to classify high risk) having a sensitivity of 99% and a specificity of 99% (Pokorny, 1983). Even under these idealized conditions, the positive predictive value (PPV) was 33%, with 66% of the high-risk stratum being false positives, thus demonstrating the absolute statistical ceiling imposed on PPV by the low prevalence of suicide, regardless of the method of stratification.

Accuracy Statistics for Predictive Instruments

The classic accuracy statistics for diagnostic, prognostic, and predictive instruments at the population level are: sensitivity, specificity, PPV, and negative predictive value (NPV). An individual patient's risk might be additionally understood by estimating the change in pre-test to post-test probability by using (positive or negative) likelihood ratios (LR+ and LR-; Attia, 2003; Furukawa, Strauss, Bucher, Thomas, & Guyatt, 2015). The clinical usefulness of any predictive test might be understood by using the (positive or negative) clinical utility index (CUI+ and CUI-; Mitchell, 2011). All of these tests of accuracy or clinical usefulness are linked. The key point is that sensitivity, specificity, LR+, and LR- are properties of the test and are usually not influenced by “disease” prevalence (Attia, 2003). Conversely, PPV and NPV are influenced by prevalence (Brenner & Gefeller, 1997), as are CUI+ and CUI- (because their calculation relies on PPV and NPV).

Three accuracy measures help to understand the high-risk classification as the basis for allocation of interventions. These are sensitivity, PPV, and LR+; although an ideal test would also consider specificity, NPV, and LR, which relate to the accuracy of a low-risk classification. Sensitivity answers the following question: How many cases of future suicide or other suicidal behaviors will be detected or missed for a given population? This is helpful for case-finding in whole populations, but not for assessing the usefulness of using high risk as a basis for offering a particular intervention. For the individual patient–clinician pair, the PPV (the proportion of the high-risk group with the outcome) is the first consideration, and a rule of thumb says a PPV of greater than 80% is necessary for the allocation of an intervention. In some circumstances, a lower level might be considered (e.g., conditions with high mortality rates; availability of low-cost, highly efficacious, low-harm interventions). The next consideration is the LR+ (how many times more likely those with future suicidal behaviors will have a high-risk classification compared with those without future suicidal behaviors) and the rule of thumb says an LR+ of greater than 10 is likely to be clinically useful.

Best Estimates of Sensitivity, PPV, and LR+

Several systematic reviews and meta-analyses meet (or nearly meet) standards for reporting of diagnostic and predictive tests (Leeflang, Deeks, Gatsonis, & Bossuyt, 2008) and have established estimates for the key accuracy statistics.

Systematic reviews assessed the predictive accuracy of individual scales for future suicidal behaviors – e.g., the SAD PERSON Scale (Warden, Spiwak, Sareen, & Bolton, 2014); the Suicide Intent Scale (Freedenthal, 2008) – and for scales commonly used in the UK (Quinlivan et al., 2016) or more widely (Runeson et al., 2017); however, in summary no scales performed well enough for routine clinical use in predicting suicidal behavior.

Eight more systematic reviews and meta-analyses have evaluated various predictive scales or individual risk factors for suicide or other suicidal behaviors: three for multiple instruments (Carter et al., 2017; Chan et al., 2016; Large et al., 2016); one for all reported risk factors over 50 years (Franklin et al., 2017); three for suicidal ideation (and prior suicidal behaviors) in adults (Chapman et al., 2015; Hubers et al., 2016; Ribeiro et al., 2016); and one in young people (Castellvi et al., 2017). Another review estimated the accuracy of clinician prediction of repeated hospital-treated self-harm (Woodford et al., 2017).

To summarize, the findings of these nine reviews would be exhaustive, but the key message can be understood from the review of risk factors. This review of 365 studies (3,428 risk factor effect sizes) found that (a) for diagnostic accuracy, prediction was only slightly better than chance for suicide or suicidal behaviors, (b) no subcategory predicted far above chance levels, (c) and prediction has not improved in 50 years (Franklin et al., 2017). Some specific results from other reviews warrant consideration. For future suicide in clinical populations, the pooled sensitivity for all instruments was 56% (Large et al., 2016), and the PPVs were 5.5% (Carter et al., 2017), 5.5% (Large et al., 2016), and ranged from 1.3 to 16.7% (Chan et al., 2016). Thus, the best pooled estimates for future suicide will miss around 44% of all cases, while identifying 94.5% false positives classified as high risk. If interventions were allocated for high-risk classification, then many true cases would not receive interventions and the vast majority of the high-risk stratum could not benefit from the interventions. The prediction of repeat nonfatal self-harm behaviors was similar. A meta-analysis of all predictive instruments found PPVs of 26.3% overall, with 16.1% for high-quality studies, 32.5% for hospital-treated self-harm, and 26.8% for psychiatric in-patients (Carter et al., 2017). In a meta-analysis of clinician prediction of repeat hospital-treated self-harm, the pooled estimates for sensitivity were 31%, PPV was 22% (weighted), and LR+ was 1.67 (unweighted; Woodford et al., 2017). For the classification of high risk for repeat self-harm, most cases are missed, most positives will be false, with only a marginal change from pre-test to post-test probability.

The most common single factor used to classify high risk for suicide is suicidal ideation. In a review of 81 studies the relative risk (RR) for suicide after 12 months in psychiatric populations was raised (3.53) for suicidal ideation; however, the absolute risk (AR) was very low (1.4%) for suicidal ideation as well as for no ideation (0.4%; Hubers et al., 2016). Suicidal ideation is an important risk factor for later suicide; however, only 1.4% go on to die by suicide (equivalent PPV = 1.4%), and the AR for no ideation is only 1% less than the ideation group. Another review reported a sensitivity range of 10–26% for various suicidal behaviors as outcomes for prior self-injurious thoughts and behaviors (Ribeiro et al., 2016). A good risk factor is not necessarily a good predictive factor (Grobman & Stamilio, 2006). Low-risk groups are usually more numerous than high-risk groups, and thus despite the lower percentage in the low-risk stratification case numbers are substantial (low sensitivity).

Resistance to Change in Clinical Practice

What factors are important for resistance to change in clinical practice? Most of us would like to believe that accurate prediction is possible. There has been an astonishing recent increase in reporting the success of various risk factors and instruments, with encouragement for clinical use (Franklin et al., 2017; Large et al., 2016). There is the general problem of implementing change with the often-quoted 17-year time lag for translating evidence-based findings into clinical practice (Zoe, Steven, & Jonathan, 2011).

There are at least two biases that may be relevant. Confirmation bias is the tendency to search for, interpret, and recall information in ways that confirm preexisting beliefs. We have noted the increase in publications reporting techniques that are better than chance (Franklin et al., 2017; Large et al., 2016), and while most original articles do not offer estimates of sensitivity, PPV, and LR+ to inform clinical usefulness, the authors usually recommend the use of their instruments.

Clinical experience also encourages confirmation bias. Consider a contrived example from the clinical population with the highest rate of subsequent suicide. A clinician makes a low-risk classification of most individuals at the general hospital and discharges them home with minimal follow-up, or a high-risk classification and admits this minority to the psychiatric hospital. For 1,000 patients, only 17 patients will die by suicide after 12 months (Carroll et al., 2014b) and few will die in the 24–72-hr time horizon of a clinician trying to keep a patient safe. Thus, a clinician is frequently reinforced by the apparent accuracy of her/his risk stratification; that is, the patient will not die by suicide 98% of the time, regardless of the allocation of intervention. We can be fooled by low prevalence disorders and our inherent tendencies to believe in our own skills and powers.

A second possible bias is confounding by indication (i.e., high-risk patients receive effective after-care, reducing the rate of suicidal behaviors, meaning that the true prevalence of future suicidal behaviors is higher than estimated). There is not much evidence to support that this phenomenon is substantial even in the hospital-treated self-harm population. A recent systematic review concluded that there is little clear evidence to suggest routine aspects of self-harm patient care, including psychosocial assessment, reduce the risk of subsequent suicide and repeat self-harm (Carroll, Metcalfe, & Gunnell, 2014a). A recent systematic review of clinician prediction (Woodford et al., 2017) used the estimated pooled efficacy of psychosocial interventions for repeat self-harm (Hetrick, Robinson, Spittal, & Carter, 2016) and concluded that even with full treatment compliance, and the highest proven treatment response, the prevalence would be increased by only 1–2% and the PPV would not improve sufficiently to be clinically useful (Woodford et al., 2017).

Conflicts of interest may drive resistance to change clinical practice. Many services use risk stratification to allocate expensive interventions to high-risk groups; the effect is to reduce service and contain cost. Clinician-researchers are producing predictive instruments and risk stratification techniques that have considerable direct costs for grants and opportunity costs for other questions. Several instruments have been commercialized and are available for a cost. The cost of applying these instruments in large populations is considerable, and the marketing departments of commercial enterprises are expected to promote the apparent benefits of their instrument.

Clinical Alternatives to Risk Stratification

Recent advice suggests we should abandon misguided attempts at risk prediction and instead encourage real engagement with the individual patient, the specific problems, and the circumstances (Mulder, Newton-Howes, & Coid, 2016). Professional bodies in the UK, Australia, and New Zealand have recommended three complementary (and sometimes overlapping) approaches for the allocation of services to reduce future suicidal behavior (Carter et al., 2016; National Institute for Health and Care Excellence, 2011; Royal College of Psychiatrists, 2010). These should be delivered in the context of consensual engagement with (a) an assessment and management plan for each patient, (b) a needs assessment, (c) identification of modifiable risk factors (not risk stratification), and (d) allocation to proven interventions that are clinically available.

Needs Assessment

The National Institute for Health and Care Excellence suggests that the assessment of needs should include: skills, strengths, and assets; coping strategies; mental (and physical) health problems or disorders; social circumstances and problems; psychosocial and occupational functioning and vulnerabilities; recent and current life difficulties; the need for psychological intervention, social care and support, occupational rehabilitation, and drug treatment; and the needs of any dependent children (National Institute for Health and Care Excellence, 2011). There may be other important needs but these seem like a reasonable place to start. The US Center for Disease Control summarizes potentially useful interventions for common needs: strengthening economic support; strengthening access and delivery of suicide care; creating protective environments; promoting connectedness; and teaching coping and problem-solving skills (Stone et al., 2017).

Identifying Modifiable Risk Factors for Every Patient and Intervening to Reduce Exposure to Them

The classic public health approach is to identify modifiable risk factors on the casual pathway to the outcome and then reduce exposure to these risk factors at clinical and population levels. After the many studies of risk factors for suicidal behaviors (Franklin et al., 2017) we have a list of risk factors but limited information about which to choose. To make the best choice of the modifiable risk factors we need to know more than the magnitude of the association: the population attributable risk (PAR) and the years of life lost (YLL) for suicide, or the quality adjusted life years (QALYs) or disability adjusted life years (DALYs) for nonfatal suicidal behaviors, and the magnitude of the efficacy for interventions that reduce exposure to these modifiable risk factors. We need to know these things after accounting at least for age, gender, and a past history of suicidal behaviors because the risk factors, PAR, and YLL vary for these groups.

We have some guidance. A recent systematic review of hospital-treated self-harm identified four risk factors for suicide, with robust effect sizes and little change when adjusted for important potential confounders: previous episodes of self-harm (hazard ratio [HR] = 1.68), suicidal intent (HR = 2.7), physical health problems (HR = 1.99), and male gender (HR = 2.05). Suicidal intent and physical health problems are potentially modifiable at the clinical level and previous self-harm at the population level. However, these estimates are drawn from only a small number of studies and we have few estimates for the PAR or YLL. A recent review calculated PARs for the repetition of self-harm in young people – borderline personality disorder (PAR = 42.4%), any personality disorder (PAR = 16.3%), any mood disorder (PAR = 42.2%), and previous sexual abuse (PAR = 12.8%) – and odds ratios (ORs) for continuous measures of severity of hopelessness (OR = 2.95) and suicidal ideation (OR = 2.01; Witt et al., 2018). Although these estimates also come from relatively few studies (and are not stratified by gender or previous self-harm) the substantial PARs for borderline personality disorder and any mood disorder are useful in identifying intervention targets for young people. Continuous measures of hopelessness and suicidal ideation do not only allow for PAR calculations, but might also be targets for intervention. Future studies should make choices of modifiable risk factors more clear-cut.

We need to develop interventions to reduce exposure to the modifiable risk factors, for example, current suicidal ideation. Interventions like the Collaborative Assessment and Management of Suicidality (CAMS) are being evaluated for reducing suicidal ideation (Jobes et al., 2017).

Make Available and Use Proven Effective Treatments to Reduce Suicidal Behaviors

We have some effective treatments for self-harm and we should make these available and enhance uptake by patients. The availability of interventions is limited by cost, training of clinicians, and lack of awareness of the potential benefits. Borderline personality disorder treatments have reduced suicidal behaviors (National Health and Medical Research Council, 2012) as well as borderline symptoms, health service use, anxiety, and depression (Cristea et al., 2017). Unselected hospital-treated self-harm populations have reduced self-harm with psychological interventions (Hetrick et al., 2016) and frequency of self-harm events by brief contact interventions (Milner, Carter, Pirkis, Robinson, & Spittal, 2015). These benefits are increasingly clear, although the number of studies and the effect sizes are modest.

Conclusion

Despite substantial effort over decades to identify risk factors and develop predictive instruments, risk stratification is too inaccurate to be clinically useful and might even be harmful (Mulder et al., 2016). Risk stratification misses many cases, has a very high false-positive rate, and provides a false sense of certainty. PPV is the simplest way of evaluating predictive risk for an individual patient, and low prevalence imposes an absolute statistical ceiling on PPV. This ceiling is independent of the methods used to generate the predictive instrument. Instead of using highly inaccurate risk stratification as the basis for differential allocation of treatment, we should focus on real engagement with each patient, their specific problems, and their circumstances. An alternative approach comprising a needs assessment, identification of modifiable risk factors, and use of effective interventions should be employed to guide management. Future research could be directed away from risk stratification and toward a better understanding of needs, specific modifiable risk factors, and effective intervention.