Health Reports
Validation of a brief measure of combat exposure among Canadian Armed Forces personnel

by Kerry A. Sudom, Robert Nesdole and Mark A. Zamorski

DOI: https://www.doi.org/10.25318/82-003-x201901100002-eng

Research has consistently shown that exposure to military combat is associated with a host of mental health problems, including posttraumatic stress disorder (PTSD), depression and alcohol misuse,^{Note 1Note 2Note 3} with PTSD showing the strongest association.^{Note 4} In addition, combat exposure has been linked to substance abuse, suicidal ideation, injury and traumatic brain injury.^{Note 1Note 5Note 6Note 7Note 8Note 9} To understand how combat affects adverse health outcomes, sound measurement of combat experiences is required. Generic trauma measures have items that map conceptually to military experiences (e.g., exposure to combat or peacekeeping), but there are several limitations to using these measures: a lack of clarity as to what precise experiences underlie a respondents’ endorsement of an item, an inability to capture the broad range of intensity of combat experiences, and an inability to tie other trauma items from the inventory to military service.

Although different measures of combat exposure exist, many recent Canadian^{Note 10Note 11} and international^{Note 12Note 13} studies have used measures based on the Mental Health Advisory Team’s Combat Experiences Scale (MHAT-CES), developed by the Walter Reed Army Institute of Research (WRAIR).^{Note 2} The original measure is meant to capture the full range of potentially traumatic combat experiences among individuals deployed to the ongoing conflicts in Southwest Asia. The instrument is lengthy, including up to 37 items in some versions.^{Note 14} Advantages of longer instruments include their being exhaustive and, when treated as a count of the different types of combat experiences,^{Note 15} precise. However, using longer instruments results in greater respondent burden, which can be problematic in the contexts of routine post-deployment screening and large, population-based mental health surveys. Briefer instruments are therefore needed. Factor analysis and principal components analysis^{Note 3Note 16} of longer scales have demonstrated that many of the items are tightly correlated with one another, providing both a justification and a basis for selecting items to make a briefer scale.

Several shorter scales to assess combat exposure have been developed and validated in other military populations. The Critical Warzone Experiences scale, a 7-item Combat Experiences Scale (CES) measure of combat experiences derived from the MHAT-CES, was found to have favourable psychometric properties and to be strongly predictive of mental health outcomes among Iraq and Afghanistan veterans.^{Note 17} In addition, the 7-item CES has been validated with Vietnam veterans.^{Note 18} An 8-item measure of combat exposure (CES-8) was recently used in the Canadian Forces Mental Health Survey (CFMHS), a population-based survey of Canadian Armed Forces (CAF) personnel that was administered in 2013. The CFMHS provides a rich source of data on mental health and enables comparison across time points within the CAF and with the civilian general population.^{Note 19} The 8 items in the CES-8 were derived from the larger 30-item CES (CES-30) used to capture deployment experiences during post-deployment screening. However, the psychometric properties of this reduced scale have not been rigorously explored in association with the parent scale and the outcomes related to mental health that the scale is meant to predict.

Therefore, the current study uses data collected during post-deployment screening to document the psychometric properties of these 8 items as a potential brief measure of combat exposure that could be used for CAF personnel returning from deployment. The goal of this study is to examine the utility of the CES-8 as a potential alternative to the CES-30 in the contexts of both screening and survey research.

Method

Participants

Participants included 16,188 CAF personnel who deployed in support of the mission in Afghanistan between 2009 and 2012. There was significant diversity in the roles and experiences of participants across phases and locations of deployment. Participants located in Kandahar, representing the majority of deployments, had the highest threat level.^{Note 15Note 20} The sociodemographic and military characteristics of the sample are presented in Table 1. The majority of participants were male, Regular Force, non-commissioned members, and from the Army element.

Procedure

Combat exposure data from the Enhanced Post-Deployment Screening (EPDS) process were used for this study. The EPDS involves a confidential, although not anonymous, questionnaire administered between 90 and 180 days following overseas deployments of 60 days or longer.^{Note 10} Data from the EPDS process are captured electronically for health surveillance purposes. The anonymized use of these administrative health data for the present analysis was approved as part of a larger research project^{Note 21} by Veritas Independent Review Board in Montreal, Quebec.

Measures

Combat exposure: Combat exposure was assessed using a modified version of the 34-item CES developed by WRAIR.^{Note 14Note 22} The scale assesses the types of events experienced during the most recent deployment (e.g., “knowing someone seriously injured or killed”). The EPDS uses a 30-item version of the WRAIR combat exposure scale, with four items of the original scale removed because of concerns that endorsing them might require investigation into potential misconduct (e.g., “witnessing mistreatment of a non-combatant”). The items for the CES-8 were derived from these 30 items using considerations such as the prevalence,^{Note 3} association with adverse mental health outcomes,^{Note 23} ability to map to a range of different subfactors identified in earlier principal components analysis,^{Note 3Note 4} and conceptual considerations of the items (M. Zamorski, personal communication, March 30, 2017). The response options were “yes” or “no” for having experienced each type of event during the most recent deployment.

Mental health problems: Several measures of mental health and well-being were used in this study to establish whether the CES-8 had the expected relationship with PTSD, depression, anxiety disorder, suicidal ideation and high risk drinking. PTSD symptomatology was assessed using the PTSD Checklist–Civilian version (PCL-C).^{Note 24} The PCL-C consists of 17 items that encompass the diagnostic symptoms of PTSD. With this test, respondents are asked to indicate how much they were bothered by each symptom in the past month, and scores of 50 or higher indicate the presence of PTSD.^{Note 24} Symptoms of depression were measured using the Primary Care Evaluation of Mental Disorders (PRIME-MD) Patient Health Questionnaire (PHQ),^{Note 25} which assesses core diagnostic symptoms of depression and involves asking respondents how often they had been bothered by problems over the last two weeks. Developers of the scale created algorithms to determine diagnoses of major depressive disorder (MDD) and minor depressive disorder (described as “other depressive disorder” by the developers).^{Note 25} Symptoms of anxiety were measured using seven items from the PRIME-MD PHQ, assessing how often the respondent had been bothered by the diagnostic symptoms of generalized anxiety disorder (GAD) over the previous four weeks. This was done using the algorithm documented by the developer for “other anxiety syndrome.”^{Note 25} Suicidal ideation was measured using one item (i.e., “thoughts that you would be better off dead, or of hurting yourself in some way”) embedded in the PRIME-MD PHQ.^{Note 25} Respondents were asked how often they had been bothered by this over the last two weeks. Symptoms of high risk drinking were measured using the 10-item Alcohol Use Disorders Identification Test,^{Note 26} developed to identify individuals whose alcohol consumption has become hazardous or harmful to their health.

Analysis

The reliability coefficients for the full scale (CES-30) and the 8-item scale (CES-8) were calculated. The factor structure of the CES-30 and CES-8 were evaluated for model fit using the following indices: chi-square, the root mean square error of approximation (RMSEA),^{Note 26} the comparative fit index (CFI),^{Note 27} the Tucker-Lewis Index (TLI),^{Note 28} and the standardized root mean square residual (SRMR).^{Note 29} Fit index cut-off levels for determining goodness of fit include X² p-value > 0.05,CFI > 0.95, TLI > 0.95, RMSEA < 0.06, and SRMR < 0.08.^{Note 29} Log likelihood estimates obtained from generalized structural equation modelling were used to compare the fit of the CES-8 with the CES-30.

The association between CES-8 scores and CES-30 scores was evaluated using point biserial correlation coefficients for dichotomous mental health outcomes. In addition, the relationship between the CES-8 and measures of health known to be associated with combat exposure from past research was assessed using logistic regression. Finally, the overall predictive value for the outcome of PTSD was compared for the two scales by calculating the area under the curve (AUC) using receiver operating characteristic (ROC) analysis. Stata version 14 was used for all analyses.

Results

Descriptive information and reliability

Table 2 presents the percentage of respondents who reported experiencing each of the events at least once. The most commonly experienced event was receiving incoming artillery, rocket, or mortar fire, reported by 63% of participants, while feeling responsible for the death of Canadian or ally personnel was reported by less than 3% of participants. The Kuder-Richardson reliability coefficient for the CES-30 was high (KR-20 = 0.92), while the coefficient for the CES-8 was moderate (KR-20 = 0.63).

Model fit

Confirmatory factor analysis model fit indices are shown in Table 3. For both scales, four out of five of the indices indicated poor fit. With the exception of a single index (SRMR), fit tended to be better, if anything, for the CES-8 than for the CES-30. However, the log likelihood test suggested that the CES-8 does not offer a statistically significant improvement in model fit:
${{\rm X}}^2\left(df\right)=-348,765.96\left(22\right),p=0.99$.

Comparison of the CES-8 and the CES-30

The CES-30 and CES-8 were highly correlated, at 0.85 (p < 0.01), indicating that both scales are likely measuring the same construct. Both CES-8 and CES-30 total scores were correlated with greater prevalence of PTSD, depression, suicidal ideation, anxiety and high risk drinking (Table 4). For both measures, the strongest associations were evident for PTSD.

Logistic regression was used to assess the associations of the CES-8 and CES-30 with mental health outcomes (Table 5). The CES-8 and CES-30 were both found to be predictive of PTSD, MDD, other depressive disorders, GAD, suicidal ideation and high risk drinking (p < 0.001), with PTSD displaying the strongest associations for both versions of the scale.

An ROC curve was developed to determine the relative predictive value of the CES-8 and CES-30 for PTSD symptoms. PTSD was chosen as the outcome variable for this analysis because of its strong association with combat exposure relative to other mental health problems.^{Note 4} The results of this analysis suggested that both the CES-8 (AUC = 0.72, p < 0.001) and the CES-30 (AUC = 0.70, p < 0.001) have similar predictive value for PTSD.

Discussion

A brief measure of combat exposure helps militaries to reduce the burden on respondents and the administration time in large population-based health surveys, as well as in post-deployment settings where personnel must also be screened for mental and physical health problems. The present study sought to describe the psychometric properties of a brief version of the CES among CAF members who recently returned from a deployment in support of the mission in Afghanistan and who had completed post-deployment screening. It was found that the 8-item measure did not fit a one-factor solution well, nor did the 30-item scale, indicating that the CES-30 likely works best when used with subscales or on more homogeneous groups, as noted in previous research.^{Note 3} In addition, the shorter scale did not offer a statistically significant improvement in model fit over the full 30-item measure. However, the association of the CES-8 with increased odds of a number of mental health outcomes indicates its potential usefulness as a brief measure of combat exposure in settings where using the full scale is not feasible. Moreover, the CES-30 and the CES-8 were found to have similar predictive value for PTSD.

The items used in the reduced CES are similar to those used in a validation study of a 7-item CES, which demonstrated high reliability and a single higher-order factor among a sample of U.S. veterans.^{Note 17} However, the 7-item CES validation used only treatment-seeking veterans, unlike the current study, which included all CAF members undergoing post-deployment screening regardless of their health status. Similarly, although Keane and colleagues^{Note 18} also found that a 7-item CES had good reliability and factorial validity, the study used only treatment-seeking veterans, and those with mental health conditions were oversampled. Other research using longer scales has also used more homogeneous samples.^{Note 14Note 16} A much larger and more diverse group of personnel returning from deployment was used in the current study.^{Note 10} Therefore, it is impossible to attribute the more favourable factor structure of these other brief scales to the scale itself rather than the greater homogeneity of the validation sample.

Similar to past research using brief scales of combat exposure,^{Note 17Note 18} the CES-8 showed strong associations with health conditions. In the present study, the strongest associations were found for PTSD. Past research has shown that, of the mental health conditions assessed following deployment, PTSD generally shows the strongest association with combat exposure.^{Note 4}

Limitations

There are a number of limitations to this study. First, post-deployment screening data are collected several months following the end of deployment, which may lead to issues with accurate recall of events, particularly for participants with mental or physical health conditions. In addition to the length of time elapsed, the respondent’s current level of mental health and functioning may affect their perceptions of events that occurred during the deployment.

The survey contains self-reported, subjective measures of combat experiences and mental and physical health conditions. Although confidentiality is assured, the EPDS is not anonymous because of its nature as a screening tool to identify individuals requiring help for health conditions following deployment. Therefore, some participants may be reluctant to admit involvement in particular combat experiences, particularly those involving perceived responsibility for the death of Canadian or ally personnel, for fear of potential career repercussions. Similarly, some respondents may be reluctant to disclose mental health problems because of perceived stigma or potential personal or career impacts. Analysis of combat exposure data from the CFMHS will alleviate this issue since the survey was conducted by a third party as part of a research study with strict confidentiality protections.

Data from post-deployment screening are collected at a single point in time. Therefore, although combat exposure was associated with a number of mental health conditions in this study, it is unclear whether combat experiences led to the development of mental health problems, or whether such experiences exacerbated existing issues among vulnerable individuals. Longitudinal research is needed to determine the temporal relationship between combat experiences and mental health. This study sought to simply examine the structure of the scale and its psychometric properties; no assumptions were made about causality.

Combat experiences were captured in dichotomous response format (i.e., “yes” or “no” to having experienced a particular type of event), which does not reflect the degree of exposure or the level of subjective stress associated with each event. Certain events may be perceived as more traumatic and exhibit greater associations with mental health problems. Indeed, previous research on CAF members following deployment has shown that certain combat experiences are associated with increased odds of PTSD, while other experiences showed no associations.^{Note 4} There is also evidence for the importance of cognitive appraisals of deployment stressors for mental health outcomes.^{Note 30}

Finally, it is difficult to reliably capture combat experiences using a single scale because of the heterogeneity of experiences of personnel deployed in support of modern military operations. Past research has alleviated this issue by using more homogeneous samples.^{Note 17} Moreover, items included in this scale may not apply to future missions, and traumatic experiences may also occur on other types of non-combat missions (e.g., peacekeeping or humanitarian missions). The heterogeneity of experiences and the changing nature of military missions reflect the difficulty in using single measures of exposure to occupational trauma in the military.

Implications

Although the 8-item scale was developed for the CFMHS, a population-based survey of CAF members, its psychometric properties were confirmed using post-deployment screening data to facilitate comparison with the full 30-item scale it was derived from and to determine whether it offered an advantage over the longer scale. However, it will also be important to examine the CES-8 scale using data from the CFMHS itself to determine whether the CES-8 exhibits similar psychometric properties in this group, which experienced a variety of deployments in addition to those in support of the mission in Afghanistan. Moreover, other shorter measures of combat exposure based on the CES-30 should be tested to determine whether a different set of items offers a greater statistical advantage and shows a more robust structure than the full scale.

Conclusions

Brief measures of combat exposure are valuable for assessing events experienced during deployment among military personnel. Such measures would be simple and convenient to administer in post-deployment settings where military personnel typically undergo screening for health issues, and thereby reduce administrative and respondent burden. Although the CES-8 assessed in the current study represents a potentially useful measure for CAF personnel, further research is necessary to improve its fit as a unidimensional scale of combat exposure.