Abstract
Objectives:
The study was designed to estimate the prevalence of mental disorders in a cohort of firefighters who had been deployed to a devastating fire in Fort McMurray, Alberta, in 2016.
Methods:
A cohort of firefighters was established and followed up by online questionnaires. The contact in October 2018 to March 2019 included the PCL-5 questionnaire screening for post-traumatic stress disorder (PTSD) and the Hospital Anxiety and Depression Scale (HADS) screening for anxiety and depression. A sample was selected comprising all scoring ≥31 on the PCL-5 or ≥12 on either scale of the HADS, 30% of those scoring 8 to 11 on the HADS, and 10% of those with lower scores on all scales. This sample was assessed through a structured clinical interview to categorize disorders as defined in Diagnostic and Statistical Manual for Mental Disorders, fifth edition (DSM-5). Interviews were carried out face-to-face or by telephone between August 2019 and February 2020. Diagnoses in the interview sample were reweighted to obtain prevalence estimates for the whole cohort. In an analysis of receiver operating characteristics (ROC), possible cut points for scores from each screening questionnaire were examined.
Results:
In 2018 to 2019, 1,000 of the cohort of 1,234 firefighters completed the HADS and 998 completed the PCL-5. Of these, 282 were identified for structured clinical interviews for DSM-5 (SCID) assessment. Interviews were carried out with 192. Among those assessed, 40.6% met the criteria for PTSD, 30.7% for an anxiety disorder, and 28.5% for a depressive disorder. When reweighted to allow for sampling and losses to assessment, cohort prevalence estimates were as follows: PTSD 21.4% (15.7% to 29.1%), anxiety disorders 15.8% (11.0% to 22.5%), and depressive disorders 14.3% (9.9% to 20.8%). Lower prevalence estimates were obtained when using the cut point with least misclassification in the ROC analysis.
Conclusion:
Using the gold-standard SCID assessment, high rates of mental disorders were found in this cohort of firefighters who had experienced a devastating fire. Fewer cases would have been identified by screening questionnaire alone.
Introduction
The Fort McMurray fire in Northern Alberta in May 2016 exposed many firefighters to high levels of heat, physical exhaustion, smoke intensity, and emotional stress, particularly in the first few days when the future of the townsite was in imminent danger, leading to the evacuation of some 80,000 inhabitants. A cohort of firefighters deployed to the fire was established because of concern about persisting effects on the respiratory system and on mental health, the focus of this article. Firefighters, together with other first responders and public safety personnel, are recognized to have an increased risk of mental disorders, particularly post-traumatic stress disorder (PTSD). Berger and colleagues 1 included 10 studies of firefighters in a meta-regression analysis of prevalence of PTSD in rescue workers, in which they estimated an overall prevalence of PTSD of 10% and a prevalence in firefighters of 7.3% (95% confidence interval [CI], 3.6 to 11.0). They also noted that rescue workers exposed to a major disaster had much higher rates than those who were not. This was explored recently in a further systematic review by Wagner and colleagues 2 of studies of mental disorders in firefighters who had been involved in a large-scale disaster. In this review, the median estimate of PTSD prevalence from 20 studies was 10.5%, somewhat higher than the 7.3% reported by Berger et al. 1 for all firefighters. The same review found many fewer studies estimating prevalence of major depression or, particularly, anxiety disorders in firefighters. 2 In a third systematic review, of depressive disorder following disasters, Bonde and colleagues 3 found an increased risk of major depression following natural disasters (odds ratio [OR] = 2.28, 95% CI, 1.30 to 3.98), but none of the studies included focused on firefighters. The current study was designed to estimate the prevalence of PTSD, anxiety, and depression in firefighters deployed to the Fort McMurray fire and to examine cut points that might be useful in defining a case for these disorders in the study population as a whole.
Methods
All firefighters, structural, industrial, and wildland, who were based in Alberta and deployed to the Fort McMurray fire, were eligible for inclusion in the cohort. Firefighters were recruited in two phases. In Phase 1, 13 fire services in the north of Alberta were visited, and firefighters were recruited face-to-face. In Phase 2, structural and industrial firefighters were identified from a “recognition list” of deployed firefighters: Fire chiefs were invited to put forward the names of all firefighters who had taken part, so that they could receive recognition from the premier of the province. Wildland firefighters who had been deployed were identified through Alberta Wildfire, the provincial agency (within the Department of Agriculture) responsible for the management of wildfires. Phase 2 firefighters were invited to join the cohort by completing a recruitment questionnaire online or by telephone. To ensure maximum engagement by firefighters based in the region of the fire, further face-to-face recruitment was carried out in the fall of 2017 and, with wildland firefighters, the spring of 2018. Firefighters signing up for the study gave a detailed account of their experiences during the fire 4 and were also asked whether they would consent to information from the provincial administrative health record for a period of 5 years (three before the fire and two after) being linked to the research record. In Canada, health care is free at the point of service, with physician payment for a consultation requiring registration of at least one diagnosis. Effectively, all residents of Alberta are part of the provincial health care system and, with consent, can be linked for research to this administrative record. The current analysis considered any physician consultation from April 1, 2013, to March 30, 2016, for which a mental health diagnosis was recorded (International Classification of Diseases, Ninth Revision [ICD-9] 290-319; ICD-10 F00-F99) as indicating mental disorders before the fire. All firefighters gave written informed consent to take part in the study.
Firefighters in the cohort were asked to complete a questionnaire on three occasions: on recruitment, in the winter and spring of 2017 to 2018, and again in the winter of 2018 to 2019. At each questionnaire, the firefighters completed the Hospital Anxiety and Depression Scale (HADS). 5,6 At the second follow-up, they also completed the PCL-5, 7 a screening questionnaire for PTSD. In completing it, they were asked to focus on how much, in the last month, they had been bothered by stressful events while they were at the Fort McMurray/Horse River Fire. Structured clinical interviews for DSM-5 (SCID) were carried out between August 2019 and February 2020 to characterize mental disorders in firefighters in the cohort, using the Diagnostic and Statistical Manual for Mental Disorders, fifth edition (DSM-5) criteria. 8 Stratified random samples were taken based on scores on the PCL-5 and the HADS completed at the second follow-up. The sampling scheme was as follows. For each scale independently, we identified all those who scored at or above suggested cut points to identify a case, taking 12 or greater for anxiety and depression on the HADS scales: As there was no generally agreed cut point for the PCL-5, we took a value (28) somewhat lower than that (31) provisionally suggested. 7 Those not meeting these scores but with borderline HADS scores (8 to 11) on the anxiety and depression scale were then identified, and a random sample of 0.3 was selected for inclusion. In addition, 10% of those scoring less than the selection point on all scales (27 or less on PCL-5 and 7 or less on the anxiety and depression scales) were randomly selected.
Structured clinical interviews (SCIs) were carried out, blind to screening scores, by a single nurse (A.M.), with wide experience in both mental health and occupational health settings. She was trained, building on published guidelines
9
to administer the research version of the SCID-5
10
using training materials from Columbia University Department of Psychiatry and the online version developed by TeleSage Inc. The interview was tailored to assess: Current major depressive episode (past month)* Persistent depressive disorder (past 2 years)* Alcohol use and disorder in the past 12 months Cannabis use and disorder in the past 12 months Panic disorder (past month)** Agoraphobia (past and recent 6 months)** Social anxiety (past and recent 6 months)** Specific phobia (past and recent 6 months)** Current generalized anxiety disorder (past 6 months)** Acute stress disorder (for an event occurring in the last month with symptoms in the last month) PTSD (symptoms in the last month) Current adjustment disorder (symptoms in the last 6 months related to a stressor but which do not meet the criteria for acute stress disorder or PTSD).
*Contributed to a diagnosis of any depressive disorder
**Contributed to a diagnosis of any anxiety disorder
The interviewer attempted to reach all firefighters selected for the SCID and thought to be still living in the province. Where possible, interviews were carried out face-to face, with the interviewer traveling to urban centers to increase the feasibility of scheduling appointments. The interviews were conducted in university-based clinical facilities in Edmonton and Calgary and in hotels, veterans’ clubs, or similar facilities elsewhere, but not at fire halls to preserve anonymity. Where numbers in a locality were too few, telephone interviews were used. This included those who, during scheduling, were found to have moved to an adjacent province but expressed a wish to complete the assessment.
Clinical assessments for respiratory ill-health were carried out in parallel to the mental health assessments described here. All firefighters in the cohort were sent a newsletter outlining the plans for these assessments and emphasizing that those both with and without symptoms would be invited to take part. This was followed by a letter advising firefighters of any assessment for which they had been selected and outlining the next steps. Those selected for the mental health assessment were invited to send back consent forms. The interviewer then contacted the participant to arrange a time for the interview. If no consent form was returned by mail, the interviewer emailed or telephoned advising that the consent could be completed online. For those interviewed face-to-face, further consent was obtained on the day of the interview. Where the firefighter agreed, a tape recording was taken for monitoring adherence to protocol. At the end of the interview, the firefighter was invited to complete, online, the Childhood Experience of Care and Abuse Questionnaire. 11 At each face-to-face interview, the firefighter was given information, tailored to the geographic region, on resources available for mental health support. Exceptionally, where in the interviewer identified an acute risk of suicide, rapid contact was made (with consent) with the family physician or other nominated caregiver.
Some weeks after the interview, a letter was sent to the firefighter and, where nominated, their physician, giving the diagnoses (if any) reached during the interview. With this, we included a brief interpretation and again reminded the firefighter where they might seek help.
Statistical Methods
Diagnoses reached during the SCI were reweighted to estimate prevalence in the whole cohort, using general linear modeling specifying a Poisson distribution within Stata Version 14.2 to estimate prevalence of each diagnostic group together with a 95% CI. The weights used were the inverse of the probability of being selected for the SCI, estimated from a logistic regression including the PCL-5, HADS anxiety, and HADS depression scores as continuous variables. The same reweighting was used to calculate receiver operating characteristics (ROC) curves to examine sensitivity and specificity and indicate cut points for “caseness” for later analysis of risk factors within the cohort.
Results
A total of 282 of the 1,000 firefighters completing the screening questionnaires were identified for inclusion in the mental health assessment. Of these, 192 completed the SCID assessment, 131 face-to-face, and 61 by telephone. The median time for the structured interview was 71 minutes (range = 19 to 202 minutes) with additional time for introduction, consent, and wrap-up. The composition of the group and the proportions in which assessment was completed are shown in Table 1. Two-thirds (68.1%) of those selected for inclusion were assessed. Apart from two small groups of firefighters with scores above 11 on the depression scale, the assessment rates were similar across groups. Of the 90 not assessed, 25 refused, 26 could not be contacted, 21 had good contact information but could not be scheduled, and 18 were not contacted because they were known to be living outside Alberta. The median time between the start of the fire (in May 2016) and the assessment was 1,283 days (3.5 years) and between completing the screening questionnaires and the assessment just over 1 year (375 days). Those selected and assessed were similar in age and sex to those not selected (Table 2). There was no significant difference in the proportions for whom a mental health diagnosis had been recorded in the 3 years before the fire, but those selected had higher mean scores on the HADS anxiety and depression scores recorded in the first contact after the fire.
Screening Scores of Those Selected and Assessed by Structural Clinical Interview.
Note. HADS = Hospital Anxiety and Depression Scale; PTSD = posttraumatic stress disorder.
Characteristics of Those Selected and Assessed.
Note. HADS = Hospital Anxiety and Depression Scale.
a HADS baseline scores available for 844 (605 not selected, 71 selected and not assessed, and 168 assessed).
The diagnoses recorded through the SCID for the 192 assessed are given in Table 3. The most frequent diagnosis was PTSD, recorded for 78, any anxiety disorder for 59, a depressive disorder for 55, and any substance use disorder for 45. No diagnosis was reached for 65. Among the 127 with a diagnosis, the number of diagnoses ranged from 1 to 6, with a median of 2. Of 78 meeting the criteria for PTSD, 10 also met the criteria for an anxiety disorder, 15 for a depressive disorder, and 30 for both an anxiety and a depressive disorder.
Diagnosis on SCID-5 among N = 192 Cases Assessed.
Note. Structured clinical interviews for DSM-5; DSM-5 = Diagnostic and Statistical Manual, fifth edition; PTSD = posttraumatic stress disorder.
The estimated prevalences in the cohort were obtained by weighting the observed diagnoses among those assessed (in Table 3) by the inverse of the probability that they would be assessed (Table 4). The estimated weighted prevalences for the whole cohort suggested more than half the cohort (51.9%) would have none of the assessed DSM-5 diagnoses, 21.4% would have PTSD, 15.8% an anxiety disorder, and 14.3% a depressive disorder. Prevalences were also estimated for a substance use disorder (17.3%) and any trauma-related diagnosis (33.9%).
Estimated Prevalence in the Whole Cohort.
Note. N = 998. Probability weighted using the inverse of the probability of being assessed where the odds of assessment = exp(−3.297 + 0.038 × PCL-5 score + 0.153 × anxiety score + 0.095 × depression score). PTSD = post-traumatic stress disorder.
The data were then examined, through ROC analysis, reweighted as above, to derive ROC curves for the whole cohort. These were used to investigate cut points for “caseness” to be used in later analyses. Table 5 shows three possible cut points for each screening scale, one derived from the literature, one from the Youden index 12 (the point at which the greatest proportion of subjects were correctly classified), and a third at the point where the prevalence would be close to that estimated from the SCID. The areas under the curve were 0.70 for PTSD, 0.84 for anxiety disorders, and 0.75 for depressive disorders. The estimated prevalence increased (as determined) with increased sensitivity and decreased with greater specificity. By definition, the highest proportion correctly classified was at the point determined by the Youden index, but prevalence estimates using this “best” cut point were lower than those estimated directly from the SCID.
Test Characteristics for Possible Cut Points with Prevalence Estimates.
Note. PTSD = post-traumatic stress disorder; HADS = Hospital Anxiety and Depression Scale.
Discussion
The study aimed to estimate the prevalence of anxiety, depression, and PTSD in a cohort of firefighters who, some 42 months before the assessment, had been deployed at a devastating and protracted fire in the north of Alberta. The outcome of the SCI suggested a high prevalence (21.4%) of PTSD in this cohort, with somewhat lower but still substantial numbers meeting the DSM-5 criteria for a depressive condition (14.3%) or anxiety disorder (15.8%). To identify fire-related causes of mental disorders in this cohort and to evaluate possible mitigating effects within the organization, it will be necessary to define cases for the whole cohort, accepting that such a diagnosis will be less valid than one obtained from the SCID. The ROC analysis included here illustrates the need for judgment (balancing sensitivity and specificity) in arriving at the most credible definitions if data from brief scales are to be used to estimate or monitor prevalence. Also, it should be noted that, in order to provide a high proportion of correct classification, the Youden-defined optimal cut point for anxiety was higher than the commonly used 12+ cut point for the HADS.
An important limitation of the study was that the screening questionnaires were completed a year before the assessment. Some 30 months had passed between the events at the start of the fire and completing the screening questionnaires, sufficient time for symptoms, if indeed resulting from the fire, to have become fully established. These might be expected to still be present the following year, at the time of the SCID. Nevertheless, events occurring between completing the screening and undergoing the SCID assessment may have masked or exacerbated symptoms. A further limitation is that the use of a single interviewer precluded assessment of interrater reliability.
The prevalence of PTSD (21.4%) found here was somewhat higher than found in the systematic review 2 of mental disorders in firefighters following large-scale disasters. In the 20 studies of PTSD prevalence included in that review, the range was 0.9% to 32.5%, with a median of 10.5%, consistent with the estimate of 11.9% found here using the Youden cut point for the PCL-5. The prevalence in almost all of the studies reviewed was estimated from questionnaire responses, with variation in the cut point used accounting for some of the variance in prevalence estimates. In 7 studies reporting prevalence of depression, the range was 3.8% to 36%, with a median of 19.5%, somewhat higher than that found here. It is important to note that our study assessed current major depressive episodes and persistent depressive disorders. In other words, we assessed active cases, but not recent episodes that had resolved, as are often included in commonly reported annual period prevalence estimates. In 3 studies reporting prevalence of anxiety, the range was 10.2% to 27.2%, consistent with our estimate. Only one study 13 included in this review used a SCID (for the DSM-III). This, reporting on the assessment of a subgroup from a cohort of Australian firefighters, found prevalences similar to those reported here (PTSD 22%, depression 16%, and anxiety 16%) 8 months after the wildfire. The structured clinical interview may be regarded as the “gold standard,” in that the approach is superior to either unstructured or fully structured clinical interview or to the use of a screening questionnaire.
It is of note that the cut point on the PCL-5 derived here for PTSD is much lower than those previously suggested. When the study was designed, it had been shown that a cut point of 31 might be equivalent to an established cut point of 44 on the earlier screening questionnaire 7 and studies in veterans have supported this. 14 A recently reported study of traumatic injuries in civilians 15 suggested very different cut points when those with intentional and unintentional injuries were considered separately, with that for those injured unintentionally (a cut point of 22) being much closer to that found here. The much higher sensitivity and lower specificity associated with the Youden cut point in the current study may in part reflect the unusually long period between completing the screening questionnaire and the SCID assessment. However, the results also support the suggestion of Geier and colleagues 15 that different cut point need to be established for different populations in which this scale is used.
The use of the SCID in the present study was undertaken to add greater certainty to estimates of mental disorders in the cohort of firefighters deployed to the Fort McMurray fire than could be achieved by use of questionnaire studies alone. The prevalence rates observed are higher, particularly for PTSD, than that reported for firefighters without exposures during a major disaster, where estimated rates were 7.5%1 and, in Canada, 13.5%. 16 The high mean scores on the anxiety and depression scales completed soon after the fire in those assessed suggest, in the absence of higher rates of mental disorder before the fire, that at least some part of the ill-health seen in this cohort is attributable to experiences during that disaster. It will be important to demonstrate a relationship between deployment factors and ill-health to support this contention and to identify factors for intervention to mitigate risk in future disasters.
Supplemental Material
Supplemental Material, sj-pdf-1-cpa-10.1177_0706743720974824 - Prevalence of Mental Ill-Health in a Cohort of First Responders Attending the Fort McMurray Fire: Prévalence De Mauvaise Santé Mentale Dans Une Cohorte De Premiers Intervenants Dans L’incendie De Fort Mcmurray
Supplemental Material, sj-pdf-1-cpa-10.1177_0706743720974824 for Prevalence of Mental Ill-Health in a Cohort of First Responders Attending the Fort McMurray Fire: Prévalence De Mauvaise Santé Mentale Dans Une Cohorte De Premiers Intervenants Dans L’incendie De Fort Mcmurray by Nicola Cherry, Jean-Michel Galarneau, Andrea Melnyk and Scott Patten in The Canadian Journal of Psychiatry
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethics Approval
Ethics approval was from the Heath Ethics Board of the University of Alberta (Pro00065284 and Pro89958).
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work reported here was supported by a nonprogram grant from the OHS Futures Program of Alberta Labour and by grants from the Canadian Institutes of Health Research (#162537 and #151027). Dr. Patten is supported by the Cuthbertson and Fischer Chair in Pediatric Mental Health at the University of Calgary.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
