Establishment of a Disaster-Related Psychological Screening Test

Abstract

Objective: To create a short screening scale for the detection of posttraumatic stress disorder (PTSD) and major depressive disorder (MDE) in earthquake survivors in Taiwan.

Method: Trained research assistants used the Disaster-Related Psychological Screening Test (DRPST) to assess 461 residents of a village that had experienced a major earthquake. The participants were also evaluated by psychiatrists using the Mini-International Neuropsychiatric Interview (MINI). Best subset regression analysis and the receiver operating characteristics curve were used to select a subset of items and cut-off points from the DRPST.

Results: A seven-symptom scale and a three-symptom analogue were selected for PTSD and MDE screening, respectively. Scores of three or more on the PTSD scale and two or more on the MDE scale were used to define a group of positive cases that provide useful information for the patient cohort and will be valuable in long-term follow-up studies of the prevalence of psychiatric diseases following a natural disaster. However, higher scores could also be used to define positive cases under limited psychiatric care resources.

Conclusion: The DRPST, which was administered for phase 1 of this two-phase study, may be used for effective and rapid screening for PTSD and MDE after an earthquake, despite the usual limitations on resources following a disaster.

Although research data concerning the mental health impact of disaster or trauma in many developed countries have been published [1–7], there are fewer reports for East Asia (with the exception of Japan). Earthquakes have caused some of the most devastating natural disasters of the 20th century [1, 8, 9]. Unlike many other catastrophic events, there is no warning, the impact is widespread and effects are ongoing. Several studies have documented the short-term psychological effects of earthquakes [1, 3, 4, 9]. Posttraumatic stress disorder (PTSD) and major depressive episode (MDE) are the most common earthquake-related psychiatric diagnoses, and the occurrence of each is strongly associated with the presence of the other [4, 10, 11].

The devastating Chi-Chi earthquake, registering 7.3 on the Richter scale, struck Taiwan in the early morning of September 21, 1999. It caused severe injury and damage, and the survivors suffered varying degrees of psychological impairment that required systemic investigation. Because the development of an appropriate, structured screening scale is an essential part of psychiatric epidemiology [12], most researchers have developed shortened versions of structured interviews for different targets [13–15]. If human resources and cost effectiveness need to be considered in the design of two-phase surveys, short screening scales are also useful as investigative tools for phase 1 of the intervention [16]. Breslau et al. designed a short screening scale for PTSD [13]. Further, it has been proposed that the use of questions developed for the evaluation of randomly selected traumas results in a short screening scale with characteristics that are representative of the entire range of traumas experienced in the community, rather than just the most severe examples or those associated with the most distressing psychological sequelae [13]. Because the catastrophic earthquake gives no warning, the attending psychiatric team should estimate and evaluate the impact on the victims' psychiatric condition as soon as possible, because the abruptness of the trauma results in more severe psychiatric impact. The main aim of the present research was to construct just such a short and rapidly administered screening test.

Method

Instrument

Five psychiatrists and two public health professionals designed the Disaster-Related Psychological Screening Test (DRPST) to collect background information on residents and check for psychological symptoms resulting from disaster-related psychiatric disorders (including 17 items of PTSD and 9 items of MDE) according to DSM-IV criteria; the questionnaires were designed to account for local language and grammar. The first part of the DRPST is used to compile background information, including date of birth, gender, education, marital status, and extent of the physical injury and economic loss suffered as a result of the earthquake. The second part investigates the psychological symptoms and included 17 items of PTSD and nine items of MDE. Because the DRPST is typically used for the first phase of a twophase survey, items that measure duration and severity are not included because of time limitations and because such items increase the sensitivity of the test.

For this study, psychiatrists also used the Mini-International Neuropsychiatric Interview (MINI, based on 5.0 English version) to assess respondents. Several benefits of the MINI are brevity, simplicity, clarity, increased sensitivity and specificity, and ease of administration [17]. Originally it was developed to provide a short diagnostic structured interview compatible with DSM-IV (which were used for this study) and question formulations are similar to those of the Composite International Diagnostic Interview. The instrument was designed to provide just enough diagnostic information to make good clinical decisions. Given these advantages, the authors feel that the MINI may also prove useful for clinical psychiatry, in addition to its use in research settings, which are very structured [11]. Though it could be used to explore the 17 Axis I diagnoses (according to DSM-IV), priority was given to identification of PTSD and MDE.

Subjects

Tong-Chi village was selected for evaluation for two reasons. First, Tong-Chi is the second largest village in Yu-Chi Township, which is the area of epicentre, and the residents were considered equally exposed to the effects of the earthquake. Second, the area had both rural and urban populations within the residents of Tong-Chi so it could be compared to the entire affected area. Household registrations indicated that the village had a total of 736 residents (357 men and 379 women) aged 16 or over. The mean age for these residents was 51.3 years (range, 16–98 years); 53.0% of the residents were aged 50 or older. Education level was primary school or below for 56.0%. Most were married (77.6%). No gender differences were demonstrated for basic information items. We later learned that the household registrations included 159 persons who had left Tong-Chi village after the earthquake and before our evaluation because of economic problems or lack of academic opportunity (manifested as vacant households).

There were 461 respondents in total (209 men and 252 women), for an overall response rate of 62.6% of the total registered population. However, when the vacant households were considered, the adjusted response rate was 79.9% (461 of 577). The mean age of the respondents was 54.3 years (range, 17–91 years). Education level was primary school or below for 63.7% of the respondent group. Most were married (79.2%). Except for age, no significant differences were demonstrated for other basic information comparing respondents and non-respondents (Table 1). Most of the non-respondents (excluding vacant households) were elderly residents who lived alone, so they worried about being cheated, which is the probable reason that the age of the non-respondents was greater than that of the respondents.

Table 1.

Demographic data for respondents and nonrespondents (excluding vacant household) aged 16 and older

Procedure

Six research assistants underwent a 2 week psychiatric training programme conducted by the psychiatric team. The psychiatric training programme included: (1) use of questionnaires to collect data; (2) ways to introduce the purpose of the study to the residents and establish rapport; (3) an introduction to psychiatric disorders; and (4) demonstrations of the interview technique by psychiatrists using 10 residents of the study village. The research assistants then used the DRPST to rate the same residents, and there was a group discussion of the results. A validation study was subsequently carried out using a subset of the respondent sample; results showed that the percentage of agreement between the structured interviews administered by the research assistants and those given by the psychiatrists were about 82.6% to 98.0%. The psychiatric team also arranged a 1 week training programme to instruct psychiatrists in the use of the Taiwanese version of the MINI. The details of this training programme were provided previously [11].

Twenty-one months after the earthquake, research assistants used the DRPST to collect basic information and evaluate psychological impairment for all residents aged 16 or older. Analogous demographic information for the non-respondents was derived from local government records. At the same time, psychiatrists interviewed these residents using the MINI. Although the rule for administering the MINI states that when the required symptoms for diagnosis of specific diseases were not present, interviewers would not attempt to identify any other symptoms to save time, psychiatrists used the MINI to interview respondents with all items of PTSD and MDE for the evaluation of correlation between psychiatrist-assessed respondents' symptoms and research assistant-assessed respondents' symptoms. During these visits, the purpose of the research project was explained in detail. If residents refused to participate at this point, only their basic information was collected using part I of the DRPST.

There were 275 persons who did not respond, including the 159 residents (21.6%) who had moved to another town for work or study following the earthquake. Of the other non-respondents, 78 refused to participate in the study (10.6%), 37 could not be contacted despite two attempts (5.0%), and 1 had died (0.1%). Frequently given reasons for refusal were that the resident was too busy, was worried about being cheated, felt well, did not like surveys, thought it would not be helpful and, most frequently, that they had no comment. Informed consents were obtained from all respondents, together with approval from the local government authorities and the Department of Health, Republic of China.

Data analysis

Demographic data for total and respondent residents was compared using the χ² test (2 × n nominal data) with Yates' correction (2 × 2 nominal data). The correlation data for the interviews was compared using the Kappa test. The best score was determined on the basis of the predicted probability of a diagnosis of PTSD or MDE, as modelled by logistic regression with conditional forward. Selected symptoms were identified as PTSD and MDE predictors according to the MINI. Seventeen items measuring PTSD criteria symptoms and nine items measuring MDE analogues were included in the analysis. Receiver operating characteristic (ROC) analysis was performed to select the most appropriative score for diagnosis. The set of fitted probabilities was evaluated in terms of sensitivity, specificity, and positive and negative predictive value. All data were analysed using SPSS 7.0 statistical software.

Results

One or more psychiatric symptoms were recorded for 54% of the respondents. For PTSD evaluation, the highest rate was noted for symptom D5, exaggerated startle response (28.0%), and the lowest for C3, amnesia for the trauma (3.0%; Table 2). For MDE evaluation, the highest rate was noted for symptom 4, insomnia or hypersomnia nearly every day (20.8%), whereas the lowest rate was determined for symptom 9, recurrent thoughts of death, recurrent suicidal ideation without a specific plan, or suicide attempt or specific plan for execution (4.3%).

Table 2.

DSM-IV criterion symptoms of posttraumatic stress disorder (PTSD) and major depressive episode (MDE) reported by 461 subjects

Table 3 presents the results of logistic regression with conditional forward for PTSD and MDE diagnoses using the DRPST. A sevensymptom scale was selected for PTSD screening. Three of these symptoms were from the intrusive group, three were from the avoidance and numbing group, and one was from the hyperarousal group. A three-symptom scale was selected for screening MDE. One symptom was from the core group and 2 were from the associated group (described more detail in Table 3).

Table 3.

Logistic regression analysis of predictors for Mini-International Neuropsychiatric Interview-diagnosed posttraumatic stress disorder (PTSD) and major depressive episode (MDE)

The selected sensitivity, specificity, and positive and negative predictive values for cut-off points for the seven-symptom PTSD screening scale and the three-symptom MDE screening analogue are presented in Table 4. A score of 3 or more on the PTSD scale was used to define the positive cases; this resulted in the most appropriate sensitivity (97.8%) and specificity (96.6%), a positive predictive value of 76.3%, and a negative predictive value of 99.8%. A score of 2 or more on the MDE scale was used to define positive cases of MDE, giving a sensitivity of 92.1%, specificity of 98.3%, positive predictive value of 83.3%, and negative predictive value of 99.3%. When a score of 4 or more on the PTSD scale was used to define the positive cases, the sensitivity decreased to 76.1% and positive predictive value increased to 97.2%. When a score of 3 on the MDE scale was used to define the positive cases, the sensitivity decreased to 60.5% and positive predictive value increased to 100.0%.

Table 4.

Validity of the Disaster Related Psychological Screening Test using MINI-diagnosis of posttraumatic stress disorder (PTSD) and major depressive episode (MDE) as the standard

Discussion

The development of screening scales has been an essential part of the growth of psychiatric epidemiology [12]. Because the catastrophic earthquake hit suddenly and without warning, different degrees of psychological impairment have been noted for most survivors [1, 2, 8–11]. Psychiatric research on disasters may offer important information with respect to the prevalence, natural course, co-occurrence and risk factors for psychiatric disorders among survivors [18, 19]. The psychiatric team visited the Tong-Chi residents as soon as practicable after the earthquake. However, the appropriateness of the mental health services provided to the affected persons also needs systematic assessment [11]. Such an evaluation should include assessment of the psychiatric manpower requirement for optimal mental health rehabilitation after a disaster, which means the psychological condition of survivors should be assessed as soon as possible after the event. The short screening test is useful in surveys using a two-phase design where a quick screening test is used to select a subset of respondents who are likely to have a disorder (phase 1) for more intensive diagnostic assessment in the next stage (phase 2) [13, 16]. Thus, we designed the DRPST for phase 1 screening of earthquake victims.

Because the effect of the base rate of a disorder on its diagnostic detection is the factor that is often of most importance to researchers, the clinician should have an appreciation of the effect of this rate on the power of assessment instruments [20]. The positive predictive value of a diagnostic test incorporates both sensitivity and specificity. An ideal test has a positive predictive value of 1, with a sensitivity of 1 (no false-negative results) and specificity of 1 (no false-positive results). When the base rate of a disorder is low, application of Bayesian theory predicts that the target condition will not be present. At low prevalence rates, the positive predictive value shrinks because a large number of subjects scoring positive for the disorder are actually falsepositives. Thus, when the PTSD prevalence is low, the rate of identification of true cases is also low [21].

In our previous paper [11], a PTSD rate of 7.9% and an MDE rate of 9.5% were determined for the survivors. Therefore, because the prevalence of postearthquake PTSD and MDE is not high, their positive predictive value is not high. A score of 3 on the seven-symptom screening scale was identified as the optimal cut-off point for PTSD diagnosis. For MDE diagnosis, a score of 2 on the three-symptom screening scale was identified as the optimal cut-off. These diagnostic cut-offs were selected to minimize the probability of false-negative results for PTSD and MDE, at the expense of a somewhat raised probability of false-positive results. This trade-off is particularly suitable for two-phase surveys in which the first phase is designed to maximize the inclusion of true cases of PTSD or MDE, and the second phase is expected to reclassify those who were wrongly assigned. Other more advantageous cut-off points may be determined for different applications of PTSD and MDE screening scales. Although the screening scale is not a substitute for a psychiatric diagnosis [13], it may be applied to identify suspected cases of PTSD and MDE (phase 1) with these individuals then selected as a fixed cohort to be followed up after a catastrophic event.

If an analysis of cost effectiveness is to have an enduring impact, it is critical that the measurement of unit costs and their ultimate translation into the cost of episodes of care are accurate, true, meaningful and valid [22]. Although research investigating the short-term psychological effects of earthquakes has revealed an elevated prevalence of psychological problems [4, 5], most of the residents of Tong-Chi, like other Taiwanese, had worried that they would be viewed as ‘mad’ if they sought psychiatric help. In these circumstances, it is difficult to identify the appropriate types of mental health care services that are needed. Most Taiwanese have the stereotypical view of a psychiatric disorder as a ‘stigma’, and it is difficult for them to accept that they or their family members have psychiatric problems. If too many false-positive results are defined using the PTSD scale, people will be less willing to accept treatment, and it will take psychiatrists too much time to explain the reasons for false-positive cases.

Taiwan does not have an adequate number of psychiatrists. Thus, to supply psychiatric services to most of the survivors, it was necessary to use a screening instrument like the DRPST for phase 1 of a two-stage survey of the large number of survivors who may have suffered psychological impairment. A score of 3 (not 4) or more on the PTSD scale and a score of 2 (not 3) or more on the MDE scale was used to define the positive cases; this resulted in both higher sensitivity and higher negative predictive value. However, for most individuals, changes in feelings, thoughts, behaviours and biologic state are mostly transitory after a catastrophe. Analysis of preliminary results revealed that most respondents (67%–80%) had one or more symptoms, and these results produced a higher false-positive rate.

Andrews showed that determining the effectiveness of treatments requires that psychiatrists demonstrate the treatment's benefit through the use of outcome measurement [23]. When we take the limited amount of psychiatric manpower into consideration, a higher score (e.g. 4 or more on PTSD) should be used to define positive cases, which will decrease the number of false-positive results and make the best use of the available psychiatric treatment resources. Although we would not identify some real cases in phase 1 of the two-stage survey (in the current study, a decrease of 21.7% on PTSD and 31.6% on MDE), we will probably be able to persuade more subjects to undergo treatment because of the higher probability of their having actual psychiatric disorders. We could then use an alternative method (e.g. home visits by public nurses) to find those with false-negative results who still need treatment.

Given the resource limitations (especially psychiatric manpower) that exist at the beginning of an earthquake, in those circumstances we would also select subjects who had higher scores as positive cases in the DRPST for phase 1 screening of earthquake victims. However, in the long term follow-up stage, we would select appropriate lower cut-off points to screen subjects and provide information useful for the patient cohort and for the long-term follow-up study of the prevalence of psychiatric diseases.

Conclusion

Faced with the impact of a catastrophic earthquake, and given the resource limitations and burden on respondents inherent to a disaster of this magnitude, the DRPST, applied for phase 1 of this two-stage study, as it is an effective and rapidly administered test for PTSD and MDE.