An evaluation of Health of the Nation Outcome Scales data to inform psychiatric morbidity following the Canterbury earthquakes

Abstract

Objective:

Following the onset of the Canterbury, New Zealand earthquakes, there were widespread concerns that mental health services were under severe strain as a result of adverse consequences on mental health. We therefore examined Health of the Nation Outcome Scales data to see whether this could inform our understanding of the impact of the Canterbury earthquakes on patients attending local specialist mental health services.

Method:

Health of the Nation Outcome Scales admission data were analysed for Canterbury mental health services prior to and following the Canterbury earthquakes. These findings were compared to Health of the Nation Outcome Scales admission data from seven other large District Health Boards to delineate local from national trends. Percentage changes in admission numbers were also calculated before and after the earthquakes for Canterbury and the seven other large district health boards.

Results:

Admission Health of the Nation Outcome Scales scores in Canterbury increased after the earthquakes for adult inpatient and community services, old age inpatient and community services, and Child and Adolescent inpatient services compared to the seven other large district health boards. Admission Health of the Nation Outcome Scales scores for Child and Adolescent community services did not change significantly, while admission Health of the Nation Outcome Scales scores for Alcohol and Drug services in Canterbury fell compared to other large district health boards. Subscale analysis showed that the majority of Health of the Nation Outcome Scales subscales contributed to the overall increases found. Percentage changes in admission numbers for the Canterbury District Health Board and the seven other large district health boards before and after the earthquakes were largely comparable with the exception of admissions to inpatient services for the group aged 4–17 years which showed a large increase.

Conclusion:

The Canterbury earthquakes were followed by an increase in Health of the Nation Outcome Scales scores for attendees of local mental health services compared to other large district health boards. This suggests that patients presented with greater degrees of psychiatric distress, social disruption, behavioural change and impairment as a result of the earthquakes.

Keywords

Disasters earthquakes mental health HoNOS

Introduction

This paper explores the impact of the Canterbury, New Zealand earthquakes on routinely collected Health of the Nation Outcome Scales (HoNOS) data. The earthquakes started with a 7.1 magnitude earthquake on 4 September 2010 and continued for more than 2 years with further major earthquakes in February, June and December 2011 and over 10,000 aftershocks. The February 2011 earthquake was the most devastating; it resulted in significant loss of life and multiple injuries (182 people died and 6659 were injured in the initial 24 hours) and widespread damage to property and infrastructure (Ardagh et al., 2012). In addition to the earthquake sequence, there have also been persistent effects on the population over a lengthy period because of infrastructure damage, insurance disputes and delays in rebuilding. These adverse psychosocial effects continue in varying degrees to the present day (approximately 6 years after the start of seismic activity).

In this context, there were widespread concerns that mental health services in Canterbury were facing heightened morbidity as a result of the earthquakes (Humphrey and Renison, 2015; McClure, 2016). Although these concerns are supported by a large number of publications linking natural disasters and earthquakes with adverse mental health consequences (see, for example, Kilic and Ulusoy, 2003; Kuo et al., 2003; Salcioglu et al., 2003), the existing literature on the mental health consequences of natural disasters also has a number of limitations. These have been noted in a series of reviews and include lack of longitudinal study design, failure to make comparisons with suitable controls and poor sample selection (Bonde et al., 2016; Bromet and Dew, 1995; Neria et al., 2008; Rubonis and Bickman, 1991).

Patients utilising psychiatric services represent a unique sub-population that may be more vulnerable to the effects of natural disasters. It is possible that patients receiving psychiatric care have more psychological distress complexity when seeking assistance following a disaster. Previous studies have evaluated severity of symptoms in psychiatric patients and mental health service use following disasters (see, for example, Boscarino et al., 2004; Bromet et al., 1982; Fried et al., 2005; Rosenheck and Fontana, 2003), but the methodological limitations applicable to the general literature on disasters remain relevant when interpreting these studies’ findings.

In Canterbury, a number of longitudinal studies have examined the impact of the earthquakes on the local population. These studies have reported a range of findings following the earthquakes, including no long-term effects following the earthquakes (Beaglehole et al., 2015a, 2015b; Thomson et al., 2015), a weak but statistically significant increase in the risk of moderate or severe mood or anxiety symptoms (Hogg et al., 2016) and small to moderate increases in rates of mental disorders and symptoms following the earthquakes (Fergusson et al., 2014; Spittlehouse et al., 2014). No study to date has evaluated the severity of patients admitted to Specialist Mental Health Services (SMHS) following the earthquakes. However, quantifying the impact of the earthquakes on SMHS is important for service provision reasons particularly as widespread media reporting suggests that the earthquakes have resulted in significant psychological morbidity and strain on services (Humphrey and Renison, 2015; McClure, 2016).

The HoNOS was developed by Wing et al. (1998) to adequately monitor clinical and social improvements in patients with psychiatric illness. It is easy to use, sensitive to change and covers common clinical problems and social functioning. The original HoNOS has spawned a series of related instruments that include the HoNOS for Children and Adolescents (HoNOSCA; Gowers et al., 1999) and the HoNOS for Elderly People (HoNOS65+; Burns et al., 1999). The use of the HoNOS total score for outcome measurement is consistently supported (Speak and Muncer, 2015, 2016), and a review of the psychometric properties of the HoNOS family of measures (Pirkis et al., 2005) concluded that the instruments performed adequately or better on most dimensions.

In addition to the use of overall scales as an outcome tool, the use of HoNOS subscales has also been proposed to further delineate symptomatic, behavioural, impairment and social concerns (Eagar et al., 2005; Gee et al., 2010; Gowers et al., 1999; Pirkis et al., 2005; Preston, 2000; Speak and Muncer, 2015, 2016; Trauer, 1999). Subscales arise from the summing of related items. For example, the social problem subscale for HoNOS includes questions on relationships, activities of daily living, living conditions, occupation and activities. Similarly, the impairment subscale comprises questions on cognitive problems and physical illness or disability. The original subscale structure for HoNOS had four components, although a five-component structure is reported to have better psychometric properties (Eagar et al., 2005; Trauer, 1999) and is used by the Ministry of Health, New Zealand for HoNOS and HoNOS65+, with HoNOSCA remaining a four-factor scale.

The use of the HoNOS scales is mandatory in SMHS in the United Kingdom, Australia and New Zealand. In New Zealand, HoNOS data are available from July 2008; therefore, it predates the onset of the earthquake sequence and provides longitudinal data that allow for the delineation of pre-existing and earthquake-related trends. Data are also collected nationally allowing comparisons to be made with non-earthquake-exposed populations. The availability of longitudinal data predating the earthquake sequence coupled with appropriate control data means that an evaluation of total HoNOS and subscale data is a powerful way to evaluate clinical severity and social factors of patients attending SMHS following the Canterbury earthquakes.

Method

The Ministry of Health New Zealand (2016) routinely collects HoNOS data through its Programme for the Integration of Mental Health Data (PRIMHD). HoNOS data are required to be collected by SMHS staff for all patients entering community or inpatient care. Entry into a service occurs from primary care or when transitioning from another setting of care (e.g. transitioning from community psychiatric care to an inpatient setting). The HoNOS collected at these points in time are designated as admission HoNOS (Beveridge et al., 2015). Admission HoNOS data were provided for HoNOS, HoNOSCA and HoNOS65+ from July 2008 to October 2015. The time period commencing July 2008 and ending August 2010 was designated the pre-earthquake period and the 5 years from September 2010 to August 2015 were designated the post-earthquake period.

Data were requested for the Canterbury District Health Board (CDHB). The CDHB provides health services to slightly over 500,000 people in the city of Christchurch, outlying towns and rural areas. Greater Christchurch (defined as the city of Christchurch and immediate outlying towns within a commuter belt) comprises 85% of the population serviced by the CDHB and was the area most affected by the series of earthquakes (Beaglehole et al., 2015b). Data were also requested for ‘other large DHBs’. These are seven predominantly urban district health boards (DHBs; Bay of Plenty, Capital and Coast, Southern, Waikato, Auckland, Counties Manukau and Waitemata) that in aggregated form constituted the comparison group and have previously been clustered by the Ministry of Health for assessment and analysis purposes. In Canterbury, there are no private inpatient facilities and very limited private community psychiatric coverage. As a consequence, HoNOS data have the potential to capture information on the majority of individuals requiring specialist psychiatric input in the community or inpatient settings.

Statistical analysis

The change in admission mean HoNOS, HoNOSCA and HoNOS65+ scores from the pre-earthquake period to the corresponding post-earthquake period was calculated independently for the CDHB and other large DHBs. These pre- and post-earthquake comparisons were then compared between the CDHB and other large DHBs using a Z-test. The magnitude of the difference in the changes between the CDHB and the other large DHB group were then calculated using Cohen’s (1969) d effect sizes. If significant differences in the changes were identified between the CDHB and the other large DHB comparison, identical analyses were undertaken using the sub-scores from the scales to determine which components were contributing to the differences. Data were examined for inpatients, general community, and alcohol and drug community populations.

Data on quarterly admission numbers collected for the HoNOS data set and related completion rates of HoNOS were provided by the Ministry of Health for the CDHB and nationally. Data were provided for admission numbers to the community or inpatient services for the age groups congruent with each of the HoNOS scales (ages 4–17, HoNOSCA; 18–64, HoNOS; and 65+, HoNOS65+). Percentage changes in admission numbers and completion rates were calculated and examined for any relevant trends to assist in providing context to the study findings.

Results

Mean scores for the admission HoNOS scales and Cohen’s d scores for the effect size of the difference in the changes between the CDHB and other large DHBs following the earthquakes are provided in Table 1.

Table 1.

Mean total HoNOS scores before and after the Canterbury earthquakes for the CDHB compared to other large DHBs.

HoNOS scale	Mean CDHB pre-earthquakes (SD)	Mean CDHB post-earthquakes (SD)	Mean other large DHBs pre-earthquakes (SD)	Mean other large DHBs post-earthquakes (SD)	Cohen’s d effect size for the difference in the change (p value)
HoNOS (inpatient)	14.32 (0.65)	14.80 (0.84)	15.05 (0.80)	13.50 (0.33)	2.10 (<0.01)
HoNOS (community)	9.72 (0.71)	11.74 (0.96)	10.18 (0.46)	9.75 (0.30)	2.64 (<0.01)
HoNOS (Alcohol and Drug)	10.70 (3.32)	10.31 (3.09)	7.79 (1.69)	10.83 (2.89)	−0.86 (<0.01)
HoNOSCA (inpatient)	20.64 (2.29)	18.74 (2.87)	20.89 (1.62)	17.23 (2.33)	0.54 (<0.01)
HoNOSCA (community)	15.77 (0.91)	14.79 (1.09)	14.88 (0.71)	13.90 (0.49)	0.00 (1.00)
HoNOS65+ (inpatient)	15.08 (2.24)	14.5 (1.47)	15.62 (1.32)	13.68 (0.96)	0.61 (<0.01)
HoNOS65+ (community)	8.58 (0.88)	10.65 (1.64)	12.20 (0.73)	11.17 (0.46)	2.08 (<0.01)

HoNOS: Health of the Nation Outcome Scales; HoNOSCA: HoNOS for Children and Adolescents; HoNOS65+: HoNOS for Elderly People; CDHB: Canterbury District Health Board; DHB: district health board; SD: standard deviation.

Inpatient and community HoNOS

The mean CDHB inpatient (Z-difference, 15.25; p < 0.01) and mean community (Z-difference, 22.42; p < 0.01) HoNOS increased compared to other large DHBs following the earthquakes. All subscales contributed significant increases to the overall effect.

Alcohol and drug HoNOS

The mean CDHB community Alcohol and Drug HoNOS scores fell compared to other large DHBs following the earthquakes (Z-difference, −7.14; p < 0.01). This finding was largely caused by an increase in mean HoNOS scores for the seven other large DHBs, while the mean CDHB score remained relatively stable.

Inpatient and community HoNOSCA

The mean CDHB inpatient HoNOSCA increased compared to other large DHBs following the earthquakes (Z-difference, 4.86; p < 0.01). All subscales contributed significantly to the overall effect with the exception of the impairment problems subscale which did not show a significant change (Z-difference, 1.25; p = 0.21). The mean CDHB community HoNOSCA did not change significantly following the earthquakes compared to other large DHBs (Z-difference, 0.00; p = 1.00).

Inpatient and community HoNOS65+

The mean CDHB inpatient HoNOS65+ increased compared to other large DHBs following the earthquakes (Z-difference, 4.03; p < 0.01). All subscales contributed significantly to the overall effect, with the exception of the delusions/hallucinations subscale HoNOS65+ which reduced (Z-difference, −2.74; p = 0.01).

The mean CDHB community HoNOS65+ increased compared to other large DHBs following the earthquakes (Z-difference, 20.39; p < 0.01). All subscales contributed significant increases to the overall effect.

Figure 1 demonstrates the change that occurred for mean admission HoNOS scores to adult community mental health services for the CDHB and other large DHB group. Adult community mental health services were chosen to demonstrate the observed differences over time because this group comprised the largest population for analysis purposes (mean quarterly admissions pre-earthquake for the CDHB, 1188.86). Similar patterns were present for the other study groups that showed significant changes over the study period (figures available on request).

Figure 1.

Mean total HoNOS scores on admission to adult community mental health services.

Effect sizes of the changes

Cohen’s d effect sizes quantifying the extent of the differences in the changes between the CDHB and the other large DHBs are provided for the HoNOS scales (see Table 1). The effect size changes for the total scales varied from −0.86 (p < 0.01) (community Alcohol and Drug services) to a magnitude of 2.64 (p < 0.01) for community HoNOS data.

Admission trends

Percentage changes in admission numbers for the CDHB and the seven other large DHBs before and after the earthquakes were largely comparable with the exception of admissions to inpatient services for the group aged 4–17 years (see Table 2). Admissions for this groups increased by 80.1% for the CDHB following the earthquakes in contrast to a 29.9% increase nationally. Admissions for this age group comprised the lowest numbers of any of the comparisons (mean quarterly admissions pre-earthquake 34.1 compared to 61.5 for the post-earthquake period). Percentage completion of HoNOS scales ranged from 17.8% to 80.6% over the study period. Consistent trends were not apparent, although completion rates for national figures tended to show greater increases following the earthquakes, and CDHB groups either showed a small decrease in completion rates or small improvements.

Table 2.

Admission numbers for HoNOS purposes and completion rates.

	Mean quarterly admissions pre-quakes	Mean quarterly admissions post-quakes	Mean change admissions (%)	Mean percentage HoNOS scale completed pre-quakes (%)	Mean percentage HoNOS scale completed post-quakes (%)	Percentage change in completion of family of HoNOS (%)
CDHB 18–64 Community	1188.9	1324.2	11.4	39.8	40.7	2.2
National 18–64 Community	10,545.0	11,713.5	11.1	17.8	27.4	54.1
CDHB 18–64 inpatient	383.3	374.8	−2.2	37.9	52.4	38.5
National 18–64 inpatient	2541.4	2583.7	1.7	40.2	63.1	56.9
CDHB 4–17 Community	397.7	504.1	26.7	43.2	39.9	−7.6
National 4–17 Community	3537.0	4427.3	25.2	28.7	42.4	47.6
CDHB 4–17 Inpatient	34.1	61.5	80.1	80.6	72.5	−10.0
National 4–17 Inpatient	178.0	231.3	29.9	44.1	54.8	24.3
National 65+ Community	508.4	561.4	10.4	19.0	18.7	−1.4
National 65+ Community	2003.1	2244.4	12.0	27.8	34.3	23.1
CDHB 65+ Inpatient	99.4	109.8	10.4	43.1	59.8	38.8
National 65+ Inpatient	311.1	344.1	10.6	41.5	60.2	44.9

HoNOS: Health of the Nation Outcome Scales; CDHB: Canterbury District Health Board.

Discussion

The principal finding of our study is that routinely collected mean admission HoNOS scores (with the exception of that from community Alcohol and Drug services and community Child and Adolescent services) increased for the CDHB compared to seven other large DHBs following the Canterbury earthquakes. Subscale analysis showed that most subscales contributed to the increases as opposed to distinct effects from specific subscales.

The clinical significance of the changes in mean admission HoNOS scores needs to be considered. The range of effect size changes for the HoNOS scores that showed a significant increase was 0.54–2.64, indicating moderate to very large effects (Cohen, 1969). Various markers and methods for establishing clinically significant change in HoNOS for treatment and follow-up purposes have been proposed (Burgess et al., 2009; Parabiaghi et al., 2005; Sharma et al., 1999), but we are unaware of similar benchmarks for studies that track admission HoNOS scores over time. In this study, the HoNOS scales that increased significantly did so by a mean of 2.47 points (range: 1.76–3.01). This can be placed in context by a comparison with typical changes that occur with treatment. In an Australian setting, Goldney et al. (1998) and Boot et al. (1997) showed that HoNOS scores typically improve by six points between admission and discharge in an inpatient setting. In an outpatient sample, Kisely et al. (2010) showed that total HoNOS scores improved by a mean of 2.09 points between baseline and final assessment (approximately 264 days later). These findings suggest that the magnitude of change in HONOS scores following the earthquakes was similar to that found over the course of outpatient care but substantially less than might be observed in the context of inpatient treatment.

A further way to explain the significance of the observed effects is to place the findings in the context of the scales themselves. Any increase in severity of an individual item results in incremental increases in the total score. For example, increasing from a sub-threshold problem to mild problem, or mild problem to moderately severe problem, or moderately severe problem to a severe or very severe problem each results in a one-point increase for the individual item addressed. Therefore, the observed increase in mean total HoNOS scores following the earthquakes could typically be accounted for by at least two individual items worsening in severity by one step or one item worsening by two steps. To summarise, the clinical significance of the increases in mean total HoNOS scores consists of moderate to very large effect sizes and represents increasing complexity and severity of presentations for affected individuals that are likely to be of significance for mental health services over time.

No significant changes for mean total community HoNOSCA were observed over the study period and mean total community Alcohol and Drug services HoNOS scores improved in comparison to other large DHBs following the earthquakes. The large effect size difference for the Alcohol and Drug HoNOS comparison was caused by a significant increase in Alcohol and Drug HoNOS scores for the control group as the CDHB Alcohol and Drug HoNOS scores were relatively stable. This suggests that confounding influences occurring nationally may provide a better explanation for the observed difference. Despite this, Alcohol and Drug HoNOS scores did not worsen for the CDHB following the earthquakes. Other studies have compared alcohol and drug use following a disaster in general populations and found temporary reductions in alcohol consumption (Knudsen et al., 2005) and no significant effects on alcohol/drug abuse and dependence (Fergusson et al., 2014). Our study was not able to provide reasons as to why the community HoNOSCA and Alcohol and Drug services HoNOS data did not show similar effects to the other comparisons, but if these findings continue to be replicated in other settings, there is value in further research to clarify whether or not specific groups respond differently in the aftermath of a disaster.

The subscale analysis was undertaken to complement the examination of the full scales. If overall increases were found in the total HoNOS scores, we were interested in whether or not these were as a result of increases in specific subscales or more globally. However, the results suggest that increases in the majority of subscales contributed to the overall increases found.

Admission numbers showed that increases in admissions for the CDHB were matched by comparable increases nationally. This suggests that the impact of the earthquakes on SMHS was confined to increases in HoNOS severity as opposed to greater numbers of individuals entering care. The large increase in CDHB admissions for inpatients aged 4–17 is an exception that deserves mention. The increase may reflect increasing demand and need for inpatient services in this group following the earthquakes. However, the low base rate of quarterly admissions for this group meant that small variations in numbers resulted in large percentage changes and suggested that caution is important in interpreting the significance of this finding.

An obvious strength of our study is the longitudinal study design and use of appropriate controls. These features have been identified in reviews of the disaster literature as important for improving the quality of disaster research (Bromet and Dew, 1995; Rubonis and Bickman, 1991). The presence of pre-disaster measures allowed initial comparisons to be made to clarify whether or not the HoNOS scores had increased relative to baseline measures. The use of long-term data extending for 5 years after the earthquakes is important because adverse mental health consequences of disasters can be both delayed and long-term (Bryant et al., 2014). The use of a control group for comparison allowed local trends to be delineated from trends occurring for other large DHBs and nationally. The importance of an appropriate control group is exemplified by the admission data to CDHB community services for ages 4–17, 18–65 and 65+. These all increased by 26.7%, 11.4% and 10.4%, respectively. It would be easy to assume that these increases were disaster related, but the national data for the same age groups showed increases of 25.2%, 11.1% and 12.0%, respectively, meaning that earthquake-related admission trends were not identified for this group.

Figure 1 suggests that the impact of the earthquakes may be diminishing by 2016. We considered attempting a three-phase analysis (pre-earthquake, post-earthquake and recovery) to examine for this effect. However, in light of further recent severe earthquakes affecting the Canterbury region, we elected to confine our analysis to a pre- and post-earthquake comparison.

Limitations

A potential limitation of this study is the quality of the HoNOS scales and whether they have the necessary properties to measure clinical symptoms and social functioning in psychiatric patients and be sensitive to change. Despite initial misgivings, these scales have now been studied in a range of inpatient and outpatient settings, and the consensus of reviewers is that the HoNOS scales and subscales have acceptable psychometric properties for the routine monitoring of psychiatric patients (Pirkis et al., 2005; Trauer et al., 1999). In this study, the scales were administered routinely as part of mandated clinical practice and not for research purposes. This means that any changes in scores have not occurred because raters changed their reporting behaviour because of enrolment in a study. Although practitioners completing the scales are likely to have been exposed to the earthquakes themselves, it seems unlikely that systematic changes in reporting of HoNOS after the earthquake scores accounted for the study findings. Similarly, it seems unlikely that the changes in HoNOS scores arose as a consequence of varying completion rates (which typically shifted to a greater degree nationally than for the CDHB), although the potential impact of this and other confounding factors is not able to be quantified by the study design.

Conclusion

In conclusion, we have demonstrated widespread increases in mean admission HoNOS scores for patients attending SMHS following the Canterbury earthquakes. Our research complements other local studies through analysing the impact of the earthquakes on mental health service consumers. The study findings support the general psychiatric literature by documenting further links between natural disasters and adverse mental health consequences. Our findings substantiate some of the concerns raised in the media relating to the impact of the earthquakes on psychiatric services. If reproduced in future disasters, the effect size of the changes represents moderate to very large effects that are likely to be discernible to mental health services, patients and clinicians. Anticipation and planning for these effects by mental health services for future disasters could lessen their impact and improve care for those affected.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

References

Ardagh

Richardson

Robinson

et al . (2012) The initial health-system response to the earthquake in Christchurch, New Zealand, in February, 2011. The Lancet 379: 2109–2115.

Beaglehole

Bell

Beveridge

et al . (2015a) Psychiatric admissions fall following the Christchurch earthquakes: An audit of inpatient data. Australian and New Zealand Journal of Psychiatry 49: 346–350.

Beaglehole

Bell

Frampton

et al . (2015b) The impact of the Canterbury earthquakes on prescribing for mental health. Australian and New Zealand Journal of Psychiatry 49: 742–750.

Beveridge

Papps

Bower

et al . (2015) National outcomes collection: Mental health outcomes information collection protocol (ICP). Available at: https://www.tepou.co.nz/uploads/files/resource-assets/mental-health-outcomes-information-collection-protocol-v2_2-june-2015.pdf (accessed 2 November 2016).

Bonde

Utzon-Frank

Bertelsen

et al . (2016) Risk of depressive disorder following disasters and military deployment: Systematic review with meta-analysis. British Journal of Psychiatry 208: 330–336.

Boot

Hall

Andrews

(1997) Disability, outcome and case-mix in acute psychiatric in-patient units. British Journal of Psychiatry 171: 242–246.

Boscarino

Galea

Adams

et al . (2004) Mental health service and medication use in New York City after the September 11, 2001, terrorist attack. Psychiatric Services 55: 274–283.

Bromet

Dew

(1995) Review of psychiatric epidemiologic research on disasters. Epidemiologic Reviews 17: 113–119.

Bromet

Schulberg

Dunn

(1982) Reactions of psychiatric patients to the Three Mile Island nuclear accident. Archives of General Psychiatry 39: 725–730.

10.

Bryant

Waters

Gibbs

et al . (2014) Psychological outcomes following the Victorian Black Saturday bushfires. Australian and New Zealand Journal of Psychiatry 48: 634–643.

11.

Burgess

Pirkis

Coombs

(2009) Modelling candidate effectiveness indicators for mental health services. Australian and New Zealand Journal of Psychiatry 43: 531–538.

12.

Burns

Beevor

Lelliott

et al . (1999) Health of the Nation Outcome Scales for elderly people (HoNOS 65+). British Journal of Psychiatry 174: 424–427.

13.

Cohen

(1969) Statistical Power Analysis for the Behavioral Sciences. New York: Academic Press.

14.

Eagar

Trauer

Mellsop

(2005) Performance of routine outcome measures in adult mental health care. Australian and New Zealand Journal of Psychiatry 39: 713–718.

15.

Fergusson

Horwood

Boden

et al . (2014) Impact of a major disaster on the mental health of a well-studied cohort. JAMA Psychiatry 71: 1025–1031.

16.

Fried

Domino

Shadle

(2005) Use of mental health services after hurricane Floyd in North Carolina. Psychiatric Services 56: 1367–1373.

17.

Gee

Croucher

Beveridge

(2010) Measuring outcomes in mental health services for older people: An evaluation of the Health of the Nation Outcome Scales for elderly people (HoNOS65+). International Journal of Disability, Development and Education 57: 155–174.

18.

Goldney

Fisher

Walmsley

(1998) The Health of the Nation Outcome Scales in psychiatric hospitalisation: A multicentre study examining outcome and prediction of length of stay. Australian and New Zealand Journal of Psychiatry 32: 199–205.

19.

Gowers

Harrington

Whitton

et al . (1999) Brief scale for measuring the outcomes of emotional and behavioural disorders in children. Health of the Nation Outcome Scales for children and Adolescents (HoNOSCA). British Journal of Psychiatry 174: 413–416.

20.

Hogg

Kingham

Wilson

et al . (2016) Spatio-temporal variation of mood and anxiety symptom treatments in Christchurch in the context of the 2010/11 Canterbury earthquake sequence. Spatial and Spatio-temporal Epidemiology 19: 91–102.

21.

Humphrey

Renison

(2015) Earthquake stress triggers mental health issues. Christchurch: Fairfax Media: The Press, 25 May. Available at: http://www.stuff.co.nz/the-press/opinion/68769392/Earthquake-stress-triggersmental-health-issues.

22.

Kilic

Ulusoy

(2003) Psychological effects of the November 1999 earthquake in Turkey: An epidemiological study. Acta Psychiatrica Scandinavica 108: 232–238.

23.

Kisely

Campbell

Cartwright

et al . (2010) Do the Health of the Nation Outcome Scales measure outcome? Canadian Journal of Psychiatry 55: 431–439.

24.

Knudsen

Roman

Johnson

et al . (2005) A changed America? The effects of September 11th on depressive symptoms and alcohol consumption. Journal of Health and Social Behavior 46: 260–273.

25.

Kuo

Tang

Tsay

et al . (2003) Prevalence of psychiatric disorders among bereaved survivors of a disastrous earthquake in Taiwan. Psychiatric Services 54: 249–251.

26.

McClure

(2016) Christchurch dilemmas: Christchurch’s mental health crisis. Christchurch: Fairfax Media: The Press, 15 August. Available at: http://www.stuff.co.nz/national/health/83124349/christchurch-dilemmas-christchurchs-mental-health-crisis.

27.

Ministry of Health New Zealand (2016) PRIMHD – Mental health data. Available at: www.health.govt.nz/nz-health-statistics/national-collections-and-surveys/collections/primhd-mental-health-data

28.

Neria

Nandi

Galea

(2008) Post-traumatic stress disorder following disasters: A systematic review. Psychological Medicine 38: 467–480.

29.

Parabiaghi

Barbato

D’Avanzo

et al . (2005) Assessing reliable and clinically significant change on Health of the Nation Outcome Scales: Method for displaying longitudinal data. Australian and New Zealand Journal of Psychiatry 39: 719–725.

30.

Pirkis

Burgess

Kirk

et al . (2005) A review of the psychometric properties of the Health of the Nation Outcome Scales (HoNOS) family of measures. Health and Quality of Life Outcomes 3: 76.

31.

Preston

(2000) The Health of the Nation Outcome Scales: Validating factorial structure and invariance across two health services. Australian and New Zealand Journal of Psychiatry 34: 512–519; discussion 520–511.

32.

Rosenheck

Fontana

(2003) Use of mental health services by veterans with PTSD after the terrorist attacks of September 11. American Journal of Psychiatry 160: 1684–1690.

33.

Rubonis

Bickman

(1991) Psychological impairment in the wake of disaster: The disaster-psychopathology relationship. Psychological Bulletin 109: 384–399.

34.

Salcioglu

Basoglu

Livanou

(2003) Long-term psychological outcome for non-treatment-seeking earthquake survivors in Turkey. Journal of Nervous and Mental Disease 191: 154–160.

35.

Sharma

Wilkinson

Fear

(1999) Health of the Nation Outcome Scales: A case study in general psychiatry. British Journal of Psychiatry 174: 395–398.

36.

Speak

Muncer

(2015) The structure and reliability of the Health of the Nation Outcome Scales. Australasian Psychiatry 23: 66–68.

37.

Speak

Muncer

(2016) Factorial structure of the Health of the Nation Outcome Scales: An ordinal confirmatory factor analysis using a national sample of clinician ratings in England. International Journal of Mental Health Nursing 25: 87–98.

38.

Spittlehouse

Joyce

Vierck

et al . (2014) Ongoing adverse mental health impact of the earthquake sequence in Christchurch, New Zealand. Australian and New Zealand Journal of Psychiatry 48: 756–763.

39.

Thomson

Seers

Frampton

et al . (2015) Sequential population study of the impact of earthquakes on the emotional and behavioural well-being of 4-year-olds in Canterbury, New Zealand. Journal of Paediatrics and Child Health 52: 18–24.

40.

Trauer

(1999) The subscale of the Health of the Nation Outcome Scales (HoNOS). Journal of Mental Health 8: 499–509.

41.

Trauer

Callaly

Hantz

et al . (1999) Health of the Nation Outcome Scales: Results of the Victorian field trial. British Journal of Psychiatry 174: 380–388.

42.

Wing

Beevor

Curtis

et al . (1998) Health of the Nation Outcome Scales (HoNOS): Research and development. British Journal of Psychiatry 172: 11–18.