Abstract
Objective:
Following the onset of the Canterbury, New Zealand earthquakes, there were widespread concerns that mental health services were under severe strain as a result of adverse consequences on mental health. We therefore examined Health of the Nation Outcome Scales data to see whether this could inform our understanding of the impact of the Canterbury earthquakes on patients attending local specialist mental health services.
Method:
Health of the Nation Outcome Scales admission data were analysed for Canterbury mental health services prior to and following the Canterbury earthquakes. These findings were compared to Health of the Nation Outcome Scales admission data from seven other large District Health Boards to delineate local from national trends. Percentage changes in admission numbers were also calculated before and after the earthquakes for Canterbury and the seven other large district health boards.
Results:
Admission Health of the Nation Outcome Scales scores in Canterbury increased after the earthquakes for adult inpatient and community services, old age inpatient and community services, and Child and Adolescent inpatient services compared to the seven other large district health boards. Admission Health of the Nation Outcome Scales scores for Child and Adolescent community services did not change significantly, while admission Health of the Nation Outcome Scales scores for Alcohol and Drug services in Canterbury fell compared to other large district health boards. Subscale analysis showed that the majority of Health of the Nation Outcome Scales subscales contributed to the overall increases found. Percentage changes in admission numbers for the Canterbury District Health Board and the seven other large district health boards before and after the earthquakes were largely comparable with the exception of admissions to inpatient services for the group aged 4–17 years which showed a large increase.
Conclusion:
The Canterbury earthquakes were followed by an increase in Health of the Nation Outcome Scales scores for attendees of local mental health services compared to other large district health boards. This suggests that patients presented with greater degrees of psychiatric distress, social disruption, behavioural change and impairment as a result of the earthquakes.
Introduction
This paper explores the impact of the Canterbury, New Zealand earthquakes on routinely collected Health of the Nation Outcome Scales (HoNOS) data. The earthquakes started with a 7.1 magnitude earthquake on 4 September 2010 and continued for more than 2 years with further major earthquakes in February, June and December 2011 and over 10,000 aftershocks. The February 2011 earthquake was the most devastating; it resulted in significant loss of life and multiple injuries (182 people died and 6659 were injured in the initial 24 hours) and widespread damage to property and infrastructure (Ardagh et al., 2012). In addition to the earthquake sequence, there have also been persistent effects on the population over a lengthy period because of infrastructure damage, insurance disputes and delays in rebuilding. These adverse psychosocial effects continue in varying degrees to the present day (approximately 6 years after the start of seismic activity).
In this context, there were widespread concerns that mental health services in Canterbury were facing heightened morbidity as a result of the earthquakes (Humphrey and Renison, 2015; McClure, 2016). Although these concerns are supported by a large number of publications linking natural disasters and earthquakes with adverse mental health consequences (see, for example, Kilic and Ulusoy, 2003; Kuo et al., 2003; Salcioglu et al., 2003), the existing literature on the mental health consequences of natural disasters also has a number of limitations. These have been noted in a series of reviews and include lack of longitudinal study design, failure to make comparisons with suitable controls and poor sample selection (Bonde et al., 2016; Bromet and Dew, 1995; Neria et al., 2008; Rubonis and Bickman, 1991).
Patients utilising psychiatric services represent a unique sub-population that may be more vulnerable to the effects of natural disasters. It is possible that patients receiving psychiatric care have more psychological distress complexity when seeking assistance following a disaster. Previous studies have evaluated severity of symptoms in psychiatric patients and mental health service use following disasters (see, for example, Boscarino et al., 2004; Bromet et al., 1982; Fried et al., 2005; Rosenheck and Fontana, 2003), but the methodological limitations applicable to the general literature on disasters remain relevant when interpreting these studies’ findings.
In Canterbury, a number of longitudinal studies have examined the impact of the earthquakes on the local population. These studies have reported a range of findings following the earthquakes, including no long-term effects following the earthquakes (Beaglehole et al., 2015a, 2015b; Thomson et al., 2015), a weak but statistically significant increase in the risk of moderate or severe mood or anxiety symptoms (Hogg et al., 2016) and small to moderate increases in rates of mental disorders and symptoms following the earthquakes (Fergusson et al., 2014; Spittlehouse et al., 2014). No study to date has evaluated the severity of patients admitted to Specialist Mental Health Services (SMHS) following the earthquakes. However, quantifying the impact of the earthquakes on SMHS is important for service provision reasons particularly as widespread media reporting suggests that the earthquakes have resulted in significant psychological morbidity and strain on services (Humphrey and Renison, 2015; McClure, 2016).
The HoNOS was developed by Wing et al. (1998) to adequately monitor clinical and social improvements in patients with psychiatric illness. It is easy to use, sensitive to change and covers common clinical problems and social functioning. The original HoNOS has spawned a series of related instruments that include the HoNOS for Children and Adolescents (HoNOSCA; Gowers et al., 1999) and the HoNOS for Elderly People (HoNOS65+; Burns et al., 1999). The use of the HoNOS total score for outcome measurement is consistently supported (Speak and Muncer, 2015, 2016), and a review of the psychometric properties of the HoNOS family of measures (Pirkis et al., 2005) concluded that the instruments performed adequately or better on most dimensions.
In addition to the use of overall scales as an outcome tool, the use of HoNOS subscales has also been proposed to further delineate symptomatic, behavioural, impairment and social concerns (Eagar et al., 2005; Gee et al., 2010; Gowers et al., 1999; Pirkis et al., 2005; Preston, 2000; Speak and Muncer, 2015, 2016; Trauer, 1999). Subscales arise from the summing of related items. For example, the social problem subscale for HoNOS includes questions on relationships, activities of daily living, living conditions, occupation and activities. Similarly, the impairment subscale comprises questions on cognitive problems and physical illness or disability. The original subscale structure for HoNOS had four components, although a five-component structure is reported to have better psychometric properties (Eagar et al., 2005; Trauer, 1999) and is used by the Ministry of Health, New Zealand for HoNOS and HoNOS65+, with HoNOSCA remaining a four-factor scale.
The use of the HoNOS scales is mandatory in SMHS in the United Kingdom, Australia and New Zealand. In New Zealand, HoNOS data are available from July 2008; therefore, it predates the onset of the earthquake sequence and provides longitudinal data that allow for the delineation of pre-existing and earthquake-related trends. Data are also collected nationally allowing comparisons to be made with non-earthquake-exposed populations. The availability of longitudinal data predating the earthquake sequence coupled with appropriate control data means that an evaluation of total HoNOS and subscale data is a powerful way to evaluate clinical severity and social factors of patients attending SMHS following the Canterbury earthquakes.
Method
The Ministry of Health New Zealand (2016) routinely collects HoNOS data through its Programme for the Integration of Mental Health Data (PRIMHD). HoNOS data are required to be collected by SMHS staff for all patients entering community or inpatient care. Entry into a service occurs from primary care or when transitioning from another setting of care (e.g. transitioning from community psychiatric care to an inpatient setting). The HoNOS collected at these points in time are designated as admission HoNOS (Beveridge et al., 2015). Admission HoNOS data were provided for HoNOS, HoNOSCA and HoNOS65+ from July 2008 to October 2015. The time period commencing July 2008 and ending August 2010 was designated the pre-earthquake period and the 5 years from September 2010 to August 2015 were designated the post-earthquake period.
Data were requested for the Canterbury District Health Board (CDHB). The CDHB provides health services to slightly over 500,000 people in the city of Christchurch, outlying towns and rural areas. Greater Christchurch (defined as the city of Christchurch and immediate outlying towns within a commuter belt) comprises 85% of the population serviced by the CDHB and was the area most affected by the series of earthquakes (Beaglehole et al., 2015b). Data were also requested for ‘other large DHBs’. These are seven predominantly urban district health boards (DHBs; Bay of Plenty, Capital and Coast, Southern, Waikato, Auckland, Counties Manukau and Waitemata) that in aggregated form constituted the comparison group and have previously been clustered by the Ministry of Health for assessment and analysis purposes. In Canterbury, there are no private inpatient facilities and very limited private community psychiatric coverage. As a consequence, HoNOS data have the potential to capture information on the majority of individuals requiring specialist psychiatric input in the community or inpatient settings.
Statistical analysis
The change in admission mean HoNOS, HoNOSCA and HoNOS65+ scores from the pre-earthquake period to the corresponding post-earthquake period was calculated independently for the CDHB and other large DHBs. These pre- and post-earthquake comparisons were then compared between the CDHB and other large DHBs using a
Data on quarterly admission numbers collected for the HoNOS data set and related completion rates of HoNOS were provided by the Ministry of Health for the CDHB and nationally. Data were provided for admission numbers to the community or inpatient services for the age groups congruent with each of the HoNOS scales (ages 4–17, HoNOSCA; 18–64, HoNOS; and 65+, HoNOS65+). Percentage changes in admission numbers and completion rates were calculated and examined for any relevant trends to assist in providing context to the study findings.
Results
Mean scores for the admission HoNOS scales and Cohen’s
Mean total HoNOS scores before and after the Canterbury earthquakes for the CDHB compared to other large DHBs.
HoNOS: Health of the Nation Outcome Scales; HoNOSCA: HoNOS for Children and Adolescents; HoNOS65+: HoNOS for Elderly People; CDHB: Canterbury District Health Board; DHB: district health board; SD: standard deviation.
Inpatient and community HoNOS
The mean CDHB inpatient (
Alcohol and drug HoNOS
The mean CDHB community Alcohol and Drug HoNOS scores fell compared to other large DHBs following the earthquakes (
Inpatient and community HoNOSCA
The mean CDHB inpatient HoNOSCA increased compared to other large DHBs following the earthquakes (
Inpatient and community HoNOS65+
The mean CDHB inpatient HoNOS65+ increased compared to other large DHBs following the earthquakes (
The mean CDHB community HoNOS65+ increased compared to other large DHBs following the earthquakes (
Figure 1 demonstrates the change that occurred for mean admission HoNOS scores to adult community mental health services for the CDHB and other large DHB group. Adult community mental health services were chosen to demonstrate the observed differences over time because this group comprised the largest population for analysis purposes (mean quarterly admissions pre-earthquake for the CDHB, 1188.86). Similar patterns were present for the other study groups that showed significant changes over the study period (figures available on request).

Mean total HoNOS scores on admission to adult community mental health services.
Effect sizes of the changes
Cohen’s
Admission trends
Percentage changes in admission numbers for the CDHB and the seven other large DHBs before and after the earthquakes were largely comparable with the exception of admissions to inpatient services for the group aged 4–17 years (see Table 2). Admissions for this groups increased by 80.1% for the CDHB following the earthquakes in contrast to a 29.9% increase nationally. Admissions for this age group comprised the lowest numbers of any of the comparisons (mean quarterly admissions pre-earthquake 34.1 compared to 61.5 for the post-earthquake period). Percentage completion of HoNOS scales ranged from 17.8% to 80.6% over the study period. Consistent trends were not apparent, although completion rates for national figures tended to show greater increases following the earthquakes, and CDHB groups either showed a small decrease in completion rates or small improvements.
Admission numbers for HoNOS purposes and completion rates.
HoNOS: Health of the Nation Outcome Scales; CDHB: Canterbury District Health Board.
Discussion
The principal finding of our study is that routinely collected mean admission HoNOS scores (with the exception of that from community Alcohol and Drug services and community Child and Adolescent services) increased for the CDHB compared to seven other large DHBs following the Canterbury earthquakes. Subscale analysis showed that most subscales contributed to the increases as opposed to distinct effects from specific subscales.
The clinical significance of the changes in mean admission HoNOS scores needs to be considered. The range of effect size changes for the HoNOS scores that showed a significant increase was 0.54–2.64, indicating moderate to very large effects (Cohen, 1969). Various markers and methods for establishing clinically significant change in HoNOS for treatment and follow-up purposes have been proposed (Burgess et al., 2009; Parabiaghi et al., 2005; Sharma et al., 1999), but we are unaware of similar benchmarks for studies that track admission HoNOS scores over time. In this study, the HoNOS scales that increased significantly did so by a mean of 2.47 points (range: 1.76–3.01). This can be placed in context by a comparison with typical changes that occur with treatment. In an Australian setting, Goldney et al. (1998) and Boot et al. (1997) showed that HoNOS scores typically improve by six points between admission and discharge in an inpatient setting. In an outpatient sample, Kisely et al. (2010) showed that total HoNOS scores improved by a mean of 2.09 points between baseline and final assessment (approximately 264 days later). These findings suggest that the magnitude of change in HONOS scores following the earthquakes was similar to that found over the course of outpatient care but substantially less than might be observed in the context of inpatient treatment.
A further way to explain the significance of the observed effects is to place the findings in the context of the scales themselves. Any increase in severity of an individual item results in incremental increases in the total score. For example, increasing from a sub-threshold problem to mild problem, or mild problem to moderately severe problem, or moderately severe problem to a severe or very severe problem each results in a one-point increase for the individual item addressed. Therefore, the observed increase in mean total HoNOS scores following the earthquakes could typically be accounted for by at least two individual items worsening in severity by one step or one item worsening by two steps. To summarise, the clinical significance of the increases in mean total HoNOS scores consists of moderate to very large effect sizes and represents increasing complexity and severity of presentations for affected individuals that are likely to be of significance for mental health services over time.
No significant changes for mean total community HoNOSCA were observed over the study period and mean total community Alcohol and Drug services HoNOS scores improved in comparison to other large DHBs following the earthquakes. The large effect size difference for the Alcohol and Drug HoNOS comparison was caused by a significant increase in Alcohol and Drug HoNOS scores for the control group as the CDHB Alcohol and Drug HoNOS scores were relatively stable. This suggests that confounding influences occurring nationally may provide a better explanation for the observed difference. Despite this, Alcohol and Drug HoNOS scores did not worsen for the CDHB following the earthquakes. Other studies have compared alcohol and drug use following a disaster in general populations and found temporary reductions in alcohol consumption (Knudsen et al., 2005) and no significant effects on alcohol/drug abuse and dependence (Fergusson et al., 2014). Our study was not able to provide reasons as to why the community HoNOSCA and Alcohol and Drug services HoNOS data did not show similar effects to the other comparisons, but if these findings continue to be replicated in other settings, there is value in further research to clarify whether or not specific groups respond differently in the aftermath of a disaster.
The subscale analysis was undertaken to complement the examination of the full scales. If overall increases were found in the total HoNOS scores, we were interested in whether or not these were as a result of increases in specific subscales or more globally. However, the results suggest that increases in the majority of subscales contributed to the overall increases found.
Admission numbers showed that increases in admissions for the CDHB were matched by comparable increases nationally. This suggests that the impact of the earthquakes on SMHS was confined to increases in HoNOS severity as opposed to greater numbers of individuals entering care. The large increase in CDHB admissions for inpatients aged 4–17 is an exception that deserves mention. The increase may reflect increasing demand and need for inpatient services in this group following the earthquakes. However, the low base rate of quarterly admissions for this group meant that small variations in numbers resulted in large percentage changes and suggested that caution is important in interpreting the significance of this finding.
An obvious strength of our study is the longitudinal study design and use of appropriate controls. These features have been identified in reviews of the disaster literature as important for improving the quality of disaster research (Bromet and Dew, 1995; Rubonis and Bickman, 1991). The presence of pre-disaster measures allowed initial comparisons to be made to clarify whether or not the HoNOS scores had increased relative to baseline measures. The use of long-term data extending for 5 years after the earthquakes is important because adverse mental health consequences of disasters can be both delayed and long-term (Bryant et al., 2014). The use of a control group for comparison allowed local trends to be delineated from trends occurring for other large DHBs and nationally. The importance of an appropriate control group is exemplified by the admission data to CDHB community services for ages 4–17, 18–65 and 65+. These all increased by 26.7%, 11.4% and 10.4%, respectively. It would be easy to assume that these increases were disaster related, but the national data for the same age groups showed increases of 25.2%, 11.1% and 12.0%, respectively, meaning that earthquake-related admission trends were not identified for this group.
Figure 1 suggests that the impact of the earthquakes may be diminishing by 2016. We considered attempting a three-phase analysis (pre-earthquake, post-earthquake and recovery) to examine for this effect. However, in light of further recent severe earthquakes affecting the Canterbury region, we elected to confine our analysis to a pre- and post-earthquake comparison.
Limitations
A potential limitation of this study is the quality of the HoNOS scales and whether they have the necessary properties to measure clinical symptoms and social functioning in psychiatric patients and be sensitive to change. Despite initial misgivings, these scales have now been studied in a range of inpatient and outpatient settings, and the consensus of reviewers is that the HoNOS scales and subscales have acceptable psychometric properties for the routine monitoring of psychiatric patients (Pirkis et al., 2005; Trauer et al., 1999). In this study, the scales were administered routinely as part of mandated clinical practice and not for research purposes. This means that any changes in scores have not occurred because raters changed their reporting behaviour because of enrolment in a study. Although practitioners completing the scales are likely to have been exposed to the earthquakes themselves, it seems unlikely that systematic changes in reporting of HoNOS after the earthquake scores accounted for the study findings. Similarly, it seems unlikely that the changes in HoNOS scores arose as a consequence of varying completion rates (which typically shifted to a greater degree nationally than for the CDHB), although the potential impact of this and other confounding factors is not able to be quantified by the study design.
Conclusion
In conclusion, we have demonstrated widespread increases in mean admission HoNOS scores for patients attending SMHS following the Canterbury earthquakes. Our research complements other local studies through analysing the impact of the earthquakes on mental health service consumers. The study findings support the general psychiatric literature by documenting further links between natural disasters and adverse mental health consequences. Our findings substantiate some of the concerns raised in the media relating to the impact of the earthquakes on psychiatric services. If reproduced in future disasters, the effect size of the changes represents moderate to very large effects that are likely to be discernible to mental health services, patients and clinicians. Anticipation and planning for these effects by mental health services for future disasters could lessen their impact and improve care for those affected.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
