Sage Journals: Discover world-class research

Abstract

Objective:

The Health of the Nation Outcome Scales was developed as an overall measure of mental health, applicable to the complete range of psychiatric disorders. Meanwhile, it is a benchmark tool for service providers and is also used for the allocation of costs and funding. The ability of the Health of the Nation Outcome Scales to assess and differentiate the severity of psychiatric disorders is largely unknown, as it is the interpretation of a change in score. We aim to establish Health of the Nation Outcome Scales cut-off and benchmark values for severity and improvement – respectively change, using equipercentile linking to the Clinical Global Impression scales.

Methods:

In a clinical sample of 30,616 individuals with a psychiatric disorder, we used a multivariate regression analysis to determine the correlation between the scales and possible confounders. We used an equipercentile linking analysis of the Clinical Global Impressions severity scale with the Health of the Nation Outcome Scales sum score to establish cut-off values for severity. The linking of the Health of the Nation Outcome Scales sum score difference and the percentage of change to the Clinical Global Impression improvement scale determined benchmark values for change (i.e. improvement or deterioration).

Results:

The Health of the Nation Outcome Scales and Clinical Global Impression scales showed a Spearman correlation of 0.38 (p < 0.000). Clinical Global Impression–Severity: ‘borderline-ill’ corresponded to Health of the Nation Outcome Scales score 3–5; ‘mildly ill’ to 6–10; ‘moderately ill’ to 11–16; ‘markedly ill’ to 17–25; ‘severely ill’ to 26–35; and ‘extremely ill’ to a score ⩾36. The Spearman correlation between the percentage change of the Health of the Nation Outcome Scales was 0.39 (p > 0.000); Clinical Global Impression–Improvement: ‘minimally improved’ corresponded to Health of the Nation Outcome Scales reduction of 4 points or 9%; ‘much-improved’ to 12 points or 48%; ‘very-much-improved’ to ⩾20 points or ⩾90%. Clinical Global Impression–Improvement: ‘minimally worse’ corresponded to an increase of 6 points or 25%; ‘much-worse’ to 12 points or 54%; and finally, ‘very-much-worse’ to >18 points or ⩾82%.

Conclusion:

Our results allow for the comparison of severity and response to treatment of patients with a psychiatric disorder, independent of diagnosis.

Keywords

Health of the Nation Outcome Scales Clinical Global Impression equipercentile analysis severity rating cut-off values

Introduction

The Health of the Nation Outcome Scales (HoNOS) was developed nearly 25 years ago; with the aim of providing a brief, accurate and relevant measure of overall mental health and social functioning; which would be applicable to the complete range of psychiatric disorders (Wing et al., 1994, 1996). Since its first publication, the psychometric properties and the performance of the HoNOS have been the object of continuous study, demonstrating both good reliability and validity, as well as clinical utility (James et al., 2018; Theodoridou et al., 2011). As a result, it has become a well-established measurement instrument and is primarily used to assess the outcome of psychiatric treatment (Delaffon et al., 2018; James et al., 2018). In the meantime, it is available in several languages (Delaffon et al., 2018; Harris et al., 2018), and in some countries, the HoNOS is a mandatory outcome measure (Delaffon et al., 2018; Speak et al., 2015).

Despite a large number of publications and the profound implications of its use, the ability of the HoNOS to assess and differentiate the severity of psychiatric disorders is still unchartered (Harris et al., 2018). This is attributable to several factors, mainly the methodological and statistical intricacy of comparing heterogeneous groups across multiple domains (Gift et al., 1980; Zimmerman et al., 2018). The interpretation of change in the HoNOS sum score can be arduous, due to the construction of the scale, where each item measures a separate issue (Boon et al., 2019; Egger et al., 2015; Harris et al., 2018; Parabiaghi et al., 2014). Clinical change may, therefore, be captured by a single item, with sub-threshold changes in other items potentially cancelling this out (MacDonald, 2009).

Like the HoNOS, the Clinical Global Impression (CGI) Scales were developed as ‘universal’ measurement instruments (Guy, 1976). They are easy to use and intuitively understood, therefore widely used in research and clinical practice (Berk et al., 2008; Busner and Targum, 2007). The CGI scales extract the raters’ judgement on the severity of illness of a given subject, considering the raters’ clinical experience within the particular diagnosis (Guy, 1976). The CGI scales serve as a reference for the definition of boundaries and cut-off values for several other psychometric instruments (Egger et al., 2019; Leucht et al., 2005, 2006).

We consider that staging of severity and change are crucial aspects for the further use and interpretation of the HoNOS. Therefore, using equipercentile linking analysis of the HoNOS to the CGI scales, we aim to develop HoNOS cut-off scores for severity from one single measurement and benchmark values for change in severity (i.e. improvement or deterioration) whenever two HoNOS measurements are available.

Methods

Sample and procedure

The Department of Psychiatry, Psychotherapy and Psychosomatics as part of the Psychiatric University Hospital of Zurich is responsible for the treatment of adult patients (aged 18–65) in the city of Zurich and its surroundings, with a catchment area of approximately 500,000 inhabitants. For each admission (and discharge), a series of psychometric measures to assess severity and need for treatment is mandatory. We used data from a full 6-year (2013–2018) cohort of consecutive admissions for treatment at our department; only patients admitted and discharged in the study period were considered.

Ethical considerations

The data collection is part of the routine quality control and clinical care procedures; accordingly, previous trial registration is therefore not required. At the time of data collection, no specific written informed consent was obtained from patients. Following current ethical standards and in compliance with current legislation, a protocol and data analysis plan were submitted to the competent Authorities. The Ethics Committee of the Canton of Zurich authorised the further use of the anonymised data for research and publication purposes (BASEC: 2018-01906).

Diagnosis and diagnostic groups

Psychiatric diagnoses were made according to the International Classification of Diseases, 10th edition (ICD-10) (World Health Organization (WHO), 1992) criteria by the clinician responsible for treatment and were confirmed by a senior psychiatrist. The diagnoses of the sample were divided into 16 diagnostic categories according to the ICD-10 classification: dementia (DEM) (ICD-10: F00-F03); neurocognitive disorders (NCD) (ICD-10: F0X; the ‘X’ denotes the remaining diagnostic codes in the respective category); alcohol use disorders (AUD) (ICD-10: F10); substance use disorders (SUD) (ICD-10: F11-F19); schizophrenia (SZP) (ICD-10: F20); transient psychotic disorder (TPD) (ICD-10: F23); schizoaffective disorder (SZA) (ICD-10: F25); schizophrenia spectrum disorder (SZS) (ICD-10: F2X); bipolar disorder (BPD) (ICD10: F30-F31); major depressive disorder (MDD) (ICD-10: F32); stress-response disorders (SRD) (ICD-10: F43); somatoform and dissociative disorders (SDD) (ICD-10: F44-F45); anxiety disorders (AXD) (ICD-10: F4X); body image disorders (BID) (ICD-10: F5); personality disorders (PPD) (ICD-10: F6) and neurodevelopmental disorders (NDD) (ICD-10: F7-F9). The diagnostic clustering was aimed to have diagnostic homogeneous groups, with a large enough sub-sample size.

Rating scales

The CGI scales (Guy, 1976) are widespread in research and clinical practice. They are brief, easy to use and pragmatic measurement tools applicable to the entire spectrum of psychiatric disorders. The CGI consists of three subscales: (1) Severity of Illness (CGI-S), (2) Global Improvement (CGI-I) and (3) Efficacy Index (CGI-E); the latter will not be used for the current analyses. CGI-S and CGI-I have a 7-point Likert-type scale response format. The CGI-S ranges from 1 representing ‘normal’ to 7 ‘extremely ill’. The CGI-I ranges from 1 ‘very much improved’ to 7 ‘very much worse’, whereby a score of 4 indicates no change. Ratings of CGI-S refer to the past week and ratings of CGI-I to the time elapsed since a previous (or first) CGI-S assessment.

The HoNOS is an observer-rated scale to assess the severity of a psychiatric disorder (Wing et al., 1994, 1996). It consists of 12 items: (1) ‘Overactive, aggressive, disruptive or agitated behaviour’, (2) ‘Non-accidental self-injury’, (3) ‘Problem-drinking or drug-taking’, (4) Cognitive problems’, (5) ‘Physical illness or disability problems’, (6) ‘Problems associated with hallucinations and delusions’, (7) ‘Problems with depressed mood’, (8) ‘Other mental and behavioural problems’ (including phobic; anxiety; obsessive-compulsive; mental strain/tension; dissociative; somatoform; eating; sleep; sexual and other), (9) ‘Problems with relationships’, (10) ‘Problems with activities of daily living’, (11) ‘Problems with living conditions’ and (12) ‘Problems with occupation and activities’. Each item is rated on a 5-point Likert-type scale from 0 ‘no problem’ to 4 ‘severe to very severe problem’ (Wing et al., 1998, 1999).

The HoNOS has four predefined dimensions: ‘behaviour’, ‘impairment’, ‘symptoms’ and ‘social problems’. Evaluation can occur on an item, a subscale and total score (Wing et al., 1998, 1999). The usual observation period embraces 14 days. However, analogous to the CGI scales, we also used a 7-day observation period for the HoNOS. Previous findings suggest that a reduction in the observation time frame does not confound the evaluation (Luo et al., 2016). Since its original publication, the scales have undergone a subtle review (James et al., 2018). For this study, we used the German HoNOS translation (Andreas et al., 2007).

Raters and rating

Raters were clinicians: psychiatrists, psychiatry residents or clinical psychologists. Training in the use and objectives of the measurement instruments was conducted in workshops, following to a standard schedule using examples and case vignettes. Trainers were available for consultation, as well as regular refresher sessions. Relevant information for the rating derives from clinical interview and behavioural observation, together with information provided by nursing staff, social workers and significant others. All measures were rated retrospectively for the 7 days prior to the assessment; measurements at admission were conducted within the first 72 hours; measurements at discharge on the day of discharge, at the latest 1-week post-discharge.

Statistical analyses

The analysis included only subjects with complete CGI and HoNOS data sets at baseline and discharge; HoNOS with three or more missing items were considered incomplete (Harris et al., 2018; James et al., 2018). To avoid an overlap of the time frame of evaluation, for the analysis of change, we excluded patients discharged the first week after admission. Demographic and clinical characteristics of the whole sample use descriptive statistics.

To begin with we conducted a multiple regression analysis to determine the correlation between the scales used for equipercentile analysis (CGI and HoNOS) and demographics (gender, age, education and marital status) together with clinical variables (primary diagnosis). Spearman’s rank coefficient determines the pairwise correlations between both scales. Agreement between measurement instruments, was assessed using the visual inspection method of a Bland–Altman plot analysis (Bland and Altman, 1986); of the mean differences and the 95% agreement interval (Giavarina, 2015).

Finally, we performed an equipercentile linking analysis to compare the HONOS with the CGI scales. Equipercentile linking is a non-parametric method where corresponding cut-off points on two different scales are determined, by identifying scores on both measures which have the same percentile rank (González and Wiberg, 2017). It does not require a specific distribution type and allows for possible measurement errors on both scales; for this reason, it is considered the preferred linking method (Wee et al., 2018).

In the first stage, the distribution of the scales of interest was generated; subsequently the distributions were log-linear smoothed, using the Kernel smoothing method for mixed data types, whereby the variables were ‘polished’ using observations close to a shifting target (González and Wiberg, 2017; Hastie et al., 2009). After smoothing, the corresponding percentiles of the scales were matched pairwise in the following manner: the CGI-S to the HoNOS sum score, with a CGI-S score of 0 (‘Normal’) and an HoNOS sum score of zero (‘0’) as anchor scores. The CGI-I score of 4 (‘no change’) anchored to the HoNOS difference score and the percentage change of zero (‘0’).

In a post hoc analysis; variables found to have a statistically significance interaction in the multivariate analysis will be manually removed using a leave-one-out method. The influence of the variables on both the Spearman rank correlation and the equipercentile analysis was recorded. A chi-square goodness-of-fit analysis compares the original CGI scales classification with the severity and change categories derived from the HoNOS cut-off scores and benchmark values.

The statistical software R (v4.0.2) was used for analysis; for multivariate regression with the package ‘np’ (v0.60-9) and for equipercentile calculations with the package ‘equate’ (v2.0.7).

Results

For the analysis, we collected data from 31,759 patients admitted for treatment at our department over a 6-year period (2013–2018). After removing 1143 (3.60%) patients due to incomplete diagnosis, CGI or HoNOS items, the sample size for analysis included 30,616 patients. The mean age was 45.10 ± 17.39 years, with 47.15% of females, 55.85% were single, 17.64% were married, 20.63% were either separated or divorced, and 4.02% were widowed. 22.99% had completed regular education, 34.52% had completed an apprenticeship and 14.95% had a college or university degree. 87.63% were either native speakers or spoke fluent German. The majority of admissions were voluntary (72.05%), with 27.50% compulsory admissions. The mean length of stay was 27.52 ± 28.47, with a right skew distribution and a median length of stay of 19 days.

Main diagnostic groups are MDD (5975, 19.52%), AUD (5414, 17.68%), SZP (4804, 15.69%), SRD (3029, 9.89) and PPD (2250, 7.35%) accounting together for over two thirds of patients. The remaining diagnostic groups are distributed as follows: SUD (1926, 6.29%), BPD (1461, 4.77%), SZA (1378, 4.50%), DEM (1069; 3.49%), NCD (924, 3.02%), TPD (748; 2.44%), AXD (560. 1.83%), NDD (411, 1.34%), SZS (406, 1.33%), SFD (175, 0.57%) and BID (86; 0.28%). Further demographic and clinical characteristics of the sample are summarised in Table 1 and Figure 1. There was no statistical difference regarding demographic characteristics between the sample included for analysis and patients excluded due to missing items (data not shown).

Table 1.

Demographic and clinical characteristics of the sample population.

Variable	n = 30,616
Age in years, mean (SD)	45.10 (17.39)
Female, n (%)	14,435 (47.15)
Marital status, n (%)
Single	17,100 (55.85)
Married	5399 (17.64)
Separated	934 (3.05)
Divorced	5382 (17.58)
Widowed	1232 (4.02)
Unknown	569 (1.86)
Education, n (%)
Regular education	7041 (22.99)
Apprenticeship	10,569 (34.52)
College/University	4577 (14.95)
Unknown	8431 (27.54)
German proficiency
Native/fluent	26,828 (87.63)
Difficult	2829 (9.24)
Interpreter needed	959 (3.13)
Admission status, n (%)
Voluntary	22,196 (72.50)
Compulsory	8420 (27.50)
Length of stay in days, mean (SD)	27.52 (28.47)
CGI-S at admission (SD)	4.98 (0.98)
CGI-I at discharge (SD)	2.59 (0.96)
HoNOS at admission (SD)	18.77 (7.61)
HoNOS at discharge (SD)	13.83 (8.18)

SD: standard deviation; CGI-S: Clinical Global Impression–Severity; CGI-I: Clinical Global Impression–Improvement; HoNOS: Health of the Nation Outcome Scales.

Figure 1.

Distribution of the CGI Severity (left, blues) and CGI Improvement (right, greens) gradings among diagnostic groups. DEM: dementia (1069; 3.49%); NCD: neuro-cognitive disorder (924; 3.02%); AUD: alcohol use disorder (5414; 17.68%); SUD: substance use disorder (1926; 6.29%); SZP: schizophrenia (4804; 15.69%); TPD: transitory psychotic disorder (748; 2.44%); SZA: schizoaffective disorder (1378; 4.50%); SZS: schizophrenia spectrum disorders (406; 1.33%); BPD: bipolar disorder (1461; 4.77%); MDD: major depressive disorder (5975; 19.52%); SRD: stress-related disorders (3029; 9.89%); SFD: somatoform disorders (175; 0.57%); AXD: anxiety (560; 1.83%); BID: body image disorder (86; 0.28%); PPD: personality disorder (2250; 7.35%); and NDD: neuro-developmental disorders (411; 1.34%).

The CGI-S score at admission was 4.98 ± 0.98 with a left skew distribution; the CGI-I score at discharge was 2.59 ± 0.96 with a right skew distribution. For analysis, we calculated the HoNOS sum score, as well as the difference score and the percentage of change between admission and discharge. The mean HoNOS sum score at admission was 18.77 ± 7.61 and 13.83 ± 8.18 at discharge, with a difference score of 5.72 ± 7.61 and a percentage of change of 26.90 ± 40.01.

Multiple regression analysis

The HoNOS and the CGI-S at admission have a Spearman rank correlation of 0.38 (p < 0.000). The multiple regression analysis found a statistically significant interaction between gender (female); diagnosis (DEM, NCD, SUD, SZP, TPD, SZA, SZS, MDD, SRD, PPD and NDD); education level (college/university) and language proficiency (difficult). The HoNOS difference score and the percentage of change have a Spearman rank correlation with the CGI-I of 0.39 (p < 0.000). The multiple regression analysis also found a statistically significant interaction between diagnosis (BPD, MDD, SZA, TPD, SRD and PPD) and education level (apprenticeship and college/university).

In the post hoc analysis, we manually removed variables with a potential interaction using a leave-one-out method. The correlation between the scales of interest (i.e. HoNOS and CGI) remained practically unchanged. Furthermore, the equipercentile analysis controlling for these variables showed no variation other than rounding differences. The afterwards performed chi-square goodness-of-fit test comparing the proportions of the HoNOS derived a comparable allocation than the CGI-S (χ² (df = 36, n = 30,616) = 46, p = 0.23) and the CGI-I (χ² (df = 28, n = 22,796) = 24, p = 0.26) categorisation.

Equipercentile linking CGI-S and HoNOS sum score

The cut-off severity values for the HoNOS sum scores are given in Table 2. The CGI-S score of 2 (‘borderline ill’) corresponded to an HoNOS sum score between 3 and 5. A CGI-S score of 3 (‘mildly ill’) corresponded to a sum score between 6 and 10. The CGI-S score of 4 (‘moderately ill’) corresponded to a sum score between 10 and 16. The CGI-S score of 5 (‘markedly ill’) corresponded to a sum score between 17 and 25. The CGI-S score of 6 (‘severely ill’) corresponded to a sum score between 26 and 35. Finally, a CGI-S score of 7 (‘extremely ill’) corresponded to a sum score of 36 or more (see Figure 2).

Table 2.

Linking of the HoNOS to the CGI scales.

Severity		Change (improvement-deterioration)
CGI S	HoNOS sum score	CGI-I	HoNOS difference score	HoNOS percentage of change
Normal		Very much improved	(−)⩽20	(−)⩽90%
Borderline	3 (0.06)	Much improved	(−)⩽12	(−)⩽48%
Mildly ill	6 (0.13)	Minimally improved	(−)⩽4	(−)⩽9%
Moderately ill	11 (0.23)	No change
Markedly ill	17 (0.35)	Minimally worse	(+)⩾6	(+)⩾25%
Severely ill	26 (0.54)	Much worse	(+)⩾12	(+)⩾54%
Extremely ill	36 (0.75)	Very much worse	(+)⩾18	(+)⩾82%

CGI-S: Clinical Global Impression–Severity; CGI-I: Clinical Global Impression–Improvement; HoNOS: Health of the Nation Outcome Scales.

Cut-off scores for severity gradings (proportion of the maximal score); and benchmark values for change (improvement or deterioration).

Figure 2.

Equipercentile linking of the HoNOS sum-score and the CGI-S. The corresponding CGI-S values: 1: ‘normal’; 2: ‘borderline ill’; 3: ‘mildly ill’; 4: ‘moderately ill’; 5: ‘markedly ill’; 6: ‘severely ill’; and 7: ‘extremely ill’.

Equipercentile linking CGI-I with the HoNOS difference score and percentage of change

The benchmarks derived from the linking of the CGI-I and the HoNOS sum score and percentage of change are summarised in Table 2. The corresponding CGI-I gradings for the HoNOS sum score are as follows: patients with a decrease of 4–11 points (or 9–48%) correspond to ‘minimally improved’ (CGI-I score of 3). Patients with a sum score decrease of between 12 and 19 points (or 49–89%) were considered as ‘much improved’ (CGI-I score of 2) and a decrease over 20 points (or greater than 90%) considered as ‘very much improved’ (CGI-I score of 1). Patients with a sum score increase between 6 and 11 points (or 25–54%) were considered as ‘minimally worse’ (CGI-I score 5). Patients with a sum score increase of 12–17 points (or 55–82%) were considered ‘much worse’ (CGI-I score of 6). Those with a sum score increase over 18 points (or greater than 83%) were considered ‘very much worse’ (CGI-I score of 7). Finally, those with a decrease of 3 points (or less than 8%) or an increase of 5 points (or less than 24%) defined as ‘unchanged’ (CGI-I score of 4).

Discussion

The clinical sample used for our analysis included almost the complete spectrum of mental and substance use disorders present in the adult population (Whiteford et al., 2013; WHO, 1992). Our results, therefore, improve the interpretation of HoNOS scores in clinical practice. Using equipercentile linking of the HoNOS with the CGI scales, we have generated cut-off scores and benchmark values for change according to clinical judgement. This is of importance, considering the use of the HoNOS as a mandatory measurement instrument, as well as a benchmark tool for quality assessment and resource allocation (Delaffon et al., 2018; Speak et al., 2015).

The multiple regression analysis demonstrated a statistically significant interaction with several variables, especially some diagnostic categories. The performed post hoc analysis with a leave-one-out method did not alter either cut-off scores or benchmark values, beyond rounding differences. Therefore, we consider that the interactions found in the multiple regression analysis are an artefact (due to the large sample size) without clinical relevance (Marsh et al., 1988; Sullivan and Feinn, 2012). These findings, strengthen the notion that the HoNOS is suitable as a trans-diagnostic scale (Delaffon et al., 2018; James et al., 2018; Wing, 2000; Wing et al., 1998) and encouraging its use as a basic psychometric instrument and benchmark tool (Delaffon et al., 2018; James et al., 2018; Kisely et al., 2010).

For our analysis, we employed the HoNOS sum score. It allows both the direct comparison of an individual patient with others, as with prior measurements. Alternative approaches for evaluation have been proposed, including a categorical change model (Crawford et al., 2017) and alternative factor structures (Speak and Muncer, 2016). We consider that the sum score is able to reflect the complex and dynamic interplays between psychopathology, behaviour and psychosocial circumstances, as well as compensatory and adaptation mechanisms. These qualities can be lost when the items or subscales are dichotomised (Kortrijk et al., 2019; Schmittmann et al., 2013; Vazquez-Bourgon et al., 2019), also affecting the latent structure of the HoNOS, potentially leading to the divergent results reported to date (Bender, 2019; James et al., 2018; Preston, 2000; Smith et al., 2017; Trauer, 1999).

Our findings, especially the cut-off scores and benchmark values, reciprocally corroborate and validate previous results, obtained in different clinical samples, with other statistical methods (Boon et al., 2019; De Beurs et al., 2018; Parabiaghi et al., 2005, 2011, 2014). The percentage of change for improvement (treatment response) and deterioration determined in our analysis are in accordance with generally accepted thresholds (Duff, 2012; Evans et al., 1998; Schennach-Wolff et al., 2010). Since the last two items measure the social environment and living conditions, some authors suggest omitting these items when analysing the HoNOS, especially during hospitalisation (Bech et al., 2003; Pirkis et al., 2005; Trauer, 1999). A subsequent analysis, leaving the last two items of the HoNOS, leads to similar results as for the complete scale.

The main limitation of our results remains the low correlation between the HoNOS and the CGI-scales (Mukaka, 2012; Schober et al., 2018). We consider that the differences in the construction of the scales, mainly the ability of the CGI to capture several subjective aspects of severity, likewise functioning and well-being (Beneke and Rasmus, 1992; Berk et al., 2008; Leon et al., 1993) explain this finding. Nonetheless, we consider precisely this capacity of the CGI adds to its flexibility and uniqueness as a reference tool. Despite the rather low correlation, the agreeability of both scales is visualised in the Bland–Altman plot (Mukaka, 2012), yielding an agreement within the 95% mean difference between HoNOS and CGI (Bland and Altman, 1986; Giavarina, 2015). Through the equipercentile analysis, especially the smoothing procedure flaws in correlation can be compensated (González and Wiberg, 2017).

A peculiarity of our sample remains a large number of compulsory admissions for treatment. Previous studies suggest this is a phenomenon related to the singularities of the legal regulations in the Canton of Zurich (Dorothee, 2013). A post hoc analysis revealed no statistically significant difference between voluntary and compulsory admissions to our department regarding severity according to CGI and sum score of the HoNOS; cut-off values were also similar.

The main strength of our study is the large and naturalistic sample population, with representative diagnostic categories resembling clinical practice. This allows for robust statistical analysis, overcoming the limitations mentioned above. The post hoc analysis, using the leave-one-out method, ruled out distortions by clinical or demographic characteristics. The corroboratory analysis showed an overlapping allocation and distribution of the severity and change categories, additionally demonstrating their validity. Nonetheless, further research is required on how clinical change and severity modifies correlations between diagnoses, symptoms, behaviour and psychosocial circumstances as determined by the HoNOS (Barbalat et al., 2019; Schmittmann et al., 2013).

Conclusion

The HoNOS is a valid and reliable scale for the measurement of global severity of psychiatric disorders. The performance of the HoNOS sum score is similar across both the severity spectrum and psychiatric diagnostic categories. The inclusion and the exclusion of factors external to the patient do not appear to influence the performance of the HoNOS regarding the assessment of severity and change. The availability of a global severity index facilitates the comparison of patients, both with other patients and their own treatment outcomes.

Footnotes

Acknowledgements

The authors thank Lorna McBroom for language editing and proofreading.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Stephan T Egger

Data Availability

The protocol presented and approved by the ethics committee does not foresee the sharing of data. Following a reasonable request, the corresponding author will submit an amendment to the ethics committee asking for approval of data sharing in that particular case.

References

Andreas

Harfst

Dirmaier

, et al. (2007) A psychometric evaluation of the German version of the ‘Health of the Nation Outcome Scales, HoNOS-D’: On the feasibility and reliability of clinician-performed measurements of severity in patients with mental disorders. Psychopathology 40: 116–125.

Barbalat

van den Bergh

Kossakowski

(2019) Outcome measurement in mental health services: Insights from symptom networks. BMC Psychiatry 19: 202.

Bech

Bille

Schutze

, et al. (2003) Health of the Nation Outcome Scales (HoNOS): Implementability, subscale structure and responsiveness in the daily psychiatric hospital routine over the first 18 months. Nordic Journal of Psychiatry 57: 285–290.

Bender

(2019) The meagre outcomes of HoNOS. Australasian Psychiatry. Epub ahead of print 10/1. DOI: 10.1177/1039856219875066.

Beneke

Rasmus

(1992) ‘Clinical Global Impressions’ (ECDEU): Some critical comments. Pharmacopsychiatry 25: 171–176.

Berk

Dodd

, et al. (2008) The validity of the CGI severity and improvement scales as measures of clinical effectiveness suitable for routine clinical use. Journal of Evaluation in Clinical Practice 14: 979–983.

Bland

Altman

(1986) Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet 1: 307–310.

Boon

de Boer

SBB

van Dorp

, et al. (2019) Reliable and clinically significant change based on the Health of the Nation Outcome Scales. Psychiatry Research 281: 112587.

Busner

Targum

(2007) The clinical global impressions scale: Applying a research tool in clinical practice. Psychiatry 4: 28–37.

10.

Crawford

Zoha

Macdonald

AJD

, et al. (2017) Improving the quality of mental health services using patient outcome data: Making the most of HoNOS. BJPsych Bulletin 41: 172–176.

11.

De Beurs

Blankers

Delespaul

, et al. (2018) Treatment results for severe psychiatric illness: Which method is best suited to denote the outcome of mental health care? BMC Psychiatry 18: 225.

12.

Delaffon

Anwar

Noushad

, et al. (2018) Use of Health of the Nation Outcome Scales in psychiatry. Advances in Psychiatric Treatment 18: 173–179.

13.

Dorothee

(2013) Hohe Einweisungsraten im Kanton Zürich. Neue Zürcher Zeitung, Available at: https://www.nzz.ch/zuerich/ein-viertel-aller-psychiatrischen-hospitalisierungen-unfreiwillig-1.17965215 (accessed 27 January 2020).

14.

Duff

(2012) Evidence-based indicators of neuropsychological change in the individual patient: Relevant concepts and methods. Archives of Clinical Neuropsychology 27: 248–261.

15.

Egger

Weniger

Muller

, et al. (2019) Assessing the severity of functional impairment of psychiatric disorders: Equipercentile linking the mini-ICF-APP and CGI. Health and Quality of Life Outcomes 17: 174.

16.

Egger

Weniger

Prinz

, et al. (2015) Health of the Nation Outcome Scales in a psychiatric inpatient setting: Assessing clinical change. Journal of Evaluation in Clinical Practice 21: 236–241.

17.

Evans

Margison

Barkham

(1998) The contribution of reliable and clinically significant change methods to evidence-based mental health. Evidence-Based Mental Health 1: 70–72.

18.

Giavarina

(2015) Understanding Bland Altman analysis. Biochemia Medica (Zagreb) 25: 141–151.

19.

Gift

Strauss

Harder

(1980) The severity of psychiatric disorder. Psychiatry Research 3: 31–40.

20.

González

Wiberg

(2017) Applying Test Equating Methods: Using R. Cham: Springer.

21.

Guy

(1976) ECDEU Assessment Manual for Psychopharmacology. Rockville, MD: US Department of Health, Education, and Welfare, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration, National Institute of Mental Health, Psychopharmacology Research Branch, Division of Extramural Research Programs.

22.

Harris

Sparti

Scheurer

, et al. (2018) Measurement properties of the Health of the Nation Outcome Scales (HoNOS) family of measures: Protocol for a systematic review. BMJ Open 8: e021177.

23.

Hastie

Tibshirani

Friedman

(2009) The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer Science & Business Media.

24.

James

Painter

Buckingham

, et al. (2018) A review and update of the Health of the Nation Outcome Scales (HoNOS). BJPsych Bulletin 42: 63–68.

25.

Kisely

Campbell

Cartwright

, et al. (2010) Do the Health of the Nation Outcome Scales measure outcome? The Canadian Journal of Psychiatry 55: 431–439.

26.

Kortrijk

Schaefer

van Weeghel

, et al. (2019) Trajectories of patients with severe mental illness in two-year contact with Flexible Assertive Community Treatment teams using Routine Outcome Monitoring data: An observational study. PLoS ONE 14: e0207680.

27.

Leon

Shear

Klerman

, et al. (1993) A comparison of symptom determinants of patient and clinician global ratings in patients with panic disorder and depression. Journal of Clinical Psychopharmacology 13: 327–331.

28.

Leucht

Kane

Etschel

, et al. (2006) Linking the PANSS, BPRS, and CGI: Clinical implications. Neuropsychopharmacology 31: 2318–2325.

29.

Leucht

Kane

Kissling

, et al. (2005) Clinical implications of Brief Psychiatric Rating Scale scores. The British Journal of Psychiatry 187: 366–371.

30.

Luo

Harvey

Tran

, et al. (2016) Consistency of the Health of the Nation Outcome Scales (HoNOS) at inpatient-to-community transition. BMJ Open 6: e010732.

31.

MacDonald

AJD

(2009) The usefulness of aggregate routine clinical outcomes data: The example of HoNOS65+. Journal of Mental Health 11: 645–656.

32.

Marsh

Balla

McDonald

(1988) Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin 103: 391–410.

33.

Mukaka

(2012) Statistics corner: A guide to appropriate use of correlation coefficient in medical research. Malawi Medical Journal 24: 69–71.

34.

Parabiaghi

Barbato

D’Avanzo

, et al. (2005) Assessing reliable and clinically significant change on Health of the Nation Outcome Scales: Method for displaying longitudinal data. Australian and New Zealand Journal of Psychiatry 39: 719–725.

35.

Parabiaghi

Kortrijk

Mulder

(2014) Defining multiple criteria for meaningful outcome in routine outcome measurement using the Health of the Nation Outcome Scales. Social Psychiatry and Psychiatric Epidemiology 49: 291–305.

36.

Parabiaghi

Rapisarda

D’Avanzo

, et al. (2011) Measuring clinical change in routine mental health care: Differences between first time and longer term service users. Australian and New Zealand Journal of Psychiatry 45: 558–568.

37.

Pirkis

Burgess

Kirk

, et al. (2005) A review of the psychometric properties of the Health of the Nation Outcome Scales (HoNOS) family of measures. Health and Quality of Life Outcomes 3: 76.

38.

Preston

(2000) The Health of the Nation Outcome Scales: Validating factorial structure and invariance across two health services. Australian and New Zealand Journal of Psychiatry 34: 512–519; discussion 520–521.

39.

Schennach-Wolff

Obermeier

Seemüller

, et al. (2010) Does clinical judgment of baseline severity and changes in psychopathology depend on the patient population? Results of a CGI and PANSS linking analysis in a naturalistic study. Journal of Clinical Psychopharmacology 30: 726–731.

40.

Schmittmann

Cramer

AOJ

Waldorp

, et al. (2013) Deconstructing the construct: A network perspective on psychological phenomena. New Ideas in Psychology 31: 43–53.

41.

Schober

Boer

Schwarte

(2018) Correlation coefficients: Appropriate use and interpretation. Anesthesia & Analgesia 126: 1763–1768.

42.

Smith

Vitoratou

McCrone

, et al. (2017) Exploring the factor structure of the Health of the Nation Outcomes Scale (HoNOS) in a sample of patients with schizophrenia, schizotypal and delusional disorders. Health and Quality of Life Outcomes 15: 141.

43.

Speak

Muncer

(2016) Factorial structure of the Health of the Nation Outcome Scales: An ordinal confirmatory factor analysis using a national sample of clinician ratings in England. International Journal of Mental Health Nursing 25: 87–98.

44.

Speak

Hay

Muncer

(2015) HoNOS – Their utility for payment by results in mental health. International Journal of Health Care Quality Assurance 28: 115–128.

45.

Sullivan

Feinn

(2012) Using effect size-or why the P value is not enough. Journal of Graduate Medical Education 4: 279–282.

46.

Theodoridou

Jaeger

Ketteler

, et al. (2011) The concurrent validity and sensitivity of change of the German version of the health of the nation outcome scales in a psychiatric inpatient setting. Psychopathology 44: 391–397.

47.

Trauer

(1999) The subscale structure of the Health of the Nation Outcome Scales (HoNOS). Journal of Mental Health 8: 499–509.

48.

Vazquez-Bourgon

Gomez Ruiz

Hoyuela Zaton

, et al. (2019) Differences between psychiatric disorders in the clinical and functional effectiveness of an acute psychiatric day hospital, for acutely ill psychiatric patients. Revista de Psiquiatría y Salud Mental. Epub ahead of print 6/5. DOI: 10.1016/j.rpsm.2019.04.001.

49.

Wee

Yeo

Chong

, et al. (2018) Mean Rank, Equipercentile, and Regression Mapping of World Health Organization Quality of Life Brief (WHOQOL-BREF) to EuroQoL 5 Dimensions 5 Levels (EQ-5D-5L) Utilities. Med Decis Making 38: 319–333.

50.

Whiteford

Degenhardt

Rehm

, et al. (2013) Global burden of disease attributable to mental and substance use disorders: Findings from the Global Burden of Disease Study 2010. The Lancet 382: 1575–1586.

51.

World Health Organization (WHO) (1992) The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines. Geneva: WHO.

52.

Wing

(2000) HoNOS update. British Journal of Psychiatry 176: 392–393.

53.

Wing

Beevor

Curtis

, et al. (1998) Health of the Nation Outcome Scales (HoNOS). Research and development. British Journal of Psychiatry 172: 11–18.

54.

Wing

Curtis

Beevor

(1994) ‘Health of the Nation’: Measuring mental health outcomes. Psychiatric Bulletin 18: 690–691.

55.

Wing

Curtis

Beevor

(1996) HoNOS, Health of the Nation Outcome Scales: Report on Research and Development, July 1993–December 1995. London: Royal College of Psychiatrists.

56.

Wing

Curtis

Beevor

(1999) Health of the Nation Outcome Scales (HoNOS). Glossary for HoNOS score sheet. British Journal of Psychiatry 174: 432–434.

57.

Zimmerman

Morgan

Stanton

(2018) The severity of psychiatric disorders. World Psychiatry 17: 258–275.

Assessing the severity of psychiatric disorders using the Health of the Nation Outcome Scales: An equipercentile linking analysis

Abstract

Objective:

Methods:

Results:

Conclusion:

Keywords

Introduction

Methods

Sample and procedure

Ethical considerations

Diagnosis and diagnostic groups

Rating scales

Raters and rating

Statistical analyses

Results

Multiple regression analysis

Equipercentile linking CGI-S and HoNOS sum score

Equipercentile linking CGI-I with the HoNOS difference score and percentage of change

Discussion

Conclusion

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iD

Data Availability

References