Sage Journals: Discover world-class research

Abstract

Objective: To present a summary of quantitative scales relevant to schizophrenia prevention studies.

Method: Fifteen scales were reviewed and summarised in terms of structure, domains assessed, previous use and psychometric properties. Instruments of symptom measurement, role functioning and global functioning were considered, along with multidimensional instruments and other scales of potential interest to research in schizophrenia prevention.

Results and conclusions: No scales of potential value in measuring premorbid risk for schizophrenia have been sufficiently tested for reliability and validity in the context of primary prevention of schizophrenia. The absence of a sufficiently sensitive and specific means for identifying those at high risk of schizophrenia before the onset of psychosis is a major barrier to valid measurement of the outcome of attempts at primary prevention. However, there have been advances in the development of instruments relevant to the goals of secondary and tertiary prevention. Most studies use instruments developed for patients with established psychoses and have applied them to early psychosis groups with some success, although possible ‘floor’ effects may confound measurement in the ‘prodromal’ period.

Keywords

outcome prevention psychosis rating scales schizophrenia

Interest in the early detection and treatment of psychosis has recently turned towards the possibility of primary prevention of schizophrenia and related psychoses. McGlashan noted that primary prevention in mental health involves reducing the occurrence of new cases (i.e. incidence), while secondary prevention aims to delay onset or prevent recurrence (i.e. reduce prevalence), and tertiary prevention aims to decrease the duration or severity of symptoms and associated disability [1]. Three strategies have been proposed to achieve the primary, secondary and tertiary prevention of schizophrenia and related psychoses [2]. These are: (i) the early detection of groups at increased risk of developing a psychotic illness; (ii) the early treatment of first episode psychosis; and (iii) targeting interventions at the early ‘critical period’ of psychosis. In terms of primary prevention, it has been proposed that if the distinguishing features of those people who subsequently develop psychosis can be identified, then it may be possible to prevent the development of frank psychotic symptoms [1],[3],[4],[5]. Therefore, the main focus of primary prevention studies of schizophrenia has been to develop strategies and criteria for the identification of groups at increased risk of developing a psychosis.

Known risk factors for schizophrenia include family history (i.e. genetic risk), season of birth, migrant/minority status, perinatal complications, neurodevelopmental deficits, and behavioural or social maladjustment. However, each of these is associated with relatively small increases in risk relative to the general population. Certainly, none alone can be taken to indicate a sufficient degree of risk of schizophrenia to warrant preventive interventions at a clinical level. This handful of risk factors of small magnitude, together with the fact that schizophrenia is a disorder of low prevalence, suggest that even if multiple risk factors were taken into account there are still such gaps in our knowledge that the development of a multifactorial risk index of sufficient sensitivity and specificity to have clinical utility represents a formidable challenge. It is not sufficient, for primary prevention in the clinical (as opposed to population) domain, to be merely at ‘increased risk’, but to be at ‘high risk.’

The attempts by McGorry and colleagues to identify young people at sufficiently increased risk to justify intervention before the onset of psychotic symptoms has relied on state and trait risk variables that can be readily identified in a clinical setting [3],[4],[6]. This has entailed identifying individuals during what may otherwise have later been regarded, retrospectively, as the prodromal period of schizophrenia.

However, there are significant problems with this approach. First, the risk categories based on these state and trait variables [3] have not been sufficiently tested for predictive validity at this stage. Indeed, very little research has been done in following the progress of such groups to evaluate whether the purportedly ‘high risk’ individuals do go on to experience a first episode of psychosis. Although the early work in this area is very promising [6], there is a need for large scale studies to determine whether, using this or other methods of identifying ‘high risk’ groups, primary prevention is indeed feasible. Rigorous longitudinal measurement of outcomes in purportedly ‘high risk’ individuals is essential for validation of the criteria for determining risk status. Without such fundamental information, systematic prospective studies of primary prevention will lack a sound scientific basis. Second, even if a risk index of acceptable sensitivity, specificity and positive preditive value were available, then for low incidence/prevalence disorders such as schizophrenia it is by no means clear that such an instrument could, both theoretically and practically, be applied in a clinical setting sufficiently early to provide opportunity for primary prevention.

Notwithstanding the above, clinical studies of putative ‘high risk’ individuals have typically used published work from studies of established psychoses to guide the investigation of possible predictive factors and domains of interest [1],[3],[7],[8], thereby influencing the choice of quantitative outcome measures of potential applicability in primary prevention. It is not yet clear whether these measures are suitable for this purpose.

By comparison, the situation with respect to outcome measurement in secondary and tertiary prevention is much clearer. Falloon et al. have indicated that the secondary and tertiary prevention of schizophrenia could best be achieved through the early detection and treatment of psychotic symptoms, and continued monitoring for any recurrence of such symptoms [9]. Outcome measurement in this context needs to involve the investigation of the efficacy of treatment interventions and detection systems for patients in the early stages of an established psychotic illness. Thus, the aims of outcome measurement in schizophrenia prevention are similar to those discussed by Stedman et al. in the context of general mental health [10]; that is, the reliable and valid measurement of outcomes is necessary for the evaluation of treatment effectiveness. In addition, outcome measurement will allow the identification of groups for whom intervention is most effective, and the description of service provider and patient characteristics associated with a positive clinical outcome. In combination, these factors influence decision making, both at a clinical level, and at a resource allocation level.

Given that outcome measurement is vital to the investigation of schizophrenia prevention, it is necessary to describe suitable dimensions to be measured, as well as the quantitative tools used to do so. Andrews and colleagues outlined important considerations in choosing appropriate outcome measures [11]. Measures should be applicable, clearly assessing the domains of interest. In psychometric terms, measures should be reliable and valid, and should also be sensitive to change in the domain of interest. Studies of outcome in groups with established psychosis have focused on change in positive and negative symptoms following treatment intervention, evaluation of social and occupational role functioning, and investigation of the predictive validity of diagnosis in longitudinal follow-up studies [12],[13].

The present paper aims to summarise a selection of quantitative measures commonly used in the field of schizophrenia research. The measures were selected on the basis of their widespread application, their use as outcome measures in schizophrenia prevention or intervention studies, and their potential applicability to studies of the ‘prodromal’ or first-episode psychosis populations. Table 1 provides a summary of the measures detailed in this article, including their structure and the domains they attempt to measure.

Table 1.

Summary of measures commonly used in schizophrenia research

Symptom measurement

Brief Psychiatric Rating Scale

The Brief Psychiatric Rating Scale (BPRS) is a widely used measure of symptoms, developed originally by Overall and Gorham [14], and also available in an expanded version (BPRS-E) [15]. The scale has most frequently been used in initial assessment and outcome evaluation in a variety of psychiatric disorders [16–18], as well as research into the structure of psychotic symptoms [19–22]. The BPRS may be used at regular intervals for research purposes, typically ranging from one day to one month. The areas covered provide a broad perspective across numerous areas of subject behaviour and symptoms. However, although the reference group against which ratings are made are normal controls, the scale may suffer from a floor effect with early psychosis or ‘prodromal’ subjects; that is, the scale may not be sensitive to the relatively mild fluctuations in symptoms of possibly ‘prodromal’ patients. In addition, there is evidence that the lack of firm definitions for the BPRS items may result in nonstandardised use in different centres [16]. Nevertheless, overall the BPRS has very good psychometric properties. It is strongly correlated with equivalent items on the Scale for the Assessment of Positive Symptoms (SAPS) (r > 0.88) [16] and Scale for the Assessment of Negative Symptoms (SANS) (r > 0.84) [16], (r = 0.89) [21], suggesting it has excellent concurrent validity. The BPRS has also been found to have good internal consistency (α = 0.69 and 0.68 for the positive and negative syndrome scales, respectively) [23].

Scale for the Assessment of Negative Symptoms

The SANS [24] was designed to assess negative symptoms in patients with psychotic disorders [18], most notably schizophrenia. It has also been used to investigate the theoretical structure of psychotic symptoms [19],[25]. The SANS provides a detailed assessment of patients' negative symptoms. However, it is possible that the use of the scale in primary prevention studies will suffer from floor effects, as most ‘prodromal’ individuals will cluster around scores of 0 or 1. The SANS has been widely used and has good psychometric properties. Its concurrent validity is demonstrated through its high correlation with the BPRS [21], as described earlier.

Scale for the Assessment of Positive Symptoms

The SAPS [26] was developed as a companion measure to the SANS, and focuses on delusions, hallucinations and thought disorder. It has also been used with the SANS to investigate the theoretical structure of psychotic symptoms [19],[25]. The SAPS provides a detailed assessment of the positive symptoms of psychosis, but may suffer from similar floor effects as the SANS when used in ‘prodromal’ individuals. Along with the SANS, the scale has been used extensively, and has sound psychometric properties.

Positive and Negative Syndrome Scale

The Positive and Negative Syndrome Scale (PANSS) is designed to assess the positive and negative symptoms associated with schizophrenia [27]. In addition, the scale aims to measure the relative contributions of positive and negative symptoms to the patient's clinical presentation. The scale has been used in studies of the theoretical negative–positive symptom dichotomy in schizophrenia [28]. The investigators reported excellent reliability and strong criterion and construct validity [29]. In one study, the PANSS demonstrated consistently better interrater reliability and predictive validity than the BPRS [23]. Assessments of the interrater reliability of the PANSS have shown intraclass correlation coefficients (ICC) of 0.72, 0.80 and 0.56 for the positive, negative and general psychopathology scales, respectively. In addition, the scale demonstrates concurrent validity with the SAPS and SANS (r = 0.70 and 0.81 for the positive and negative scales, respectively) [30].

Positive and Negative Syndrome Scale for children and adolescents

The Positive and Negative Syndrome Scale for children and adolescents (Kiddie-PANSS) was developed as an instrument based on the adult PANSS, but for use in the assessment of children and adolescents with psychiatric disorders [31]. Owing to the age group for which the scale is designed, it is possible that the Kiddie-PANSS may be useful in primary prevention or ‘prodromal’ studies, which frequently target younger age groups than studies of first episode or chronic schizophrenia. The investigators reported that the Kiddie-PANSS demonstrated good interrater reliability, with ICC of 0.69, 0.89 and 0.52 for the positive, negative and general psychopathology scales, respectively. The instrument was also successful in distinguishing patients with schizophrenia from those with other psychiatric conditions.

Manchester Scale

The Manchester Scale (MS), also known as the Psychiatric Assessment Scale (PAS) [32], has been used to assess positive and negative symptoms in subjects with schizophrenia. It has also been used in conjunction with the Global Assessment Scale (GAS) to provide a comprehensive assessment and evaluation of treatment progress in terms of symptoms and general functioning [32],[33]. A strength of the MS is that it can be used to obtain regular ratings to map progress over time. It has sound psychometric properties, correlating with comparable items on the BPRS, SAPS and SANS, suggesting acceptable concurrent validity [33].

Symptom Checklist-90-Revised

The Symptom Checklist-90-Revised (SCL-90-R) is a standardised self-report measure in which patients rate 90 common symptoms on a five-point Likert scale [34]. The instrument consists of 10 clinical scales of somatisation, depression, anxiety, obsessive-compulsiveness, hostility, interpersonal sensitivity, phobic anxiety, paranoid ideation and psychoticism. In addition, three global symptom indices are also assessed. While the author of the scale reported acceptable validity and reliability, others have suggested that certain populations are not adequately identified by the scale. For example, in patients with obsessive-compulsive disorder the scale was found to have poor divergent and criterionrelated validity [35]. In patients with substance abuse plus psychiatric disability, the dimensional structure of the SCL-90-R was found to vary across genders [36]. The norms of the scale for adolescents have also been called into question [37], and in a study of malingering using the scale, 100%% of untrained volunteers were able to fake psychopathology at a level comparable to psychiatric patients [38]. These findings suggest that the SCL-90-R may not have sufficient specificity to assist in accurately identifying early psychosis patients.

Role functioning measurement

Quality of Life Scale

The Quality of Life Scale (QLS) was developed for use both as an outcome measure and a measure of change for patients with schizophrenia [39]. The measure systematically elicits detail on a wide range of functioning, focusing on deficit symptoms rather than positive symptoms. The use of an additional instrument is therefore necessary to monitor positive symptoms. The researchers reported strong interrater reliability when users were given appropriate training in the use of the instrument (intraclass correlation coefficients 0.91 for intrapsychic foundations, 0.94 for interpersonal relations, 0.97 for instrumental role, 0.94 for commonplace objects and activities, and 0.94 for total score).

Role Functioning Scale

The Role Functioning Scale (RFS) [40] was designed as an outcome measure for people with chronic mental disability and established schizophrenia [10]. Although it has not been extensively reported in the literature [10], it includes most of the dimensions of role functioning relevant to patients of mental health services [11]. It has been reported that the scale has excellent internal consistency (α = 0.86) [40], adequate test–retest and interrater reliability, and adequate construct and criterion validity, and correlates negatively with the Health of the Nation Outcome Scales (HoNOS) (r = –0.58, p < 0.01) [10]. The scale also has modest convergent validity with the GAS [41].

Life Skills Profile

The Life Skills Profile (LSP) was developed to assess functioning of patients with chronic severe mental illness [42]. The scale is focused towards community-based rather than inpatient services. Owing to its focus on chronic mental illness and established schizophrenia [10], the utility of the measure is uncertain for schizophrenia prevention studies other than tertiary prevention. However, the psychometric properties of the LSP are sound, with excellent item reliability and adequate test–retest reliability and internal consistency (α = 0.93 for the total score; subscales range from α = 0.67 to α = 0.88). The scale also demonstrates adequate content and criterion validity, as well as sensitivity to change [10].

Global scales

Global Assessment Scale

The Global Assessment Scale (GAS) has been frequently used for psychiatric inpatient planning and evaluation [43]. It has also been used in conjunction with the MS to provide an overall picture of functioning [33], and to monitor treatment progress [44]. Owing to the structure of the scale, it is theoretically applicable to a wide range of functions, from those suffering from severe mental illness-related disability to those with superior functioning in most areas. It also takes into account a broad range of areas, not just psychiatric symptoms. Therefore, it is useful as a summary statement of functioning and could be applied to studies of schizophrenia prevention. Its wide use may be partly attributable to its relative simplicity, ease of coding and speed. However, although the scale is useful in recording change and monitoring overall progress, it is unable to monitor specific symptom change over time, and cannot provide the detail that may be achieved with multidimensional rating scales. The scale has been reported to have modest convergent validity with the RFS [41], and good interrater reliability and concurrent validity. The GAS was modified for inclusion in the DSM-III-R as the Global Assessment of Functioning (GAF) scale, which was again modified for the DSM-IV as the SOFAS, where it is used to evaluate adaptive functioning (axis V) [45].

Multidimensional scales

Health of the Nation Outcome Scales

The HoNOS was developed for use as a routine measure of mental health consumer outcomes [46]. It is widely used as a clinical instrument to measure patients on a range of dimensions. A study by Amin and colleagues, which followed first-onset psychosis patients after 3 years, reported that the HoNOS demonstrated strong concurrent validity through significant correlations with other scales, such as the SANS and GAF [47]. Their study also found moderate to good interrater reliability (intraclass correlation coefficients ranged from 0.47 to 0.85 for the subscales, and 0.59 overall). Similarly, the Victorian field trial of the HoNOS reported acceptable interrater reliability (intraclass correlation coefficient of 0.71) [48]. However, Bebbington and coworkers have suggested that the HoNOS has poor interrater reliability, possibly related to the training and experience of professionals using the scales [49]. In their study, no item had a k value above 0.40 (‘fair’).

Mental Health Inventory

The Mental Health Inventory (MHI) was originally designed as a measure of mental health for the general population, although it has also been used in studies of mental health in a wide range of physical disorders [50]. Stedman et al. noted that the instrument takes little time to administer, and collects information about a wide range of areas [10]. However, as the scale has been used mostly in nonpsychiatric populations, its relevance for people with severe mental illness is uncertain [10], as is its applicability in early or ‘prodromal’ schizophrenia. The psychometric properties of the instrument are favourable, with sound test–retest reliability (r = 0.97) and internal consistency (α = 0.97), and adequate content and criterion validity.

Other measures of interest

Early Signs Scale

The Early Signs Scale (ESS) [51] has been used in studies attempting the early identification of relapse in schizophrenia [52],[53]. Jorgensen reported that the scale had excellent predictive validity, and sound sensitivity and specificity [53]. Given these factors, the scale would appear to be extremely useful for studies in secondary prevention. The measure also takes little time and is simple to administer.

Insight Scale

The Insight Scale (IS) was designed as a self-report measure to investigate patient levels of insight [54]. The psychometric properties of the scale were investigated in a population of patients with nonaffective psychoses. Results suggested that the IS demonstrated excellent test–retest reliability (total scale r = 0.90), as well as sound construct, concurrent and criterion validity. The IS was also found to be sensitive to change, and may therefore be used as a longitudinal measure of progress over time from the ‘prodromal’ period onwards.

Knowledge About Schizophrenia Questionnaire

The Knowledge About Schizophrenia Questionnaire (KASQ) was designed to evaluate patients' knowledge regarding schizophrenia and its management [55]. Psychometric evaluation revealed excellent reliability as assessed by internal consistency (α = 0.89 and α = 0.85 in two samples) and test–retest (r = 0.83, p < 0.005) procedures. In addition, the measure was found to be sensitive to change. However, the test was specifically designed to evaluate a particular psychoeducation program, and may not be readily applicable to prevention studies.

Discussion

The above constitutes a summary of selected outcome measures that are potentially useful in schizophrenia prevention studies. Many of the instruments presently available to researchers have been designed for people with an established diagnosis of schizophrenia, and tend to have been developed with groups of mentally ill patients. Therefore, it is likely that the scales will not adequately account for the experience of the premorbid or ‘prodromal’ individual who is not experiencing symptoms of psychosis other than, perhaps, in attenuated form. It is also likely that floor effects of the scales will limit the utility of such instruments in this context.

We could find no scale suitable for evaluating the validity of categories used to allocate people to groups at putative ‘high risk’ of psychosis, which would be relevant in the primary prevention arena [3],[4],[7]. Yet evaluating the validity of the ‘high risk’ categories is vital if the goal of primary prevention of schizophrenia is to be achieved. In the absence of ‘high risk’ categories of demonstrable reliability and validity, the efficacy of primary prevention strategies cannot be properly tested. This is because, if putative ‘high risk’ patients do not proceed to a psychotic episode, it will be impossible to know whether this is due to successful intervention, lack of predictive validity in the determination of risk status, or failure to follow the patients through the period of highest likelihood for the onset of psychosis. Therefore, it is imperative that the ‘high risk’ criteria be empirically validated, and that measures that focus specifically on the assessment and longitudinal monitoring of premorbid or ‘prodromal’ patients be developed and tested.

A particular problem in developing a measure of ‘high risk’ of schizophrenia is that none of the known risk factors, several of which were listed in the introduction, are specific to schizophrenia. Similarly, none of the state or trait features that define the ‘high risk’ categories currently in clinical use [3] are specific to schizophrenia. Other conditions in which such risk indicators are commonly found premorbidly include affective psychoses; other psychoses; some personality disorders and disturbances of conduct; substance use disorders; learning disorders; and even some neuroses. Furthermore, the substantial overlap between the phenomenology (and other features) of schizophrenia and many other psychiatric syndromes is well known. It is possible that the state of our current knowledge in this area may accurately reflect the true nature of psychoses and related disorders, and that the present nosological system in psychiatry may not map well onto risk factors or antecedent factors. If this is the case, it may be time to take a broader view of primary prevention and focus not just on schizophrenia but on the spectrum of severe mental disorders that share common risk factors.

While the goal of primary prevention is yet to be realised, secondary and tertiary prevention of schizophrenia is far less complex and may be adequately measured by instruments presently available. Although most of the scales have been developed for use in populations with established psychotic illness, they have also been used in studies of first-episode psychosis. In particular, the measures of symptoms appear to be useful in evaluating early treatment of patients experiencing a first episode of psychosis [56],[57]. In addition, there have been indications in the literature of the development of quantitative scales, which are of particular relevance to working with preventing relapse in first episode patients, such as the ESS [51] and the IS [54].

Measurement of outcome in the context of secondary and tertiary prevention is not beset by the conceptual and methodological problems that presently apply in the domain of primary prevention. Fifty years ago there was very little written about recovery from schizophrenia. Nowadays, recovery is a more commonplace event and is widely reported in the outcome literature. The concept of any form of prevention of schizophrenia 50 years ago would have been virtually unthinkable. Yet, today we are in a strong position to prevent much of the morbidity, disability and suffering associated with this illness, provided that interventions of demonstrated effectiveness in secondary and tertiary prevention are broadly implemented. Beyond this, the goal of primary prevention appears more remote. Advances in neuro-imaging such as high resolution structural MRI and functional MRI, electrophysiology, neuropsychology and, perhaps, genetic screening, may help to bring that goal closer to realisation.

Footnotes

Acknowledgements

The authors are grateful to Terry Lewin and Ulrich Schall for their comments in the preparation of this paper.

References

1. McGlashan

. Early detection and intervention in schizophrenia: research. Schizophrenia Bulletin 1996; 22: 327–345.

2. Birchwood

McGorry

Jackson

. Early intervention in schizophrenia. British Journal of Psychiatry 1997; 170: 2–5.

3. Yung

McGorry

McFarlane

Jackson

Patton

Rakkar

. Monitoring and care of young people at incipient risk of psychosis. Schizophrenia Bulletin 1996; 22: 283–303.

4. Yung

McGorry

. The prodromal phase of firstepisode psychosis: past and current conceptualizations. Schizophrenia Bulletin 1996; 22: 353–370.

5. Amminger

Pape

Rock

. Relationship between childhood behavioural disturbance and later schizophrenia in the New York High-Risk Project. American Journal of Psychiatry 1999; 156: 525–530.

6. Yung

Phillips

McGorry

. Can we predict the onset of first-episode psychosis in a high-risk group?. International Clinical Psychopharmacology 1998; 13(Suppl. 1)S23–S30.

7. Olin

Mednick

. Risk factors of psychosis: identifying vulnerable populations premorbidly. Schizophrenia Bulletin 1996; 22: 223–240.

8. Vaglum

. Earlier detection and intervention in schizophrenia: unsolved questions. Schizophrenia Bulletin 1996; 22: 347–351.

9. Falloon

IRH

. Early intervention for first episodes of schizophrenia: a preliminary exploration. Psychiatry 1992; 55: 4–15.

10.

10. Stedman

Yellowlees

Mellsop

Clarke

Drake

. Measuring consumer outcomes in mental health. Department of Health and Family Services, Canberra 1997.

11.

11. Andrews

Peters

Teeson

. Measurement of consumer outcome in mental health: a report to the National Mental Health Information Strategy Committee. Clinical Research Unit for Anxiety Disorders, Sydney 1994.

12.

12. Mason

Harrison

Croudace

Glazebrook

Medley

. The predictive validity of a diagnosis of schizophrenia. British Journal of Psychiatry 1997; 170: 321–327.

13.

13. Tsuang

Coryell

. An 8-year follow-up of patients with DSM-III-R psychotic depression, schizoaffective disorder, and schizophrenia. American Journal of Psychiatry 1993; 150: 1182–1188.

14.

14. Overall

Gorham

. Brief Psychiatric Rating Scale. Psychological Reports 1962; 10: 799–812.

15.

15. Lukoff

Nuechterlein

Ventura

. Manual for the Expanded Brief Psychiatric Rating Scale. Schizophrenia Bulletin 1986; 12: 594–602.

16.

16. Inch

Crossley

Keegan

Thorarinson

. Use of the Brief Psychiatric Rating Scale to measure success in a psychosocial day program. Psychiatric Services 1997; 48: 1195–1197.

17.

17. Nicholson

Chapman

Neufeld

RWJ

. Variability in BPRS definitions of positive and negative symptoms. Schizophrenia Research 1995; 17: 177–185.

18.

18. Stuart

Malone

Vurrie

Klimidis

Minas

. Positive and negative symptoms in neuroleptic-free psychotic inpatients. Schizophrenia Research 1995; 16: 175–188.

19.

19. Silver

David

Kaplan

. Factor analysis of schizophrenic symptoms and comparison of different rating scales. Schizophrenia Research 1993; 10: 67–75.

20.

20. Burger

Calsyn

Morse

Klinkenberg

Trusty

. Factor structure of the expanded Brief Psychiatric Rating Scale. Journal of Clinical Psychology 1997; 53: 451–454.

21.

21. Czobor

Bitter

Volavka

. Relationship between the Brief Psychiatric Rating Scale and the Scale for the Assessment of Negative Symptoms: a study of their correlation and redundancy. Psychiatry Research 1991; 36: 129–139.

22.

22. Dingemans

PMAJ

Linszen

Lenior

Smeets

RMW

. Component structure of the expanded Brief Psychiatric Rating Scale (BPRS-E). Psychopharmacology 1995; 122: 263–267.

23.

23. Bell

Milstein

Beam-Goulet

Lysaker

Cicchetti

. The Positive and Negative Syndrome Scale and the Brief Psychiatric Rating Scale. Reliability, comparability, and predictive validity. Journal of Nervous and Mental Disease 1992; 180: 723–728.

24.

24. Andreasen

. Scale for the Assessment of Negative Symptoms (SANS). University of Iowa, Iowa City 1983.

25.

25. Klimidis

Stuart

Minas

Copolov

Singh

. Positive and negative symptoms in the psychoses. Re-analysis of published SAPS and SANS global ratings. Schizophrenia Research 1993; 9: 11–18.

26.

26. Andreasen

. Scale for the Assessment of Positive Symptoms (SAPS). University of Iowa, Iowa City 1984.

27.

27. Kay

Fiszbein

Opler

. The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophrenia Bulletin 1987; 13: 261–276.

28.

28. Kay

Opler

. The positive-negative dimension in schizophrenia: its validity and significance. Psychiatric Developments 1987; 2: 79–103.

29.

29. Kay

Opler

Lindenmayer

. Reliability and validity of the Positive and Negative Syndrome Scale for schizophrenics. Psychiatry Research 1988; 23: 99–110.

30.

30. Peralta

Cuesta

. Psychometric properties of the Positive and Negative Symptom Scale (PANSS) in schizophrenia. Psychiatry Research 1994; 53: 31–40.

31.

31. Fields

Grochowski

Lindenmayer

. Assessing positive and negative symptoms in children and adolescents. American Journal of Psychiatry 1994; 151: 249–253.

32.

32. Krawiecka

Goldberg

Vaughan

. A standardized psychiatric assessment scale for rating chronic psychiatric patients. Acta Psychiatrica Scandinavica 1977; 55: 299–308.

33.

33. Munoz

. Evaluating progress among hospitalised patients with psychosis. Annals of Clinical Psychiatry 1992; 4: 197–200.

34.

34. Derogatis

. SCL-90-R: administration, scoring and procedures manual II. Clinical Psychometric Research, Towson, MD 1983.

35.

35. Woody

Steketee

Chambless

. The usefulness of the obsessive compulsive scale of the Symptom Checklist-90-Revised. Behaviour Research and Therapy 1995; 33: 607–612.

36.

36. Carpenter

Hittner

. Dimensional characteristics of the SCL-90-R: evaluation of gender differences in dually diagnosed inpatients. Journal of Clinical Psychology 1995; 51: 383–390.

37.

37. Todd

McKenna

Deane

. Appropriateness of SCL-90-R adolescent and adult norms for outpatient and nonpatient college students. Journal of Counseling Psychology 1997; 44: 294–301.

38.

38. Lees-Haley

. Malingering emotional distress on the SCL-90-R: toxic exposure and cancerphobia. Psychological Reports 1989; 65: 1203–1208.

39.

39. Heinrichs

Hanlon

Carpenter

. The Quality of Life Scale: an instrument for rating the schizophrenic deficit syndrome. Schizophrenia Bulletin 1984; 10: 388–398.

40.

40. Goodman

Sewell

Cooley

Leavitt

. Assessing levels of adaptive functioning: the Role Functioning Scale. Community Mental Health Journal 1993; 29: 119–131.

41.

41. Brekke

. An examination of the relationships among three outcomes scales in schizophrenia. Journal of Nervous and Mental Disease 1992; 180: 162–167.

42.

42. Rosen

Hadzi-Pavolvic

Parker

. The Life Skills Profile: a measure assessing function and disability. Schizophrenia Bulletin 1989; 15: 325–327.

43.

43. Endicott

Spitzer

Fleiss

Cohen

. The Global Assessment Scale. A procedure for measuring overall severity of psychiatric disturbance. Archives of General Psychiatry 1976; 33: 766–771.

44.

44. Drury

Birchwood

Cochrane

Macmillan

. Cognitive therapy and recovery from acute psychosis: a controlled trial. I. Impact on psychotic symptoms. British Journal of Psychiatry 1996; 169: 593–601.

45.

45. Goldman

Skodol

Lave

. Revising Axis V for DSM-IV. A review of measures of social functioning. American Journal of Psychiatry 1992; 149: 1148–1156.

46.

46. Wing

Curtis

Beevor

. Health of the Nation Outcome Scales: report on research. Royal College of Psychiatrists, London 1996.

47.

47. Amin

Singh

Croudace

Jones

Medley

Harrison

. Evaluating the Health of the Nation Outcome Scales. Reliability and validity in a three-year follow-up of first-onset psychosis. British Journal of Psychiatry 1999; 174: 399–403.

48.

48. Trauer

Callaly

Hantz

Little

Shields

Smith

. Health of the Nation Outcomes Scales. Results of the Victorian field trial. British Journal of Psychiatry 1999; 174: 380–388.

49.

49. Bebbington

Brugha

Hill

Marsden

Window

. Validation of the Health of the Nation Outcome Scales. British Journal of Psychiatry 1999; 174: 389–394.

50.

50. Veit

Ware

JE.

Jr . The structure of psychological distress and well-being in general populations. Journal of Consulting and Clinical Psychology 1983; 51: 730–742.

51.

51. Birchwood

Smith

Macmillan

. Predicting relapse in schizophrenia: the development and implementation of an early signs monitoring system using patients and families as observers, a preliminary investigation. Psychological Medicine 1989; 19: 649–656.

52.

52. Jorgensen

. Early signs of psychotic relapse in schizophrenia. British Journal of Psychiatry 1998; 172: 327–330.

53.

53. Jorgensen

. Schizophrenic delusions: the detection of warning signals. Schizophrenia Research 1998; 32: 17–22.

54.

54. Birchwood

Smith

Drury

Healy

Macmillan

Slade

. A self-report Insight Scale for psychosis: reliability, validity and sensitivity to change. Acta Psychiatrica Scandinavica 1994; 89: 62–67.

55.

55. Ascher-Svanum

. Development and validation of a measure of patients' knowledge about schizophrenia. Psychiatric Services 1999; 50: 561–563.

56.

56. Birchwood

Macmillan

. Early intervention in schizophrenia. Australian and New Zealand Journal of Psychiatry 1993; 27: 374–378.

57.

57. Vazquez-Barquero

Cuesta

Castanedo

Lastra

Herran

Dunn

. Cantabria first-episode schizophrenia study: three-year follow up. British Journal of Psychiatry 1999; 174: 141–149.