Global Measures of Impairment in Children and Adolescents: Results from a Swiss Community Survey

Abstract

Objective: Based on data from a Swiss epidemiological survey, both the parent and the child versions of two global measures of impairment, the Columbia Impairment Scale (CIS) and the Children's Global Assessment Scale (CGAS) were analysed with regard to validity and the effects of gender.

Method: Concurrent and discriminant validity were evaluated by examining the associations with other indicators of impairment. Gender effects on impairment were considered in each type of analysis.

Results: Based on the total sample and on the girls' data, concurrent validity of the various impairment scores as computed by correlations with other indicators of psychological dysfunction was good. When based on the boys' data it was less convincing. Discriminant validity was estimated by comparing impairment scores between those using and not using services and was proven for all impairment scores. There were significant gender effects for almost all scores, indicating that girls in contrast to boys were more impaired and functioned less well when professional contacts and psychiatric diagnosis were considered.

Conclusions: The two impairment measures are well suited to community studies. Gender and informant differences have to be considered when defining thresholds for caseness. The two scales could also be used clinically, for example for training and evaluation purposes and for the decision whether or not a child or adolescent requires treatment.

Besides the traditional categorical approach to assessing psychopathology by describing groups of symptoms or diagnoses, there is also a dimensional approach that intends to quantify behaviours and symptoms by use of rating scales and questionnaires. The latter approach offers the advantage of a quantitative, standardized definition of characteristic psychopathological features.

Usually, scales of this type are multidimensional reflecting symptoms such as hyperactivity, emotional or conduct problems.

In contrast to this multidimensional approach, a global method records the severity of a disorder in a given patient in terms of a single measurement. Thus, the total knowledge about the psychiatric and social impairment of a patient is condensed into a single index on a given scale. Although these global assessments cannot replace the detailed evaluation of symptoms, their value lies in supplementing information on impairment that is not directly derived from the assessment of symptoms. As a consequence, patients with different diagnoses can be compared as to their impairment, or a single patient can be evaluated according to the impact of treatment or developmental change.

The multi-axial versions of the two major classification schemes, DSM-IV and ICD-10, include scaled impairment ratings on separate axes. These measures were developed in the recent past by modification of the Global Assessment Scale (GAS) for adults [1]. Soon after, Shaffer et al. [2] devised the Children's Global Assessment Scale (CGAS) and tested this instrument for reliability and validity. Other groups found either goodto- excellent psychometric properties [3, 4] or moderate interrater reliability [5]. Recent epidemiological studies have used the CGAS in order to guard against an overestimation of prevalence rates for child and adolescent psychiatric disorders [6, 7]. In addition, the original CGAS, which had been developed for clinical purposes, was simplified to facilitate scoring by non-clinicians in largescale studies [8]. The CGAS rates the lowest level of functioning of a subject during a specified period.

In order to allow trained lay persons to make ratings of impairment in large-scale studies, Bird et al. [9] developed the Columbia Impairment Scale (CIS), which was used in addition to the GCAS in the National Institute of Mental Health (NIMH) Methods for the Epidemiology of Child and Adolescent Mental Disorder (MECA) study in the USA [10]. This study addressed various aspects of validity and established thresholds. It concluded that both measures are useful for clinical as well as for epidemiological purposes. The CIS is a 13-item scale that can be administrated by a lay interviewer. The items tap four major areas of functioning: interpersonal relations, broad psychopathological domains (e.g. anxiety, depression or behaviour problems), functioning in job or school and use of leisure time.

Both the CGAS and the CIS were used to estimate prevalence rates in our epidemiological and longitudinal study of the prevalence of child and adolescent psychiatric disorders in the Canton of Zürich, Switzerland. However, given a different cultural background, it was necessary to replicate, extend and modify previous analyses. More specifically, we examined concurrent and discriminant validity of the two scales, and the interaction of impairment and gender, which has not been studied so far.

Method

Sample

In this community survey, 1239 adolescents aged 15–19 years (mean = 16.3, SD = 15) were approached in 1997, and a total of 1089 (88%) finally participated in the study: 506 (46.5%) boys and 583 (53.5%) girls. According to the educational and vocational status of the main bread-winner, there were 153 (14%) lower-class families, 829 (76.2%) middle-class families, and 97 (8.9%) upper-class families. The large majority of adolescents were living in rural areas (n = 814, 74.7%) and a minority (n = 275, 25.3%) in one of the two major cities of the area. The total cohort was screened with various instruments (see below), resulting in 286 adolescents who were screen positive on one or more screens. In addition, 150 screen negatives were randomly selected for stage two of the study so that a total of 436 adolescents were selected for stage two interviews. Stage two interviews were performed after 6–9 months with 203 adolescents and their parents in 1998. In this sample there were 82 boys and 121 girls, and the mean age was 16.4 (SD = 1.5) years. The larger representation of girls was due to an overselection by the specific screens. There were significantly higher rates for girls on the two internalizing scores and the eating disorder score but no gender effects for the externalizing scores, and significantly higher scores for boys on the alcohol and other drug screens. Among the 203 probands, 137 were screen positive and 66 were screen negative. Despite the relatively large attrition rate, participants and dropouts did not differ significantly according to age, gender, type of school, socioeconomic status, urban versus rural residence, or for any of the variables that were used for screening (see below), so that the final sample interviewed was representative of the total stage two sample.

Procedures

Parent and youth information was obtained. These included the internalizing and the externalizing scores from the Child Behaviour Checklist (CBCL) [11] and the Youth Self-Report (YSR) [12], a 12- item questionnaire on eating disorders, and a 22-item questionnaire on alcohol and drug abuse, which were used as screens. The latter two questionnaires were developed for the present study. Following Verhulst et al. [7], the scores of the resulting seven screening modules were z-converted and averaged for each subject. The 75th percentile of the resulting score was used in order to define subjects for stage two assessment. Furthermore, the following five predictor variables were included into the analyses: CBCL total score, YSR total score, two items from the parents questionnaire indicating a learning problem and/or repetition of grade, and one item from the youth questionnaire indicating that the adolescent considered himself to have a mental health problem.

In stage two, the parents and the adolescents were interviewed at home by different interviewers and on different occasions. Interviewers were specially trained undergraduate clinical psychology students who were ‘blind’ to the subjects' screening scores. Both the parent and the child version of the Diagnostic Interview Schedule for Children (DISC 2.3) [13–16] were used. The interviews resulted in DSM-III-R diagnoses. The following four predictor variables were also considered for the present analyses: total symptom scores from the parent (P) and child (Y) versions of the DISC, professional contact of the adolescent with child psychiatry and child guidance clinics within the past 6 months, and divorce of the parents.

In addition, the CIS and the CGAS were administered. The CIS items are scored on a four-point scale. Total scores can range from 0 to 52 with high scores indicating greater impairment. The CGAS ratings range from 1 (most impaired) to 100 (best level of adaptive functioning). Behavioural descriptions are provided as anchor points for each decile. A single global score is given for a given subject at any point within this continuum, reflecting the subject's psychological and social functioning. Consequently, CGAS and CIS total scores have opposite meaning. The parents and the adolescents were introduced to the two scales by the interviewers and then made their rating. The interviewers blindly rated the adolescent using the information from the interview.

Thus, the following five impairment ratings were obtained: (i) parent CIS (PCIS, mean = 4.6, SD = 4.8); (ii) parent CGAS (PCGAS, mean = 86.2, SD = 8.0); (iii) parent interviewer CGAS (PICGAS, mean = 83.9, SD = 8.4); (iv) youth CGAS (YCGAS, mean = 82.2, SD = 9.0); and (v) youth interviewer CGAS (YICGAS, mean = 81.8, SD = 9.7).

Results

Concurrent validity

As a test of concurrent validity, correlations of CIS and CGAS scores with the other indicators of psychological dysfunction were computed. In the total sample, almost all correlations of the CIS and CGAS scores with other indicators, except professional contact, were significant. These ranged from 0.06 for learning problems and YCGAS to 0.56 for CBCL-Total scores and PICGAS. Correlations were higher for girls than boys. The pattern of correlations for boys was less strong, as indicated by a smaller number of significant correlations and lower correlations than for girls for various indicators. The correlations between CIS and CGAS scores with the other measures of psychological dysfunction provide support for the concurrent validity of the impairment measures.

Discriminant validity

This was estimated by comparing impairment scores between those using and not using services, assuming that the latter would be less disturbed and would have a higher level of functioning. All differences of CIS and CGAS scores in adolescents with or without professional contact in the previous 6 months were significant, indicating that those who had been brought to the attention of a professional had greater impairment and poorer psychosocial functioning. In addition, there were significant gender effects and a significant interaction of professional contact with gender, indicating that girls had greater impairment scores than boys on three of the five impairment scores: the PCIS, the PCGAS and the YICGAS scores. The interaction of professional contact by gender was non-significant only for the YCGAS score.

The effect of gender on impairment scores was also studied in a series of analyses of variance with presence or absence of any DISC diagnoses as an additional factor. Among the 82 boys, 16 had a DSMIII- R diagnosis according to the parent interview. The proportion in girls was 27 out of 121. Findings are shown in Fig. 1. The diagnosis factor was significant for all five scores. With the exception of the PCGAS score, gender was also significant for all other scores. In each instance, girls with a DISC diagnosis were rated as being more impaired and as functioning less well psychosocially. As a consequence, the respective interactions between diagnosis and impairment scores were significant. Low frequencies for one of the two genders prevented further analyses as to the differential effect of gender on various types of diagnoses.

Figure 1.

Gender effect on impairment scores according to presence of psychiatric diagnosis. Boys (n = 82), – – –; girls (n = 121), —. a, mean PCIS; b, mean PCGAS; c, mean PICGAS; d, mean YCGAS; e, mean YICGAS; f, ANOVA (F-values) for diagnosis, gender, interaction for all impairment scores

Discussion

This study, based on a Swiss epidemiological survey, tried to replicate and extend previous USA analyses of two measures of impairment [9, 10]. It corroborated the findings on the validity of both the CIS and the CGAS using a sample from a varied cultural background as well as different predictors of impairment.

In general, our findings support the concurrent validity of the two scales. This is particularly true for the girls' data but is less convincing for the boys'. Discriminant validity was excellent for all five impairment scores. Irrespective of gender, all subjects with professional contact or a formal DSM-III-R diagnosis had higher impairment scores than subjects for whom these two categories did not apply.

Perhaps most importantly, the present study found remarkable gender differences with respect to impairment scores. In general, girls with professional contact or with any DISC diagnosis were significantly more impaired than their counterpart boys. This finding suggests potential differences between the two genders in the consequences of psychiatric disorders. If girls are symptomatic in terms of a psychiatric diagnosis, they may be more severely disturbed and impaired.

If replicated, the marked gender differences for thresholds on all impairment scores make it mandatory to use these thresholds separately when calculating prevalence figures in epidemiological surveys. Interestingly, none of the previous studies based on CGAS ratings [5–7] has taken this issue into consideration.

Although these findings were derived from a community survey they also have clinical implications. Despite the fact that structured interviews have been designed primarily for research purposes, they are gaining increasing acceptance among clinicians, specifically for training and evaluation purposes. However, only the additional evaluation of impairment allows a relatively solid definition of caseness and, perhaps even more importantly, the decision of whether or not a child or adolescent requires treatment. If the present findings were to be replicated in clinical samples, clinicians must also be aware of these gender differences: girls, in general, need to be more impaired than boys in order to receive a psychiatric diagnosis.

Two limitations need to be acknowledged. Due to a limited sample size, it cannot yet be determined whether or not this general statement on gender-dependent impairment thresholds applies to specific psychiatric diagnoses. Further, a lower attrition rate for stage two would have been desirable. However, there is no evidence that the sample examined is not representative for the entire stage two sample.