Abstract
Keywords
The rate of mental illness in children and adolescents, estimated to be 14% in Australia [1], is a major concern. Early mental health problems can result in significant hardship to the child and his or her family including educational, financial, vocational, social and personal difficulties. Despite the general recognition of the problems associated with neuropsychiatric disorders, the overall burden to society of mental illness in children and adolescents is probably under-estimated [2].
In Australia, medical services are funded on the basis of diagnostic-related groups that reflect the type and chronicity of the problem. However, the diagnosis of mental health conditions in children and adolescents is problematic, and a contentious issue for paediatricians, general practitioners, and even for clinicians working in multidisciplinary mental health settings. Furthermore, the value of categorical diagnosis is often debated, especially within mental health teams.
Current operational diagnostic criteria for child mental health disorders include the impairment caused by the symptoms [3]. That is, diagnosis is often dependent on relevant symptoms resulting in substantial distress or social impairment for the young person and his or her family [4], [5]. According to Simonoff et al. [6] mental health disorders should only be diagnosed if symptoms result, at the very least, in partial impairment in two areas, or severe impairment in one. Not surprisingly, the social impact of the child's behavioural and emotional problems predicts the likelihood of a referral to a service specializing in child and adolescent mental health, with the burden on the family being particularly important [7].
Parent and teacher-rated measures of childhood behavioural and emotional functioning are available to assist clinicians screen and assess important mental health problems. Measures such as the Child Behaviour Checklist (CBCL) [8] and the Behavioural Assessment System for Children (BASC) [9] have good psychometric properties and are considered sensitive to common childhood behavioural problems. However, the CBCL and BASC are comprehensive instruments requiring 20–30 minutes to complete, and as such, are often inappropriate for screening purposes. Furthermore, measures of childhood behaviour problems such as the CBCL and BASC fail to take into account issues such as social impairment, as well as family distress and burden, which are important for diagnostic purposes. While the CBCL includes a ‘social competence’ scale, which examines how the child functions in different domains of life, it does not frame this in the context of impairment or burden. The BASC, another popular measure, does not measure social impairment at all.
The Strengths and Difficulties Questionnaire (SDQ) [10–12] is a relatively new measure that has features that may make it more appealing than traditional questionnaires. First, it is a short questionnaire that requires only 5 minutes to complete. It was designed for parents and teachers of 4–16-year-olds, and there is a self-completion version for young people aged 11–16 years. The SDQ has been shown to be a useful screening instrument that captures the main childhood behaviours and emotions that are problematic [13]. For example, a previous report suggests that the SDQ predicts diagnostic categories reliably, correctly identifying 81–91% of the children with a clinical diagnosis [16]. The SDQ has an impact supplement that assesses chronicity, distress, social impairment and burden to others [11], providing additional information for clinicians that is necessary for the diagnosis of mental health disorders and for prioritizing referrals to mental health services.
The SDQ includes a scoring algorithm that identifies three broad mental health diagnoses, namely conduct disorders, emotional disorders and hyperactivity disorders. This algorithm has been used in studies in London and Dhaka clinics, which reported a high correlation between SDQ generated diagnoses and diagnoses made by clinicians [16]. The aim of this study was to replicate these studies in a community child and adolescent mental health clinic in Australia to determine whether the level of agreement between clinical diagnoses and diagnoses generated by the SDQ is as high in Australia as that reported overseas, albeit in a naturalistic setting rather than a research setting.
Method
A multidisciplinary child and adolescent mental health team in Werribee (outer south-west Melbourne), Victoria, administered the SDQ routinely to parents, teachers and 11–16-year-olds prior to the first clinic attendance. The diagnoses generated by the SDQ algorithm was compared with the diagnoses of clinicians, replicating the methodology used by Goodman et al. [16].
Sample
The sample consisted of 130 consecutive new referrals to the clinic aged 4–15 years who had a parent SDQ completed, and who were subsequently assessed at the clinic. Teacher SDQs were completed for 101 (78%) participants, while the self-report SDQ was available for 38 of the 49 participants (76%) aged 11–16 years. The mean age of the sample was 9.3 years (SD = 2.9; range 4–14 years). Eighty-two were boys (63%) and 48 were girls (37%).
Clinical diagnoses
Clinicians made a clinical diagnosis for each participant according to the DSM-IV diagnostic manual [4], as is routine clinical practice in this CAMHS. No additional training was undertaken for the specific purpose of this study. Blinded to the identity of participants, the chief investigator (JM) assigned the team clinician-rated diagnoses into one of three diagnostic categories adopted in Goodman et al.'s study [16]: hyperactivity/inattention disorders; conduct disorders; and emotional disorders. Diagnoses that clearly did not fit into these categories (e.g. autistic disorders and psychosis) were excluded. In addition, an independent clinician (psychiatrist) at the Royal Children's Hospital, who was unrelated to the study and blind to the SDQ findings, examined each participant's file and made a clinical diagnosis after reading the case notes. For all cases, each disorder was rated as absent, borderline or present by the independent clinician.
Instruments
Questionnaires
The parent and teacher versions of the SDQ for children aged 4–16 years were used. The SDQ self-report version (children aged 11–16 years) was completed by older children.
All forms of the SDQ contain 25 items which are summed to generate five clinical scale scores (hyperactivity, emotional symptoms, conduct problems, peer problems and prosocial [positive] behaviour). The questionnaire has been tested extensively in the UK, and the items are in the form of brief statements about the child, for example, ‘many worries, often seems worried’ or ‘has at least one good friend’. Parents are asked to rate behavioural observations on a three-point scale with the responses ‘Not true’, ‘Somewhat true’ and ‘Certainly true’. The SDQ was normed on a large British population of children and clinical cut-off scores are available. It has good agreement with validated instruments such as the Rutter questionnaires [14] and the CBCL [15].
Diagnostic algorithm
The SDQ algorithm was designed to identify four broad categories: conduct disorders (including oppositional disorder); emotional disorders (including anxiety, depression, obsessive-compulsive disorder, phobias); and hyperactivity disorders (including attention deficit hyperactivity disorder and attention deficit disorder), as well as an ‘any psychiatric disorder’ category [16]. As mentioned above, the algorithm excludes diagnoses of autism or psychotic disorder. The algorithm combines scores on each scale from the different raters and incorporates the impact scores to arrive at a statistical prediction as to the likely or unlikely presence of the disorder. The predictive algorithm generates ‘unlikely’, ‘possible’, or ‘probable’ ratings for the four categories.
The original a priori algorithm predicted that a disorder was probably present on the basis that the relevant symptom score was above the 95th centile and the impact score was two or more (i.e. impact was ‘quite a lot’ in two or more domains or ‘a great deal’ in one domain). The criteria could be met according to just one rater (the parent) for the prediction of conduct or emotional disorders, but had to be met for both parent and teacher raters for the prediction of a hyperactivity disorder.
Results
Table 1 illustrates the diagnoses predicted by the SDQ algorithm for this sample. In terms of hyperactivity/inattention, 56% of the sample had a possible or probable diagnosis. For conduct disorders, 74% of the sample was predicted to have a possible or probable diagnosis, while 65% of the participants had a possible or probable diagnosis of emotional disorder. The algorithm also predicted that 91% of the sample had a possible or probable psychiatric disorder in any one of these three categories. These findings supported the view that nearly all the children presenting at this clinic during the timeframe of this data collection had a clinically important mental health problem.
Strengths and Difficulties Questionnaire generated diagnoses
Level of agreement between clinical team diagnoses and SDQ diagnoses
One hundred and nineteen patient files (92%) were analyzed. Of the remaining files, two were unavailable for review, and the remainder (n = 9) excluded according to the exclusion criteria. To examine the level of agreement, team clinical diagnoses were cross-tabulated with SDQ diagnoses. Agreement was assessed using Kendall's tau-b statistic, a measure of correlation suitable for ordinal data. For each disorder, the correlation between team clinical diagnoses and SDQ prediction was statistically significant (see Table 2).
Level of agreement between Strengths and Difficulties Questionnaire—generated diagnoses and clinical team diagnosis
The diagnoses given by the independent clinician correlated significantly with SDQ-generated diagnoses. However, it is important to note that his level of agreement was higher with the clinical team across all diagnoses (see Table 3).
Level of agreement (Kendall's tau-b) between the independent clinician's diagnoses with the Strengths and Difficulties Questionnaire diagnoses and the clinical team diagnoses
Sensitivity of the SDQ
The sensitivity (true positive rate) of the SDQ for the present sample, or how accurate the SDQ is at detecting broad mental health disorders was assessed by cross-tabulating SDQ diagnoses based on the algorithm with clinical team-rated diagnoses which were categorized into one of three broad categories. The sensitivity of probable SDQ diagnoses was 36% for emotional disorders, 44% for hyperactivity disorders and 93% for conduct disorders. In contrast, the sensitivity of combined possible and probable SDQ diagnoses was 81% for emotional disorders, 93% for hyperactivity disorders and 100% for conduct disorders. Serious false negatives, that is children who had a definite disorder but who were rated unlikely by the SDQ algorithm, were rare for conduct disorders (n = 0) and hyperactivity disorders (n = 2), but more frequent for emotional disorders (n = 7).
Discussion
In an Australian sample of children aged 4–15 years referred to a community child and adolescent mental health clinic, the SDQ identified a psychiatric disorder in 91% of participants. Based on the diagnoses generated by the SDQ algorithm, this sample had a particularly high rate of conduct disorders (74%), while the prevalence of emotional disorders (65%) and hyperactivity disorders (56%) was also significant. In terms of predicting mental health problems, it is proposed that the SDQ is more appropriate than other instruments given that in addition to assessing behavioural and emotional problems, it assesses the social impairment and family burden of the child's symptoms.
We found that the SDQ algorithm had reasonable to high levels of agreement across all mental health domains, with the lowest correlation being with emotional disorders (Kendall's tau-b = 0.39). In general, externalizing behaviours are easier to detect due to their greater ‘visibility’, possibly explaining the high rate of agreement for conduct disorders and the moderate rate of agreement for emotional disorders which represent internalizing behaviours. In this study, the sensitivity of the SDQ-generated diagnoses varied from 36% for emotional disorders to 93% for conduct disorders. These findings differ from those reported in London and the Dhaka (Bangladesh) clinics [16] as can be seen in Table 4. While the sensitivity of SDQ-generated diagnoses was consistently high across clinics for conduct disorders, the sensitivity for emotional and hyperactivity disorders in this Australian sample was substantially lower. Possible explanations for the differences in sensitivity rates between Melbourne, London and Dhaka clinics may be due to cultural factors such as parental perceptions, differences in professional training on diagnostic criteria, or sampling characteristics. However, it should be noted that we found the SDQ to be significantly more sensitive when possible and probable ratings were combined (81% for emotional disorders, 93% for hyperactivity disorders, 100% for conduct disorders).
The sensitivity of the Strengths and Difficulties Questionnaire in the Werribee clinic sample
It is important to note that the level of agreement generated in this study was based on the assumption that clinical judgement and diagnosis is the ‘gold standard’. This of course has its limitations, as has been widely reported in the literature [17]. Furthermore, this study was conducted in a naturalistic setting, which has methodological limitations when compared to similar studies undertaken in a research setting in which the clinicians were specifically trained to assign diagnoses (London, UK). In the present study, diagnoses were assigned by each patient's case manager as part of routine clinical practice. All future studies looking at the sensitivity of this instrument are advised to consider using standardized assessment procedures such as a structured psychiatric interview (e.g. DISC-IV) [18].
Nevertheless, these results substantiate the utility of the SDQ as an instrument to guide clinicians in screening and diagnosis. Specialist services are stretched for resources, and screening with the SDQ may aid this process. For example, knowing in advance that a particular child is at high risk of ADHD, conduct disorder or emotional disorder, should help streamline the assessment process and increase the diagnostic sensitivity and the efficiency of the clinic. Although the SDQ is useful for screening purposes and assisting diagnostic decisions, it is important to emphasize that the SDQ should not be utilized in isolation for making mental health diagnoses. The SDQ diagnoses had good agreement with clinical diagnoses when possible and probable categories were combined; however, it was also over-inclusive. Despite being broad, the SDQ does not assess all childhood psychiatric disorders and consequently it is inappropriate for screening children with suspected autistic disorders, psychotic disorders and other developmental disorders.
The use of the SDQ in child and adolescent psychiatry, paediatrics, as well as general practice and adult psychiatry in screening children of mentally ill parents, is recommended as it captures many of the disorders that are seen in children presenting with mental health problems. Mental health assessment and diagnosis is also difficult for many in non-medical disciplines and this questionnaire could be a useful aid for allied health and nursing professionals working with children with behavioural and emotional problems. Since the SDQ is brief, valid, captures behaviours from multi-informants, and has an impact scale which measures burden and social impairment, it is considered a useful screening tool for children presenting with mental health issues.
Footnotes
Acknowledgements
This study was funded by the Clinical Initiative Fund of the Royal Children's Hospital. We thank all participating children, parents and teachers, and all staff at Werribee CAMHS. We also thank Prakash Chidambaram, Rhea Hornby and our colleagues who provided feedback on earlier drafts of this paper.