Abstract
Epidemiological studies have provided substantial information about the prevalence and distribution of childhood mental disorders, the factors in children's social and family environments which influence the onset and maintenance of disorders and the factors which influence patterns of service utilisation (for brevity, the term children is used to refer to children and adolescents). The availability of better quality information in these areas has facilitated rational planning of prevention and treatment programs for children with mental disorders [1–3].
In Australia and New Zealand, several studies have examined the prevalence of mental health problems experienced by children and adolescents [2],[4],[5]. In Queensland, Connell et al. [4] surveyed 986 children aged 10–11 years using the same methodology as that employed by Rutter et al. in the Isle of Wight Study [6]. Connell et al. [4] reported an overall prevalence of psychiatric disorder of 14%%, with children living in rural areas having a somewhat lower prevalence of disorders than children living in urban areas. Sawyer et al. [2] surveyed 528 children aged 10–11 years or 14–15 years living in metropolitan Adelaide using the Child Behaviour Checklist [7] and reported that 21%% of children in the younger group and 15%% in the older group scored above the recommended cut-off score on the checklist. These authors drew attention to the very small number of children with problems who had attended child and adolescent mental health services and contrasted this with the much larger numbers seen by general practitioners, hospital-based services and school counsellors.
To date, the most comprehensive study undertaken in Australia is the Western Australian Child Health Study [5],[8],[9]. In this study, 2737 children aged 4–16 years living in 1462 households were assessed using the Child Behaviour Checklist [7] and the Teacher Report Form [10]. Zubrick et al. [5] reported that 18%% of the children had a mental health problem, defined as a score above the recommended cut-off on either the parent or teacher checklist. The most frequently reported problems were on the ‘delinquency problems’, ‘thought problems’ and ‘attention problems’ scales on the two checklists. Consistent with the results from the Adelaide study, Zubrick et al. [5] reported that during the 6 months prior to the study, only 2%% of children with mental health problems had attended a mental health service while 65%% had been seen by a general practitioner and 9%% by a school counsellor.
Aims and initial planning
Consistent with the adult component of the National Survey of Mental Health and Wellbeing, the child and adolescent component was designed to answer three questions:
How many children and adolescents in Australia have which mental disorders?
What is the degree of disability associated with these disorders?
What is the pattern of service utilisation by children and adolescents with and without mental disorders?
Initial discussion involving the Mental Health Branch of the Commonwealth Department of Health and Aged Care (formerly the Department of Health and Family Services) and the Strategic Management Group (responsible for the three surveys comprising the National Survey of Mental Health and Wellbeing) established a number of parameters within which the child and adolescent survey was to take place. These included: (i) establishing clear aims for the study; (ii) identifying the budget for the study; (iii) commissioning the University of Adelaide as the lead agency to conduct the child and adolescent survey; and (iv) deciding on the use of a tendering process rather than commissioning to identify the organisation which would conduct the fieldwork. The research group based at the University of Adelaide was advised by a National Collaborating Group which consisted of senior researchers drawn from around Australia.
It was planned that both diagnostic and dimensional approaches would be employed to assess childhood disorders in the survey. The diagnostic approach ensured consistency with results reported in the survey of adults and helped to ensure that the results would be readily understood by clinicians and health administrators. The dimensional approach provided a broad assessment of childhood emotional and behavioural problems and has facilitated comparisons with results from other studies. The self-report questionnaires employed with the dimensional approach also made it possible to obtain information from older children as well as parents.
An established aim for the present study was that it should obtain national estimates of the prevalence of mental disorders. To achieve this, a sample size of 4500 children aged 4–17 years was identified for the study. The sample size allowed for acceptable precision in prevalence estimates at a national level with confidence intervals of ± 1%%, assuming a prevalence of childhood mental disorders of approximately 15%%. The lower age limit for the survey was determined by the capacity of the survey instruments to obtain valid and reliable information from younger children. The upper age limit of 17 years was determined by the lower age limit of the survey of adults.
Three other key issues need to be addressed when planning epidemiological studies. First, the assessment burden imposed on participants must be kept to manageable proportions. During the planning phase of such studies, there is a temptation to expand the focus to embrace a wide range of issues, thus imposing an unreasonable burden on participants. To address this issue, early in the planning for the present study, it was agreed that a household survey would be employed for data collection and an average of 1 hour would be allowed in each household. Second, it is important to avoid the perception that a study represents an unwarranted intrusion into participants' private lives. For the purpose of data collection, the Women's and Children's Hospital in South Australia was identified as the auspicing agency for the study. This approach was adopted because it was felt that a children's hospital would be recognised by participants as an organisation which is concerned about the welfare of children and adolescents.
Finally, epidemiological studies which employ household samples are vulnerable to biases arising from the possibility that certain types of families may be more difficult to contact at home than others. This necessitated a decision about the number of attempts which would be made to contact families who were not home at the time of an earlier attempted contact. In the present study, it was specified that four call backs should be made to a household where on a previous visit, contact was not made with the occupants.
Survey instruments
An extensive review of available survey instruments was undertaken. In addition, advice was obtained from researchers at the National Institute of Mental Health and the Centers for Disease Control and Prevention in the USA about the survey instruments and the methodology to be employed in the study.
From the range of alternatives, the Diagnostic Interview Schedule for Children (DISC-IV) was selected as the diagnostic instrument which would be employed to identify children suffering from mental disorders [11]. Although the DISC-IV is suitable for use with both children and their parents, time constraints in the present study restricted its use to the parents of identified children only. On average, the complete interview takes approximately 70min. However, the interview employs a modular format which made it possible to restrict the number of mental disorders to depressive disorders, attention deficit hyperactivity disorder, conduct disorder and eating disorders. These disorders were chosen for inclusion by the National Collaborating Group because of their public health significance for child and adolescent mental health in Australia. Due to time constraints, it was not possible to include other modules.
The Child Behaviour Checklist was chosen as the self-report instrument to be completed by parents to assess the presence of childhood mental health problems [7]. Its companion checklist, the Youth Self-Report was chosen as the instrument to be completed by adolescents aged 13–17 years [12]. Both these measures have been widely used in previous studies in Australia and overseas.
Disability was assessed using the Child Health Questionnaire (CHQ) which has parallel versions designed for independent completion by either parents or children aged 10 years or older [13]. The CHQ assesses the functioning of children in a range of areas including their physical health, role/social limitations due to mental health problems and the impact of children's health on their families. Ratings on the parent version of the questionnaire also provide information about the impact of children's illness on the quality of life of parents. Information about the validity and reliability of the CHQ is available [13]. In the present study, the CHQ was completed by parents of all children aged 6–17 years and by the 13–17-year-olds. The lower age of 6 years was selected because assessment in several areas is based on children's school functioning and in some states of Australia, children aged 4 or 5 years have not commenced school.
No good-quality questionnaires could be identified to assess children's service utilisation. The absence of standard measures with a clear conceptual base and known psychometric properties is a continuing problem in this area. As a result, different approaches have been used in previous studies, making it difficult to compare results across the studies. For the purpose of the present study, the approach used in the parallel survey of adults was largely adopted, although children's service utilisation was assessed over a 6 month period rather than over 12 months. This shorter period was adopted because of concern about the ability of parents to accurately recall service use by their children over a longer period.
Adolescents aged 13–17 years also completed the Center for Epidemiological Studies Depression Scale (CES-D) [14] and the Youth Risk Behaviour System Questionnaire (YRBS) [15]. The CES-D was added to the questionnaire battery because of concern about the prevalence of adolescent depression and suicide, and the possibility that parents may not be able to accurately identify subjective symptomatology experienced by adolescents [16]. In the present study, information about drug use and suicidal behaviour was obtained from the 13–17-year-olds using the YRBS which was developed by the Centers for Disease Control and Prevention in the USA to measure health risk behaviours of high school students.
Study sample
A multistage sampling scheme was used to obtain a representative sample of 4500 children aged 4–17 years. ‘Clusters’ of 10 fully responding households with children and adolescents in the required age range were sampled from each of 450 collectors' districts (CDs) across Australia. The number of CDs sampled within each state or territory were in proportion to the size of the target populations within those areas, and were also distributed proportionately across metropolitan and non-metropolitan areas.
Within each CD, a random starting point was nominated and interviewers then approached the household at this point. From this starting point, interviewers were requested to select every third adjacent household as potential participants until a total of 10 households were identified in the CD. Only one child was selected to participate from each household. The method of selection utilised the ‘birthday technique’ whereby the child with the birthday nearest the date of the interview is selected from all the eligible children in the household.
Four call backs were to be made to households where, on a previous visit to the area, interviews were not obtained. These call backs were to be made on different days of the week and different times of the day. If no contact was made with members of the household following this procedure, or if the household did not contain a child of the appropriate age, a replacement household was to be identified.
Study progress
The Australian Bureau of Statistics was commissioned to conduct the survey of adults. The advantage of this approach was the high quality of the services which could be offered by the Bureau. However, a disadvantage of commissioning is that it limits the opportunity for other organisations to compete for the work being commissioned. Furthermore, in the survey of adults, privacy considerations meant that the mental health researchers responsible for much of the design of the study were limited in their access to the fieldwork and to the raw data collected in the study.
For the child and adolescent study a process of tendering was employed to identify the organisation which would conduct the fieldwork. This approach offered several potential advantages. First, it provided an opportunity for a wider range of organisations to undertake the fieldwork than typically occurs with a commissioning process. Second, it offered the potential for the university research group to be more closely involved in the fieldwork and analysis of results using the raw data collected in the study. It was envisaged that a collaboration would be established between a government department (Mental Health Branch, Department of Health and Aged Care) which would fund the survey, a university (the University of Adelaide) which would monitor the quality of the survey and analyse the results, and a survey company which would carry out the fieldwork. Finally, the tendering approach offered the possibility of cost savings.
Following a meeting of the National Collaborating Group in Adelaide in late 1996, the contract between the University of Adelaide and the Commonwealth Government was finalised in January 1997. Ethics approval for the study was obtained from the Research Ethics Committee at the Women's and Children's Hospital in South Australia (affiliated with the University of Adelaide's Human Research Ethics Committee). The successful tender was accepted from the survey company in June 1997. A pilot study of 100 families randomly selected in Adelaide was conducted in August 1997. The pilot study was employed to assess the training program for interviewers, the participation rate using the protocol designed to introduce the study to households and the acceptability of the DISC-IV and the self-report questionnaires to parents and adolescents. Results from the pilot study suggested that all these elements of the study worked well.
Prior to collecting data for the main survey, all interviewers participated in a 3 day training program conducted jointly by research staff from the University of Adelaide and Senior Project Managers from the survey company. Fieldwork for the main survey took place from early February 1998 to late May 1998. Initial analyses by the university researchers found that the data files sent to the University of Adelaide in July 1998 contained a number of errors. As a result, all the self-report questionnaires and the interviews had to be returned to the survey company. The re-keyed data files and questionnaires were returned to the University of Adelaide in April 1999.
Data quality
A significant advantage of including a university research group in the study program was that high standards were set in the area of study design and data quality. Considerable time was spent preparing a detailed tender document which included clear specifications about the sampling methods to be employed and the checks to ensure high-quality data would obtained in the study. Subsequently, data quality has always been given the highest priority, even though this has at times caused a delay in the study timetable. For example, fieldwork for the main survey was delayed until February 1998, rather than it taking place during the summer months when many families would have been away from home on holidays. Similarly, although the requirement that the survey company re-key the data files delayed the release of results, it ensured that a high standard of data quality was achieved.
Overall, the participation rate in the study based on the number of households identified as having children which agreed to participate was 86%%. However, the ‘worst case’ estimate of the response rate based on the number of households which agreed to participate as a percentage of the total number of households where contact was attempted on at least one occasion was 70%%. The response rate in an epidemiological study is an important issue because of the possibility that the characteristics of non-participants differ from those of participants. If this occurs, the findings based on the analysis of results of the participants will not accurately reflect the pattern for the entire population from which the sample was drawn. The response rate reflects the effect of various factors including active refusal by potential subjects to participate and the number of potential subjects who cannot be contacted despite repeated attempts. The major factor which affected the response rate in the present survey was the replacement of households after only one or two attempted contacts by an interviewer (although the contract had specified that four contacts were to be made to all households). Once this factor was identified, the challenge for the research group at the University of Adelaide was to assess the impact of this on the results from the study. The concern was that children living in households where family members were more frequently at home, and thus more easily contacted by the interviewers, may differ from children in households where family members are less often home.
Two approaches have been used to identify possible biases in the results. Initial analyses compared the demographic characteristics of children and families who participated in the study with the characteristics of the population from which they were selected (based on the 1996 Australian census). In these analyses, particular attention was paid to potential biases which might arise if interviewers had enrolled more frequently those households where family members were more often at home. For example, there was concern that the sample might contain a disproportionately high number of two-parent families, families with younger children and families with higher rates of unemployment. When these issues were examined, however, it was found that although there were somewhat higher numbers of 4- and 5-year-olds in the sample, there was little evidence of other biases in the demographic characteristics of the participants.
The second approach to assess data quality involved making comparisons between the Child Behaviour Checklist and the Youth Self-Report scores obtained in the present study with scores reported in the West Australian Child Health Survey [5] and in the manuals describing the use of the two questionnaires [7],[12]. The results of these analyses showed that the distribution of scores on all the behaviour scales of both the Child Behaviour Checklist and the Youth Self-Report were very similar in the two Australian studies. Consistent with previous studies, however, there is some evidence in both studies that a higher proportion of Australian children scored in the normal range compared with children in North America [17]. Overall, the demographic characteristics of the study sample and the distribution of scores on the Child Behaviour Checklist and Youth Self-Report scales suggest that the study is representative of 4–17-year-old children in Australia and that the pattern of childhood emotional and behaviour problems is consistent with the pattern reported in previous comparable studies in Australia.
Conclusions
The Child and Adolescent Component of the National Survey of Mental Health and Wellbeing is the largest study of child and adolescent mental health conducted in Australia and one of the few national studies of child and adolescent mental health conducted in the world. A total of 4 509 children aged 4–17 years living in metropolitan and country regions of Australia were surveyed using a comprehensive battery of instruments designed to assess the prevalence of mental disorders, the disability associated with the disorders and the pattern of service utilisation exhibited by children with disorders.
The decision to conduct a survey of child and adolescent mental disorders has provided a unique opportunity to obtain a better understanding of the prevalence of child and adolescent mental disorders at a national level. While earlier Australian and New Zealand studies have provided considerable information about the prevalence of childhood mental disorders, they have all been limited to children living in a single state or region and the studies used varying methodologies to identify children with mental disorders. As a result, there has been no coherent picture of the prevalence and distribution of childhood mental disorders at a national level.
Several key elements contributed to the success of this study. First, the early identification of a concise set of aims and clear principles for the study, together with the decision to limit data collection to ‘1 hour in the household’ provided a clear reference point against which to test the content of the assessment measures and the nature of the study methodology. Second, the inclusion of an experienced university research group ensured that high standards were set for the survey methodology and for the quality of the survey data. The close involvement of this group in the training of the survey company's field staff also provided important insight into the strengths and limitations of the fieldwork. Third, the availability of a clear tender greatly facilitated the resolution of problems in areas such as the accuracy of sampling procedures and the verification of data entry. Fourth, the availability of advice from members of the National Collaborating Group was extremely important at points when key decisions had to be made about the study. The Group's existence has also created a sense of ownership of the study at a national level. Finally, ongoing support from staff in the Mental Health Branch, Commonwealth Department of Health and Aged Care has been invaluable in sustaining the university research staff through the various challenges that have arisen during the course of the study.
The initial report describing the results of the child and adolescent survey will be released in early 2000. Following this, a series of bulletins will be released that will focus on specific disorders and other key aspects of the study. These bulletins will be written in a style which ensures the results of the study can be easily understood by the lay public and by staff working in health, welfare and educational services.
Bringing the Child and Adolescent Component of the National Survey of Mental Health and Wellbeing to a successful conclusion has required an enormous amount of work by a team of people. It is hoped that the release of results from the study during 2000 will help professionals, consumers and carers advocate for better prevention and treatment services for the large number of children and adolescents with mental disorders in Australia.
Footnotes
Acknowledgements
The Child and Adolescent Component of the National Survey of Mental Health and Wellbeing was funded by the Commonwealth Department of Health and Aged Care. The authors would like to express their gratitude to D. Casey for his ongoing advice and encouragement during the course of the study. We would also like to thank E. Brinkley and the Australian Bureau of Statistics for advice about the study methodology, and M. Drake, P. Hazell, S. Henderson, R. McKelvey, S. Silburn, V. Storm, H. Winefield and H. Whiteford for their valuable contributions to the study.