Abstract
Objective:
To describe the study design of Young Minds Matter: The second Australian Child and Adolescent Survey of Mental Health and Wellbeing. The aims of the study, sample design, development of survey content, field procedures and final questionnaires are detailed.
Method:
During 2013–2014, a national household survey of the mental health and wellbeing of young people was conducted involving a sample of 6310 families selected at random from across Australia. The survey included a face-to-face diagnostic interview with parents/carers of 4- to 17-year-olds and a self-report questionnaire completed by young people aged 11–17 years.
Results:
The overall response rate to the survey was 55% with 6310 parents/carers of eligible households participating in the survey. In addition, 2967 or 89% of young people aged 11–17 years in these participating households completed a questionnaire. The survey sample was found to be broadly representative of the Australian population on major demographic characteristics when compared with data from the Census of Population and Housing. However, adjustments were made for an over-representation of younger children aged 4 to 7 years and also families with more than one eligible child in the household.
Conclusion:
Young Minds Matter provides updated national prevalence estimates of common child and adolescent mental disorders, describes patterns of service use and will help to guide future decisions in the development of policy and provision of mental health services for children and adolescents. Advancements in interviewing methodology, addition of a data linkage component and informed content development contributed to improved breadth and quality of the data collected.
Keywords
Introduction
Young Minds Matter: The second Australian Child and Adolescent Survey of Mental Health and Wellbeing (YMM) was conducted in 2013–2014 by the Telethon Kids Institute, The University of Western Australia (UWA) on behalf of the Australian Government Department of Health. The survey is part of the National Survey of Mental Health and Wellbeing initiative and complements data collected in the population surveys of adults and of people living with psychotic disorders conducted in 2007 and 2010 respectively (Morgan et al., 2012; Slade et al., 2009).
The first child and adolescent component of the National Survey of Mental Health and Wellbeing was conducted in 1998. The 1998 survey was the first of its kind, both nationally and internationally, to measure prevalence and burden of mental disorders in children and adolescents. The survey provided important data on the prevalence of mental disorders, the burden associated with these disorders and the services used in their management. It reported that 14% of children and adolescents in Australia had experienced mental health problems in the previous 6 months, determined by a score in the clinical range on the Child Behaviour Checklist (CBCL), and only approximately one quarter of these children had received professional support during the last 6 months (Sawyer et al., 2001). For the past 17 years, these data were the only source of Australian data at a population level that were available to inform mental health policy and service planning for children and adolescents. Over that period, there has been substantial growth in mental health services designed to meet the needs of Australia’s child and youth population.
The second Child and Adolescent Survey of Mental Health and Wellbeing (YMM) provides updated prevalence estimates, and new information on service use. It provides data on the prevalence of disorders in the population and the impact of these, as well as whether those children have received services for mental health problems. The types of services used, whether their needs have been met and whether additional services were wanted were also explored. Findings have been designed to inform the development of future policy and provision of services in the area of child and adolescent mental health.
Methods and materials
Survey methodology
Sample design
YMM was a national survey involving a random probability-based sample of 5500 children aged 4–17 years. The sample size of 5500 was chosen to deliver reliable estimates of prevalence, burden and service use by gender and age groups 4–11 years and 12–17 years. A multi-stage, area-based sample selection procedure was employed to select the areas for interviewing. First, Statistical Area Level 2s (SA2s) as defined by the Australian Bureau of Statistics (ABS) (2011), representing a community both socially and economically with an average population of 20,000, were stratified by state/territory and by metropolitan versus rest of the state to ensure proportional representation of geographic areas across Australia. Then from these selected SA2s, 550 Statistical Level Area 1s (SA1s) were selected. SA1s are smaller than SA2s with an average population of 400 people. It was anticipated that on average, each SA1 would achieve 10 interviews to produce the required sample size of 5500 families. SA1s were randomly selected with probability proportional to the number of children aged 4–17 years from the 2011 Census of Population and Housing.
Oversampling
An oversample of 16- to 17-year-olds was included in the survey. This sample is of particular interest due to onset of risk behaviours related to mental health in this age group and an increasing level of contact with mental health services (Igra and Irwin, 1996). In addition, past evidence has shown poor response rates to surveys in this age group. The oversample allowed for more precise prevalence estimates of individual disorders to be generated for this age group. The resulting data also enable some basic comparison to be made with the findings on 16–17-year-olds in the 2007 National Survey of Mental Health and Wellbeing of Adults (age range of survey 16–85 years). The oversample of 800 16–17-year-olds was recruited through additional screening in approximately 50% of the SA1s selected in the main sample.
Field procedures
Interviewer training
Fieldwork was undertaken by Roy Morgan Research (RMR). Training of interviewers was conducted by RMR with close and involved support from the UWA survey team. The 3-day training course included information on survey content and mental health literacy. Lay interviewers were trained not only to standardise the administration of the interviews, but also to improve their mental health literacy to better cope with responses.
Maximising response rates
Several key strategies were employed to maximise response rates. Targeted respondent material was used to inform and encourage participation. These included a primary approach letter, parent/carer and young person’s brochure and survey website along with a celebrity endorsement flyer. Interviewers offered reimbursement payments to participants in recognition of their time commitment in completing the survey, ($40 for parents/carers interview and $20 for young people).
Interviewers were required to make six call attempts to establish contact with households. Contact attempts were required to be spread over weekdays, weekends and a range of different times of the day. Once contact was made with a household, up to six further call backs were specified to complete the survey with the selected household. If it was not possible to gain participation from the selected household, the interviewer tried to obtain some basic information about the household. This included collecting information about the location of the dwelling, the reason for refusal and if the respondent was willing, information about the basic structure of the household including number and ages of in-scope children and the family structure to allow for comparison between survey participants and non-participants.
Additional component
Data linkage and matching
Linking survey data to relevant administrative data can provide important additional information about participants without increasing respondent burden. Medicare, Pharmaceutical Benefits Scheme (PBS) and the National Assessment Programme – Literacy and Numeracy (NAPLAN) data were identified as useful sources of additional information for the survey.
Medicare and PBS data will supplement information captured in the parent/carer and youth service use modules. Written consent was sought to access data from birth and prospectively up to 2 years following the survey period. Consent for linkage to Medicare and PBS data was provided by 76% of parents/carers participating in the survey.
NAPLAN data will complement information captured in the parent/carer and youth education modules by providing a further measure of school performance. Parental/carer consent was sought to access all available retrospective NAPLAN results from 2008–2013 and prospective results from 2014 through to 2016. Parents/carers of 86% of all 5–17 year olds participating in YMM provided consent for linkage to NAPLAN data.
Ethics
The research protocol for the study was approved by the Australian Government Department of Health Human Research Ethics Committee, and The University of Western Australia Human Research Ethics Committee.
Content of the survey
The survey comprised two components: a parent/carer interview and a questionnaire to be completed on a tablet-computer by young people aged 11–17 years.
Parent/carer interview
The interview was conducted via Computer Assisted Personal Interview (CAPI) with the primary carer of the survey child. All participants were asked questions in all modules. These included the following:
Family structure;
Child health Diagnostic Interview Schedule for Children (DISC-IV) modules (Shaffer et al., 2000);
Level of functional impairment questions;
Strengths and Difficulties Questionnaire (SDQ) (Goodman, 1997) Parent Report Measure Baseline versions for Children (4–10 years) and for Youth (11–17 years);
Service use in the past 12 months;
Perceived need for services;
Education;
Family characteristics;
Kessler Psychological Distress Scale (K10) (Kessler et al., 2003) for primary parent/carer;
Demographics.
See Table 1 for a full description of modules included in the parent/carer interview.
Content of the parent/carer interview.
DISC-IV: Diagnostic Interview Schedule for Children-IV.
The parent/carer questionnaire in the first national survey consisted of the Diagnostic Interview Schedule for Children IV (DISC), the CBCL, Child Health Questionnaire (CHQ), basic demographics, a module on service use and questions on physical health.
Diagnostic module
To maintain consistency with the first child and adolescent national survey and allow for maximum comparability of prevalence estimates, the DISC-IV was utilised (Shaffer et al., 2000). The DISC-IV operationalises the diagnostic criteria as specified in the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) (American Psychiatric Association, 2000). In 2013, the DISC-IV remained the best tool world-wide for assessing 12-month prevalence of mental disorders. The modular format of the DISC-IV questionnaire allows relevant disorders to be selected for inclusion.
DISC-IV modules were selected for inclusion on the basis of prevalence and impact of the disorder. Conduct disorder, major depressive disorder and Attention-deficit/Hyperactivity disorder (ADHD), which were collected in the first survey, were included. Anxiety disorders, not included in the first national survey, were also covered through four diagnostic modules – social phobia, separation anxiety disorder, generalised anxiety disorder and obsessive-compulsive disorder.
Measure of emotional and behavioural difficulties
The first national survey included both a parent/carer and youth-reported CBCL as the key measure of emotional and behavioural difficulties. The SDQ (Goodman, 1997) was selected to replace the CBCL in the second survey. While having equivalent discriminatory and psychometric properties to the CBCL, the SDQ is substantially shorter taking an average of 5 minutes to administer rather than 15–20 minutes for the CBCL. Since the first survey, the SDQ has also been widely adopted in data collections across Australia including state and territory telephone surveys, the Longitudinal Study of Australian Children (LSAC) (Department of Family and Community Services, 2004) and the National Outcomes and Casemix Collection established in state and territory mental health services. The SDQ has also been designated by the Australian Institute of Health and Welfare (AIHW) as the preferred source of a national headline indicator for measuring social and emotional wellbeing of children and adolescents (AIHW, 2012). Its wide use in Australia allows for a range of ‘crosswalks’ to be made between the survey and other relevant data collections.
Impact on functioning
Severity of the impact of mental disorders on functioning was assessed using a specially developed measure. Seventeen items were selected based on those used for the assessment of impairment in the DISC-IV (Shaffer et al., 2000), and some of the items were based on items used in the Brief Impairment Scale (Bird et al., 2005). These assessed the impact of symptoms across four domains: the child or adolescent’s educational or workplace participation, their social development, family activities and other family members and on the child or adolescent themselves, particularly distress.
Ordinal covariance methods, along with Item Response Theory, were used to assess these items and their suitability for use as a composite scale. A final Graded Response Model was fitted to the item data to construct a composite score measuring severity of impact on function. The score was standardised and categorised into mild, moderate and severe impact consistent with the epidemiological estimates underpinning work in progress on the National Mental Health Services Planning Framework (G Stewart, 3 October 2014, personal communication).
An overall level of severity was then assigned taking into account diagnosis of mental disorders, the impact on function category and if there had been any suicide attempt in the previous 12 months. It should be noted that a suicide attempt in the previous 12 months, if indicated, was a sufficient indicator for a ‘severe’ classification to be assigned although most young people with a suicide attempt would have been classified as severe based purely on impairment to functioning.
Service use
Capturing information on the types of services used by children and adolescents for the treatment of emotional and behavioural problems was a core aim of YMM. Service use questions from the first survey were updated to reflect current service provision. The parent/carer questionnaire covered questions on health services, school services, telephone counselling services and other online services for emotional and behavioural problems used over the 12 months prior to the survey. Medication use in the previous two weeks was also captured. Additionally, parents/carers were asked if they thought that their child required more help and if there were any barriers to seeking or receiving help for emotional or behavioural problems over the 12 months prior to the survey.
Demographic characteristics
Information was obtained on the demographic characteristics of participants and their families including parent/carer education, job status, household income, home ownership and indicators of financial stress. For comparability, where possible, demographic questions were harmonised with the Australian Bureau of Statistics Census standards. Interviewers also provided a rating of the condition of houses and yards in the area, availability and condition of open space areas such as parks, the attitude of households approached and the presence of security features on the houses in the area.
Child health
A number of questions were included in the parent/carer questionnaire to capture chronic health conditions, disability and emotional or behavioural problems as diagnosed by a health professional. These questions replaced questions from the CHQ (Landgraf et al., 1996), as used in the first survey.
Additional content
Modules on the survey child’s education and family information were also included to provide contextual information regarding child and adolescent development and wellbeing. The education module in the parent/carer questionnaire captured the educational status and school performance of the child or adolescent. The family characteristics module included questions on the parent/carer’s mental health, assessed via the Kessler 10+ (Kessler et al., 2003). A 6-item version of the general functioning subscale of the McMaster Family Assessment Device assessed family functioning (Boterhoven de Haan et al., 2015). In addition to this, a list of life stress events, aligned with questions from the Longitudinal Study of Australian Children (Department of Family and Community Services, 2004) was included as well as questions on parent/carer substance use.
Youth self-report questionnaire
The youth self-report questionnaire was completed by young people in private on a tablet-computer. Modules included:
Demographics;
Education;
Use of internet and electronic games;
Strengths and Difficulties Questionnaire (SDQ) Youth Self Report Baseline version (Goodman, 1997);
Child health utility-9D (CHU-9D) (Stevens, 2012);
Kessler Psychological Distress Scale (K10+) (Green et al., 2010);
DISC-IV major depressive disorder module (Shaffer et al., 2000);
Level of functional impairment questions;
Service use in past 12 months;
Youth risk behaviours;
Presence of symptoms of psychosis;
Level of self-esteem and protective behaviours.
See Table 2 for a full description of modules included in the youth self-report questionnaire.
Content of youth self-report questionnaire.
DISC-IV: Diagnostic Interview Schedule for Children-IV.
The lower age limit of the youth self-report questionnaire in the second survey was 11-years-old in order to capture important changes as children transition through puberty. Due to the perceived sensitive nature of some questions, children younger than 12 years were not asked about self-harm and suicidal behaviours or sexual behaviours and answered limited questions on alcohol, cigarettes and other drugs.
In the first survey, a self-report module was included for all adolescents aged 13 to 17-years-old. The questionnaire included the CBCL, Child Health Questionnaire (CHQ-CF87), the Centre for Epidemiological Studies Depression Scale (CES-D) along with questions on drug use and suicidal behaviour from the Youth Risk Behaviour Surveillance System (YRBSS) questionnaire (Kolbe et al., 1993).
Diagnostic module
There was limited scope to include diagnostic modules in the youth self-report within the parameters of acceptable response times. Just one module on major depressive disorder was included. A ‘self-report youth major depressive disorder diagnostic module’ was developed specifically for the survey in collaboration with Associate Professor Prudence Fisher at Columbia University, who played a key role in the development of the DISC-IV. To minimise respondent burden, this module was limited to diagnose major depression, and did not include questions for dysthymic disorder. The module provides information on this disorder in adolescence and evidence of the sometimes disparate views of various informants on young people’s mental health.
Measures of emotional and behavioural difficulties
In line with the removal of the CBCL-YSR from the parent/carer questionnaire, the youth-report CBCL-YSR was replaced with the self-report SDQ. In addition, to both supplement the inclusion of the youth diagnostic module and to provide a crosswalk to the adult survey and other national collections, the K10+ was added to the youth self-report questionnaire. This included a question on days out of role, additional questions on aggression and attention and a positive mental health question (Green et al., 2010).
Low prevalence was the major barrier to the inclusion of diagnostic modules to measure psychosis. Items that could be used as possible indicators of prodromal states congruent with psychosis were included. Questions on psychosis symptoms were drawn from the DISC-IV diagnostic instrument. These questions were included in the youth self-report questionnaire for adolescents aged 14 years and above.
Service use
Due to the more complex language of the service use module and the relatively small proportion of 11- and 12-year-olds utilising mental health services independently without a parent/carer, this module was administered to young people 13 years and above. Similar to the parent/carer interview, questions covered health services, school services, telephone counselling services and other online services for emotional and behavioural problems used over the previous 12 months. Young people were also asked if they thought that they needed more help and if there were any barriers to seeking or receiving help for emotional or behavioural problems over the 12 months prior to the survey.
Quality of life
The Child Health Utility-9D was includedin the youth self-report questionnaire (Stevens, 2012). It captures functioning over nine different dimensions, for example, if the child or adolescent had problems with sleep, ability to do their homework or carry out their usual routine. This replaced the health-related quality of life measure included in the CHQ (Landgraf et al., 1996) in the first national survey.
Education
To supplement the education module in the parent/carer interview, the youth self-report questionnaire included questions on education status, school connectedness and academic performance.
Internet use
Technology has become a much more important part of young people’s lives since the first national survey was conducted. Questions were included to assess levels of Internet and electronic game use and any possible negative impact this may have on young people. A tailored questionnaire based on the European Union (EU) Kids Online survey was developed for this purpose (Livingstone and Haddon, 2009).
Youth risk behaviours
The first national survey utilised the Youth Risk Behaviour Surveillance System (YRBSS) to capture information on youth risk behaviours (Kolbe et al., 1993). Updates made to the YRBSS questionnaire since the first survey were reviewed and the following areas of content were included in the questionnaire: tobacco use, alcohol and other drugs, deliberate self-harm (without intent to end their own life) and suicide, bullying, eating disorders and sexual behaviour. Content was tailored to be age appropriate and age cut-offs or restrictions of the number of items were set for particular questions. With regards to smoking and alcohol, young people aged 11–12 were asked screening questions about ever using these, while young people 13 years and older were asked more extensive questions about use and amount. Other drug use was asked of young people 13 years and older, self-harm and suicidal behaviour was asked of young people 12 years and older and sexual behaviour of young people 13 years and older.
Self-esteem and protective factors
The Adolescent Self-Esteem Questionnaire was used to assess self-esteem. This 13-item questionnaire, targeted specifically to adolescents, was developed by the UWA survey team. This questionnaire replaced the CHQ (Landgraf et al., 1996) which was used to measure self-esteem in the first survey.
Questions relating to protective factors, and in particular positive mental health and resilience, were included in the youth self-report questionnaire. Questions drawn from the National Comorbidity Survey – Adolescent Supplement about social support formed the basis of this module (Merikangas et al., 2009).
Testing
A comprehensive pilot and cognitive testing process was conducted to inform the development of survey content, assess its meaning and suitability to respondents and finalise field administration procedures prior to main fieldwork. This included cognitive testing of the parent/carer and youth interview with participants from both general and clinical populations, and a more extensive pilot test followed by a formal dress rehearsal. Feedback from these processes resulted in improvements to the parent/carer interview and youth questionnaire as well as the procedural and operational aspects of the survey. Further detail regarding testing of survey content and procedures can be found in the Survey User’s guide (Telethon Kids Institute, 2015).
Main survey
Fieldwork for the main wave of data collection was conducted between 31 May 2013 and 10 April 2014.
Screening
As there is no list of households with children aged 4- to 17-years-old, the number of households selected in each SA1 was calculated based on the proportion of households with children 4–17 years at the 2011 census. During the fieldwork, a consistent trend was observed to find about 19% fewer households than expected in-scope. Despite investigations, no reason for this could be determined. Sample size per SA1 was adjusted for the second half of the fieldwork for the difference in the projected number of households with 4–17 year olds from the 2011 Census and the reality of numbers found in the first half of the survey fieldwork. In total, 76,606 households were approached during the survey.
Interview duration
The Parent/carer interview ranged from 20 to 255 minutes with a mean duration of 66 minutes and a median duration of 60 minutes. This excluded the time taken to complete the consent booklet and the initial screening of the household. Overall average interviewer time in the household was 75 minutes. Time taken to complete the youth self-report questionnaire ranged from 11 to 172 minutes, with a mean of 36 minutes and a median of 33 minutes.
Response rates
The overall response rate by selected households was 55%. In total, 6310 parents and carers of eligible households participated in the survey. In addition, 2967 or 89% of young people aged 11–17 years, for whom their parents or carers had given permission, completed a youth self-report questionnaire. Response rates for national surveys have been declining in recent years (Tourangeau and Plewes, 2013). The response rate for the 2007 Adult National Survey of Mental Health and Wellbeing was 60% (Slade et al., 2009). The length of the survey interview may have been a contributing factor with the most common reasons cited for non-participation being too busy or insufficient time. Due to the lower-than-anticipated response rate, it was particularly important to assess the representativeness of the sample and test for possible non-response bias.
Sample representativeness
Comparison with 2011 Census data showed that the YMM sample was broadly representative of the Australian population in terms of major demographic characteristics. Data was compared on Socio-Economic Indexes for Areas (SEIFA), population distribution, age, sex and country of birth of the total population of 4- to 17-year-olds in Australia and demographic characteristics including household income, family type, household tenure, parent/carer education and labour force status of families with children aged 4–17 years. Selected indicators are included in Table 3, no major differences were observed for those indicators not included in this table.
Sample representativeness.
SEIFA: Socio-Economic Indexes for Areas.
‘Original family’ has at least one child living with their natural, adoptive or foster parents and no step children.
‘Other family’ is where all children are not the natural, adopted, foster or step child of one or both carers.
Note: that the over-representation of young people 16–17 years in Table 3 is due to the inclusion of the oversample of 16–17-year-olds in the figures.
However, the sample was found to include a higher proportion of children aged 4–7 years than would be expected based on random sampling with 34.1% of the main sample aged 4–7 years, compared with the 28.8% in the 2011 Census. This pattern has been observed repeatedly in social surveys including the first national survey, the 1993 WA Child Health Survey and the WA Aboriginal Child Health Survey. This is most likely due to families with younger children having lower workforce participation, and therefore being more likely to be available and more willing to make time available to participate.
There was also a lower proportion of families with only one eligible child, with 37.7% of all participating families having one child compared with 45.8% of those in the 2011 Census. As the survey was based on a random sample of families from which one in-scope child per household was randomly selected, the unweighted sample numbers would be expected to have a higher proportion of children in families with just one child aged 4–17 years. This pattern has also been observed repeatedly in other social surveys and is also most likely to be due to workforce participation. Families with multiple children may be more likely to have a parent or primary carer who does not work or works part-time and therefore are more likely to be available and willing to participate in the surveys.
While the response rate to the survey reflects the trend to declining participation in social surveys over the last few decades, analyses of the final survey sample against the 2011 Census found it to be representative of the Australian population on SEIFA, population distribution, family structure, parent/carer country of birth and housing tenure,housing income, education status and labour force status of parents/carers.
Weighting
The survey data were weighted to represent the Australian estimated resident population of children and adolescents aged 4–17 years as at 30 June 2013 provided by the ABS (2014). Weights were calculated using the generalised raking procedure of Deville and Sarndal (1992). Weights were calibrated by sex, age (in single years), family size and household income. A separate set of weights was calculated for the youth self-report questionnaire data.
Conclusion
Australia is distinguished by its initiatives in the measurement of the mental health status of its population. The first child and adolescent component of the National Survey of Mental Health and Wellbeing represented the first time any nation had attempted to undertake a population survey of the mental health of its young people. The first survey provided a basis for planning and development of child and adolescent mental health services in Australia. Not unreasonably, it might be expected that the new findings from the second Australian Child and Adolescent Survey of Mental Health and Wellbeing will be critical in underpinning the ongoing progress in mental health sector reform by providing contemporary estimates on the prevalence of mental health problems, the burden associated with these disorders and the services used in their management, and the consumer response to their adequacy in meeting their needs.
The data collected in the survey will be made available to interested researchers through access to the confidential unit record file (CURF). This will provide a continuing source of high-quality information through the onward work of scientists, scholars, practitioners, policy makers and service providers and produce continuing benefits to guide the next generation of Australian mental health care.
Footnotes
Acknowledgements
The authors would like to thank the over 6000 families that gave their time to participate in the survey, Rajni Walia, Gerry Bardsley and the team at Roy Morgan Research who undertook the data collection, and the Survey Reference Group for their input into the design and conduct of the survey. Professor Zubrick is supported by a Centre of Excellence grant from the Australian Research Council (CE40100027).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The second Australian Child and Adolescent Survey of Mental Health and Wellbeing was funded by the Australian Government Department of Health.