Abstract
Objective:
To assess the screening accuracy of information obtained from parents of 4–5-year-old children for the purpose of identifying the children who have teacher-reported mental health problems when they are aged 6–7 years.
Method:
The study used data from the Longitudinal Study of Australian Children (LSAC) obtained when children were aged 4–5 years and 6–7 years. The level of children’s mental health problems was assessed using the Strengths and Difficulties Questionnaire (SDQ) completed by parents when children were aged 4–5 years and by teachers when children were aged 6–7 years (n=2163). When children were aged 4–5 years, parenting skills were assessed using three questionnaires developed for the parent-completed LSAC questionnaire and maternal mental health was assessed using the Kessler Psychological Distress Scale (K6).
Results:
When the level of parent-reported childhood mental health problems at 4–5 years old was used to identify children with teacher-reported mental health problems (i.e. a score in the “abnormal” range of the teacher-reported SDQ Total Difficulties Scale) when the children were aged 6–7 years, sensitivity was 26.8%, positive predictive value was 22.8%, and specificity was 92.9%. The addition of further information about the characteristics of children and their parents made only a small improvement to screening accuracy.
Conclusions:
Targeted interventions for preschool children may have the potential to play an important role in reducing the prevalence of mental health problems during the early school years. However, current capacity to accurately identify preschoolers who will experience teacher-reported mental health problems during the early school years is limited.
Introduction
In 2011 it was proposed that the “Healthy Kids Check” in Australia be extended to include screening for early signs of mental health problems. The aim was to identify preschool children who could benefit from early targeted interventions to ensure that they are healthy, fit, and ready to learn when they start school (Department of Health and Ageing, 2011). The proposal generated considerable controversy both in the general media and in academic journals (Alexander et al., 2013; Daubney et al., 2013; Jureidini and Raven, 2012; Kowalenko, 2012; Prior, 2012). In this journal, some authors highlighted the potential benefits of early screening for mental health problems (Newman, 2012) while others suggested caution (Jureidini and Raven, 2012; Prior, 2012). However, all agreed that better quality information about the accuracy of early screening for mental health problems was needed to guide policy development.
At a population level, there are two broad groups of interventions that can be employed to help reduce mental health problems experienced by children. These are called “universal” and “targeted” interventions (Lynch et al., 2010; Mrazek and Haggerty, 1994). Universal interventions are provided to all children in a designated population, regardless of whether or not they have problems. Targeted interventions are provided to children identified as needing help because of emerging problems and/or the presence of contextual risk factors (e.g. parental mental health problems) that are known to increase the risk for future mental health problems. By focusing on fewer children, targeted programmes may be more efficient than universal programmes. However, to be effective, targeted interventions require screening programmes that can enable early and accurate identification of children who will experience future mental health problems. Overidentification of false-positives in screening programmes can result in wasted resources and possible stigmatization of children who do not require help. Overidentification of false negatives may deny children access to help that may benefit them. However, if targeted interventions are low cost, highly effective, and non-stigmatizing then screening protocols with higher rates of false-positives may still be acceptable as children correctly identified will benefit and children falsely identified will not experience harms (Jones et al., 2002).
Several previous studies have investigated the extent to which screening assessments can accurately identify children who could benefit from targeted interventions due to their increased risk for future mental health problems (Bennett et al., 1999; Dwyer et al., 2006; Hill et al., 2004; Kim-Cohen et al., 2005, 2009; Lochman and The Conduct Problems Prevention Research Group, 1995). Three studies used self-report questionnaires completed by parents and teachers to screen children attending kindergartens in the USA and Canada (Bennett et al., 1999; Hill et al., 2004; Lochman and The Conduct Problems Prevention Research Group, 1995). The length of follow up in these studies varied from 12 months to 30 months and results focused on the specificity achieved (i.e. 1=false-positive rate) when cut offs on the questionnaire scores were chosen to achieve a sensitivity of 50%. At this level of sensitivity, specificity in the three studies ranged from 85% (Lochman and The Conduct Problems Prevention Research Group, 1995) to 93% (Bennett et al., 1999). In effect, this means that 7–15% of the children who screened positive in these studies did not have mental health problems when reassessed at follow up. In Australia, Dwyer et al. (2006) investigated the screening capability of a “family risk factor checklist” completed by parents and teachers to screen 7-year-old children in Australia (n=766) for the presence of externalizing and internalizing problems at 1-year follow-up assessment. The authors reported a somewhat lower level of screening accuracy than earlier studies with sensitivity ranging from 26% to 69% and specificity from 68% to 78%, depending on the outcomes being predicted and the informant reports utilized in the screening protocol.
The strongest predictor of future mental health problems is current level of mental health problems being experienced by children (Hill et al., 2004). However, a number of family and child characteristics can further increase the risk of children experiencing future mental health problems including parenting quality, maternal mental health, and children’s gender (Jenkins, 2008). It is possible that including information about these characteristics in a screening protocol may improve the protocol’s predictive accuracy. However, a major challenge is the large developmental variability in children’s mental health problems, with only 30–50% of children experiencing mental health problems that persist (Pihlakoski et al., 2006; Pine and Klein, 2008). A successful screening protocol must be able to accurately discriminate between children who will develop new problems or experience persisting problems and those whose problems will remit at a time when there is large variability in children’s rates of development. In addition a successful screening protocol needs to be practical for target users: in this case, general practitioners who have limited time and are not able to perform lengthy diagnostic assessments on all children.
The goal of the present study was to assess the screening accuracy of information obtained from parents and used to identify children aged 4–5 years who will have teacher-reported mental health problems when the children are aged 6–7 years. To do this, the study utilized data from the Longitudinal Study of Australian Children (LSAC) to assess the predictive accuracy of potential screening protocols employed with 4–5-year-old children to identify the children who had teacher-reported mental health problems (defined as a score in the “abnormal” range of the teacher-completed SDQ) 2 years later when children were aged 6–7 years. In the initial analyses, we employed children’s level of parent-reported mental health problems as the screening variable. Subsequent analyses assessed the extent to which inclusion of additional information describing child, parent, and family characteristics at age 4–5 years improved the predictive accuracy of our screening protocols.
Methods
Participants
The study utilized data for children aged 4–5 years at recruitment (mean±standard deviation 56.9±2.6 months), from LSAC, a nationally representative prospective study of young children’s development. The methodology, recruitment, and response rates have been described elsewhere (Soloff et al., 2006). In brief, LSAC used a two-stage cluster sampling design. Firstly, Australian postcodes were randomly sampled and stratified by state and rurality to ensure that the sample was geographically representative. Secondly, children born from March 1999 to February 2000 within each postcode were randomly sampled from their Medicare enrolment data. This process identified 10,275 children who were eligible to participate in the cohort aged 4–5 years at recruitment. Of these, 48.5% (n=4983) participated in the baseline face-to-face interview when children were 4–5 years.
In LSAC, information from parents was collected using face-to-face interviews and self-completed questions. Information from teachers was collected using self-completed questionnaires. When children were aged 4–5 years, 4983 parents completed face-to-face interviews and 4229 parents returned self-completed questionnaires (containing the SDQ and K6). At ages 4–5 and 6–7 years respectively, 3776 and 3609 teachers returned self-completed SDQ questionnaires. Parent-rated questionnaires were completed by the child’s biological mother in 99.3% of cases. The LSAC was approved by the Australian Institute of Family Studies Ethics Committee.
Characteristics of the children who participated in this study are shown in Table 1. The analyses for this manuscript are based on children described in the column labelled “Teacher sample” (n=2163). These were the children for whom teacher-rated SDQ data for 6–7 year olds and all screener variable data were available. The column labelled “Response sample” shows the number and characteristics of children for whom LSAC data were available from face-to-face interviews and/or self-complete questionnaires when children were aged 4–5 years. Comparisons with the characteristics of the children in the “Response” sample suggest that although a substantial number of participants were lost to the study sample (largely due to noncompletion of the self-complete questionnaires, which included the SDQ and K6, see Table 1), differences between the child and family characteristics across the groups were small. Furthermore, we conducted multiple imputation of missing data to address the potential bias due to missing data and analyses using the multiply imputed data did not substantially change the pattern of the results (see section Missing data).
Distribution of exposure and outcome for response sample and complete case sample (i.e. participants with no missing data at years 4–5 and 6–7 years).
Values are mean±standard deviation (%), mean±standard deviation, or %.
The substantial loss of participants with K6 scores is likely due to this questionnaire being presented in the “self-complete” questionnaire rather than during the “face to face interview”.
Measures
Outcome at age 6–7
Teacher-reported mental health problems
When children were 6–7 years, teachers completed the Strengths and Difficulties Questionnaire (SDQ; Goodman, 2001). The SDQ is a 25-item questionnaire that provides ratings of children’s emotional problems, hyperactivity/inattention, conduct problems, and peer problems and also a pro-social behaviour score. Scores in the four problem areas can be combined to generate a total difficulties score. For the purpose of this study, we dichotomized teacher-reported total difficulties scores using recommended cut offs (Goodman, 2001) to identify children scoring in the “abnormal” range (coded as “1”) vs. children scoring in the “borderline” or “normal” ranges (coded as “0”).
Screening variables at age 4–5
Parent and teacher-reported mental health problems
When children were 4–5 years, parents and teachers completed the SDQ (Goodman, 2001). The parent-completed SDQ is a parallel version of the questionnaire completed by teachers with the same number of items and scoring procedures. The total difficulties score was dichotomized as per the teacher version above.
Parenting practices (warmth, inductive reasoning, consistency, and self-efficacy)
Parent questionnaires employed in LSAC (Australian Institute of Family Studies, 2011) were used to assess parental warmth, consistency, and self-efficacy when children were aged 4–5 years. The warmth and consistency scales each have five items and the inductive reasoning questionnaire has two items. On each scale, parents respond using a 5-point Likert scale (1=“Never/Almost never” to 5=“Always/Almost always”). For calculating the sensitivity and specificity of the individual variables, total scores on each scale were dichotomized (1=“<average group score”; 0=“≥average group score”). Parents rated their level of parental self-efficacy using one item: “Overall, as a parent, do you feel that you are: (1) Not very good at being a parent; (2) a person who has some trouble being a parent; (3) an average parent; (4) a better than average parent; or (5) a very good parent.” For calculating the sensitivity and specificity of the individual variables, scores on this item were dichotomized (1=“not very good” “some trouble”, or “average”; 0=“better than average” or “very good”).
Parental concern about children’s socio-emotional development
LSAC utilized a single item to assess parent’s level of concern about their child’s socio-emotional development. This is an important issue because of evidence that parental perceptions play a key role in children’s service use. The LSAC item asked parents to rate their level of concern about their child’s emotional wellbeing, happiness, and behaviour using a 5-point scale. For calculating the sensitivity and specificity of the individual variable, responses to the item were dichotomized as 1=“a little bit”, “some”, “quite a bit” or “a lot”, and 0=“none”.
Psychological distress
Mothers reported their current symptoms of psychological distress using the Kessler K6 scale (Kessler et al., 2002). The K6 is a six-item measure of non-specific psychological distress during the past four weeks (Kessler et al., 2003, 2010). The total score is the sum of scores on the six items with higher scores indicating more problems. A score ≥13 is considered to indicate “possible serious mental illness” (Kessler et al., 2003, 2010). For calculating the sensitivity and specificity of the individual variable, the total score was dichotomized (1=“≥13”; 0=“<13”).
Parent and child demographic characteristics
Information about child and family demographic characteristics was obtained from parents. Financial hardship was assessed by asking parents to rate whether or not they had experienced any of seven hardships during the last 12 months (e.g. due to shortage of money being unable to pay for gas, electricity, or telephone on time). Responses were summed to generate a total score which ranged from 0 to 7. For calculating the sensitivity and specificity of the individual variable, the score was dichotomized (1=“≥1”; 0=“0”).
Statistical analyses
Initially we assessed the ability of screening variables to identify children with teacher-rated mental health problems at age 6–7 years by calculating sensitivity, specificity, and positive predictive value using the dichotomized screening variables. Next multiple logistic regression analyses were used to examine the strength of the association between screening variable scores when children were 4–5 years and the presence of teacher-reported mental health problems (defined as a teacher-reported SDQ total difficulties score in the “abnormal range”) when children were 6–7 years. For the purpose of these analyses, we also used dichotomized screening variable scores to facilitate comparisons between the size of odds ratios obtained from the logistic regression analysis and the predictive accuracy of the individual variables based on sensitivity, specificity, and positive predictive value.
The accuracy with which the screening variables could discriminate the 6–7-year-old children with teacher-reported mental health problems from the children without teacher-reported mental health problems was characterized using the area under receiver operating characteristic (AUROC) curves. For these analyses, we used the screening variable scores in their original continuous format. Receiver operating characteristic (ROC) curves plot the fraction of true positives (i.e. sensitivity) vs. the fraction of false-positives (i.e. 1 – specificity), corresponding to each value of a prediction model. In the present study, the area under the ROC curve (or AUROC) is an indication of how well the score on a screening variable when children are aged 4–5 years discriminates between children with mental health problems at age 6–7 years and children without mental health problems at 6–7 years. An AUROC of 0.5 would indicate that the screening protocol has no ability to discriminate; an AUROC of 1.0 would indicate perfect discrimination (all children with mental health problems having a screening protocol score above the chosen cut off). An AUROC of 0.74, for example, indicates that a randomly selected “case” (i.e. a child with mental health problems at age 6–7) has a 74% chance of scoring higher at age 4–5 than a randomly selected “non-case” (i.e. child without mental health problems at age 6–7).
We tested the capacity of three different screening models to identify children with teacher-reported mental health problems at age 6–7: model 1: parent-reported problems at age 4–5 years; model 2: model 1 plus child’s gender, mother’s relationship status and employment, financial hardship, parental concern about social-emotional development, parenting (warmth, inductive reasoning, consistency, and self-efficacy), and mother’s level of psychological distress; and model 3: model 2 plus teacher-rated mental health problems at age 4–5 years.
To compare the AUROC for the three models, we used the “roccomp” command in Stata 12.1. The “roccomp” command uses the permutation test described in DeLong et al. (1988). The permutation test assumes that the ROC curves are arising from two or more measures applied to the same person and follows a chi-squared distribution. Thus, the results from the three ROC curves were compared using a chi-squared test.
Missing data
Multiple imputation by chained equations was performed using the “mi” command in Stata 12.1 to address the potential bias from missing data. Missing data were imputed for the full sample and analysed for the 3609 children with complete data for the outcome – teacher-rated SDQ scores at 6–7 years (i.e. we did not analyse imputed outcomes; von Hippel, 2007). The imputation model included all variables in the analysis and seven additional predictors of missingness including factors such as parental education, ethnicity of parents, number of siblings, and number of times the family had moved house. The imputation process assumed that data were “missing at random” and the extra predictors of missingness accounted for the probability of having missing values (Rubin, 1976). We generated 20 data sets and undertook 50 cycles of regression switching (Royston, 2004). Characteristics of the response sample, complete case and imputed samples are shown in Appendix A in Table 1A (available online).
We analysed the data using both multiply imputed and complete-case data (i.e. data from children for whom SDQ data at 4–5 years and 6–7 years, and all screening variable data were available – the teacher sample). At this stage, it is not possible, as far as the authors are aware, to combine ROC curves across imputed data sets to present these graphically, and combining rules do not exist to calculate chi-squared tests to compare the average estimate of the AUROC across imputed datasets. Therefore, we present the complete-case analysis in the manuscript and include the results of the multiple imputation in Appendix A. As noted earlier, the results using the imputed data were not substantively different from the complete-case analysis and would not change the conclusions of the study.
Results
In Figure 1, it can be seen that 42 of the 157 children with teacher-reported mental health problems at 6–7 years had “abnormal” scores at 4–5 years (i.e. sensitivity=26.8%), while 42 of the 184 children who had “abnormal” scores at 4–5 years had teacher-reported mental health problems at 6–7 years (i.e. positive predictive value, PPV=22.8%). As well, 1864 of the 2006 children who screened negative at 4–5 years, did not have teacher-reported mental health problems at 6–7 years (i.e. specificity=92.9%). Finally, 142 of the 2006 children who did not have problems at 6–7 years screened positive at age 4–5 years (i.e. false-positive rate=7.1%).
For children with teacher-reported problems at age 6 years, the percentage who had persisting problems from age 4 years as rated by parents in comparison to the percentage who developed problems after 4 years of age (n=2163).
We also examined the capacity of parent-reported internalizing and externalizing problems on the SDQ at age 4–5 years to identify teacher-reported internalizing and externalizing problems at age 6–7 years (Appendix B). Sensitivity and PPV were somewhat higher for externalizing problems than internalizing problems, but there was little difference in the level of specificity between the two problem areas.
Table 2 shows the sensitivity, PPV, and specificity values for a range of child and family characteristics that could be utilized for screening purposes when children are 4–5 years old to identify teacher-reported mental health problems when children are aged 6–7 years. The challenges faced by a screening programme for 4–5-year-old children are clearly highlighted by the results in this table. For example, it can be seen that use of parental concerns as a screener in the LSAC cohort identified 76.4% of the children who would later be found to have teacher-reported mental health problems at 6–7 years. However, even though sensitivity was relatively high, 55.6% of the children identified using this screener would not have teacher-reported mental health problems at 6–7 years (i.e. they would be “false-positives”). Furthermore, only 9.7% of the children who screened positive at age 4–5 years would later be found to have teacher-reported mental health problems at 6–7 years. This demonstrates that from among the screening variables, although parental concerns identifies the largest percentage of the children who will have problems at age 6–7 years, only a small percentage of the total group of children who screen positive with this screener at 4–5 years will experience teacher-reported mental health problems at 6–7 years.
Proportion of children with teacher-reported problems at age 6–7 years that would be identified on the basis of predictors at age 4–5 years (n=2163).
Parenting (including self-efficacy, warmth, inductive reasoning, consistency) were all assessed as “0”=average and above; “1”=below average.
PPV, positive predictive value.
Results from the logistic regression analyses (Table 3) show that, with the exception of maternal psychological distress, parenting self-efficacy, inductive reasoning, maternal unemployment, and warmth, at a bivariable level all of the child and family characteristics examined at age 4–5 years were associated with increased risk for teacher-reported mental health problems when children were aged 6–7 years. It is important to note that the strength of such associations when assessed in this way provides little information about the predictive accuracy of variables when used for screening purposes. For example, while the parent-reported SDQ score at age 4–5 years was associated with a 4.8-fold increase in risk for teacher-reported mental health problems at age 6–7 years, as a screening variable only 22.8% of children identified as “at risk” at age 4–5 years on the basis of this variable will have teacher-rated mental health problems at age 6–7 years. After adjustment for the effect of the other screening variables, the size of the odds ratios assessing the strength of the association between the child and family characteristics and teacher-reported mental health problems was attenuated. For example, screening positive on the parent-reported SDQ was associated with a 2.7-fold increase in the risk of teacher-reported problems at 6–7 years (Table 3).
Predictors of teacher-reported (n=2163) mental health problems for children at age 6–7 years.
Variables were adjusted for all other variables entered into the model.
Parenting (including self-efficacy, warmth, inductive reasoning, consistency) were all assessed as “0”=average and above; “1”=below average.
PPV, positive predictive value.
The predictive accuracy of the three potential screening measures used to identify children at age 4–5 years who had teacher-reported mental health problems at age 6–7 years was compared using ROC analyses (Table 4). Model 1 achieved an AUROC of 0.71, which improved slightly in model 3 to 0.81. A sensitivity of 50% was associated with false-positives (1 – specificity) that ranged from 24.3% in model 1 to 8.5% for model 3. While, the chi-squared tests comparing model 1 vs. 2 and model 2 vs. 3 suggest that adding additional screening variables improved the predictive accuracy, it is important to note that the additional information that could be obtained from parents (model 2) did not greatly improve the predictive accuracy over that achieved in model 1. Figure 2 contains the ROC curves for the three models tested to identify teacher-rated mental health problems at age 6–7 years. The figure shows that there was some degree of overlap in the curves for model 1 vs. 2, and model 2 vs. 3.
Comparison of models to identify children with teacher-reported (n=2163) mental health problems on the SDQ at age 6–7 years.
Model 1: parent-rated problems at 4–5 years; model 2: model 1 plus gender, mother’s relationship status and employment, financial hardship, parental concern about social-emotional development, parenting (warmth, inductive reasoning, consistency, and self-efficacy), and mother’s level of psychological distress; model 3: model 2 plus teacher-reported problems at age 4–5 years.
AUROC, area under the curve; SE, standard error.
p-value tests null hypothesis that there is no difference in the AUROC of the models tested.
Receiver operating characteristic curves assessing screening accuracy for mental health problems rated by teachers at age 6–7 years utilizing (n=2163): model 1: parent-rated problems at 4–5 years; model 2: model 1 plus gender, mother’s relationship status and employment, financial hardship, parental concern about social-emotional development, parenting (warmth, inductive reasoning, consistency, and self-efficacy), and mother’s level of psychological distress; model 3: model 2 plus teacher-reported problems at age 4–5 years.
Discussion
The aim of this study was to assess the predictive performance of three potential screening protocols that utilized information from parents of 4–5-year-old children and their families in the LSAC cohort with the goal of identifying children who had teacher-reported mental health problems (defined as scoring in the “abnormal” range on the teacher-reported SDQ total difficulties score) when children were aged 6–7 years.
In the LSAC cohort of children, a potential screening protocol based on whether children had screened positive for parent-reported mental health problems at age 4–5 years (i.e. they scored in the “abnormal” range on the parent-reported SDQ) had a sensitivity of 26.8%. That is, when children were 4–5 years old, the protocol identified 42 of the 157 children who had teacher-reported mental health problems when children were aged 6–7 years. However, only 22.8% of children identified at age 4–5 years had teacher-reported mental health problems at age 6–7 years. This means that 77.2% of the children who were identified as “at risk” at age 4–5 by this screening protocol were falsely being labelled “at risk”. From the perspective of prevention, in a world where interventions were 100% effective and were available to help these children, use of this screening protocol would enable prevention of problems in 26.8% of 6–7-year-old children. However, a further 77.2% of children who didn’t have problems at age 6–7 years would receive a preventative intervention that they did not require and be placed at risk for stigmatization and being incorrectly “labelled” as having mental health problems from a young age.
The difficulty of accurately identifying young children at risk for future mental health problems poses a major challenge for the implementation of effective targeted intervention programmes. Several different approaches have been used to try and improve predictive validity. These include the use of additional information describing the characteristics of children’s parents and families (Bennett et al., 1999; Hill et al., 2004), combined use of parent and teacher reports (Lochman and The Conduct Problems Prevention Research Group, 1995), and broadly based screening measures designed to identify multiple family risk factors (Dwyer et al., 2006). While these approaches have resulted in some improvement in the sensitivity and specificity of screening techniques, they are all associated with relatively high rates of misclassification. For instance, in the present study, improvements in screening accuracy for teacher-reported mental health problems at age 6–7 could be achieved if teacher-reported SDQ scores were utilized when children were 4–5 years old, but this would substantially increase the workload of general practitioners undertaking Healthy Kids Checks.
Bennett et al. (1998, 1999) recommended that to be useful, at a minimum a screening protocol should identify at least half of the children who have later problems (sensitivity=50%) and at least half of the children identified by the protocol should subsequently be found to have problems (PPV=50%). When we simulated a screening protocol based on parent-reported mental health problems at 4–5 years and utilized cut-off scores that identified at least half of these 6–7-year-old children (i.e. sensitivity=50%, and PPV=50%), we would falsely identify 526 4–5-year-old children (i.e. false-positive rate=24.3%). This false-positive rate was much higher than that required to meet Bennett’s criteria (i.e. a false-positive rate of ≤4%, in a sample with a prevalence of 7.3% for the condition of concern). Thus, the potential screening protocols tested in the present study do not meet Bennett’s suggested levels of sensitivity and specificity. It is important to note that the criteria suggested by Bennett are somewhat arbitrary. Also, the sensitivity, specificity, and PPV estimated in the present study are likely to be higher than those achieved in a screening programme implemented through routine clinical practice as it is unlikely that the whole population of children would attend for screening assessments.
More effective use of targeted interventions to help ensure that children are healthy, fit, and ready to learn when they start school is the key goal of the extended Healthy Kids Check (Department of Health and Ageing, 2011). However, when screening protocols are developed for such purposes, careful consideration needs to be given to the trade offs between sensitivity and specificity. This is important because the relative advantages of high sensitivity vs. high specificity depend on the nature of the problems being identified, the likelihood that identified children will receive effective treatment, the potential stigma associated with treatment, and the costs of treatment programmes (Glascoe et al., 1997; Jones et al., 2002). For example, if treatment is expensive and large numbers of children are wrongly identified as being at risk for future problems (i.e. low specificity), the costs of wasted resources may be high. On the other hand, if costs of treatment are low and treatment is not intrusive or stigmatizing, low specificity may be of less concern. Jones et al. (2002) noted that as long as the negative consequences of inclusion in targeted interventions are avoided, children who are mistakenly labelled as high risk might still gain benefit from their participation in a targeted intervention. Similarly, if the personal and financial costs of a mental disorder to society are high, and treatment is effective and relatively inexpensive, lower levels of sensitivity may be acceptable because of the potential benefits of early intervention that interrupts pathways leading to a chronic and expensive mental disorder.
Our findings draw attention to several issues relevant to the proposal to extend the Healthy Kids Check. First, in Australia, there are relatively few clinicians with the skills needed to help young children with mental health problems. Indeed, many services struggle to cope with the number of children and families who already seek help on the basis of self-referral or referral from a primary practitioner. In these circumstances, there is a risk that many children who screen positive in a Healthy Kids Check will be unable to access appropriate care or treatment. There is also a risk that young children screening positive may be labelled or stigmatized as being at risk for mental health problems without having the opportunity for a diagnostic review of their problems and circumstances. Finally, although there is increasing evidence about the effectiveness of treatment regimens for child and adolescent mental health problems, information about the effectiveness of targeted interventions for preschoolers is limited. Furthermore, while there are programmes that are developing effectiveness for this age range (e.g. de Graaf et al., 2008), they are yet to be available widely and at low cost to parents.
One approach which could be used to improve our ability to accurately identify children who will develop future mental health problems or experience persisting problems is to better integrate screening and clinical practices. During early childhood, there is large variability in rates of development and the age when mental health problems both emerge and persist. Distinguishing between transient and persisting problems is a common challenge for clinicians during this period. Screening practices typically place emphasis on maximizing the accuracy of a single assessment procedure. In contrast, clinicians utilize their opportunity to reassess children on several occasions and observe developmental progress over time. This allows clinicians to identify whether a child’s development is following a normal or aberrant course over time. More explicitly linking these approaches has the potential to improve the effectiveness of early identification programmes designed to reduce the incidence and prevalence of child and adolescent mental health problems.
Strengths of this study are its use of a large representative cohort of children in the general community and the relatively high retention rate of participants. Limitations include the use of teacher-report questionnaires rather than a diagnostic assessment to identify children with mental health problems at 6–7 years. Furthermore, the data utilized in the present study was collected in children’s homes from parents who had given informed consent for their children to participate in a longitudinal study of child health and wellbeing. As such, it is possible that the quality of this data and the screening results identified in our analyses exceed those that might be achieved in a busy general practice. However, it is also possible that direct observation of children in clinical practice, such as would be possible by a general practitioner completing a “Healthy Kids Check”, may add information not included in the present study that could improve screening accuracy.
In conclusion, targeted interventions directed at the preschool age group have the potential to play an important role in reducing the prevalence of childhood mental health problems. However, accurate identification of individual preschool children who in the absence of an effective intervention will have future problems is an essential prerequisite for the effective use of such interventions. Findings from the present study show that accurate identification continues to pose a major challenge for targeted interventions aimed at achieving “real world” effectiveness in reducing childhood mental health problems in the Australian community. As well, screening programmes can only be purposeful and valuable for children and families if all identified children have access to low-cost, effective, and non-stigmatizing interventions. In Australia, this goal is yet to be achieved.
Footnotes
Acknowledgements
This paper uses unit record data from Growing Up in Australia, the Longitudinal Study of Australian Children (LSAC). The study is conducted in partnership between the Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA), the Australian Institute of Family Studies (AIFS) and the Australian Bureau of Statistics (ABS). The findings and views reported in this paper are those of the author and should not be attributed to FaHCSIA, AIFS.
Funding
JWL is supported by an Australia Fellowship from the National Health and Medical Research Council of Australia (570120). ACPS and MNM are also supported by funds from the Australia Fellowship awarded to JWL.
Declaration of interest
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.