Abstract
Background:
Adverse childhood experience (ACE) exposure is associated with increased risk of poor health outcomes. Several tools to identify ACEs during pediatric practice exist, but few include all 10 ACEs from the original ACE study and none have established predictive validity.
Objectives:
Assess predictive validity of the ACE score reported during routine pediatric practice using the Whole Child Assessment (WCA).
Methods:
This retrospective cohort study included children ages 3 to 8 years presenting for well-child care at a low-income resident clinic between May 25, 2016 and March 31, 2018, and ages 5 to 8 years presenting for well-child care at a private insurance clinic between November 1, 2017 and March 31, 2018. Patients with chronic health problems were excluded to prevent confounding by preexisting health problems. The charts of children with 0 to 1 ACEs (lower risk) and 2+ ACEs (higher risk) at baseline were reviewed to collect data on health and psychosocial outcomes at follow-up from diagnoses documented in the medical record and parent-reported outcomes on the WCA. Logistic regression models adjusted for age, gender, and clinic were used to analyze differences in outcomes. We hypothesized that children in the higher risk group at baseline would have more health and psychosocial problems at follow-up.
Results:
The initial cohort (n = 907) had 669 children with 0 to 1 ACEs and 238 children with 2+ ACEs. At follow-up (mean 718 days, range 329-1155 days), children in the higher risk group had statistically significantly higher rates of ADHD/ADD, school failure/learning problem, and other behavioral/mental problems. Parents of these children also reported on the WCA higher rates of children being nervous or afraid, feeling sad or unhappy, having trouble paying attention or staying still, getting angry or into fights, bullying, poor sleep, and healthcare utilization. There were no statistically significant differences in various physical health concerns measured.
Conclusion:
This study supports the predictive validity of the WCA to identify subpopulations at risk of developing poor mental health and social-emotional outcomes. While more research is needed to facilitate translation into pediatric practice, these results highlight the strong influence of ACEs on mental health outcomes.
Keywords
Introduction
Adverse childhood experiences (ACEs) include child physical, sexual, or emotional abuse; child physical or emotional neglect; and household dysfunction relating to incarceration, domestic violence, mental illness, substance abuse, or parental divorce or separation. 1 Leading causes of morbidity and mortality in adults, including chronic conditions such as heart disease, cancer, stroke, and diabetes, have been associated with ACEs.1-3 ACEs are also correlated to various health concerns in childhood, such as developmental delays, behavior problems, unhealthy weight, and early tobacco/alcohol use.4-8 The mechanisms through which ACEs increase risk of developing poor health and social outcomes have not been completely elucidated, but 1 pathway is speculated to be through activation of the body’s stress response, which can be damaging in excess and if not countered by protective factors.9,10 Early identification of ACEs through screening in pediatric practice can enable primary care providers to identify those at risk for developing poor health and social outcomes and to counsel or refer these patients to promote protective factors and build resilience. 11
Most research on health outcomes associated with ACEs has been done on adults who recalled early childhood experiences. The risk of poor outcomes in adulthood has been shown to increase with increasing ACE scores.12-14 There are several tools that have been developed to screen children for one or more ACEs, but only a subset have included all ten ACEs and none have demonstrated predictive validity.15,16 The current tool that we are using to screen for ACEs in pediatric practice, the Whole Child Assessment (WCA), has demonstrated concurrent validity using cross-sectional data.4,17-20 However, we have not yet examined the predictive validity of this tool based upon prospective data. We began screening for ACEs at all well-child visits in May 2016 at our resident clinic, and at all well-child visits for children ages 5 to 11 years at our faculty clinic in November 2017. Our early adoption of ACEs screening during routine pediatric practice provides us with a unique opportunity to examine the potential association between parent-reported ACEs and future child outcomes.
We aimed to test the predictive validity of the ACE score reported during routine pediatric practice using the WCA by analyzing differences in follow-up health and psychosocial outcomes between the lower risk group (0-1 reported ACE exposures at baseline) and the higher risk group (2+ reported ACE exposures at baseline). We hypothesized that children in the higher risk group would have more health and psychosocial problems at follow-up compared to children in the lower risk group.
Methods
Setting
Patients in the study were seen at 1 of 2 university-affiliated clinics in California. One clinic was staffed by residents with supervision by general pediatric faculty and served primarily low-income patients on Medicaid. The resident clinic adopted screening for ACEs using the WCA at all well-child visits in May 2016. The second clinic was staffed by general pediatric faculty and served patients with private insurance. The faculty clinic adopted screening for ACEs using the WCA at well-child visits for ages 5 to 11 years old in November 2017. Questions about ACEs, lifestyle, and clinical symptoms were included on the WCA, which was filled out by caregivers at baseline and follow-up well-child visits, reviewed with the pediatrician, and then scanned into the electronic medical record.
Pediatricians were offered self-directed reading and an algorithm on how to respond to ACEs, including a handout that could be shared with families about ACEs, resilience, and community resources. The self-directed reading included a brief Canvas module summarizing the literature on ACEs and resilience, and a link to the original ACE study. 1 The algorithm provided guidance as to how to counsel families with ACEs and included use of a patient handout with relevant community resources. The algorithm and patient handout are available online, https://lluch.org/health-professionals/whole-child-assessment-wca, and more information is described in a previous publication. 17
Subjects
This retrospective cohort study was approved by the IRB at our academic institution (IRB #5200139). Chart reviews were conducted for all children ages 3 to 8 years old who presented for well-child care at the low-income resident clinic between May 25, 2016 and March 31, 2018. To increase the socioeconomic diversity of our sample, we also reviewed charts of all children ages 5 to 8 years old who presented for well-child care at a private insurance clinic between November 1, 2017 and March 31, 2018. At follow-up, we reviewed charts of the initial cohort sample presenting between January 1, 2019 and March 31, 2020. The baseline period started with adoption of ACE screening at the respective clinics and ended at least 8 months prior to the start of the follow-up time period, which gap was intentional to make sure that there was a lapse of time between our calculation of baseline ACEs and our prediction of future health outcomes. The follow-up period ended at the time the clinics switched to video visits due to COVID-19 precautions. Since caregivers complete the WCA for children through age 11 years, whereas children ages 12 and older self-report, we intentionally chose a patient age group that would have caregiver report at both baseline and follow-up.
Patients with chronic health problems were excluded from the initial cohort to eliminate the possibility of confounding by the presence of preexisting health problems when evaluating the impact of ACEs on health outcomes. A chronic health condition was defined as any health problem that is likely to persist for more than 3 months (e.g., asthma) or acute problem that persisted over 3 or more months based upon medical record documentation (e.g., constipation that was documented as a diagnosis at least twice over 3 months or longer), and this included all diagnoses that we included as outcomes in our follow-up data (see Procedure). Patients were also excluded if an entire page of the WCA was not filled out or if 6 or more responses to the Child-ACE exposure questions were missing (see Statistical Analysis).
Procedure
To minimize measurement bias, 1 researcher conducted the baseline chart reviews and a second conducted the follow-up chart reviews without knowledge of baseline ACE scores. All charts were reviewed for total ACE score based upon the WCA. At follow-up, charts were also reviewed for health and psychosocial outcomes that have been previously reported to be associated with ACEs in pediatric patients. 8 These included allergic rhinitis/allergies, asthma, eczema/atopic dermatitis, urticaria, constipation, enuresis, encopresis, stomachaches, headaches, failure to thrive/poor growth, poor dental health, speech delay, other development delay, school failure/learning problem, attention deficit hyperactivity disorder (ADHD) or attention deficit disorder (ADD), autism, aggression/anger/poor impulse control/conduct, anxiety, depression/frequent crying, sleep disturbance, other behavior/mental problem, and other chronic disease. In addition, we evaluated health care utilization between the initial well-child visit and follow-up well-child visit by looking at the number of primary care problem visits, subspecialty visits, hospitalizations or surgeries, and emergency/urgent care visits documented in the electronic medical record during the 6 months prior to the follow-up visit. Parents were also asked about healthcare utilization in these categories on the WCA, and this data was used to capture health care visits outside of the medical records accessible by the researchers.
Statistical Analysis
Statistical analyses were performed using R version 4.0.2 and the dplyr package version 1.0.7. Descriptive statistics were used to calculate prevalence rates of responses to each question about exposure or risk of Child-ACEs. Charts with 6 or more missing responses to the Child-ACE exposure questions were excluded from the analysis, in keeping with previous research on the WCA. 18 For the remaining charts with 5 or fewer missing responses to the Child-ACE exposure questions, missing data was assumed to be negative, in keeping with standard statistical procedure to input the most common response for missing data and with previous research on the WCA.18,21 We substituted a negative value for all missing ACE scores, which was the most common value for all combinations of gender, age, and clinic for all ACE questions. We believed that the bias introduced would be minimal given the low number of missing values for most ACE questions and the small percentage of positive cases for most ACE questions.
For questions on the WCA, “unsure” was a response which was supposed to prompt providers to review the question with the caregiver. However, some scanned forms did not have any annotation from the physician to explain the answer or change it to a clear affirmative or negative answer, so we had to account for the “unsure” answer when performing an analysis of a binary variable. The “unsure” responses were dichomotized based upon the assumption of higher risk compared to a negative response. The same approach was used in previously published work on the WCA. 18
Total ACE score was used as an outcome in the analysis since we were examining the impact of accumulated ACEs. We used baseline data on total ACEs to divide the sample into a lower risk group (0-1 reported ACE exposures at baseline) and a higher risk group (2+ reported ACE exposures at baseline). Analysis of variance was used to evaluate whether patients in the low-risk and high-risk groups had similar rates of loss to follow-up. To evaluate the predictive validity of the baseline Child-ACE score, we used logistic regression models adjusted for age, gender, and clinic.
Results
There were 1267 eligible well-child visits in the baseline time period, of which 31 were excluded for pre-existing chronic conditions, 3 were excluded for insufficient WCA data at baseline, and 326 were excluded due to loss to follow-up (lack of data after the baseline visit). There was no statistical difference in loss to follow-up between the low-risk and high-risk groups (34% vs 42%, respectively, P = .124).
The resulting cohort group (n = 907) consisted of 669 (74%) children in the lower risk group (0-1 reported Child-ACE exposures at baseline) and 238 (26%) children in the higher risk group (2+ reported Child-ACE exposures at baseline) (Table 1). The most frequently reported ACE exposures were Parents Separated and Lack of Family Closeness. There was not a significant difference in baseline Child-ACE scores across age groups or clinics. However, there was a statistically significant greater proportion of male patients in the higher risk category (30% vs 23%, P = .019).
Prevalence of Baseline Reported Child-ACE Exposures by Covariates.
Indicates P-value ≤.05.
At follow-up (mean 718 days, range 329-1155 days), children in the higher risk group had statistically significantly higher rates of ADHD/ADD (OR: 4.11, 95% CI: 1.124-15.033, P = .033), school failure/learning problem (7.403, 1.921-28.527, P = .004), and other behavioral/mental problems (2.084, 1.087-3.995, P = .027) as documented in the electronic medical record (Table 2). At follow-up, parents of children in the higher risk group also reported on the WCA higher rates of children being nervous or afraid (1.807, 1.029-3.176, P = .040), feeling sad or unhappy (2.122, 1.123-4.012, P = .021), having trouble paying attention or staying still (1.978, 1.26-3.105, P = .003), getting angry or into fights (3.943, 2.158-7.206, P < .001), getting bullied (3.112, 1.638-5.913, P = .001), getting poor sleep (2.796, 1.81-4.317, P < .001), and healthcare utilization at other clinics (1.934, 1.092-3.424, P = .024) (Table 3).
Association of Reported Child-ACE Total (Exposures plus Risk) With Child Outcomes From Chart Review (n = 907).
Indicates P-value ≤.05.
Association of Reported Child-ACE Total (Exposures plus Risk) With Child Outcomes as Reported on Whole Child Assessment.
Indicates P-value ≤.05.
There were no statistically significant differences in other outcomes based on caregiver report (new illness, school grades) or medical record review of physical health concerns (autism, anxiety, allergic rhinitis/allergies/allergic conjunctivitis, asthma, eczema/atopic dermatitis, urticaria, constipation, dysuria, enuresis, stomachaches, headaches, failure to thrive/poor growth, poor dental health, any developmental delay, sleep disturbance) and healthcare utilization (primary care problem visits, subspecialty problem visits, hospitalizations/surgeries, urgent care/ED visits) (Tables 2 and 3). We did not track any outcomes for children who were excluded from the baseline cohort for pre-existing chronic conditions.
Discussion
Implications
This study demonstrates that ACE scores reported on the WCA during routine pediatric practice can be used to identify a pediatric subpopulation at higher risk of developing subsequent poor mental health and social-emotional outcomes. Our baseline sample were generally healthy pediatric patients ages 3 to 8 years old who did not have any chronic health problems. At follow-up, children with higher baseline ACE scores had statistically significantly higher rates of ADHD/ADD, school failure/learning problem, and other behavioral/mental problems as documented in the medical record. Although these are often co-morbid conditions, our findings across these conditions are consistent with prior studies of correlations with early ACE exposure.4,5,22 Also at follow-up, caregivers of children in the higher risk group reported on the WCA higher rates of children being nervous or afraid, feeling sad or unhappy, having trouble paying attention or staying still, getting angry or into fights, bullying, and poor sleep. While these symptoms do not necessarily mean that the children had higher rates of diagnosed mental and/or behavioral health issues, the parent-reported symptoms can indicate the early stages of these diagnoses. In combination, the medical diagnoses and caregiver-reported symptoms suggest the increased risk of psychosocial dysfunction after exposure to ACEs.
We did not demonstrate an association between baseline ACE scores and follow-up physical health outcomes. One possible reason is because the strength of association between ACEs and physical outcomes is smaller than mental health outcomes. In general, ACE exposure is associated with a smaller (about 2 times) risk for various physical health outcomes, but a greater (about 4-12 times or higher) risk for mental health and substance use problems.1,4-6,12 Another possible reason why we did not demonstrate associations with physical health outcomes is because of insufficient follow-up time to demonstrate effects on somatic systems. While the mean follow-up time was approximately 2 years, the range was between approximately 11 months and 3 years, which may have been an insufficient amount of time to cause physical health problems in generally healthy children. However, this relatively short duration of follow-up underscores how remarkable it is that we found any statistically significant differences between the lower and higher ACE groups and highlights the strong influence of ACEs on mental health outcomes.
We used a retrospective cohort study design to evaluate the impact of ACEs on future health outcomes, which was the strongest design available to us given that it would have been unethical to assign ACEs. Some strengths of our methodology include our careful exclusion of children with chronic health problems from the baseline sample in order to assure the temporal ordering of ACE exposure prior follow-up health problems. Another strength of this study was that data on ACEs was collected during the course of routine pediatric practice. While this likely resulted in underreporting of ACEs, it also means that our results are directly relevant to general pediatric practice. In addition, we minimized measurement bias by having different researchers collect baseline and follow-up data, we used both subjective (caregiver-reported) and objective measures (physician diagnoses) of health outcomes, and we verified that loss to follow-up did not differ for groups with higher and lower baseline ACE scores.
Limitations
Some limitations should be considered in interpreting the results of this study. We did not have information about race or ethnicity and so cannot comment on potential differences by race and ethnicity. We also had different but overlapping age groups at the clinic sites, due to the differences in implementation of WCA screening. However, all of our logistic regression models were adjusted for age, gender, and clinic site which overlapped with insurance type, an indicator of socioeconomic status.
It should also be noted that we used a lower cutoff than the cutoff of 3 or 4+ ACEs that is more typically reported in the literature. The reason we used a lower cutoff was because our data was collected during the course of routine pediatric practice, and therefore skewed toward lower ACE scores in contrast to surveys done in other settings (e.g., anonymous telephone surveys). However, the effect of using a lower cutoff would have biased our results toward the null, but despite this we still found differences in mental health outcomes at follow-up.
Lastly, it was beyond the scope of this study to measure the support or referrals that our pediatricians made in response to ACEs, and therefore we cannot comment on the effectiveness of pediatric interventions. However, even if our pediatricians started counseling and offering referrals at all well-child visits based upon review of documented ACEs (which is quite unlikely given the limited amount of physician training and the considerable challenges in changing physician practice), this would have biased our results toward the null. That is, this would have made it more difficult for us to find any differences between the lower and higher ACE groups. The fact that we still found differences in outcomes at follow-up suggests that any clinical intervention that did occur was insufficient to eliminate the impact of ACEs.
Further Research
This retrospective cohort study provides pilot data for the predictive validity of the WCA in identifying a pediatric subpopulation at risk for poor mental health and social-emotional outcomes. A multi-center study with longer follow-up time could further validate the predictive validity of the WCA for both physical and mental health outcomes.
Future research is also needed to evaluate the sensitivity and specificity of individual ACE scores in order to determine implications for clinical practice. In addition, clinical trials are needed to compare screened and unscreened populations in order to evaluate the potential benefits (or harms) of physician interventions to reduce the physical and mental health sequelae of exposure to ACEs.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
