Abstract
Pediatricians are encouraged to screen for adverse childhood experiences (ACEs). The current study developed and implemented a tool to screen for Child-ACEs at a pediatric resident clinic in San Bernardino, California. Development of the tool, named the Whole Child Assessment (WCA), was based on an iterative process that incorporated triangulation of references, patient data, and physician feedback. Implementation of the WCA occurred over the course of 6 improvement cycles that involved obtaining and responding to stakeholder feedback, streamlining paperwork and workflow, and providing physician education. Over the course of our study, we reviewed 1100 charts from well-child visits. We demonstrated that use of the WCA increased identification of multiple Child-ACEs compared with no screening and that reports of multiple Child-ACEs increased with age. These results suggest that use of the WCA provides an acceptable and feasible way to screen for Child-ACEs during routine pediatric practice.
Keywords
Introduction
Adverse childhood experiences (ACEs) as defined by the landmark ACE study include childhood exposures to abuse, neglect, and household dysfunction including parental separation, substance abuse, mental illness, domestic violence, and incarceration. 1 The majority of adults (52% to 70%) report exposure to 1 ACE, and many adults (6% to 21%) report exposure to 4 or more ACEs.1-5 High rates of ACEs exist in all socioeconomic strata and geographic locations, although the prevalence of ACEs is highest in low-income, impoverished regions. 3
A robust and growing body of evidence suggests that ACEs have a profound negative impact on virtually all aspects of health, including physical, mental, and emotional well-being. 6 In adults, ACEs are associated with leading causes of morbidity and mortality including heart disease, cancer, stroke, lung disease, and diabetes.1,3,7 In children, ACEs correlate with developmental delays, behavior problems, weight status, and injuries.8-10 If not countered by protective factors, exposure to ACEs can cause prolonged activation of the body’s stress response resulting in poor health outcomes, or so-called “toxic stress.”11,12
Given the prevalence and potential impact of ACEs, it behooves pediatric providers to take an active role in preventing and identifying childhood adversity in order to reduce the health consequences of toxic stress. 13 While the need for prevention is clear, what is less clear is how to translate knowledge about ACEs into pediatric practice. Effective preventive interventions require a feasible and valid method for identifying high-risk patients, along with treatment options that are successful in improving outcomes for high-risk patients. 14 For addressing ACEs through pediatric practice, an acceptable and feasible tool to screen for ACEs must include questions about sensitive topics that caregivers or youth are willing to answer in the context of a pediatric visit, and are able to answer and review with the pediatric provider in a timely manner.
Two recent literature reviews summarized methods to assess for ACEs in pediatric populations. One review identified 14 tools, the majority of which were tools used for research or nonclinical practice. 15 A second review identified 32 tools based on broader criteria. 16 Of these 32, only 11 tools screened for all ACEs in accordance with the ACE study. Of these 11, only 4 were feasible for use in pediatric practice based on length (10 minutes or less to complete), reporting method (caregiver or self-report as opposed to interview), and age range (includes young children). Between both reviews, a total of 5 tools were potentially feasible to screen for ACEs in pediatric practice, but none to date (including an earlier version of our own tool) have published rates of ACEs reported during routine pediatric practice. Context is important to consider in evaluating the applicability of any survey tool, but is particularly important to consider when evaluating the utility of screening for ACEs in pediatric practice. A question may accurately inquire about ACE exposure, but is useless if caregivers are reluctant to answer that question due to concerns about being judged or reported to child protective services.
Our quality improvement (QI) project attempted to address the specific issue of feasibility of screening for ACEs in pediatric practice. Our goal was to develop and implement a tool to screen for ACEs in children (Child-ACEs) that was acceptable and feasible to adopt during well-child visits at a general pediatric practice.
Methods
Description of Resident Practice
We conducted our project at a resident-driven, ambulatory pediatric clinic affiliated with a university hospital in San Bernardino, California. Seventy-two pediatric residents rotated through this clinic. General pediatric faculty provided oversight as preceptors for the clinic. Residents performed approximately 10 000 well-child visits per year. All of the patients were low-income recipients of Medicaid and approximately 65% were Hispanic or Latino. Patients in our chart review samples were 47% female (range 43% to 54%).
Initial Development of the Whole Child Assessment Tool
The Whole Child Assessment (WCA) was developed and modified over the course of 6 improvement cycles. Our primary reference was the adult 1-page calculator of ACEs based on questions used in the original ACE study. 1 A second reference was a 7-item parent-report survey that was shown in a research setting to be associated with behavioral problems, developmental delays, and injuries in children aged 4 to 5 years. 17 Prior to implementation in a pediatric setting, we reviewed the literature for other instruments that demonstrated feasibility in a primary care setting. Key references that we identified included the Safe Environment for Every Kid (SEEK) model, which screens for risk factors for child abuse.18,19 We also identified the Patient Health Questionnaire that screens for depression, 20 a Generalized Anxiety Disorder screen, 21 and the National Institute on Alcohol Abuse and Alcoholism guidelines for screening in primary care. 22 In addition, we considered the ACE module questions used as part of the Behavioral Risk Factor Surveillance System. 23 In keeping with the original ACE study, our focus was on interpersonal risk factors rather than social risk factors. 24
Based on the above-mentioned references, we created a survey draft for feedback from caregivers and professionals. We solicited feedback from caregivers using waiting room questionnaires that included the survey draft and follow-up questions about whether questions were difficult to understand, answer choices were sufficient, caregivers would be comfortable completing the survey, a doctor should know these things, and questions should be added.
We collected feedback from professionals via recorded interviews or focus groups using a semistructured interview guide. We invited faculty members in our Pediatrics Department involved in ambulatory clinic leadership or resident education, as well as faculty involved in related research in our Psychology and Social Work Departments. We invited all current pediatric residents to participate in a focus group, but attendance was limited to those who were interested and available. Key topics for the professional interview guide were feedback on the survey questions (1—Which questions were confusing, if any? How would you modify? 2—Which answer choices were limited, if any? How would you modify? 3—Would you be comfortable completing this survey if given to you at your child’s physical? 4—What additional questions do you think should be added?), role of the pediatrician in addressing ACEs, and need for community resources.
Implementation of Child-ACE Screening
We used the Plan-Do-Study-Act (PDSA) approach to QI to implement Child-ACE screening during well-child visits, which meant beginning with small tests of change and expanding to bigger tests of change based on evidence of effectiveness. 25 We chose to focus our PDSA cycles on age 5- to 11-year-old well-child visits to minimize impact on clinic workflow (fewer visits than for the 0-4 age group). Because our practice did not have colocated behavioral health professionals, we developed a workflow that did not depend on calculation of a Child-ACE score. Rather, pediatric residents reviewed responses to individual questions and counseled, provided handouts, or referred as indicated.
Our implementation strategies included collecting stakeholder feedback, providing physician support and education, translating for Spanish speaking caregivers, and evaluating caregiver response rates. 26 Specifically, for PDSA #1 (January 2015), nurses handed out the Child-ACE survey at a consecutive series of 5- to 11-year-old well-child visits. The primary feedback from caregivers and pediatricians was that there were too many questionnaires to complete. Therefore, we combined our Child-ACE questions with 2 other forms used by our clinic at well-child visits, the California Staying Healthy Assessment and tuberculosis screening questions. The California Staying Healthy Assessment included questions about nutrition, physical activity, safety (including in the neighborhood), dental health, and tobacco exposure. For PDSA #2 (May 2015), nurses handed out the combined survey to a consecutive series of 5- to 11-year-old well-child visits. Caregivers and pediatricians no longer complained about the number of questionnaires. However, caregivers needed more time to complete the survey. In addition, pediatricians requested easy access to information about local resources to address ACEs. Therefore, we spent some time working with allied health professionals at our clinic to develop a resource list that ultimately included national and local references for basic needs, domestic violence, substance abuse treatment, mental health treatment, parenting education, and resources for caregivers who are single, divorced, separated from an incarcerated parent, or grandparents parenting. We also had our combined survey tool and resource list translated into Spanish, and educated residents about screening for Child-ACEs and using International Classification of Diseases, 10th Revision, codes to document risk factors. For PDSA #3 (June 2016), we had the combined survey and resource lists handed out by the front desk for all patients aged 5 to 11 years who checked in for their well-child visit. There were no caregiver or pediatrician complaints. Pediatricians requested adoption of Child-ACE screening at well-child visits for all age groups.
In California, use of a state-approved Individual Health Education Behavioral Assessment is required when providing health assessments for Medicaid patients in managed care health plans. Therefore, we had to discontinue use of our combined questionnaire until we received state approval. In October 2016, the California Department of Health Care Services approved our combined questionnaire and we named it the Whole Child Assessment. For PDSA #4 (October 2016), the WCA and resource list were distributed at well-child visits for all age groups. There were no caregiver complaints. Pediatricians requested additional education on use of the WCA. For PDSA #5 (January 2017), we provided additional education on the use of the WCA, use of the resource handouts, and International Classification of Diseases, 10th Revision, code documentation. There were no caregiver or pediatrician concerns about the WCA tool. For PDSA #6 (July 2017), minor edits to questions about domestic violence and child abuse were made in an attempt to improve response rates.
Measures of Implementation Outcomes
Acceptability and feasibility for revisions of the WCA were evaluated using caregiver response rates, physician willingness to use the WCA, and visit duration data. 27 Residents participated in medical record reviews of the well-child history, assessment, plan, and problem list, as well as the scanned WCA form. In our initial medical record review prior to implementing screening using a structured tool (no screening September 2014), we reviewed a consecutive series of well-child visits for patients of any age. We agreed to discontinue chart reviews after 21 charts since there was no evidence of any risk factor documentation. We assessed risk factor identification during PDSA #1 and #2 for a consecutive series of 5- to 11-year-old patients who were given the Child-ACE screening tool when roomed for their well-child visit (N = 16 and 18, respectively). Prior to implementing screening using a structured tool for all 5- to 11-year-old well-child visits, we reviewed all well-child visits for patients aged 5 to 11 years in the indicator month (no screening April 2016, N = 210). For PDSA cycles #3 to #5, we also reviewed all well-child visits for patients aged 5- to 11-year in the indicator months (N = 101, 113, and 93, respectively). In July 2017 (PDSA #6), we expanded our chart reviews to include all well-child visits of any age that month (N = 96, age 0-11 months; N = 149, age 1-2 years; N = 75, age 3-4 years; N = 116, age 5-8 years; and N = 92, age 9-11 years).
We assessed resident and faculty feedback on the WCA tool and screening process using a physician satisfaction survey after PDSA #1. However, the burden of extra paper work was too much, so for the remaining PDSA cycles we assessed physician satisfaction informally about twice per year at noon conferences and at monthly QI team meetings, and tracked the willingness of physicians to expand usage of the survey.
As an important factor for feasibility, we also collected patient-reported cycle times (time elapsed from check-in to check-out) without use of the WCA (September 2016, N = 41) and after implementing use of the WCA at all well-child visits (November 2017, N = 76).
Analysis
This was a QI project that used a series of PDSA cycles with mixed methods to achieve the goal of implementing ACEs screening. As part of the initial survey development, we collected qualitative data. Recruitment for initial caregiver waiting room surveys and professional interviews continued until saturation of responses (ie, until no new information was observed). Survey results and transcribed interview recordings were reviewed by the principal and resident investigators. Concerns raised by professionals and caregivers were discussed by the investigators and changes to the survey were made based on consensus. We used an iterative process to revise the survey that incorporated triangulation of references, patient data, and physician feedback. We continued revisions until no further concerns were raised.
As part of our assessment of acceptability and feasibility, we collected quantitative data on caregiver response rates. We conducted analyses on caregiver response data using Minitab 18. From our chart reviews, we tracked over time the proportion of patients aged 5 to 11 years with 3 or more Child-ACEs documented. We compared these proportions to upper and lower confidence limits for each time point. For July 2017 data, we used binary logistic regression to compare proportions across age groups for all caregiver-reported questionnaires (0-12 months, 1-2 years old, 3-4 years old, 5-8 years old, and 9-11 years old). For before and after patient cycle time data, we conducted a 2-sample t test with unequal and equal size distributions.
Ethical Approval and Informed Consent
The research committee at the resident clinic approved this QI project, and the institutional review board at our university approved publication of our data (institutional review board # 5130368 and 5160368). We obtained verbal consents for the waiting room surveys and interviews, but were not required to obtain consents for the chart reviews.
Results
Table 1 shows a summary of the Child-ACE questions included in our final tool with a slash to indicate slightly different wording for ages 0 to 11 months versus 1 to 11 years.
Child-Adverse Childhood Experiences Questions Included in the Whole Child Assessment for Ages 0 to11 Years (June 2, 2017, Version).
Note: Bold font used to provide clarity for caregiver.
Qualitative Data
Thirty caregivers completed our survey in the waiting room and provided feedback to a research assistant on our initial survey draft. The majority (90%) of caregivers did not express any concerns about completing the survey, and stated that it was helpful for doctors to know these things about the child (“helpful to know in case the child was in any type of danger,” “important to better understand the overall well-being of the child”). Concerns raised focused on the theme of discomfort with letting the pediatrician know about socioeconomic factors, such as food insecurity (“I wouldn’t want the doctor to treat me differently because he knew I waited to buy milk”) and economic stressors (“parents might think that they would be judged by the doctors if they were destitute and therefore their child might not receive quality care”).
Fifteen pediatric faculty and residents, as well as allied health professionals, gave us feedback on our initial survey draft via interviews or focus groups. One theme focused on giving caregivers an opt-out option, such as “don’t know” or “leave blank” in case a caregiver was not comfortable answering a specific question. A second theme was to use less explicit language (eg, when asking about sexual abuse). All of the professionals supported the value of pediatricians screening for Child-ACEs. After implementation of screening, we addressed concerns raised by pediatricians during informal feedback until no further concerns were raised and clinic leadership requested expanding use of the WCA to well-child visits for all age groups.
Quantitative Data
Without use of a screening tool for Child-ACEs, our documentation of Child-ACEs in the medical record was close to zero, as shown in September 2014 and again in April 2016 (Table 2). When Child-ACE screening was conducted at well-child visits (PDSA cycles #1-#6), documentation of Child-ACEs or risk for Child-ACEs increased (Figure 1). Confidence limits were wide at the beginning of our QI project, which utilized small tests of change, and narrowed as our QI project progressed and sample sizes increased. Two time points were outside the confidence limits suggesting special cause effects (meaning that the variation was unusual and not consistent with the historical pattern): April 2016 when we suspended use of a screening tool for Child-ACEs and July 2017 when we completed revisions to the WCA.
Child-ACE Reporting Data by Improvement Cycles for Age 5 to 11 Years.
Abbreviations: ACE, adverse childhood experience; PDSA, Plan-Do-Study-Act.
All ages.

Proportion of 3 or more Child-ACEs over time.
In July 2017, we evaluated total Child-ACE scores across age groups. There was no statistically significant difference in the proportion of patients with Child-ACE scores of 3 or more for infants aged 0 to 12 months compared with children aged 1 to 2 years (odds ratio [OR] = 2.2, 95% confidence interval [CI] = 0.8-6.2). There was a greater odds of 3 or more ACEs for older age groups compared with 0 to 12 months age group as follows: age 3 to 4 years (OR = 4.5, 95% CI = 1.6-13.2), age 5 to 8 years (OR = 3.8, 95% CI = 1.4-10.5), and age 9 to 11 years (OR = 6.1, 95% CI = 2.2-16.8). Children in the 9- to 11-year-old group were also statistically significantly more likely to have higher Child-ACE scores compared with age 1 to 2 years (OR = 2.8, 95% CI = 1.4-5.6). There were no other statically significant differences between age groups.
There was a significant difference in patient cycle time minutes for a well-child visit from check-in to check-out before implementing the WCA (mean = 132, SD = 34) and after implementing the WCA (mean = 117, SD = 27) using unequal (t[74] = 2.87, P < .05) and equal size distributions (t[76] = 2.21, P < .05).
Discussion
We demonstrated that it is acceptable and feasible to adopt use of the WCA to screen for Child-ACEs, based on caregiver response rates, physician willingness to expand use of the WCA, and visit duration data. We were able to increase caregiver disclosure about sensitive topics and support physician willingness to use the WCA without a negative impact on visit length by using an iterative process that included obtaining and responding to stakeholder feedback, streamlining paperwork and workflow, and providing physician education. Our well-child visit durations post-implementation of the WCA were actually shorter, which could have been due to greater efficiency in paper work and/or other parallel efficiency initiatives. Our results also demonstrated that caregivers reported more Child-ACEs in older children, which was expected and supports the face validity of the WCA.
A particular strength of the WCA is that it integrates assessment of lifestyle and other risk factors that need to be addressed at well-child care. This reduces the burden of multiple questionnaires for families and providers, and illustrates our response to qualitative data gathered during our first PDSA cycle. In addition, pediatricians can play a key role in mitigating the impact of Child-ACEs by counseling families on healthy nutrition, exercise, and sleep habits, which are key components for addressing toxic stress. 28 Since the WCA includes assessment of nutrition, exercise, and sleep, the tool itself provides information to guide pediatricians in formulating a treatment plan with families.
Another important strength of the WCA is that it does not just assess exposure to potentially toxic stressors but also inquires about risk of exposure. For example, there is a direct question about exposure to mental health problems in the household (“Did your baby/child ever live with anyone who was depressed, mentally ill or suicidal?), as well as questions about risk of exposure to mental health problems (2 items each from the Patient Health and Generalized Anxiety Disorder Questionnaires). During the patient history, providers follow-up on positive responses to questions about risk in order to determine if there is a current need to counsel or refer. Thus, use of the WCA offers the opportunity to identify exposure to Child-ACEs, as well as opportunities to prevent exposure to Child-ACEs by counseling families on self-care and parenting, and connecting families to supportive resources. This is key for capitalizing on the opportunity within pediatric practice to both prevent and reduce the impact of Child-ACEs.
A few other self-report tools screen for child abuse, child neglect, and household dysfunction and have versions for all pediatric age groups.29-33 We do not know how these tools compare with the WCA in identifying Child-ACEs since published data are not available yet for these tools. We do know that these alternative tools are separate screens for ACEs and do not have the advantage of being integrated with other key components of well-child care, nor the advantage of assessing risk for exposure in addition to history of exposure. One tool, the ACE-Questionnaire, has the unique feature of providing de-identified assessment of Child-ACEs. 29 This means that respondents can report the total number of Child-ACE experiences rather than specifying individual Child-ACEs. This is useful for pediatric practices that have integrated behavioral health and so can offer warm handoffs to behavioral health for children with high Child-ACE scores. For practices without integrated behavioral health or for providers who want to be able to follow-up on the many different ways in which ACEs can present and affect children, the WCA may provide a practical option.
Our project has some limitations. First, all of our data were collected at a low-income general pediatric clinic serving mostly Hispanic/Latino patients. It is likely that risk factor detection using the WCA will vary with the demographics of the clinic, although we would predict that this variation would be consistent with the variance in population prevalence of Child-ACEs. Second, we collected our data at a resident pediatric practice with a high rate of discontinuity due to resident schedules. We would expect higher response rates at pediatric practices with higher rates of continuity of care and, therefore, more trusting relationships between pediatricians and families. Third, we were unable to formally assess acceptability to physicians due to concerns for the impact of a formal assessment on clinic workflow. Important next steps are to evaluate the impact of the WCA on referral rates and child outcomes, as well as the validity and reliability of the tool. We are optimistic that the WCA will demonstrate good psychometric properties given that it is based on prior research, including our own demonstration of association with child outcomes. 17
Conclusion
Our report describes the development and implementation of the WCA, a tool that provides an acceptable and feasible way to screen for Child-ACEs and risk for toxic stress during routine pediatric practice. Our data demonstrate that using the WCA improves identification of Child-ACEs compared with not using a screening tool and without a negative impact on visit length. Future implementation studies are needed to evaluate whether use of the WCA also changes pediatric practice and contributes to improved clinical outcomes.
Footnotes
Author Contribution
AMM: Contributed to conception and design; contributed to analysis and interpretation; drafted manuscript; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.
JL: Contributed to conception; contributed to acquisition and analysis; drafted manuscript; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.
CS: Contributed to conception; contributed to acquisition and analysis; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.
FC: Contributed to conception; contributed to interpretation; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.
SR: Contributed to conception; contributed to interpretation; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.
CV: Contributed to conception; contributed to interpretation; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
