Sage Journals: Discover world-class research

Abstract

Objective:

The purpose of the study was to implement a comprehensive adverse childhood experiences (ACEs) screening protocol in a rural Integrated Primary Care Clinic serving Latino Children utilizing implementation science as a framework.

Methods:

The Exploration, Preparation, Implementation, Sustainment (EPIS) framework was utilized to guide and evaluate the implementation. There were 2,347 children who were screened with the developed protocol which included screening for ACEs, psychosocial problems, adolescent depression, and parental depression. Several implementation outcomes were collected and are reported, including a list of screening barriers and adaptations, a list of developed staff training, a clinician stakeholder survey and focus group, and a patient stakeholder survey.

Results:

Clinicians reported being satisfied with the protocol and indicated that screenings provided new clinical information, changed their clinical care and helped to inform the medical needs of patients. Barriers included inadequate time to complete screenings and over-screening for prevalent stressors in the community.

Conclusions:

Implementation science provides a reliable framework for the implementation of a comprehensive ACE screening process by identifying the necessary conditions for high-quality implementation. With the appropriate resources in place, screenings are possible and enhance patient care. More attention is needed to identify primary care interventions for ACEs.

Implications for Impact Statement

Implementation science provides models for implementation that can improve the uptake of ACE screenings in Pediatric Primary Care settings. Integrated Care presents a comprehensive interdisciplinary approach that aids in providing appropriate responses to positive screenings.

Adverse childhood experiences (ACEs) are an important public health issue. ACEs are potentially traumatic events occurring between 0 and 17 years of age and include experiences like abuse, neglect, and household dysfunction. The seminal ACE study found a correlation between 10 specific childhood experiences and physical health outcomes related to high morbidity and mortality in adulthood (Felitti et al., 1998). Subsequent studies have expanded research to consider other adverse experiences like bullying, community violence, neighborhood safety, and racism (Cronholm et al., 2015). Adults who have experienced one or more ACEs are more likely to engage in behaviors such as smoking, substance use, physical inactivity, and suicide attempts and have a higher likelihood of being diagnosed with chronic diseases (Felitti et al., 1998; Hughes et al., 2017). These diseases include cancer, obesity, chronic bronchitis, liver, and cardiovascular diseases (Felitti et al., 1998; Hughes et al., 2017). Adults with ACE exposure are also at risk for depression, anxiety, and intimate partner violence (Hughes et al., 2017).

ACEs and Childhood Development

Adult health problems associated with ACEs are tied to disruptions in child development. Persistent adversity can consistently activate the stress response system and increase cortisol levels during childhood (Miller et al., 2011). This chronic activation affects future health through pathways of disrupted neurodevelopment leading to social and cognitive impairment and the adoption of health risk behaviors (US Centers for Disease Control and Prevention, 2021). Several studies have demonstrated that specific types of ACEs have distinctive impact on neurodevelopment. For instance, parental verbal abuse seems to significantly impact the arcuate fasciculus, a region of the brain that interconnects Broca and Wernicke's area and is involved in producing and understanding language (Choi et al., 2009). Another study highlighted that maltreatment at 10–11 years of age was associated with sensitivity in the right amygdala, a part of the brain that has a strong link with negative emotions such as fear and sadness (Pechtel et al., 2014).

Screening for ACEs

While the impact of high ACEs on child development and later adult health is well documented, ACE screenings have not been widely incorporated into routine Pediatric Primary Care (Rariden et al., 2021). Barriers to screening implementation initially centered around concerns of patient discomfort; however, research has mainly shown that families are receptive to screenings (Rariden et al., 2021). A greater concern has been the feasibility of incorporating ACE screenings in health care settings. Pediatricians often express concerns about the amount of time it would take to complete and respond to screenings, discomfort around how to discuss ACEs with families and overall low levels of confidence on the topic (Gillespie & Folger, 2017). Health care settings may be unprepared to screen particularly if they lack access to resources like multidisciplinary behavioral health support to help respond to positive screening reports (Flanagan et al., 2018; Rariden et al., 2021). There are several approaches that may help advance the identification of ACEs in Pediatric Primary Care settings. The first is utilizing an Integrated Care model where a practice team of primary care and mental health clinicians work together with patients and families to address mental health, health behaviors (and their contribution to chronic illness), and life stressors (Peek, 2013). Integrated health care teams include a myriad of interprofessional health care professionals including but not limited to social workers, mental health consultants, care managers, nurses, primary care physicians (PCPs), psychiatric mental health nurse practitioners, psychologists, and psychiatrists (Brown et al., 2021). Such a multidisciplinary team could help the concern about managing positive ACE reports. A second approach that may help advance the identification of ACEs is the utilization of implementation models in the development and application of screenings to ensure feasibility, acceptability, and sustainability (Silver et al., 2017). While there is a significant research base to addressing ACEs in Pediatric Primary Care, there is less empirical data about which strategies lead to successful implementation in this setting. Implementation of such efforts should not be spontaneous. Implementation science provides a reliable framework for integrating practices into routine care by identifying the necessary conditions for high-quality implementation (Silver et al., 2017). This is particularly important when developing processes with considerations for diverse populations. The current study describes the development and implementation of a comprehensive ACE screening protocol (ACEsPlus+) for children ages 6–16 in a rural Pediatric Primary Care setting. The specific aims of the study were to (a) utilize implementation science and the Exploration, Preparation, Implementation, Sustainment (EPIS) model (Moullin et al., 2019) as a guiding framework for high-quality implementation of a screening protocol, (b) to develop a screening process with adaptations to maximize the contextual fit within a Federally Qualified Health Center (FQHC) setting serving Latino families from an agricultural community, and (c) to use the Standards for Quality Improvement Reporting Excellence guidelines (SQUIRE 2.0; Ogrinc et al., 2016) to report findings from the implementation process.

Method

Implementation Strategy

The implementation strategy was to initiate the development and adoption of a comprehensive ACE screening protocol, ACEsPlus+, in a Pediatric Primary Care setting with Integrated Behavioral Health. The implementation process was informed by the EPIS framework, a framework commonly used in health service systems to guide implementation practices for evidence-based programs (Moullin et al., 2019). The methods for each of the EPIS phases are described below and outcomes for each of the phases will be reported in the Results section.

Exploration Phase

The primary goal of this initial phase was for stakeholders to consider the prevalence of ACEs in the patient population to later identify the best measures and methods for screening. There were three core activities that took place during this phase. First, an Implementation Resource Team (IRT; Moullin et al., 2019) was formed to guide the implementation, the result of a public–academic collaborative partnership between a FQHC and a public medical school. Members included administrators and physicians from the health center and clinical psychology faculty from the medical school who met regularly to guide the implementation efforts throughout each of the EPIS phases.

Next, a needs assessment was completed which focused on the prevalence of ACEs in the target population (Rosado et al., 2021). Finally, the IRT held meetings with clinical staff from the health center's Pediatrics department to introduce the concept of comprehensive ACE screening and to elicit their receptiveness as well as any foreseeable barriers. This included introductory meetings with all levels of staff to ensure that the unique perspectives of each position within the pediatric department were given consideration. Both nonclinical (operations director, office manager, receptionists, schedulers, billing staff) and clinical staff (nurses, medical providers, behavioral health providers) participated. Important components of this step were to (a) give staff a voice in the implementation process and (b) consider nonclinical factors that could influence the screening process (i.e., staff buy-in to follow through with implementation, scheduling, and operations factors).

Preparation Phase

The primary objective of this phase was to identify potential barriers to implementing screenings and corresponding needed adaptations, or innovation factors, that would maximize the contextual fit of the screening measures and methods to be implemented, including their fit within the service setting and patient characteristics (Moullin et al., 2019). The IRT utilized information gathered during the Exploration phase to generate a list of potential barriers and recommended adaptations to improve uptake of the screenings. Additionally, there were two key innovations completed during this phase. The first was to adapt the original ACE Questionnaire used in the CDC-Kaiser ACE Study (Felitti et al., 1998) to meet the literacy needs of the target patient population. The questionnaire is designed to screen for 10 ACEs, including physical, emotional, and sexual abuse, physical and emotional neglect, mental illness in the household, witnessing intimate partner violence, separation from parents, having an incarcerated relative and substance abuse in the household. The original questionnaire, which was designed for an adult to self-report childhood ACEs, was adapted to include a parent version used by caregivers to endorse any ACEs experienced by their child, and a youth self-report version used by youth ages ≥12. Additionally, adaptations included changes to simplify the text, such as substituting complex words (e.g., replacing “A parent often or very often humiliates me” with “…makes me feel very badly about myself”). Items were also broken down for clarity by asking about one adversity at a time. For example, separating multiple items like “Does a parent often or very often swear at you, insult you, put you down, or humiliate you?” into several sub-questions asking about each behavior individually (i.e., “Does a parent … swear at you?” Does a parent … insult you?” “Does a parent … put you down?” etc.).

The second innovation completed during the Preparation Phase was the development of an electronic tool, the Multilingual Automated Screening System (MASS; Rosado et al., 2018). MASS utilizes an electronic platform to administer the ACE questionnaire and other measures included in the screening protocol. MASS runs on touchscreen devices and is programmed to administer screening questions in a choice of languages (English, Spanish, and Creole). The software utilizes a graphical user interface that allows users to independently interact with the touchscreen device (without staff assistance) through graphical icons and audio indicators. This interface was designed to address screening barriers associated with the linguistic needs and reading/literacy levels of the patient population. MASS provides automated scoring of responses to screening items and generates a report that can be imported into an electronic medical record (EMR). This feature was designed to address implementation barriers associated with workflow limitations including challenges in scoring screening results during an office visit.

Additional measures were included given guidelines emphasizing the need for comprehensive ACE screenings that not only assess for experiences of adversity but also behaviors and associated symptoms (NCTSN, 2021). The Pediatric Symptom Checklist (PSC; Jellinek et al., 1988) was used as a measure of child psychosocial functioning. Two versions of the PSC were included in the screening protocol: the original parent-completed version and the Youth Self-Report (Y-PSC) completed by adolescents (ages >12). Additionally, the Patient Health Questionnaire (PHQ-9; Kroenke & Spitzer, 2002) was completed by children ages 12 and older as a measure of adolescent depressive symptoms and by parents of younger children (ages 6–11) to screen for parental depression. The final step of the Preparation Phase was to develop staff training and formulate an implementation plan.

Implementation Phase

There were three notable components of the implementation plan. First, a caseworker position was added to the Primary Care team to help respond to positive screenings. If ACE screenings resulted in the identification of social service needs (i.e., food, housing, financial assistance, etc.) or patient navigation needs (i.e., access to specialty mental health services) the caseworker was available to work directly with families. The second component of the implementation plan was the development of a comprehensive protocol (ACEsPlus+) so that a team of providers could share the responsibility of responding to positive screenings according to the risk level identified. Third, training was provided to staff (including nurses, receptionists, etc.) on how to include the ACEsPlus+ screening into the regular workflow. Patient stakeholder surveys and a clinician survey were also administered with the goal of assessing experience with the protocol. Additionally, a focus group with pediatricians was conducted to measure their experience with the screening process.

Sustainment Phase

The sustainment phase had three main components with the goal to ensure the screening process became routine care and sustainable. First, leadership at the clinic adopted the established workflow into the operating procedures. Second, an effort was made to bill for services provided as a result of the positive screenings. Finally, the collaboration between the FQHC and the academic clinicians was ongoing and facilitated continuous data collection and analysis providing opportunities for updates and dissemination.

Data Collection

This study was approved by the Florida State University Institutional Review Board. Researchers approached pediatricians and invited them to participate in the study (i.e., clinician stakeholder survey and focus group). Written informed consent was obtained. A subset of patients was invited to complete a patient stakeholder survey by a bilingual research assistant following the Well-Child visit. Those who agreed completed a written informed consent form. There were six pediatricians (four female, two male) and three psychologists at the FQHC who participated in the implementation. There were a total of 2,347 children who were screened during the implementation process. Approximately 51% of the children were male with a mean age of 11 years old (SD = 3.1). Most children were Hispanic/Latino (87%) from Spanish-speaking households (63%). Approximately 88% of participants were covered by Medicaid.

Measures

Implementation Measures

Clinician Stakeholder Survey

Items from a screening feasibility survey were modified to measure pediatricians' experience with the ACEsPlus+ protocol (Glowa et al., 2016). The survey included 12 items on a 4-point Likert-type scale across two domains which assessed the clinician's attitudes and satisfaction with the protocol and evaluated the clinician's perception of the added value of screening for ACEs.

Clinician Focus Group

A 90-min focus group was facilitated with the six pediatricians participating in the implementation practice. An independent evaluator facilitated the group using an open-ended interview protocol. Interview questions were developed by the IRT and covered two general domains related to the ACE screening implementation. First, clinicians were asked to recall their practice pre-implementation and reflect on how/if they obtained information related to a child's history of adversity during a clinical encounter. They were asked to compare that practice to information obtained from the newly implemented screening practice. Second, clinicians were asked to discuss any added value they saw in incorporating screenings into Well-Child visits. Finally, they were questioned about any implementation barriers and concerns. Audio recordings from the focus groups were transcribed and then analyzed using a multistep, successive approximation coding process (Neuman, 2013).

Patient Stakeholder Survey

This survey included 13 questions adapted from a Parent Exit Survey (Fothergill et al., 2013) designed to measure patient comfort with screening questions included in the protocol. Parents and adolescents (ages >12) were randomly selected and asked to complete the survey at the end of an office visit.

Results

Implementation Outcomes (by EPIS Phase)

Preparation Phase Outcomes

Barriers and Solutions

A list of potential barriers to implementing screenings and recommended adaptations were derived by the IRT (see Table 1). These were organized into five categories: workflow barriers, workforce competencies, screening tool characteristics, family perceptions about screenings, and needed resources. Workflow barriers centered around time to administer and score the screening measures. Barriers around screening tool characteristics focused on the literacy and language needs of the population. These barriers were primarily addressed using the electronic assessment tool that provided automated screenings and scoring in patient's preferred language and with graphical icons and audio indicators to aid users with low literacy levels.
Table 1
Screening Implementation Barriers and Solutions

Potential barriers Recommended adaptations/solutions

Workflow barriers

Adequate time to complete screening before the pediatric visit

Complete screenings only during Well-Child visits

Adjust time slots for Well-Child visits

Challenges in scoring results during the office visit

Develop an electronic assessment tool to provide automated screenings and scoring

Staffing and time allocation to review screening results

Utilize integrated care program and add caseworker to workflow

Workforce competencies

Provider understanding, interpreting, and responding to results

Provide professional development to all staff (clinical and non-clinical)

Screening tool characteristics

Linguistic needs

Select measures available in multiple languages (English, Spanish, and Creole)

High reading level for patient population

Adapt ACE screening tool

Family perceptions about screenings

Burdensome length of screening process and family perceptions

Patient education around the importance of comprehensive screenings

Resources

Equipment: tablets, printer

Secure funding to cover programming and staffing expenses (grant funding and operational funds)

Software programming needs

Staff Training

A series of staff training was provided to all pediatric clinic personnel during the preparation phase (see Table 2). The functional purposes of training were to promote an implementation climate that welcomed and supported the screening practice by all staff levels and followed recommendations from national organizations, such as the National Child Traumatic Stress Network, around appropriate screening approaches for childhood adversity (NCTSN, 2021).
Table 2
Staff Training by EPIS Phase

EPIS phase Training component Training description Staff trained

Preparation phase Leadership orientation IRT meets with FQHC leadership to describe implementation strategy FQHC leadership (medical/operations directors, finance, etc.)

All staff orientation Orientation to ACEs and introduction of screening concept All staff (pediatrician, psychologists, nursing, reception staff, caseworker)

Medical provider training Understanding the relationship between ACEs and medical conditions, responding to positive screenings, integrated care Pediatricians

Behavioral health provider training Strategies to communicate with families about ACEs; techniques to address the ACE psychological effects Psychologists

Implementation phase Technical/software training Technical training around use of the MASS software Reception staff

Implementation training Training around the implementation of the ACEsPlus+ screening protocol (focus on workflow) All staff (pediatrician, psychologists, nursing, reception staff, caseworker)

Implementation Phase Outcomes

Screening Protocol

A primary outcome of the Implementation Phase was the development of a comprehensive ACE screening protocol, ACEsPlus+ (Figure 1). The protocol included a team of Primary Care Providers (pediatricians), Primary Care Behavioral Health Providers (psychologists), and the caseworker that shared the responsibility of responding to positive screenings. Reception staff asked families to use the MASS software to complete the screening at check-in. An automated electronic report with results was imported into the EMR for review at intake/vitals by a nurse. The nurse reviewed the results to determine the level of risk as indicated by the screening protocol (Figure 1). Levels of risk were defined by the IRT during the Implementation Phase and were informed in part by recommended cutoff scores from developers of the PHQ-9 (Kroenke & Spitzer, 2002) and the PSC (Jellinek et al., 1988), clinical guidance around assessing adolescent suicide risk (Substance Abuse and Mental Health Services Administration, 2020) and child abuse and neglect (Flaherty et al., 2019) and recommendations for the interpretation of ACE screening data (NCTSN, 2021). Low Risk (Level 1) screenings were those with ACE scores of zero, depression PHQ-9 scores below the moderate range (≤9), and scores below the cutoff for psychosocial impairment on the PSC (<28) and Y-PSC (<30). Moderate to Significant Risk (Level 2) screenings were those with an ACE score >0, and/or depression PHQ-9 scores at or above the moderate range (≥10), and/or scores above the cutoff for psychosocial impairment on the PSC (≥28) or Y-PSC (≥30). Severe Risk (Level 3) screenings were those with ACE endorsement of abuse or neglect and/or domestic violence and/or substance abuse as well as any suicidal ideation reported on the PHQ-9.
Figure 1
Note. See the online article for the color version of this figure.

For Severe Risk screenings, the nurse alerted the psychologist who would begin an evaluation in the examination room prior to the pediatrician entering the visit. The psychologist's evaluation focused on a risk assessment to identify any imminent safety concerns. Initial findings were shared with the pediatrician prior to initiating the Well-Visit. Both providers worked together to formulate a treatment plan based on the family's needed level of care (i.e., Behavioral Health services in integrated Primary Care setting vs. referral to specialty mental health). Moderate to Significant Risk screenings were reviewed by the pediatrician during the Well-Visit. If a history of ACEs was endorsed but no psychosocial impairment or depression symptoms were identified, the pediatrician would review ACE responses with the family to determine if there were any concerns or unaddressed effects of the adversity. Families with ACE concerns and/or symptoms of depression and/or psychosocial impairment received a same-day warm hand off from the pediatrician to the psychologist following the Well-Visit. The psychologist initiated an evaluation, then shared preliminary findings with the pediatrician and both providers worked together to formulate a treatment plan based on the family's needed level of care. Low Risk screenings received anticipatory guidance and education on ACEs by the pediatrician. Anytime a social service need was identified based on positive screening results, families would receive a same-day warm hand off to the caseworker who would respond to identified needs. The caseworker would also help navigate patients referred for specialty mental health services. In their review of ACE reports, providers would discuss any discrepancies between parent-report of child ACEs and adolescent self-report. Providers would often follow up on any sensitive discrepancies (i.e., parental substance use, physical abuse, domestic violence, etc.) during portions of the office visit when they were alone with the adolescent.

Implementation Training

The Implementation Phase yielded additional training for staff (Table 2). This included technical training around the use of the MASS software and an implementation training module around the ACEsPlus+ protocol and workflow.

Clinician Stakeholder Survey

Pediatricians' responses to the Clinician Stakeholder Survey were analyzed to understand their experience with the screening protocol. Table 3 summarizes responses provided by the six pediatricians participating for each of the two domains measured by the survey (satisfaction and value). All reported being satisfied with the screening protocol despite the majority indicating that screenings interrupted their workflow (pace of work prior to implementation). None of the pediatricians preferred to address ACEs on their own (vs. using the protocol). All indicated that utilizing the protocol provided new (clinical) information and changed their clinical care. One pediatrician reported that the protocol made their work more difficult.
Table 3
Clinician Stakeholder Survey Responses

Domain Item % agree or strongly agree (N = 6)

Satisfaction I am satisfied with screening protocol 100

Screenings interrupted my workflow 83

Parents appreciated discussing ACEs 50

Adolescents appreciated discussing ACEs 50

Families said MASS was difficult to use 50

I would rather address ACEs on my own 0

Value Using the ACEsPlus+ screening protocol…

Provided new info 100

Changed my clinical care 100

Changed my patient follow-up plan 83

Changed the referrals I made 83

Saved me time 67

Made my work more difficult 17

Clinician Focus Group

All six pediatricians participated in a 90-min focus group during the Implementation Phase. From the coding process, several specific subthemes emerged and were organized around two domains. For Domain 1, pediatricians were asked to reflect on their practice pre-implementation. Three themes emerged and are described in Table 4. Pediatricians felt that prior to the implementation, they were inconsistent about addressing ACEs and felt they were missing important information about children's environment. Furthermore, while components of a treatment team were in place, there was a poor collaboration between providers.
Table 4
Clinician Focus Group Domains and Themes

Themes Theme description

Domain 1 Themes: Reflection on Level of Attention given to ACEs Pre-Implementation

Limited attention was given to ACEs during clinical encounters Inconsistency in addressing ACEs due to lack of routine screening method; feeling like they were asking the wrong questions

“Whole-health” assessments were not being provided Well-Child visits were not providing “whole-health” assessments; described missing important information about child's environment

Team-based care was fragmented Clinicians described feeling “alone” and “without support” in their endeavor to address a child's adversity

Domain 2 Themes: ACE Screening Added Value, Barriers, and Concerns

Added value

Helped inform and support patient medical needs An “unexpected” source for informing a patient's medical treatment (vs. primarily a source to develop a behavior health treatment plan)

A life-saving procedure for some patients A life-saving procedure for high-risk patients with chronic adversity and associated symptoms (i.e., suicidal ideation)

Enhanced team-based care and facilitated patient access to services An avenue by which medical and behavior health providers could collaborate and created a pathway for patients to accept behavior health

Barriers/concerns

Inadequate time to complete screenings Concern about the length of the screening measures and the amount of time it took to complete the screenings

Difficulty integrating psychologists On busy clinical days, it was difficult to develop joint treatment plans with psychologists who were occupied with treating scheduled patients

False positives Concern that subset of patients misunderstood several ACE items

Over screening Concern about screening for ACEs experienced by majority of population

Patient discomfort A minority of patients disclosed discomfort with ACE items

For Domain 2, pediatricians were asked to discuss any added value they found and implementation barriers and concerns. Table 4 describes three themes that emerged for “added value” and five themes for barriers and concerns. In general, the added value was to inform and support the medical needs of patients and increased team-based care. Pediatricians noted that screening results were an unexpected source of information for informing a patient's medical treatment and at times the screenings were a life-saving procedure for high-risk patients with chronic adversity and associated symptoms like suicidal ideation. Pediatricians' concerns included the length of the screening, amount of false-positive screening results, and how to best integrate the psychologist into the established workflow. The IRT utilized this information, along with Clinician Stakeholder Survey results, to work with the FQHC to modify provider schedules to address some of the challenges around integrating psychologists into the workflow. Psychologists' schedules were modified so that they had blocks of time without scheduled follow-up appointments so that they had more availability to collaborate with pediatricians.

Patient Stakeholder Survey

Over the first several months of the Implementation Phase, parents of screened patients were randomly selected and asked to complete the Patient Stakeholder Survey at the end of the office visit. Overall, parents reported feeling comfortable with questions across the different screening domains (see Table S1 in the online supplemental materials). Parents felt least comfortable with questions related to incarceration. In comparison, adolescents endorsed lower levels of comfort levels with the screening questions and endorsed feeling least comfortable with questions about family issues, incarceration, and sexual abuse. These Patient Stakeholder Survey results were utilized to create and program an introductory video into the MASS software that provided details about the nature of the screening questions being asked and why their responses were important and relevant to the child's health.

Discussion

Summary

The current study used implementation science and the EPIS model to develop and implement a comprehensive ACE screening protocol in a rural Pediatric Primary Care (FQHC) clinic serving a predominantly Latino population. There are four key findings of the present research. First, implementation science and models like EPIS, offer useful strategies that can lead to the successful implementation of ACE screening efforts in Pediatric Primary Care settings. The implementation strategies applied in this study led to the successful screening of 2,347 children during the implementation process and to the adoption of the screening protocol into the operating procedures at the FQHC. Second, commonly cited ACE screening barriers, such as insufficient time to administer and score screening measures, can be addressed with electronic assessment tools like the MASS software utilized in this study. This type of technology automated screening and scoring and adjusted for literacy levels to maximize contextual fit. Third, staff training in the implementation setting promotes an implementation climate consistent with national recommendations. Fourth, Integrated Care teams can significantly facilitate the screening process and help ensure patient access to needed resources in response to positive screening results.

Interpretation

While there is a strong research base to addressing ACEs in Pediatric Primary Care there have been concerns about the feasibility of incorporating screenings to the extent that they have not been widely incorporated into routine care (Rariden et al., 2021). Bringing evidence to routine clinical care is often challenging and is a well-recognized gap across multiple key healthcare areas (Li et al., 2018). Results from this study support previous findings that have focused on implementation models as effective strategies to promote the successful uptake and sustainability of evidence-based practices (Tucker & Gallagher-Ford, 2019). The use of implementation models to bring evidence-based practices into community-based healthcare settings can have a significant positive impact on both the system and the people who are served. At a system level, the implementation in this study led to changes in workflow, introduced training to address workforce competencies, and changed service delivery through team-based care and billing. Pediatricians noted that screening results became a source of information for informing a patient's medical treatment (and not just their mental health treatment) and at times were a life-saving procedure for high-risk patients that presented with chronic adversity and associated symptoms.

The implementation phase in this study centered on the creation of a comprehensive ACE screening protocol that relied on an interprofessional team who responded to screening results. This team approach helped to address known implementation barriers related to providers feeling that they alone would not be able to manage screenings either due to lack of time, knowledge, or access to referral resources (Emerson, 2019). Integrating mental health professionals into the pediatric care team can help identify treatment needs, and create unified treatment plans (Hagan, 2017). Practice settings must be mindful that while Integrated Care teams can be very effective, it is important to establish procedures that facilitate their collaboration.

Limitations

A key component of this study in the early stages of implementation was the development of the IRT, which was the result of a partnership between the FQHC and a medical school. Without a similar partnership to bring together resources, it may be difficult to replicate the implementation process followed in this study. This could limit current inferences about feasibility within the Primary Care system. However, this underscores the need to develop sustainable community–academic partnerships has been cited as an essential factor for the successful implementation of interventions aimed at addressing important public health issues (Noel et al., 2019).

Conclusions

Clinician and patient stakeholder data from this study highlight the usefulness and value of implementation models when developing the practice of screening for ACEs. Clinicians found clinical utility in screening data to address ACEs. The model helped verify the feasibility of parents' using technology and recognizing its value. Data collected using this model contributed to sustainability as it informed FQHC stakeholders of the prevalence of the problem and highlighted the need for Integrated Care. Furthermore, it motivated FQHC stakeholders to work through identified barriers to ensure sustainability. Using the implementation model, staff connections with program developers (i.e., MASS software developers, medical school faculty) were formed. Staff were invested in the efforts that had gone into implementation and wanted to see those efforts continue long-term.

ACE screenings can be particularly useful given that they allow for rapid assessment of cumulative adversity (Bucci et al., 2016). However, ACE screenings need to be comprehensive and assess not just exposure to adversity but also associated symptoms and more importantly should result in linking children with needed interventions. Implementation research is a powerful tool to use for this end.

References
1.
Brown M. Moore C. A. MacGregor J. , & Lucey J. R. (2021). Primary care and mental health: Overview of integrated care models. The Journal for Nurse Practitioners, 17(1), 10–14. 10.1016/j.nurpra.2020.07.005
2.
Bucci M. Marques S. S. Oh D. , & Harris N. B. (2016). Toxic stress in children and adolescents. Advances in Pediatrics, 63(1), 403–428. 10.1016/j.yapd.2016.04.002
3.
Choi J. Jeong B. Rohan M. L. Polcari A. M. , & Teicher M. H. (2009). Preliminary evidence for white matter tract abnormalities in young adults exposed to parental verbal abuse. Biological Psychiatry, 65(3), 227–234. 10.1016/j.biopsych.2008.06.022
4.
Cronholm P. F. Forke C. M. Wade R. Bair-Merritt M. H. Davis M. Harkins-Schwarz M. Pachter L. M. , & Fein J. A. (2015). Adverse childhood experiences: Expanding the concept of adversity. American Journal of Preventive Medicine, 49(3), 354–361. 10.1016/j.amepre.2015.02.001
5.
Emerson A. J. (2019). Screening for adverse childhood experiences in primary care: Recommendations for practice [Doctoral dissertation]. North Dakota State University.
6.
Felitti V. J. Anda R. F. Nordenberg D. Williamson D. F. Spitz A. M. Edwards V. Koss M. P. , & Marks J. S. (1998). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. American Journal of Preventive Medicine, 14(4), 245–258. 10.1016/S0749-3797(98)00017-8
7.
Flaherty E. Legano L. , & Idzerda S. (2019). Ongoing pediatric health care for the child who has been maltreated. Pediatrics, 143(4), Article e20190284. 10.1542/peds.2019-0284
8.
Flanagan T. Alabaster A. McCaw B. Stoller N. Watson C. , & Young-Wolff K. C. (2018). Feasibility and acceptability of screening for adverse childhood experiences in prenatal care. Journal of Women's Health, 27(7), 903–911. 10.1089/jwh.2017.6649
9.
Fothergill K. Gadomski A. Solomon B. Olson A. Gaffney C. dosReis S. , & Wissow L. (2013). Assessing the impact of a web-based comprehensive somatic and mental health screening tool in pediatric primary care. Academic Pediatrics, 13(4), 340–347. 10.1016/j.acap.2013.04.005
10.
Gillespie R. J. , & Folger A. T. (2017). Feasibility of assessing parental ACEs in pediatric primary care: Implications for practice-based implementation. Journal of Child & Adolescent Trauma, 10(3), 249–256. 10.1007/s40653-017-0138-z
11.
Glowa P. T. Olson A. L. , & Johnson D. J. (2016). Screening for adverse childhood experiences in a family medicine setting: A feasibility study. The Journal of the American Board of Family Medicine, 29(3), 303–307. 10.3122/jabfm.2016.03.150310
12.
Hagan G. (2017). Pediatric perspective on the role of integrated care in the treatment of adverse childhood experiences (ACEs). Journal of the American Academy of Child & Adolescent Psychiatry, 56(10), S61. 10.1016/j.jaac.2017.07.802
13.
Hughes K. Bellis M. A. Hardcastle K. A. Sethi D. Butchart A. Mikton C. Jones L. , & Dunne M. P. (2017). The effect of multiple adverse childhood experiences on health: A systematic review and meta-analysis. The Lancet Public Health, 2(8), e356–e366. 10.1016/S2468-2667(17)30118-4
14.
Jellinek M. S. Murphy J. M. Robinson J. Feins A. Lamb S. , & Fenton T. (1988). Pediatric Symptom Checklist. The Journal of Pediatrics, 112(2), 201–209. 10.1016/S0022-3476(88)80056-8
15.
Kroenke K. , & Spitzer R. L. (2002). The PHQ-9: A new depression diagnostic and severity measure. Psychiatric Annals, 32(9), 509–515. 10.3928/0048-5713-20020901-06
16.
Li S. A. Jeffs L. Barwick M. , & Stevens B. (2018). Organizational contextual features that influence the implementation of evidence-based practices across healthcare settings: A systematic integrative review. Systematic Reviews, 7(1), Article 72. 10.1186/s13643-018-0734-5
17.
Miller G. E. Chen E. , & Parker K. J. (2011). Psychological stress in childhood and susceptibility to the chronic diseases of aging: Moving toward a model of behavioral and biological mechanisms. Psychological Bulletin, 137(6), 959–997. 10.1037/a0024768
18.
Moullin J. C. Dickson K. S. Stadnick N. A. Rabin B. , & Aarons G. A. (2019). Systematic review of the EPIS framework. Implementation Science, 14(1), Article 1. 10.1186/s13012-018-0842-6
19.
NCTSN . (2021). Beyond the ACE score: Perspective from the NCTSN on child trauma and adversity screening and impact. The National Child Traumatic Stress Network.
20.
Neuman W. (2013). Social research methods: Qualitative and quantitative approaches. Pearson Education.
21.
Noel L. Phillips F. Tossas-Milligan K. Spear K. Vanderford N. L. Winn R. A. Vanderpool R. C. , & Eckhardt S. G. (2019). Community-academic partnerships: Approaches to engagement. American Society of Clinical Oncology Educational Book, Annual Meeting (Vol. 39, pp. 88–95). American Society of Clinical Oncology.
22.
Ogrinc G. Davies L. Goodman D. Batalden P. Davidoff F. , & Stevens D. (2016). SQUIRE 2.0 (Standards for Quality Improvement Reporting Excellence): Revised publication guidelines from a detailed consensus process. BMJ Quality & Safety, 25(12), 986–992. 10.1136/bmjqs-2015-004411
23.
Pechtel P. Lyons-Ruth K. Anderson C. M. , & Teicher M. H. (2014). Sensitive periods of amygdala development: The role of maltreatment in preadolescence. NeuroImage, 97(15), 236–244. 10.1016/j.neuroimage.2014.04.025
24.
Peek C. J. (2013). The National Integration Academy Counsel Executive summary—Lexicon for behavioral health and primary care integration: Concepts and definitions developed by expert consensus (Report No. 13-IP001-1-EF). Agency for Healthcare Research and Quality.
25.
Rariden C. SmithBattle L. Yoo J. H. Cibulka N. , & Loman D. (2021). Screening for adverse childhood experiences: Literature review and practice implications. The Journal for Nurse Practitioners, 17(1), 98–104. 10.1016/j.nurpra.2020.08.002
26.
Rosado J. I. MacDonnell G. Fernandez T. , & Bauer M. (2018). Multilingual automated screening system (Version 1) [Computer software]. Florida State University.
27.
Rosado J. I. Ramirez A. Montgomery J. Reyes E. , & Wang Y. (2021). Adverse childhood experiences and its association with emotional and behavioral problems in U.S. children of Latino immigrants. Child Abuse & Neglect, 112, Article 104887. 10.1016/j.chiabu.2020.104887
28.
Silver R. B. Newland R. P. Hartz K. Jandasek B. Godoy L. Lingras K. A. Low C. M. Dickstein S. Campagna K. Berger B. , & Seifer R. (2017). Integrating early childhood screening in pediatrics: A longitudinal qualitative study of barriers and facilitators. Clinical Practice in Pediatric Psychology, 5(4), 426–440. 10.1037/cpp0000214
29.
Substance Abuse and Mental Health Services Administration . (2020). Treatment for suicidal ideation, self-harm, and suicide attempts among youth (SAMHSA Publication No. PEP20-06-01-002). National Mental Health and Substance Use Policy Laboratory, Substance Abuse and Mental Health Services Administration.
30.
Tucker S. J. , & Gallagher-Ford L. (2019). From strategy to implementation. American Journal of Nursing, 119(4), 50–52. 10.1097/01.NAJ.0000554549.01028.af
31.
US Centers for Disease Control and Prevention . (2021). About the CDC-Kaiser ACE study. Reviewed April 6, 2021. Accessed June 30, 2022. https://www.cdc.gov/violenceprevention/aces/about.html

Potential barriers	Recommended adaptations/solutions
Workflow barriers
Adequate time to complete screening before the pediatric visit	Complete screenings only during Well-Child visits
	Adjust time slots for Well-Child visits
Challenges in scoring results during the office visit	Develop an electronic assessment tool to provide automated screenings and scoring
Staffing and time allocation to review screening results	Utilize integrated care program and add caseworker to workflow
Workforce competencies
Provider understanding, interpreting, and responding to results	Provide professional development to all staff (clinical and non-clinical)
Screening tool characteristics
Linguistic needs	Select measures available in multiple languages (English, Spanish, and Creole)
High reading level for patient population	Adapt ACE screening tool
Family perceptions about screenings
Burdensome length of screening process and family perceptions	Patient education around the importance of comprehensive screenings
Resources
Equipment: tablets, printer	Secure funding to cover programming and staffing expenses (grant funding and operational funds)
Software programming needs

EPIS phase	Training component	Training description	Staff trained
Preparation phase	Leadership orientation	IRT meets with FQHC leadership to describe implementation strategy	FQHC leadership (medical/operations directors, finance, etc.)
	All staff orientation	Orientation to ACEs and introduction of screening concept	All staff (pediatrician, psychologists, nursing, reception staff, caseworker)
	Medical provider training	Understanding the relationship between ACEs and medical conditions, responding to positive screenings, integrated care	Pediatricians
	Behavioral health provider training	Strategies to communicate with families about ACEs; techniques to address the ACE psychological effects	Psychologists
Implementation phase	Technical/software training	Technical training around use of the MASS software	Reception staff
	Implementation training	Training around the implementation of the ACEsPlus+ screening protocol (focus on workflow)	All staff (pediatrician, psychologists, nursing, reception staff, caseworker)

Domain	Item	% agree or strongly agree (N = 6)
Satisfaction	I am satisfied with screening protocol	100
	Screenings interrupted my workflow	83
	Parents appreciated discussing ACEs	50
	Adolescents appreciated discussing ACEs	50
	Families said MASS was difficult to use	50
	I would rather address ACEs on my own	0
Value	Using the ACEsPlus+ screening protocol…
	Provided new info	100
	Changed my clinical care	100
	Changed my patient follow-up plan	83
	Changed the referrals I made	83
	Saved me time	67
	Made my work more difficult	17

Themes	Theme description
Domain 1 Themes: Reflection on Level of Attention given to ACEs Pre-Implementation
Limited attention was given to ACEs during clinical encounters	Inconsistency in addressing ACEs due to lack of routine screening method; feeling like they were asking the wrong questions
“Whole-health” assessments were not being provided	Well-Child visits were not providing “whole-health” assessments; described missing important information about child's environment
Team-based care was fragmented	Clinicians described feeling “alone” and “without support” in their endeavor to address a child's adversity
Domain 2 Themes: ACE Screening Added Value, Barriers, and Concerns
Added value
Helped inform and support patient medical needs	An “unexpected” source for informing a patient's medical treatment (vs. primarily a source to develop a behavior health treatment plan)
A life-saving procedure for some patients	A life-saving procedure for high-risk patients with chronic adversity and associated symptoms (i.e., suicidal ideation)
Enhanced team-based care and facilitated patient access to services	An avenue by which medical and behavior health providers could collaborate and created a pathway for patients to accept behavior health
Barriers/concerns
Inadequate time to complete screenings	Concern about the length of the screening measures and the amount of time it took to complete the screenings
Difficulty integrating psychologists	On busy clinical days, it was difficult to develop joint treatment plans with psychologists who were occupied with treating scheduled patients
False positives	Concern that subset of patients misunderstood several ACE items
Over screening	Concern about screening for ACEs experienced by majority of population
Patient discomfort	A minority of patients disclosed discomfort with ACE items

From Planning to Implementation: Developing an ACE Screening Protocol in a Rural Integrated Primary Care Clinic Serving Latino Children

Abstract

Objective:

Methods:

Results:

Conclusions:

Implications for Impact Statement

ACEs and Childhood Development

Screening for ACEs

Method

Implementation Strategy

Exploration Phase

Preparation Phase

Implementation Phase

Sustainment Phase

Data Collection

Measures

Implementation Measures

Clinician Stakeholder Survey

Clinician Focus Group

Patient Stakeholder Survey

Results

Implementation Outcomes (by EPIS Phase)

Preparation Phase Outcomes

Barriers and Solutions

Screening Implementation Barriers and Solutions

Staff Training

Staff Training by EPIS Phase

Implementation Phase Outcomes

Screening Protocol

Implementation Training

Clinician Stakeholder Survey

Clinician Stakeholder Survey Responses

Clinician Focus Group

Clinician Focus Group Domains and Themes

Patient Stakeholder Survey

Discussion

Summary

Interpretation

Limitations

Conclusions

References