Assessment of the Effects of Pediatric Attention Deficit Hyperactivity Disorder on Family Stress and Well-Being: Development of the IMPACT 1.0 Scale

Abstract

Medications may lessen core symptoms of attention deficit hyperactivity disorder (ADHD), yet families continue to report stress and have a low quality of life. Primary care providers manage almost half of all children with ADHD but do not have a brief measure to assess ADHD impacts on family in the context of everyday family life. The IMPACT (Impact Measure of Parenting-Related ADHD Challenges and Treatment) 1.0 Scale was codeveloped with input from parent advisors and administered to 79 parents of children with ADHD. Exploratory factor analysis, correlations with validated instruments, and test-retest reliability were examined. Exploratory factor analysis resulted in 4 subscales (Misbehavior, Siblings, Time, School), which demonstrated moderate to high test-retest reliability. Scale domains were related to severity and change in ADHD symptoms. Significant correlations were found between IMPACT scores, adaptive functioning in the home, and ADHD-related quality of life. The IMPACT 1.0 Scale provides a novel, reliable, and valid method to assess family impact of ADHD.

Keywords

primary care quality of life pediatrics measurement patient-centered outcomes

Introduction

Raising a child with attention deficit hyperactivity disorder (ADHD) can be stressful for parents. Core symptoms related to hyperactivity, impulsivity, and/or inattention create significant functional impairments at home and school. While stimulant medications can ameliorate symptoms of ADHD, families report ongoing stress related to everyday activities and a lower quality of life compared with those raising children without ADHD.¹

Primary care providers are increasingly taking on primary management of almost half of all children with the diagnosis of ADHD.^2,3 ADHD guidelines developed by professional organizations^4,5 emphasize the importance of accurate diagnosis and use of well-established, relevant rating instruments.

The Vanderbilt ADHD Diagnostic Rating Scale (VADRS) is used in primary care to identify not only the core symptoms of ADHD but also its impact on the child’s daily functioning.⁶ The items focus on academic subjects and interpersonal relationships, with global items related to “relationship with parents” and “overall school performance.” These functional items expand the understanding of the child’s ADHD symptoms from core behaviors to the impact on daily functioning. Still, families report unmet needs not captured by broad functional categories.⁷ While some ADHD-related quality of life measurements evaluate specific areas of stress-related functional outcomes,^8,9 these instruments are used mostly in research and not for clinical purposes. Brief tools to quickly identify areas of ADHD-related stress, problems, and daily functioning challenges during assessment and/or treatment visits to tailor treatment recommendations are lacking.

This study aimed to address this gap in clinical assessment of ADHD-related family stresses and daily functioning by developing a novel instrument for use in primary care settings. We implemented 2 phases to develop a scale with relevant content and strong psychometrics. The first phase was to determine key domains of ongoing stress for families and compare these domains to existing validated measures. The first phase uncovered the need for a new measurement tool suitable for primary care, and one was created by the team. The second phase was to assess whether this tool could be implemented in actual primary care settings and to understand how the domains measured by the tool relate to ADHD symptoms.

Methods

Participants

Three family advisors participating in a larger comparative effectiveness trial examining group visit care of ADHD in primary care agreed to participate in Phase I (Content Development). Advisors were from a subset of 21 families who (1) were willing to provide consultation on an ad hoc basis to the study team throughout the larger trial, (2) had a child aged 6 to 12 years old with ADHD, and (3) were enrolled in the larger study.

Participants in Phase II (Psychometrics) were composed of families who met study criteria and enrolled in the larger trial. Families were eligible if they received care at a participating study clinic and had a 6- to 12-year-old child with ADHD. Families were excluded if the child had a diagnosis of conduct disorder, autism, or moderate to severe intellectual disability that precluded active participation in group visit discussions. Families were assigned to receive ADHD follow-up care in a group setting or with providers receiving decision support mirroring clinical care guidelines based on the clinic attended; results are reported elsewhere.¹⁰ The focus of this article is on the development and psychometrics of the measure of family stresses related to ADHD impact. Institutional review board approval was obtained prior to the initiation of study procedures. All recruited participants provided written informed consent.

Measures

Impact Measure of Parenting-Related ADHD Challenges and Treatment (IMPACT) 1.0 Scale

The IMPACT 1.0 Scale is a parent-completed questionnaire of ADHD-related family stresses and challenges comprising 12 items. Although all 12 items on the scale were identified as domains of family stress that are exacerbated by ADHD symptoms, these 12 stress domains also occur in families of children who do not have ADHD. Therefore, families are asked to complete IMPACT 1.0 Scale items based on their general experiences, without making specific judgments about how much of each domain of stress is specifically/uniquely related to ADHD. This characteristic reduces subjective attribution of stress to ADHD versus other causes and allows for IMPACT 1.0 scores to be compared across families with ADHD and other diagnoses (or no diagnosis). Parents were asked to indicate whether the child was on medication at the time of completion. The final IMPACT 1.0 Scale consisted of 4 subscales related to Misbehavior, Siblings, Time, and School (see the appendix).

Vanderbilt ADHD Diagnostic Rating Scale

The VADRS is a screening tool for ADHD based on the Diagnostic Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria and used in primary care to identify and monitor ADHD symptoms.⁶ Symptom-based VADRS items fall onto 2 subscales: Inattention and Hyperactivity-Impulsivity. Parents rated symptoms on a 4-point Likert-type scale with 0 = never and 3 = very often. Severity was assessed as the sum of subscale items with higher scores indicating increasing severity.

Pediatric Quality of Life (PedsQL)

The PedsQL Scale is a caregiver-completed questionnaire to measure health-related quality of life.⁸ The PedsQL generic core scales contain 23 items and capture physical, emotional, social, and school functioning. Higher scores indicate better quality of life. For this study, the parent versions were administered.

Home Situations Questionnaire (HSQ)

The HSQ is a parent-completed questionnaire that assesses behavioral noncompliance across situations in the home, community, and school.¹¹ Parents rate the presence of a particular problem behavior and, when present, rate the severity. A mean severity score is obtained based on a sum. Higher scores indicate higher severity and impairment.

Procedure

Phase I: Content Development

To understand ADHD-related stresses, family advisors were led through 2 activities (the “5 Whys” and “Distillation Game”) used in design research to engage participants quickly and easily. The “5 Whys” is an iterative technique to understand cause and effect relationships to uncover the perceived root cause of a problem.¹² Each answer to the question “why” becomes the basis of the next round of questioning. Participants were asked to consider, “What specific things make it a good day for you in managing your child’s ADHD,” followed by, “Why does X make a day good?” Each subsequent response generated up to 4 additional “why” questions with each response written on post-it notes. The activity was allowed to go on for approximately 20 minutes to allow for multiple rounds of questioning. All responses were examined and grouped based on similarities. Each grouping became a domain. Participants were asked to pick the 5 domains deemed most important.

The second activity, the “Distillation Game,” utilizes techniques from popular party games such as Pictionary, Charades, and Taboo to facilitate expression of concepts in the simplest terms possible, such as one word, gesture, or image.¹³ These viewpoints are further probed to gain better understanding. Each of the 5 domains chosen at the end of the first activity was written on an index card, and cards were dealt to participants. Participants were asked to describe the domain on the card without using the printed words on the card, while other participants tried to guess what the card said. Cards were collected, shuffled, and dealt to participants. In rounds 2 and 3, the process was repeated but participants had to draw and then use only one word or sound to denote the domain on the card. In this way, the study team was able to better understand what parents saw as key indicators within each domain. For example, drawings for “child can cooperate with siblings” and “child can take part in family functions” both featured the child sitting and concentrating. The engagement session lasted 2 hours. The IMPACT Scale was derived based on findings and described in further detail below.

Phase II: Psychometrics

The IMPACT 1.0 Scale, along with the other measures, was administered to a total of 79 parents of children with ADHD participating in a larger comparative effectiveness trial of 2 different primary care–based interventions for ADHD. All measurement tools were administered at 2 time points: Baseline (T0) and 12 months after the initiation of intervention (T1).

Data Analysis

Phase I

The discovery session was audio-recorded and transcribed. Qualitative analysis of the data was based on Ackoff’s Data-Information-Knowledge-Wisdom scheme to distinguish between 4 different levels of sense making.¹⁴ Notes, along with materials generated in session (eg, images and text), were analyzed, and themes were extracted based on importance using a blend of research design and qualitative descriptive methodologies. Keywords were written on post-it notes, displayed visually on a wall for analysis, and coded by grouping items of significance into themes and hierarchies representing patient-centered domains of importance. Two members of the study team (CM and DL) compared domains generated to those captured in validated instruments, and disagreements were handled with input from 2 additional study team members (SEW and NSB). Several of the domains of familial importance were not adequately identified with the existing measurement tools, so a separate tool was developed to be piloted within the larger study.

Phase II

Descriptive statistics were calculated for all measures used in the study. To develop subscales for the IMPACT 1.0 Scale, exploratory factor analysis (EFA) was performed using the Time 0 (pre-intervention) data to generate groups of statistically related individual items. The number of components was selected using a combination of the Eigenvalue >1 convention, scree plot, and inspection of component loadings to produce the best differentiation of clusters of items. Loadings were examined following varimax rotation, and each item was assigned to the component with which it had the highest loading. Subscale scores were then developed by adding raw scores of constituent items on each component. Internal consistency values for subscales were calculated using T1 data, and test-retest reliability of subscales was examined using correlations of T0 and T1 data. Subscale validity and utility in ADHD assessment were investigated using Pearson correlations of IMPACT 1.0 subscale scores with other validated measures (HSQ, PedsQL) for T0, T1, and change (T1-T0).

Results

Phase I

Analysis of the discovery session led to identification of gaps in the measurement of patient-centered outcomes within proposed validated tools so a new scale was constructed, the IMPACT 1.0 Scale. A heuristic model of caregiver challenges and stress related to ADHD was developed to reflect these domains (Figure 1). The most salient themes that arose reflect challenges concerning Misbehavior (in public or with others during family events, during mealtimes); School (worries about negative perceptions from teachers and receiving calls from school); Time (perception of not having enough time for self, household duties, or other children); and Siblings (fighting or verbal arguments, not wanting to be around the child with ADHD, inability to work cooperatively). Themes mapped to domains not captured in existing tools were then used as the basis for scale item development, resulting in a 12-item scale. As noted earlier, the intent of the IMPACT 1.0 Scale was not to identify family stresses uniquely related to ADHD (eg, not present for other conditions or no diagnosis) but rather to identify family stresses specifically exacerbated by ADHD, as identified in the Phase I analysis. As such, all IMPACT 1.0 Scale items are present to some degree in all families but were rated as particularly stressful by families of children with ADHD. Each item was rated on a 5-point frequency scale with 0 = not at all to 5 = every day/almost every day; the 5-point scale was chosen to differentiate between levels of mild, moderate, and severe stress (which may not be detected on a 2- or 3-point scale) while also providing a small number of response options to facilitate rapid completion by parents.

Figure 1.

Heuristic model of caregiver challenges and stress related to attention deficit hyperactivity disorder.

Phase II

A total of 79 parents from diverse backgrounds had complete data for IMPACT items at T0. The sample characteristics are presented in Table 1. Sixty-eight subjects who had at least 9 non-missing items at T0 and T1 were included in the analysis of internal consistency and test-retest reliability. EFA of the 12 IMPACT 1.0 Scale items at T0 produced 4 factors based on Eigenvalue >1 and scree plot inspection. Varimax (orthogonal) rotation revealed a pattern of loadings supporting content for the following 4 subscales: Siblings, Misbehavior, Time, and School (Table 2). Factor analysis using Maximum Likelihood with Quartimin rotation was also explored, and similar patterns were found (data not shown). Subscale scores were created by adding raw scores for items assigned to each of the 4 factors. Internal consistency values for subscales were then calculated using T1 data, and test-retest reliabilities of subscales were examined using correlations of T0 and T1 data (see Table 2). Descriptive statistics for all measures used in the study, including the IMPACT subscale scores, are presented in Table 3.

Table 1.

Sample Characteristics (N = 79)^a.

Characteristics	Group Visit Condition (N = 46), n (%)	Individual Visit Condition (N = 33), n (%)
Child’s race
African American	19 (41.3)	28 (84.8)
White	12 (26.1)	4 (12.1)
Other	15 (32.6)	1 (3.0)
Child’s ethnicity
Hispanic	17 (37.0)	3 (9.1)
Child’s gender
Male	34 (73.9)	20 (60.6)
Child’s age
N (mean ± SD)	42 (9.2 ± 1.8)	25 (9.2 ± 2.1)
Total children in home
N (mean ± SD)	37 (2.5 ± 1.3)	31 (2.6 ± 1.3)
Child on medication T0
Yes	34 (73.9)	28 (84.8)
Parent’s race
African American	16 (34.8)	23 (69.7)
White	10 (21.7)	7 (21.2)
Other	20 (43.5)	3 (9.1)
Parent’s ethnicity
Hispanic	14 (35.9)	3 (9.4)
Parent’s gender
Female	38 (97.4)	29 (93.5)
Parent’s preferred language
Spanish	11 (23.9)	0 (0.0)
Highest grade in school
High school or less	26 (65.0)	19 (59.4)
2 year college+	9 (22.5)	9 (28.1)
Other	5 (12.5)	4 (12.5)
Marital status
Married	14 (35.9)	8 (24.2)
Never married	16 (41.0)	17 (51.5)
Other	9 (23.1)	8 (24.2)
Health literacy (SILS)
Limited	9 (19.6)	3 (9.1)
Parent diagnosed/treated for ADHD
Yes	3 (7.5)	8 (24.2)

Abbreviations: SILS, Single Item Literacy Screener; ADHD, attention deficit hyperactivity disorder.

Values may not equal 100% due to missing data.

Table 2.

Exploratory Factor Analysis (EFA) Loadings After Varimax Rotation.

	Component
Item	I	II	III	IV
EFA Loadings	Siblings	Misbehavior	Time	School
Phone calls from school	0.045	0.102	−0.076	0.936
Worry if teachers think negative of child	0.078	0.337	0.467	0.473
Misbehavior at social/family events	0.082	0.790	0.213	0.228
Misbehavior at meal times	0.098	0.898	0.118	−0.079
Misbehavior in public	0.058	0.877	0.140	0.155
Feeling no time for other children	0.440	0.363	0.545	−0.070
Feeling no times for yourself	0.188	0.104	0.847	0.047
Feeling no time for household duties	0.158	0.170	0.877	−0.051
Physical fights with siblings	0.831	0.043	0.207	−0.036
Verbal arguments with siblings	0.851	0.0278	0.154	0.163
Siblings do not want to be around child	0.799	0.144	0.174	−0.046
Siblings cannot work together	0.852	0.061	0.040	0.061

Eigenvalue	4.536	2.164	1.284	1.009
Cronbach’s α at Time 1	0.858	0.931	0.849	0.640

Mean (SD) at Time 0	7.3 (5.6)	4.4 (3.4)	4.4 (3.9)	1.6 (1.8)
Mean (SD) at Time 1	6.8 (5.5)	3.9 (3.6)	3.3 (3.8)	1.9 (2.2)
Test-retest correlation	0.630***	0.497***	0.670***	0.639***

***

Bolded results indicate significance p<0.0001.

Table 3.

Descriptive Summary of Measures.

	T0, N = 68	T1, N = 68	Change (T1 − T0)
IMPACT Sibling
Mean ± SD	7.3 ± 5.6	6.9 ± 5.5	−0.3 ± 4.7
Median (Min, Max)	7.0 (0.0, 20.0)	6.0 (0.0, 20.0)	0.0 (−20.0, 11.0)
IMPACT Misbehavior
Mean ± SD	4.4 ± 3.4	3.9 ± 3.6	−0.5 ± 3.6
Median (Min, Max)	3.0 (0.0, 12.0)	3.0 (0.0, 12.0)	0.0 (−9.0, 11.0)
IMPACT Time
Mean ± SD	4.4 ± 3.9	3.3 ± 3.8	−1.1 ± 3.2
Median (Min, Max)	3.0 (0.0, 12.0)	1.5 (0.0, 12.0)	−0.5 (−9.0, 9.0)
IMPACT School
Mean ± SD	1.6 ± 1.8	1.9 ± 2.2	0.2 ± 1.7
Median (Min, Max)	1.0 (0.0, 7.0)	1.0 (0.0, 8.0)	0.0 (−4.0, 7.0)
VADRS Hyperactive-Impulsive Symptom Severity
Mean ± SD	15.3 ± 7.6	13.8 ± 7.8	−1.5 ± 5.3
Median (Min, Max)	16.0 (1.0, 27.0)	14.0 (0.0, 27.0)	−1.0 (−12.0, 14.0)
VADRS Inattentive Symptom Severity
Mean ± SD	15.4 ± 6.0	14.5 ± 6.3	−1.1 ± 5.9
Median (Min, Max)	16.0 (2.0, 26.0)	14.0 (2.0, 27.0)	−1.5 (−15.0, 18.0)
VADRS Hyperactive-Impulsive Symptom Count
Mean ± SD	5.0 ± 3.2	4.4 ± 3.4	−0.5 ± 2.7
Median (Min, Max)	5.0 (0.0, 9.0)	4.0 (0.0, 9.0)	0.0 (−7.0, 8.0)
VADRS Inattentive Symptom Count
Mean ± SD	5.1 ± 3.1	4.7 ± 3.1	−0.5 ± 2.9
Median (Min, Max)	5.0 (0.0, 9.0)	5.0 (0.0, 9.0)	0.0 (−7.0, 7.0)
Parent PedsQL Physical Functioning Score
Mean ± SD	67.8 ± 21.4	70.2 ± 21.1	2.4 ± 25.6
Median (Min, Max)	71.9 (18.8, 100.0)	71.4 (12.5, 100.0)	0.0 (−65.6, 78.1)
Parent PedsQL Emotional Health Score
Mean ± SD	60.4 ± 20.7	65.7 ± 20.3	5.1 ± 18.0
Median (Min, Max)	60.0 (10.0, 100.0)	65.0 (25.0, 100.0)	10.0 (–50.0, 55.0)
Parent PedsQL Social Functioning Score
Mean ± SD	62.9 ± 23.6	64.1 ± 24.2	1.5 ± 24.5
Median (Min, Max)	60.0 (0.0, 100.0)	65.0 (0.0, 100.0)	0.0 (−65.0, 60.0)
Parent PedsQL School Functioning Score
Mean ± SD	55.1 ± 15.2	53.8 ± 17.0	−1.5 ± 16.9
Median (Min, Max)	55.0 (5.0, 85.0)	55.0 (15.0, 90.0)	−5.0 (−45.0, 45.0)
Parent PedsQL Psychosocial Health Summary Score
Mean ± SD	59.4 ± 14.5	61.1 ± 15.5	1.6 ± 13.4
Median (Min, Max)	60.0 (21.7, 88.3)	62.5 (26.7, 93.3)	1.7 (−35.0, 23.3)
Parent PedsQL Total Score
Mean ± SD	62.4 ± 14.1	64.3 ± 14.3	1.9 ± 14.5
Median (Min, Max)	63.0 (30.4, 87.0)	65.2 (37.0, 92.4)	1.1 (−41.3, 41.3)
Median (Min, Max)	69.2 (16.7, 100.0)	66.7 (20.0, 93.3)	−1.7 (−40.0, 50.0)
HSQ Total Number Problems
Mean ± SD	8.1 ± 3.2	7.0 ± 3.7	−1.1 ± 3.6
Median (Min, Max)	8.8 (0.0, 13.0)	6.8 (0.0, 13.0)	−1.0 (−8.0, 13.0)
HSQ Number Problems ≥5
Mean ± SD	4.3 ± 3.6	3.4 ± 3.3	−1.0 ± 4.1
Median (Min, Max)	4.0 (0.0, 12.0)	2.0 (0.0, 12.0)	0.0 (−12.0, 10.0)
HSQ Severity
Mean ± SD	3.1 ± 1.9	2.6 ± 2.0	−0.6 ± 2.0
Median (Min, Max)	2.8 (0.0, 6.9)	2.1 (0.0, 8.2)	−0.6 (−4.9, 4.9)

Abbreviations: IMPACT, Impact Measure of Parenting-Related ADHD (attention deficit hyperactivity disorder) Challenges and Treatment; Min, minimum; Max, maximum; VADRS, Vanderbilt ADHD Diagnostic Rating Scale; PedsQL, Pediatric Quality of Life; HSQ, Home Situations Questionnaire.

Subscale validity and utility in ADHD assessment were investigated using Pearson correlations of IMPACT 1.0 subscale scores with other validated measures for T0, T1, and change (T1 − T0; see Table 4). IMPACT 1.0 subscale scores correlated with VADRS scores; however, the same was not true across correlations with the PedsQL or HSQ. IMPACT 1.0 Sibling scores at Time 1 were significantly positively correlated with VADRS Hyperactivity-Impulsivity severity and HSQ severity scores but were not consistently correlated with PedsQL scores (only one positive correlation with physical functioning was found). IMPACT 1.0 Misbehavior scores at each time point were positively and significantly correlated with ADHD symptoms (VADRS Hyperactivity-Impulsivity severity and VADRS Inattention severity scores) and behavioral noncompliance in the home (HSQ Severity score) and negatively correlated with quality of life in multiple domains (PedsQL Emotional Health, Social Functioning, and School Functioning scores). IMPACT 1.0 Time scores at each time point were positively and significantly correlated with inattentive ADHD symptoms (VADRS Inattention severity) but were correlated positively with VADRS Hyperactive-Impulsive severity scores only at Time 1. Time scores were not significantly correlated with most of the PedsQL and HSQ measures, with the exception of a negative correlation at Time 0 with PedsQL Emotional Health. IMPACT 1.0 School scores were significantly positively correlated with VADRS Hyperactivity-Impulsivity severity at each time point but were not significantly correlated with most PedsQL or HSQ severity scores.

Table 4.

IMPACT 1.0 Subscales Correlations With Other Outcomes.

	I. Sibling			II. Misbehavior			III. Time			IV. School
	T0	T1	Change	T0	T0	T0	T0	T1	Change	T0	T1	Change
VADRS Hyperactive-Impulsive Symptom Severity	0.13	0.263*	0.390**	0.582**	0.585**	0.407**	0.227	0.385**	0.311*	0.393**	0.486**	0.382**
VADRS Inattentive Symptom Severity	0.004	0.154	0.292*	0.294*	0.443**	0.332**	0.417**	0.343**	0.327**	0.21	0.244	0.346**
VADRS Hyperactive-Impulsive Symptom Count	0.135	0.24	0.286*	0.536**	0.562**	0.276*	0.199	0.423**	0.300*	0.423**	0.504**	0.376**
VADRS Inattentive Symptom Count	0.039	0.127	0.205	0.22	0.373**	0.191	0.354**	0.301*	0.259*	0.224	0.172	0.365**
Parent PedsQL Physical Functioning Score	−0.001	0.288*	0.048	−0.126	0.12	0.048	−0.102	0.236	0.165	0.146	−0.022	−0.035
Parent PedsQL Emotional Health Score	−0.169	−0.12	−0.067	−.307*	−.368**	−.329**	−.375**	−0.161	−0.225	−0.085	−0.235	−0.018
Parent PedsQL Social Functioning Score	−0.037	−0.073	−0.18	−.369**	−.354**	−.332**	−0.119	−0.024	0.007	−0.009	−.467**	−0.123
Parent PedsQL School Functioning Score	0.069	0.053	−0.161	−.265*	−0.161	−0.03	−0.206	−0.222	−0.134	−.263*	−0.06	−0.07
Parent PedsQL Psychosocial Health Summary Score	−0.077	−0.072	−0.198	−.437**	−.407**	−.343**	−.314**	−0.158	−0.155	−0.137	−.364**	−0.115
Parent PedsQL Total Score	−0.052	0.096	−0.084	−.361**	−0.225	−0.174	−.265*	0.011	0.013	−0.014	−.269*	−0.085
HSQ Total Number Problem Items	0.228	0.296*	0.103	0.531**	0.498**	0.454**	0.395**	0.299*	0.399**	0.235	0.246*	0.039
HSQ Number Problems ≥5	0.196	0.399**	0.266*	0.549**	0.608**	0.442**	0.478**	0.366**	0.324**	0.166	0.402**	0.105
HSQ Severity	0.222	0.329**	0.172	0.564**	0.577**	0.479**	0.490**	0.371**	0.383**	0.227	0.370**	0.108

Abbreviations: IMPACT, Impact Measure of Parenting-Related ADHD (attention deficit hyperactivity disorder) Challenges and Treatment; VADRS, Vanderbilt ADHD Diagnostic Rating Scale; PedsQL, Pediatric Quality of Life; HSQ, Home Situations Questionnaire.

Correlation significant at the .05 level (2-tailed). **Correlation significant at the .01 level (2-tailed).

Importantly, changes in all IMPACT 1.0 subscale scores mirrored changes in all VADRS subscale scores, indicating that change in the IMPACT 1.0 assessed level of family stress was sensitive to rated change in ADHD symptoms. For the Misbehavior and Time subscales, change in IMPACT 1.0 scores was significantly correlated with change in HSQ-assessed behavioral noncompliance in the home.

Discussion

The IMPACT 1.0 Scale was developed using design thinking methods to capture salient family stresses and challenges that affect well-being and daily functioning. While existing tools capture broad impairment in home and school, they may miss more nuanced everyday stresses and challenges, which are crucial for treatment decision-making, global psychological adjustment, and satisfaction with care. The IMPACT 1.0 Scale includes items to improve understanding of the extent to which ADHD interferes with children’s behavior, siblings, school, and caregiver well-being. Because ADHD is a significant public health issue affecting the child, family, and community at large, better understanding of the functional stresses in these areas is a critically important component of clinical and research evaluation.¹⁵ IMPACT item content is not specific to ADHD, but content analyses suggest that family stresses captured on IMPACT items are particularly important and relevant for understanding the effect of ADHD on the family. Hence, IMPACT-assessed family stresses may also present with less frequency among children without any mental health condition,¹⁶ allowing for comparison of families of children with ADHD to other families. The IMPACT Scale was designed for clinician use during primary care follow-up visits in conjunction with the Vanderbilt to provide a more comprehensive and salient assessment of child and family functioning going beyond focused assessment of ADHD symptoms, in order to guide understanding, communication, and decision-making. Nevertheless, further work is needed to ensure that the scale distinguishes between children with and without ADHD.

Design thinking techniques elicited both tacit and latent knowledge that ultimately allowed for the creation of the IMPACT 1.0 Scale items.¹⁷ Design thinking has roots in business and engineering and used to explore stakeholders’ experiences and co-creating solutions that match stakeholders’ needs, add value to experiences, or improve interactions with a product.¹⁸ The approach has been used increasingly in cancer care and inpatient health care service delivery.^19,20 It has also been used in technologies to improve diabetes and depression self-care.^21,22 Thus, the approach to co-creation of the IMPACT 1.0 Scale with families struggling with ADHD is noteworthy, increases its face validity, and can lead to conversations that are practical and family-centered.

IMPACT 1.0 subscale scores were empirically developed using EFA and showed good internal consistency (particularly considering the brevity of the subscales and that α is related to subscale length) and test-retest reliability. These findings support the constructs of misbehavior, time, school, and siblings as sources of family stress as rated by parents of children with ADHD. In support of the validity of the IMPACT 1.0 Scale, we found a positive relationship between core ADHD symptoms on the Vanderbilt and IMPACT scores, such that increased ADHD severity was related to greater family stresses in all IMPACT subscale areas. As such, IMPACT represents a way for providers to efficiently identify, address, and monitor family stresses as a part of treatment that is related to but not redundant with core ADHD symptoms. Even for children successfully treated with ADHD medication, lingering family stresses may significantly affect quality of life (consistent with the significant correlations between IMPACT 1.0 and PedsQL scores). Discussion of family stress related to ADHD can create an opportunity for providers to more holistically assess the effect of current treatment, provide critical support, and facilitate referrals to family and social services that may improve overall quality of life. For example, it is unlikely that a family will be fully satisfied with treatment if symptom ratings improve but family stress levels show little concurrent change.

In addition to scale development, core constructs were identified for understanding the family impact of ADHD, by developing subscales in consultation with parents: Misbehavior, Time, Siblings, and School. An area of ongoing concern for parents of a child with ADHD pertains to the strife between siblings. Qualitative studies have shown siblings often have negative feelings toward the child with ADHD, and these experiences are often overlooked or minimized.²³ It is not uncommon for family conflict to arise between the child with ADHD and sibling(s), which can contribute to severe strain within family relationships and parental stress.^24-26 The IMPACT 1.0 Sibling subscale includes 4 items to capture the unique contribution of sibling conflict and ADHD. Sibling subscale scores were related to hyperactive-impulsive symptoms and to parental perception of the child’s behavior at home. Positive responses to Sibling items may lead to tailored counseling around sibling relationships and referrals for parenting training or family therapy and may spur discussions between the child with ADHD and/or sibling(s).

Children’s behavior and functioning at school are important to monitor to ensure that ADHD does not impede academic success. Executive functioning deficits and coexisting learning disabilities increase the risk of poor educational outcomes, including poor grades and increased detention and expulsion rates.²⁷ Children with ADHD, particularly those with coexisting oppositional or aggressive behaviors, often experience problem behaviors at school that are stressful for teachers and parents.²⁸ The IMPACT School subscale includes 2 items: “worrying that teachers thought negatively of their child” and “receiving phone calls from school.” School subscale scores were strongly related to ADHD hyperactive-impulsive symptoms, suggesting that externalizing symptoms of ADHD are particularly salient in generating school-related stresses for the family.

The last 2 subscales of IMPACT pertain to family stress related to time commitments and to functional effects and environments of misbehavior (eg, social events, in public, and meals). Misbehavior scores were strongly related to ADHD symptoms regardless of subtype. Time scores were strongly related to ADHD Inattentive symptoms and less consistently related to hyperactive-impulsive symptoms. Scores on the Misbehavior and Time subscales were also strongly related to parental perception of child behavioral issues at home, and Misbehavior scores in particular were highly related to parental report of child psychosocial quality of life.

The cumulative effects of family stresses related to misbehavior, time, siblings, and school may put the well-being of parents at risk. Parents experience high levels of stress when caring for a child with overt behavioral problems.^29-31 Parents may experience depression, feelings of isolation, and poorer health.^32,33 Positive responses to any items may produce targeted discussions about the parents’ support network, opportunities for self-care, and perceptions of parenting practices.

The findings of this research must be considered in light of the methodological characteristics used and IMPACT 1.0 Scale content. As with any parent-reported tool, IMPACT scores may be influenced by rater factors including social desirability, catastrophizing, limited awareness/insight into behavior, and insufficient understanding of content. All questionnaires were completed by the same rater (parent), raising potential effects of method bias. In addition, content and wording might reflect biases and experiences of the small number of parent advisors. While a larger number of families were invited to participate, the final number was due to scheduling and logistics during the winter. A cognitive interviewing method with additional parent advisors is being done with the plan to revise (including adding or subtracting items) and retest the scale as needed. Moreover, although challenges in the IMPACT Scale are common and stressful,^1,34,35 specific items may not always be applicable (eg, Sibling scale). Further research is planned to optimize the utility of the scale in primary care settings, as well as examining pediatricians’ and pediatric providers’ perceptions of this tool for use in practice. Therefore, use of the scale should be done with caution despite strong psychometric support, pending further investigation. Last, no counterbalancing measures of resilience were utilized. However, as with any screening tool in pediatrics, the clinician should use these tools as a way to discuss strengths, in addition to parental concerns with families.

Conclusion

The IMPACT 1.0 Scale allows examination of a domain of family functioning (family stress) not captured by existing tools suitable for primary care practice. It is a valid, reliable, and novel tool that can guide discussions so treatment and services can be prescribed or refined to meet the unique needs of each family. Further research will explore ways to optimize the tool for primary care, including minor changes to content and wording, as well as examination to the feasibility of its implementation as perceived by pediatric providers.

Footnotes

Appendix

Acknowledgements

The authors wish to thank the clinical support staff, nurses, and physicians at the study clinics within Eskenazi Health and Riley Hospital for Children at Indiana University Health who supported this study. We wish to thank all the parents and children who participated in the study. We also wish to thank the Pediatric Research Network (PResNet) for their tireless efforts to reach mothers and for working closely with the clinical staff to coordinate scheduling and research. A special thanks to the following PResNet research assistants: Amy Pottenger, Elaine Cuevas, Vickie Cater, Jennifer Stanton, and Cathy Luthman (PResNet executive director). A special thanks to Gina Claxton MPH, RD, program manager of the Indiana CTSI Community Health Partnerships, for her role in helping coordinate engagement sessions and study team meetings. The study team would also like to thank Dr Scott Denne and the Indiana CTSI Pediatrics Project Development Team for their thoughtful critiques in the early stages of this project after the initial patient engagement sessions that led to the development of the IMPACT Scale.

Author Contributions

NSB: Contributed to conception/design; contributed to analysis; drafted manuscript; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.

SO: Contributed to conception/design; contributed to analysis; drafted manuscript; critically revised manuscript; gave final approval.

CM: Contributed to conception/design; contributed to analysis; drafted manuscript; critically revised manuscript; gave final approval.

DL: Contributed to conception/design; contributed to analysis; drafted manuscript; critically revised manuscript; gave final approval.

SEW: Contributed to conception/design; contributed to analysis; critically revised manuscript; gave final approval.

SMD: Contributed to conception/design; critically revised manuscript; gave final approval.

AEC: Contributed to conception/design; critically revised manuscript; gave final approval.

WGK: Contributed to conception/design; contributed to analysis; drafted manuscript; critically revised manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded, in part, by R24 award (R24-HS022434-01) from the Agency for Healthcare Research and Quality (Principal Investigator: Aaron E. Carroll, MD, MS). This project was supported, in part, with support from the Indiana Clinical and Translational Sciences Institute funded, in part by Grant Number UL1TR001108 from the National Institutes of Health, National Center for Advancing Translational Sciences, Clinical and Translational Sciences Award. Dr Bauer is an investigator with the Implementation Research Institute (IRI) at the George Warren Brown School of Social Work, Washington University in St. Louis, MO; through an award from the National Institute of Mental Health (5R25MH08091607) and the Department of Veteran Affairs, Health Services Research & Developmental Service, Quality Enhancement Research Initiative (QUERI).

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the Ethical Standards of the Institutional and/or National Research Committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Approval for the protocol outlined in the article was approved prior to initiation of the study.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Welfare of Animals

This article does not contain any studies with animals performed by any of the authors.

ClinicalTrials.gov Registration

Comparative Effectiveness of Primary Care-based Interventions for Pediatric ADHD (NCT02105142).

ORCID iD

Nerissa S. Bauer

References

Klassen

Miller

Fine

Health-related quality of life in children and adolescents who have a diagnosis of attention-deficit/hyperactivity disorder. Pediatrics. 2004;114:e541-e547.

Williams

Klinepeter

Palmes

Pulley

Foy

JM.

Diagnosis and treatment of behavioral health disorders in pediatric practice. Pediatrics. 2004;114:601-606.

Hoagwood

Jensen

Feil

Vitiello

Bhatara

VS.

Medication management of stimulants in pediatric practice settings: a national perspective. J Dev Behav Pediatr. 2000;21:322-331.

Subcommittee on Attention-Deficit/Hyperactivity Disorder; Steering Committee on Quality Improvement and Management; Wolraich

et al . ADHD: clinical practice guideline for the diagnosis, evaluation, and treatment of attention-deficit/hyperactivity disorder in children and adolescents. Pediatrics. 2011;128:1007-1022.

American Academy of Pediatrics; Subcommittee on Attention-Deficit/Hyperactivity Disorder and Committee on Quality Improvement. Clinical practice guideline: treatment of the school-aged child with attention-deficit/hyperactivity disorder. Pediatrics. 2001;108:1033-1044.

Wolraich

Lambert

Doffing

Bickman

Simmons

Worley

Psychometric properties of the Vanderbilt ADHD Diagnostic Parent Rating Scale in a referred population. J Pediatr Psychol. 2003;28:559-567.

Sikirica

Flood

Dietrich

et al . Unmet needs associated with attention-deficit/hyperactivity disorder in eight European countries as reported by caregivers and adolescents: results from qualitative research. Patient. 2015;8:269-281.

Varni

Burwinkle

Seid

The PedsQL as a pediatric patient-reported outcome: reliability and validity of the PedsQL Measurement Model in 25 000 children. Expert Rev Pharmacoecon Outcomes Res. 2005;5:705-719.

Rentz

Matza

Secnik

Swensen

Revicki

DA.

Psychometric validation of the Child Health Questionnaire (CHQ) in a sample of children and adolescents with attention-deficit/hyperactivity disorder. Qual Life Res. 2005;14:719-734.

10.

Bauer

Sullivan

Szczepaniak

et al . Attention deficit-hyperactivity disorder group visits improve parental emotional health and perceptions of child behavior. J Dev Behav Pediatr. 2018;39:461-470.

11.

Altepeter

Breen

MJ.

The Home Situations Questionnaire (HSQ) and the School Situations Questionnaire (SSQ): normative data and an evaluation of psychometric properties. J Psychoeduc Assess. 1989;7:312-322.

12.

Manktelow

Carlson

. 5 whys: getting quickly to the root of the problem. https://www.mindtools.com/pages/article/newTMC_5W.htm. Accessed February 21, 2019.

13.

Brandt

Messeter

Facilitating collaboration through design games. Paper presented at: Proceedings of the Eighth Conference on Participatory Design: Artful Integration: Interweaving Media, Materials, and Practices—Volume 1; July 27-31, 2004; Toronto, Canada.

14.

Ackoff

From data to wisdom. J Appl Syst Anal. 1989;16:3-9.

15.

Perou

Bitsko

Blumberg

et al . Mental health surveillance among children—United States, 2005-2011. MMWR Suppl. 2013;62(suppl 2):1-35.

16.

Minuchin

Families and Family Therapy. Cambridge, MA: Harvard University Press; 1974.

17.

Orlowski

Matthews

Bidargaddi

et al . Mental health technologies: designing with consumers. JMIR Hum Factors. 2016;3:e4.

18.

Asch

Rosin

Innovation as discipline, not fad. N Engl J Med. 2015;373:592-594.

19.

Patel

Moore

Blayney

Milstein

Transforming cancer care: are transdisciplinary approaches using design-thinking, engineering, and business methodologies needed to improve value in cancer care delivery?

J Oncol Pract. 2014;10:e51-e54.

20.

Carmel-Gilfilen

Portillo

Designing with empathy: humanizing narratives for inspired healthcare experiences. HERD. 2016;9:130-146.

21.

Petersen

Hempler

NF.

Development and testing of a mobile application to support diabetes self-management for people with newly diagnosed type 2 diabetes: a design thinking case study. BMC Med Inform Decis Mak. 2017;17:91.

22.

Yarosh

Schueller

SM.

“Happiness inventors”: informing positive computing technologies through participatory design with children. J Med Internet Res. 2017;19:e14.

23.

Kendall

Sibling accounts of attention deficit hyperactivity disorder (ADHD). Fam Process. 1999;38:117-136.

24.

Smith

Brown

Bunke

Blount

Christophersen

Psychosocial adjustment and peer competence of siblings of children with attention-deficit/hyperactivity disorder. J Atten Disord. 2002;5:165-177.

25.

Harpin

VA.

The effect of ADHD on the life of an individual, their family, and community from preschool to adult life. Arch Dis Child. 2005;90(suppl 1):i2-i7.

26.

Mikami

Pfiffner

LJ.

Sibling relationships among children with ADHD. J Atten Disord. 2008;11:482-492.

27.

Loe

Feldman

HM.

Academic and educational outcomes of children with ADHD. J Pediatr Psychol. 2007;32:643-654.

28.

Greene

Beszterczey

Katzenstein

Park

Goring

Are students with ADHD more stressful to teach? Patterns of teacher stress in an elementary school sample. J Emot Behav Disord. 2002;10:79-89.

29.

Hinojosa

Fernandez-Baca

Knapp

Thompson

LA.

Parental strain, parental health, and community characteristics among children with attention deficit-hyperactivity disorder. Acad Pediatr. 2012;12:502-508.

30.

Podolski

Nigg

JT.

Parent stress and coping in relation to child ADHD severity and associated child disruptive behavior problems. J Clin Child Psychol. 2001;30:503-513.

31.

Tzang

Chang

Liu

SI.

The association between children’s ADHD subtype and parenting stress and parental symptoms. Int J Psychiatry Clin Pract. 2009;13:318-325.

32.

Murphy

Christian

Caplin

Young

PC.

The health of caregivers for children with disabilities: caregiver perspectives. Child Care Health Dev. 2007;33:180-187.

33.

DosReis

Barksdale

Sherman

Maloney

Charach

Stigmatizing experiences of parents of children with a new diagnosis of ADHD. Psychiatr Serv. 2010;61:811-816.

34.

Whalen

Henker

Ishikawa

et al . An electronic diary study of contextual triggers and ADHD: get ready, get set, get mad. J Am Acad Child Adolesc Psychiatry. 2006;45:166-174.

35.

Whalen

Henker

Jamner

et al . Toward mapping daily challenges of living with ADHD: maternal and child perspectives using electronic diaries. J Abnorm Child Psychol. 2006;34:115-130.