Abstract
Behavior disorders (BDs) are common and, without treatment, can have long-term impacts on child and family health. Behavioral Parent Training (BPT) is the standard of care intervention for early-onset BDs; however, structural socioeconomic barriers hinder treatment outcomes for low-income families. While digital technologies have been proposed as a mechanism to improve engagement in BPT, research exploring the relationship between technology use and outcomes is lacking. Thus, this study with 34 low-income families examined the impact of parents’ use of adjunctive mobile app components on treatment efficiency in one technology-enhanced (TE-) BPT program, Helping the Noncompliant Child (HNC). While parent use of the TE-HNC app and its impact on the efficiency of service delivery varied across specific components, increased app use significantly reduced the number of weeks required for families to achieve skill mastery. Implications for the design and development of behavior intervention technologies in general, as well as for BPT in particular, are discussed.
Approximately 113 million children globally experience clinically elevated externalizing symptoms consistent with a diagnosis of conduct disorder and oppositional defiant disorder (i.e., behavior disorders, BDs), as well as commonly co-occurring attention-deficit/hyperactivity disorder (ADHD; Merikangas et al., 2009; Polanczyk et al., 2015). These externalizing symptoms are often characterized by frequent, long-lasting, dysregulated behavior that occurs across contexts, in conjunction with irritability, impulsivity, or aggression. Further, considerable research has documented an association between untreated symptoms in early childhood and adverse developmental outcomes, including delays in cognitive, social, and emotional functioning, which, in turn, predict long-term risk for later educational, employment, and health problems (Fergusson et al., 2005; Odgers et al., 2007; Piquero et al., 2016).
Behavioral Parent Training (BPT; also called Behavioral Parenting Interventions and Parent Management Training) is an evidence-based intervention for early-onset (3–8 years old) BDs, and its efficacy is well established (e.g., Lundahl et al., 2006; McMahon & Forehand, 2005; Piquero et al., 2009). BPT encompasses a “family” of programs that target the “coercive cycle” of parent-child interactions implicated in early-onset BDs (e.g., Kaehler et al., 2016; Long et al., 2017; Southam-Gerow & Prinstein, 2014). While BPT is the standard of care for early-onset BDs, its effectiveness is hindered by poor engagement, especially for families with low incomes who are overrepresented in statistics on BDs compared to families with relatively higher income (see Chacko, Jensen, et al., 2016; Gardner et al., 2009; Lundahl et al., 2006; Reyno & McGrath, 2006; Shaw & Taraban, 2016 for reviews). With sufficient engagement, outcomes are comparable regardless of family income; however, research suggests that insufficient engagement (e.g., compromises in commitment to and completion of treatment, such as attendance, compliance, or retention) is associated with less beneficial treatment outcomes for families in evidence-based interventions, including BPT (Gardner et al., 2009; Jones et al., 2013; Shaw & Taraban, 2016). Consistent with economic family stress theory (see Conger & Donnellan, 2007, for a review) and the poverty-related stress model (Wadsworth et al., 2011), socioeconomic disadvantage increases the likelihood that parents experience persistent financial (e.g., difficulty paying bills), work (e.g., unemployment and shiftwork), and parenting-related (e.g., lack of or low-quality child care) stressors and cope with those stressors with relatively higher levels of disengagement and withdrawal (Santiago et al., 2011). Thus, socioeconomic disadvantage and associated stressors may both exacerbate the coercive cycle and compromise engagement in BPT interventions, highlighting the critical importance of innovative strategies to decrease the number of sessions required for families to master program skills and complete services more efficiently (Cohen & Piquero, 2009; Jones et al., 2013).
Given technology-enhanced models have been proposed as a potential tool to improve engagement in and efficiency of evidence-based treatments, recent work has extended the long history of technology enhancements in BPT (e.g., video modeling) to include those targeting clinicians (e.g., online therapist training) and families (e.g., telehealth and other behavioral intervention technologies; Bausback & Bunge, 2021; Chacko, Isham, et al., 2016; Comer et al., 2015; Ortiz et al., 2020; Sullivan et al., 2021). Despite well-established disparities in access to and use of information and communication technology (i.e, “digital divide”; Van Dijk, 2020), estimates suggest that the majority (76%) of households with low income own a smartphone and approximately a quarter (27%) are smartphone-dependent (or rely on smartphone devices for online access (Pew Research Center, 2021; Vogels, 2021). Accordingly, families with relatively lower income are more likely than other groups to rely on mobile phones to access health information and as their primary means of communication (Anderson-Lewis et al., 2018; Vangeepuram et al., 2018). Building upon such trends, smartphones have the potential to cost-effectively extend BPT treatment with a range of technical features, especially in the case of mobile applications (apps), which are increasingly influential in how parents consume information about child rearing (Guerra-Reyes et al., 2016; Kubb & Foran, 2020).
By capitalizing on the increased functionality afforded by access to smartphones, Technology-Enhancements (TE-) to one BPT program, Helping the Noncompliant Child (HNC; McMahon & Forehand, 2005), were developed to improve engagement in BPT with families experiencing financial hardship (Jones et al., 2014, 2020). Consisting of a mobile app for parents and web portal for clinicians, TE-HNC was designed, in part, to increase the frequency and reliability of parenting skill use at home and in the context of the family’s daily life. Findings across two randomized control trials (RCT) suggest that TE-HNC has the potential to efficiently (i.e., fewer weeks for families to achieve skill mastery and complete treatment) improve aspects of low-income families’ engagement in BPT while achieving clinically significant gains in parenting skill use and child behavior after treatment (Jones et al., 2014, 2020; Parent et al., 2022). Further, a preliminary within-group analysis of families (n = 9) who used the TE-HNC prototype in the initial pilot RCT evidenced a relatively high level (52%–92%) of overall mobile app use during the active treatment phase and suggested the potential for adjunctive behavioral intervention technologies to improve BPT service delivery models (Anton et al., 2016). As such, this study aimed to replicate and extend those analyses with a larger sample and the updated version of the app by examining patterns in parent use of TE-HNC mobile app components and their differential impact on the efficiency of service delivery, which is critical to understanding opportunities for tailoring treatment. It was hypothesized that parent use of the mobile app components (Skills Videos, Surveys, and Videorecording Skills Practice) during treatment would be associated with the efficiency of service delivery (i.e., weeks to master program skills and complete treatment) in TE-HNC (see Methods for more detail on the mobile app).
Methods
Overview
The current study represents secondary data analysis of a randomized controlled trial with 101 low-income families, who were randomized to HNC (McMahon & Forehand, 2005; n = 54) or TE-HNC (n = 47). To be eligible for the study, families had to meet criteria for low-income (100%–250% of the federal poverty guidelines), as well as have a child between 3 and 8 years old who met clinical cut-offs for significant externalizing behavior as defined by the Problem and/or Intensity Subscales of the Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999), a parent-report measure of common child behavior problems. Exclusion criteria included a pending or prior substantiation of child abuse or neglect; current parental substance abuse/dependence, psychotic disorder, or severe depression/manic episode diagnosis; or child psychotic disorder, mood disorder with psychotic features, or developmental or other disability that would require significant adaptations to the standard treatment protocol and materials to achieve fidelity.
Participants
Participant characteristics are summarized in Table 1. The current analyses focused on families randomized to the TE-HNC group who completed treatment (n = 34). As previously reported, drop-out (27.7%) in TE-HNC was significantly lower than previously reported in BPT work with low-income families (Jones et al., 2020). On average, caregivers in this group were 31.4 years old (SD = 5.2). The majority of caregivers self-identified as female (97.1%) and a fourth self-identified as ethnic or racial minorities (26.5%). According to parent-report, children were 4.6 years old on average (SD = 1.3) and 58.8% were boys. The majority of the participants (94.1%) owned a personal smartphone and reported using Android (55.9%) or Apple iOS (38.2%) operating systems at baseline.
Sample Demographics.
Procedures
Families were recruited via word-of-mouth, social media sites, and advertisements targeting local schools and community centers, as well as healthcare, social service, and other agencies that primarily serve low-income families in north central North Carolina (NC). Phone screening was conducted to determine eligibility for the study. After the initial phone screen, interested and eligible caregivers were scheduled for a baseline assessment where parents provided consent for themselves and their child, eligibility was confirmed, and a series of demographic and psychosocial information about the family was collected. Eligible caregiver-child dyads were randomized to HNC or TE-HNC during the first session. Given data that suggests 44% of low-income users let service plans lapse due to finances (Smith, 2015), caregivers randomized to TE-HNC received an iPhone device with the TE-HNC mobile app (i.e., Tantrum Tamers) pre-downloaded for the duration of the study to ensure consistent access to the app. All families received $50 per assessment and TE-HNC families received an additional $100 for returning the project phone. All procedures were approved by the university’s institutional review board.
Intervention
Families randomized to the TE-HNC group in the parent RCT received the standard HNC program, as well as technology enhancements via a HIPAA-compliant technical system, which allowed clinicians (via a web portal) to monitor and reinforce caregiver progress (via native iOS application, Tantrum Tamers©). HNC is a mastery-based BPT program that includes weekly in-person sessions and brief midweek calls with clinicians. Progression through and completion of the HNC program, which consists of two phases, is dependent upon parents meeting criteria for each new skill. Mastery of parenting skills is determined by weekly observations and coding of parent skill use by therapists in a clinic-based setting and some work suggests this type of format may be optimal with low-income families (Lundahl et al., 2006; see Kaminski et al., 2008 for a meta-analysis). In the first phase, Differential Attention, the caregiver is taught methods for (re)establishing a positive and mutually reinforcing parent-child relationship, including attending, rewarding, and active ignoring skills. During treatment, caregivers practice these skills in session in the context of child-directed play (Child’s Game) and are instructed to practice at home for a minimum of 15 minutes per day. Once parents master Phase I skills, they progress to Phase II, Compliance Training. In Phase II, caregivers are taught skills to maximize opportunities for child compliance (i.e., clear instructions) and implement non-physical consequences for noncompliance and inappropriate, nonignorable behaviors (i.e., Time-Out). Phase II skills are taught in the session using caregiver-directed activities (i.e., clean-up task). To maintain mastery of Phase I skills, caregivers are encouraged to practice earlier skills in later sessions and instructed to practice Child’s Game at home throughout both phases.
An adjunct to the standard HNC program (e.g., weekly clinic-based sessions, midweek calls, daily home practice), the Tantrum Tamers mobile app was originally developed by an interdisciplinary team consisting of researchers with expertise in BPT for underserved families, an advisory panel of five clinicians who practiced at least one BPT program, an industry partner with experience developing health-related software apps, and health economists with expertise in health care evaluation, efficiency, and effectiveness (Jones et al., 2014). Building upon the functionality and content tested in the pilot study, the Tantrum Tamers mobile app allowed clinicians to integrate additional behavior change strategies associated with improvements in intervention effectiveness, including three features initiated by caregivers, independently, to promote the acquisition and use of parenting skills at home—Skills Videos, Surveys, and Videorecording Skill Practice (Jones et al., 2020; Webb et al., 2010). The Skills Videos corresponded to each of the program skills (e.g., Attends, Rewards, and Ignoring) and included both psychoeducation and skill demonstrations. Surveys assessed parents’ use of skills outside of sessions, as well as their attitude, mood, and stress management during the day. In response, parents were provided tailored feedback via the app (e.g., positive reinforcement for the completion of home practice and watching Skills Videos) and by clinicians during sessions to adapt and reinforce skills practice. Finally, parents were instructed to upload a 15-minute videorecording of skills practice with their child each week for clinician review and feedback during sessions. Caregivers were only able to access the Tantrum Tamers app through the study phone which was provided to families at randomization.
Therapist Training and Fidelity
In the parent RCT, Master’s-level clinicians treated families in the HNC and TE-HNC groups. Training included reviewing treatment manuals, establishing reliability with the HNC coding system, roleplays, observations and discussion, and supervision and feedback by two licensed clinical psychologists. Approximately 30% of sessions were coded by a Master’s-level or doctoral-level coder to ensure treatment fidelity and 61% of those were double-coded. Analyses yielded a treatment fidelity of 98% in the TE-HNC group.
Measures
Demographics
Participating parents provided demographic information including age, race, ethnicity, and gender of self and the child, as well as family income.
Mobile App Component Use
Parents’ use of mobile app components (Surveys, Skills Videos, and Videorecording Skills Practice) was operationalized as the frequency of component use contextualized by opportunities for use. The frequency of component use was determined via passively collected app data and opportunities for use were defined as the number of times parents’ were instructed to engage with the component at randomization, thus providing an indicator of component use in a manner consistent with treatment specifications. Opportunities for use were defined as weekly for submitting a videorecording of skills practice, and daily for watching a video and completing a survey. For example, Survey use during this phase was defined by the number of surveys completed divided by the total number of days in treatment. An overall mobile app use variable was calculated by averaging use across the three components.
Efficiency of Service Delivery
The efficiency of service delivery was operationalized as the number of weeks required for each family to master program skills and complete the program. Families were considered to have completed the program once the parent met behavioral skill criterion for each program skill, which in HNC and other mastery-based BPT programs determine progress from one skill to the next, Phase I to II, and treatment completion. HNC mastery criteria were assessed by clinician observation and coding of parent and child behavior during weekly sessions.
Plan of Analyses
Descriptive analyses were used to examine demographics, mobile app use, and the efficiency of service delivery. First, given the mobile app was configured for iOS operating systems and may have influenced app use depending on mobile device ownership prior to treatment, an independent samples t-test was conducted to examine whether overall app use differed significantly between participants who owned an Android and iPhone at baseline. Second, correlation and multiple linear regression analyses were conducted to detect if use of the mobile app (Skills Video, Survey, and Videorecording Skills Practice components) during treatment predicted the efficiency of service delivery (i.e., weeks to treatment completion) in TE-HNC. For the latter, a priori sensitivity power analysis determined a minimum detectable effect size of f2 = 0.246 for the two-tailed test given the sample size (n = 34), alpha (.05), power (0.80), and number of predictors (3). Data were analyzed using IBM SPSS version 27.0. Diagnostics were performed on the data to ensure all assumptions of statistical tests were met and power analyses were conducted using G*Power version 3.1 (Faul et al., 2007).
Results
Mobile App Use
All caregivers used the mobile app at least once during treatment. The average overall mobile app use across components was 41.3% (SD = 22.1); in other words, caregivers used the mobile app components, on average, slightly less than half of the number of times they were instructed to during treatment. Overall use did not significantly differ as a function of whether caregivers owned an Android or iPhone device prior to treatment, t(30) = 0.669, p = .509 (Table 2). The Videorecording Skills Practice component was used most frequently on average (62.8%; SD = 33.2), followed by Survey use (35.1%; SD = 23.8) and Skills Video use (26.0%; SD = 23.4). During treatment, 22 (64.7%) parents used a component of the app at least once 25% of the days, with 14 (41.1%) and 3 (8.8%) parents using a component at least once more than 50% and 75% of the days, respectively.
t-Test Comparing Overall Mobile App Use Between iPhone and Android Smartphone Ownership.
Relation Between Mobile App Use and Efficiency of Service Delivery
On average, families completed treatment in 11.98 weeks (SD = 4.4). A correlation matrix was initially conducted with each of the main study variables (component use and efficiency of service delivery). While the three components were significantly intercorrelated (.444–.620, Table 3), only use of the Survey and Videorecording Skills Practice components was correlated with the outcome variable.
Correlations Between Mobile App Component Use and Efficiency of Service Delivery.
p < .05. **p < .01.
A multiple regression with the three usage variables entered simultaneously was subsequently performed. The overall model for the efficiency of service delivery was statistically significant, F(3, 30) = 7.41, p = .001, with an R2 of .426. Videorecording Skills Practice was significantly associated with efficiency of service delivery (B = 0.70, p = .001), but use of the Survey (B = 1.03, n.s.) and Skills Video (B = 1.02, n.s.) components did not add significantly to the predictive model. Additionally, the model predicted that the number of weeks to achieve sufficient parent skill use and child behavior scores decrease by 7 weeks (~5 days) for every use of the Videorecording Skills Practice component (i.e., ratio of actual component use to opportunities for use). The adjusted R2, which is a corrected goodness-of-fit measure that considers the number of predictors in the model, is .368. Use of the Skills Video and Survey components did not significantly influence the relationship between use of the Videorecording Skills Practice component and the efficiency of service delivery. Results from the model are reported in Table 4.
Multiple Regression Analyses Predicting the Efficiency of Service Delivery.
Note. Model was run with log-transformed variables to meet regression assumptions. Beta from the log-transformed model (Log B) is presented to illustrate the directionality of association. The remaining variables presented in the table have been converted to linear scale via exponentiation for interpretability.
Discussion
Despite the increase in the development of behavior intervention technologies for BPT, few studies to date have examined actual use of technology components in programs (e.g., Breitenstein et al., 2017), especially for technologies designed to augment, rather than substitute, face-to-face treatment (e.g., Cefai et al., 2010; Nixon et al., 2003). Thus, the current study addressed this gap by examining parent use of an adjunct behavioral intervention technology to clarify the distinct and differential effects of engagement with technology components on the treatment efficiency of TE-HNC in low-income families.
Notably, app usage was relatively high during the active treatment phase in comparison to previously documented trends in mHealth and mobile app retention (e.g., 45%–70% reduction in users after 1 week; Bradway et al., 2018; Lattie et al., 2016; Sigg et al., 2016), which may be due, in part, to the additional clinician (e.g., weekly sessions), technology (e.g., automated reminders), and administrative (e.g., technical support) level supports not characteristic of most standalone or commercially available behavioral intervention technologies in prior studies. Consistent with discussions about the caution that is warranted in overinterpreting pilot effect sizes (e.g., Kistin & Silverstein, 2015; Kraemer et al., 2006; Lancaster et al., 2004), overall mobile app use was lower in the present study than with the pilot version in the preliminary study. However, similar trends emerged with Videorecording Skills Practice used most frequently relative to opportunities for use, followed by Survey and Skills Video use (Anton et al., 2016).
Behavioral intervention technologies in BPT have demonstrated efficacy in prior research. Less work has considered if and how parents use these technologies or their impact on treatment efficiency (Bausback & Bunge, 2021), which has been shown to impact implementation in routine practice settings (e.g., Álvarez Lorenzo et al., 2016; Breitenstein et al., 2010; Keyworth et al., 2018). As hypothesized, the current study demonstrates that use of specific mobile app components in TE-HNC predicted the efficiency of service delivery, or how quickly parents met mastery criteria for each of the program skills and completed the program. Further, findings suggest that videorecording parenting skills practice at home and associated feedback, which is not a feature of the majority of behavioral intervention technologies for BPT, may be particularly beneficial for these purposes and additive to traditional and technology-enhanced BPT treatments. One potential explanation for these findings is that the Videorecording Skills Practice component requires that parents engage in an activity (i.e., practicing new skills with the child at home) that is an existing ingredient in traditional BPT programs like HNC, such that the benefits of increased use may simply reflect the effects of greater at-home practice. Although this possibility cannot be ruled out given the study design, open-ended feedback from TE-HNC families after treatment suggests two alternative explanations. For one, the component may serve as a source of accountability for parents to engage in home practice as instructed during treatment, as illustrated by quotes from parents about the specific component during the post-treatment assessment (e.g., “[Videorecording skills practice] forced me to ensure I was having that one-on-one time [with my child]”) and consistent with findings that TE-HNC families completed home practice at significantly higher rates than families who only received the standard treatment in the parent RCT (Jones et al., 2020). This potential benefit may be especially important to explore in future research, as some evidence suggests highly variable caregiver engagement in home practice (39%–89%), which is a common practice element in BPT programs and associated with treatment outcomes (Danko et al., 2016; Berkovits et al., 2010; Högström et al., 2015; Jones et al., 2020; Lyon & Budd, 2010). Additionally, the component may allow clinicians to acquire a more nuanced understanding of parent skill use and child behavior at home and, accordingly, allow parents to receive tailored support on home practice from their clinician (e.g., “The therapist could really see what was going on and [I could] get real feedback on what we needed to improve on”). In other words, Videorecording Skills Practice may facilitate the generalization of the HNC skills from the weekly sessions to the family’s home. Although both possibilities confer an advantage given the importance of consistent practice and skill generalization for behavior change, this component requires a slightly more sophisticated technical system (e.g., video recording and storage capabilities) and use of clinical resources (e.g., clinician time) than the other components. Yet, such functions could be augmented in a range of different formats as technologies and clinical practice evolve, including as a guided behavioral intervention technology with adjunctive human support (e.g., self-directed BPT program delivered online with weekly interactions with a clinician to review video content) or as a standalone technology (e.g., self-directed BPT program with artificial intelligence algorithms to analyze and provide feedback on video content; see Entenberg et al., 2021 for a recent example application of AI chatbox in BPT). Given clinician time and resources may be especially limited in community settings serving low-income communities, future research is necessary to identify specifications (e.g., optimal dosage of the component) and implementation strategies for delivery of TE-HNC in routine practice, as well as evaluate the feasibility, effectiveness, and ethical considerations for the integration of novel technologies (e.g., artificial intelligence) in BPT.
The results of the current study must also be interpreted in the context of its limitations. First, the relatively small sample size limited the opportunity for theoretically-relevant analyses of patterns of mobile app use and treatment efficiency across and within dimensions of socioeconomic status, race and ethnicity, and diverse family compositions. In addition, iPhones with service plans were provided for each family and some families experienced technical difficulties using the mobile app, which may have impacted engagement and real-world implications of the findings.
This study also had several strengths. For one, there has been significant and increasing investment in the development of mental health technologies in recent years across industry and academia, with relatively less attention toward assessing how outcomes improve as a function of engagement with these technologies in ecologically-valid settings (National Institute of Mental Health, 2019; Shah & Berry, 2021;). Thus, this study represented a timely advancement beyond between-group analyses in randomized controlled trials (e.g., HNC vs. TE-HNC) to within-group analyses that further explore the potential mechanisms underlying observed benefits of one adjunct behavioral intervention technology for BPT (Jones et al., 2014, 2020). Additionally, evidence suggests that self-reported measures of technology use have been subject to over- and underreporting (Boase & Ling, 2013; Scharkow, 2016; Short et al., 2018). Thus, this study examined an objective measure of use in analyses. Third, there is a dearth of work on BPT-related behavioral intervention technologies for underserved populations, and this study focused on low-income families who are more likely to have a child with an early-onset behavior disorder and less likely to engage in mental health services (Bausback & Bunge, 2021). Finally, given that HNC is one example in a family of evidence-based BPT programs with similar treatment elements, findings may generalize to and inform research and clinical practice across other interventions as well (Kaehler et al., 2016; Southam-Gerow & Prinstein, 2014).
Conclusion
There is a clear need to cost-effectively improve the efficiency of BPT for low-income families. Despite increased attention to behavior intervention technologies in BPT, very little research has examined actual use of technology components in programs and their differential impact on treatment efficiency. With the impact of technology use varying across specific mobile app components, results demonstrate that parents who used the Videorecording Skills Practice component more frequently required fewer weeks to achieve skill mastery and, in turn, complete treatment. While study limitations should be considered in their interpretation, findings represent a step toward understanding how behavioral intervention technologies may be used to improve BPT service delivery for families facing structural socioeconomic barriers to effective mental health care.
Footnotes
Acknowledgements
The authors would like to thank the families who participated in the study for their contributions to this research.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the NIMH R01MH100377 (ClinicalTrials.gov: NCT01367847) with additional support from the National Institute of Mental Health (R21MH113887; ClinicalTrials.gov Identifier: NCT03597789; 3R21MH113887-02S1), the National Science Foundation (DGE-1650116), and RWJF Health Policy Research Scholars Program.
