Expert review of child and caregiver critiques of a therapeutic guided imagery therapy mobile application targeting disorders of gut–brain interaction in children

Abstract

Background

A guided imagery therapy mobile application (GIT App) is a novel platform for treating children with disorders of gut–brain interaction (DGBI). Previous feedback from child/caregiver dyads suggested modifications for our App prototype. However, their feedback had the potential to affect the intervention's efficacy. Thus, we aimed to have their critiques vetted by relevant experts prior to further App refinement.

Objective

Compare expert reviews of the GIT App with end-users’ (i.e., child/caregiver dyads’) feedback.

Methods

This mixed-methods study with experts included a hands-on App evaluation, a survey assessing usability, and focus groups comparing their perspectives with those previously provided by end-users.

Results

Eight medical and technology experts were enrolled. Their average usability survey score of the GIT App was 69.0 ± 27.7, which was marginally above the 50th percentile. While the expert and end-user usability assessments were generally favorable, both groups agreed that the App's reminder notification feature location was not intuitive, detracting from its usability. Experts agreed with end-users that the App's aesthetics were acceptable and suggested increasing icon and font sizes. Like the end-users, the experts did not achieve consensus regarding the ideal session length or inclusion of background sounds and screen animations. Lastly, the experts agreed with end-users that gamification techniques (e.g., gift cards and virtual badges) would promote user engagement.

Conclusion

An expert review of our therapeutic App revealed findings consistent with end-users and provided insight for modifying the interface and GIT sessions. Based on this experience, we recommend expert vetting of end-user suggestions as a routine checkpoint when developing therapeutic Apps.

Introduction

Disorders of gut–brain interaction (DGBI), formerly known as functional abdominal pain disorders such as irritable bowel syndrome, are chronic abdominal pain conditions not clearly associated with biochemical or anatomical abnormalities.^1,2 These disorders affect about 15% of school-age children and adolescents worldwide^3–6 and are associated with increased psychosocial distress like anxiety and depression and poorer health-related quality of life.^3,7–10 The pathophysiology of DGBI appears to be multifactorial, with evidence supporting the role of diet, genetics, gut microbiome, intestinal hyperpermeability, inflammation, and psychological distress.^{3–5,11–13} The interplay of biological, psychological, and environmental mechanisms on gastrointestinal symptoms form the basis of the biopsychosocial model for DGBI. ¹³ The literature also supports the role of the bidirectional communication of the gut–brain axis in DGBI, which partly explains how psychological factors contribute to chronic abdominal pain and how psychological interventions can help affected patients. ¹³ Hence, professional societies’ clinical guidelines and the Rome Committee support psychological therapies as ideal treatments for affected patients.^14–18

Though medical literature supports psychological therapies for children with DGBI, the sheer number of those affected does not make it practical. Patients routinely encounter barriers to receiving psychological therapies, like poor mental health insurance coverage and needing to meet for repeated in-person sessions. Our research group is motivated by the potential of remotely delivered psychological therapies like guided imagery therapy (GIT), a variant of cognitive behavioral therapy, as an alternative means for children with DGBI to receive an evidence-based treatment. ¹⁹ GIT uses one's imagination to generate sensory images that serve as a mental representation of their pain to promote self-modulation through suggestions and metaphors.^20–22 To overcome barriers associated with the traditional delivery of psychological therapies, we are developing a mobile application (App) that provides GIT sessions tailored for children affected by DGBI. ²³

This alternative therapeutic approach is finding application in adult patients with DGBI, but no product currently on the market targets pediatric patients with DGBI.^24,25 To address this gap, we are designing and creating a GIT App platform for children ages 7–12 years old. Our primary goal is to decrease abdominal pain symptoms in children with DGBI by facilitating the delivery of a proven therapy; therefore, our ultimate goal is to receive Federal Drug Administration approval to target our therapeutic App for pediatric patients with DGBI. This is in contrast to other Apps on the market that have not been scientifically vetted for this indication. Our GIT App program is based on previous positive results using compact discs. ¹⁹ We are updating the therapy sessions and making use of newer, more ubiquitous technology (i.e., smartphones and tablets) that are accessible to nearly all young children.

Creating a successful therapeutic app requires user engagement, which can be evident through users’ attention, intrinsic interest, curiosity, and motivation. ²⁶ To be successful, our App needs to be both (1) engaging for young children for sustained use and (2) capable of delivering high-quality therapeutic GIT sessions targeting DGBI.

To achieve these goals, our App development process has embraced the user-centered design approach, which implies that end-users will influence the final technology product through their direct input. ²⁷ This methodology, although resource-intensive, has the benefit of creating a technology that is more likely to meet end-users’ needs and is technically robust post-development (i.e., fewer technical issues).^28,29

The user-centered design for mobile health application development outlined by Schnall et al. is an ideal framework for creating electronic therapeutic interventions to improve health outcomes.^30,31 Their study summarized the development of an App that aimed to support HIV prevention among men who have sex with men, and their research methods targeted both end-users and experts who provided meaningful insight for the final App design and its usability. ³⁰ Based on Schnall and colleagues’ findings, therapeutic Apps, like the GIT App, should include expert vetting of end-user critiques during the formative development process to minimize compromising the intended intervention's efficacy in future clinical trial testing. Given our unique App development challenges and outcome goals, their modified framework serves as a scientific guide for our GIT App development process. ³⁰

Thus, we previously conducted a study with young children with DGBI and their caregivers to capture their feedback regarding the GIT App prototype's usability (i.e., the ease by which an application can be learned and used), aesthetics, GIT session characteristics, and incentives to increase routine use. ³² However, to assure that their preferences do not compromise the overall App intervention's efficacy, we recognized the need to vet their preferences through additional expert feedback prior to further App refinement. The purpose of this study was to determine the consistency of findings between the GIT App end-users and medical and technology experts regarding the App's usability, aesthetics, GIT session characteristics, and incentives to promote routine use. ³³ Cumulatively, this information will inform the refinement of our GIT App prior to clinical trial efficacy testing. ³²

Methods

Prior child/caregiver dyad user-centered design formative work

Without any end-user input, we developed a generic App prototype that allows prerecorded GIT audio sessions to be uploaded onto a server and streamed to electronic devices like smartphones and tablets on demand. Next, to tailor our App for our targeted audience, we invited dyads of children ages 7–12 years with DGBI and their caregivers to participate in a mixed-methods research study (i.e., both qualitative and quantitative data collection) to provide their input about the GIT App prototype after software evaluation. ³² Participants were informed that the intervention would require children to listen to a GIT session at least 5 out of 7 days per week for a total of 8 weeks, which is the same protocol used in the original remotely delivered (via compact disc) GIT study for pediatric DGBI. ¹⁹ Children were encouraged to retreat to a quiet space where they can sit or lie down and avoid external interruptions while they engage in their audio GIT session.

The dyads then completed a usability assessment (Systems Usability Scale [SUS]) and participated in separate individual interviews commenting on the App's usability, aesthetics, GIT session characteristics, and end-user incentives to promote routine use. ³² The results of the study, including screenshots of the GIT App prototype, were previously published. ³² End-user suggestions of the App modifications were evaluated in the current study.

Participants

Medical and technology experts from a single tertiary pediatric medical center associated with a medical school in the southwestern United States were recruited via email to participate in this study. The medical specialties targeted were general pediatrics, pediatric gastroenterology, child psychology, and developmental and behavioral pediatrics. We also recruited two other experts: a GIT session creator with a background in child psychology research and an App software developer. All participants had multiple years of experience in their respective fields. All participants signed an informed consent form approved by Baylor College of Medicine's Institutional Review Board.

GIT App software evaluation

The experts were informed that they would review the preferences reported by the children and their caregivers from the previous study. ³² For the experts’ evaluation, a 2019 Samsung Galaxy Tab 10.1 containing the GIT App and a task list to guide their navigation through the App was provided by our research group 1 week before participating in a focus group session; this same task list was utilized in the previous child and caregiver dyad formative study. ³² The task list included commands to open the App, log into the App with a preassigned username and password, open a submenu with the GIT session listed, select a particular GIT session for playback, set a reminder notification for 7:30 P.M., and exit the App. The software evaluation also asked the experts to listen to an 8½-min GIT session.

Quantitative usability measure

Experts then completed the SUS questionnaire to assess the GIT App's usability.^34,35 This is a 10-item validated questionnaire that objectively scores the usability of various technologies. The SUS questionnaire uses a 5-point Likert scale to rate 10 survey items: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. Each item score is transformed to calculate the overall score. Odd-numbered items were transformed by subtracting 1 from their response value. Even-numbered items were transformed by subtracting the item response value from 5. The transformed item responses were summed and then multiplied by 2.5 to obtain the overall SUS score. ³⁵ The overall SUS score ranges from 0 to 100, with higher scores indicating higher usability. A SUS score of 68 correlates to the 50th percentile. ³⁶ Adjectives subjectively described the respective numerical SUS score result to give the readership a more straightforward interpretation of the GIT App prototype's usability. ³⁷

Focus groups

Expert focus groups were conducted via teleconferencing about 1 week after their self-guided software evaluation. A moderator used semi-structured scripted questions for the sessions, which were designed to last no longer than 90 min. The objective of the expert focus groups was two-fold: (1) assess perspectives of the GIT App prototype's usability, aesthetics, GIT session characteristics, and means for incentivizing end-users to use the App routinely and (2) compare their perspectives of these same topics with those the children and their caregivers captured in our previous study. ³² An assistant moderator was present to document the session setting and interpersonal interactions. Focus groups were video-recorded, transcribed verbatim, and reviewed for accuracy prior to initiating data analysis. The corrected transcript served as the unit of analysis for the qualitative data.

Data analysis

We calculated the average score and its standard deviation for the SUS. Calculations were performed in IBM SPSS Statistics (version 28; IBM Corporation, Armonk, New York). We compared these results to the affected children's and caregivers’ scores from the previous study. ³²

For the focus groups, trained coders (JMH and TAS) reviewed the corrected transcripts for the analysis. A codebook was constructed prior to analysis to include a priori codes and their corresponding definitions based on the semi-structured script. We used a constant comparison method for our qualitative data analyses using the child and caregiver app modification recommendations captured previously as a reference. ³² During the analysis, additional codes and their respective definitions were added to ensure that participants’ perspectives were captured. Coders compared codes throughout the analysis to ensure consistency in the application of the codebook and met regularly to discuss coding and resolve discrepancies. NVivo for Windows (version 10, Lumivero, Denver, Colorado) was used to organize participant responses and codes. Representative quotes are provided to elaborate on our study findings; speakers were identified by their profession to protect participants’ confidentiality.

The experts provided input on the GIT App's usability both quantitatively through the SUS questionnaire and qualitatively through the focus groups. This mixed-methods research approach offers the opportunity to examine the App's usability further. After completing the quantitative and qualitative analyses, we utilized methodological triangulation to determine whether the results of both research methods converged, complemented, or contradicted one another.^38–40 These data were then integrated and summarized accordingly. ⁴¹

Results

Participant demographics

We reached out to 10 participants and enrolled eight. The average age of the participants was 47.9 ± 12.3 years old. The group's self-reported race/ethnicity was 4 (50%) non-Hispanic White, 3 (37.5%) non-Hispanic Black, and 1 (12.5%) non-Hispanic Asian. Five (62.5%) participants were female.

Systems usability scale questionnaire

The experts’ SUS average score was 69.0 ± 27.7 out of 100, which is marginally above the 50th percentile and can be interpreted as in-between “OK” and “good” usability. ³⁷ Below-average scores were submitted by a behavioral and developmental pediatrician and a mobile app developer (25 and 27.5, respectively). The experts’ average SUS score was lower than the previously obtained children's and caregivers’ average SUS scores, which were deemed between “good” and “excellent” (78.2 and 78.0, respectively).^32,37

Focus group

Two focus groups were held with three and five participants, respectively. The session climate for both sessions was similar, with a cordial exchange of ideas among the participants, even while opposing viewpoints were discussed. The focus groups lasted 77 and 88 min, respectively. Each focus group discussed the previously noted end-user themes of App usability, aesthetics, GIT session characteristics, and incentives to promote regular use. ³²

App usability

In the previous study, the children and caregivers thought the App had favorable usability but identified needed changes primarily related to aesthetic issues. ³² For example, participants had difficulty setting the reminder notification feature, which is designed to serve as an alarm to alert users to listen to a GIT session at a specific time. The children's and caregivers’ perspective on the current location of the reminder notification feature was not ideal because the App images and icons were too small. ³² The experts agreed with the children's and caregivers’ usability assessments and expressed similar thoughts about the settings image and the corresponding reminder notification location in the GIT App.

I just went into that setting little thing on the top, and that's the way I could do it [set the reminder notification feature]. It's not very clear ‘cause it's very small compared to everything else. (GIT Creator)

App preferences

The App Preferences theme encompassed critiques related to the GIT App's components, including its aesthetics and overall GIT session content. Our expert focus groups reviewed the children's and caregivers’ feedback on the App's aesthetics, GIT session characteristics, and incentives to promote routine use.

App aesthetics

This topic covered critiques about the GIT App interface appearance. Our previous study revealed that children liked the current prototype's pictures and colors but also recommended changing the background to a baby blue background and adding various images like a grassy meadow, forests, islands, smiling kids, and animals like butterflies, fish, birds, and monkeys. ³² Caregivers mentioned they liked the current pictures and colors and thought the font size was too small. ³² Our experts agreed with both children and caregivers regarding these App aesthetic perspectives and recommended addressing the small font and image sizes.

Again, I would make whatever the function buttons gears are larger, all the fonts…. (General Pediatrician)

GIT session characteristics

Our expert focus groups reviewed user comments about the App's GIT sessions. After reviewing a sample GIT session, children and their caregivers previously offered their perspectives about the session and what they would prefer to have in these sessions. ³² This study asked the experts to explore these end-user comments covering session setting locations, length, and background sounds and animations.

GIT session setting locations

Children preferred GIT sessions that featured animals like dogs, cats, elephants, and horses. They also liked sessions located at beaches, jungles, playgrounds, and forests. Caregivers also recommended sessions with animals like dogs, cats, and unicorns. They also recommended sessions located at beaches with activities like surfing and boating, forests, biking adventures, and superheroes. ³² The experts agreed with the children and caregivers regarding these session locations.

GIT session length

There was no consensus among children and caregivers about the ideal session length. Most children found a session length ranging from 5 to 10 min was acceptable. Their responses ranged from 54 s to 30 min. Caregivers thought sessions ranging from 10 to 15 min would be acceptable. ³²

There was also no consensus on the ideal length of therapeutic GIT sessions among the experts. Some were concerned that shorter sessions would be more conducive to young children's attention span. However, there was concern that shorter sessions would not be long enough to allow children to create the narrated imagery in their minds and modulate their pain through therapeutic suggestions.

Okay, I could probably use this in shorter segments, but eight minutes, for me, is just too long. (General Pediatrician)

It's really hard to do three to six minutes to get children to relax and then, also, get some therapeutic imagery in there as well. (GIT Creator)

Session background sounds and animations

The children and caregivers did not reach a consensus regarding inclusion of background sounds and animations while the session was playing. Children had recommended sounds like ocean waves, rain, and birds chirping. ³² A couple of children had recommended adding animations to the screen while the session was playing. Similarly, caregivers recommended adding background sounds to the sessions (e.g., classical or instrumental music, nature sounds, and sounds related to the theme). They also had mixed reviews on incorporating animations into the GIT App. ³²

Experts similarly had mixed input about integrating session background sounds and animations into the GIT App. One argument for this change was to help the listener to be more engrossed with the narrated session. The counterpoint stated that the listener's lack of participation in creating the narrated imagery in their minds may affect the intervention's efficacy, and these extra stimuli could distract end-users during the session.

Just simple sounds in the background that enhance the story. If there is this sled, then maybe you hear children laughing in the background while they are also sledding. (Behavioral and Developmental Pediatrician)

Because that's engaging to all their senses [GIT sessions with background sounds and animations], then, and they just don’t have to do it themselves…. They just watch it and listen to it, and their brains would look like they do when they’re watching TV, which is not ideal. (Child Psychologist)

Incentives for regular use

In the previous study, we previously informed children and caregivers that children would ideally need to listen to a GIT session about 5 out of 7 days of the week for about 8 weeks to treat their DGBI based on previous clinical trial protocols.^19,32 Both groups noted incentives could be used to promote regular App use. Children recommended both tangible (e.g., money, food, and toys) and intangible (e.g., virtual points, coins, and unlocking new GIT sessions) rewards would be ideal. Caregivers agreed with the children's recommendations. ³²

The experts agreed that tangible and intangible rewards would be ideal for the GIT App. They were familiar with virtual rewards from other Apps, and this approach would be more sustainable long-term compared to tangible rewards.

A lotta the Apps that we have, they [have virtual] badges. Your first time completing one, “Yay, you get this [virtual] badge.” (General Pediatrician)

Mixed-methods integration of usability measure

Triangulation of qualitative and quantitative findings signaled agreement. The SUS score was slightly above average (69.0), and candid critiques about the illogical location of the reminder notification feature and icons and font size that were too small supported their score. In the previous study, the affected children and their caregivers stated that these same critiques influenced usability and thus affected their App prototype usability scores (78.2 for children and 78.0 for caregivers). ³²

Discussion

Our current results indicate that our experts generally agreed with children with DGBI and their caregivers regarding the GIT App session content and usability. However, among our experts, the attributes for ideal sessions, though avidly discussed, did not reach a consensus. Nonetheless, these medical and technology expert focus groups permitted diverse viewpoints about the App's refinement to be addressed and debated. This process gave our research team additional considerations to contemplate and vet before finalizing app refinement. The initial probe of GIT app prototype preferences with children with DGBI and their caregivers focused on the app interface and GIT session preferences to keep the end-users engaged with the App long-term. This external review of end-user preferences by our medical and technology experts provided a second level of review that confirmed some aspects of the end-users’ observations but importantly, ensured that efficacy would have priority. These results support the importance of obtaining external expert feedback after end-user feedback.

The literature suggests that psychological therapies like GIT, even when delivered remotely via compact discs, are efficacious for children with DGBI.^19,42 Psychological therapy clinical trials for pediatric DGBI, both in-person and remotely delivered, require participants to partake in frequent sessions over several weeks.^19,42 A practical, contemporary approach for remotely delivered therapy should utilize Apps given the ubiquity of smartphones and tablets, even among those with lower socioeconomic status. ⁴³ To develop an App that will effectively deliver GIT to children, we used a user-centered design approach to co-deisgn the software with end-users. ²⁷ The current study was the next step after obtaining feedback from children with DGBI and their caregivers by comparing their feedback with medical and technology experts to promote the development of a GIT App that is both engaging and therapeutic.

It was important to investigate if the previous feedback we obtained from the children and their caregivers would sacrifice the overall intervention's efficacy for the sake of improving user engagement. ³² The experts scored the App's usability somewhat lower than the children and caregivers in the previous study. ³² We acknowledge that this study's smaller sample contributed to the usability score's large standard deviation; nonetheless, these experts echoed the same usability concerns during the focus groups. Overall, the experts’ findings, in addition to usability, were similar to the children's and caregivers’ comments regarding the App's aesthetics and incentives for regular use. Detailed discussions with the experts about the GIT session characteristics provided key considerations for creating new ones during the next refinement phase.

Though avidly discussed by the experts, the attributes for ideal sessions did not reach a consensus. Specifically, session characteristics that may negatively influence intervention effectiveness revealed opposing views. The preference for shorter sessions by most experts was in opposition to other experts who counterargued the need for sessions long enough to deliver the vital components of a GIT session, including induction into a trance-like state, therapeutic suggestions for modulating pain symptoms, and exiting the trance-like state. ⁴⁴ Based on this feedback, our future goal is to create GIT sessions that contain each required GIT component (efficacy) but minimize session length times for brevity (engagement).

Based on this study's findings, therapeutic Apps, like the GIT App, should include expert vetting of end-user critiques during the formative development process to minimize compromising the intended intervention's efficacy in future clinical trial testing. In research of this type, developers and researchers should be wary if end-users recommend App modifications to core intervention components of therapeutic Apps (like the GIT App) and take steps to ensure suggestions do not compromise intervention efficacy. The literature suggests that GIT session modifications for sessions as long as 15 min would be developmentally appropriate for children as young as seven. ⁴⁵ Furthermore, GIT sessions as long as 25 min were utilized for the original compact disc study deemed efficacious for this same age group and patient population. ¹⁹ These end-user and expert study findings enlightened our development team about end-user GIT session preferences and reinforced the need to ground our intervention components in evidence-based principles. Similarly, developers and researchers must be cognizant of the user burden of repeated psychological therapy sessions over time for pediatric psychological research.

Limitations

This study summarizes the expert review of critiques by young children with DGBI and their caregivers toward our GIT App prototype. Our findings may be limited to our therapeutic GIT App prototype targeting young children and their unique developmental capabilities. However, their feedback highlights the importance of involving expert review in the development cycle. Our study methodology did not include repeated expert focus groups until data saturation was obtained. However, the qualitative data obtained from this heterogeneous group of experts offered diverse insight into what should be considered to address various App refinement topics and informed our research group about key decisions related to this App intervention.

Conclusion

Mobile health apps with therapeutic intent must meet the challenge of being engaging for end-users, but these Apps must also deliver the intended treatment to its target patient population. Medical and technology experts reviewed the affected children's and their caregivers’ critiques about our GIT App prototype and confirmed areas needing App refinement based on usability, aesthetics, GIT session characteristics, and the need for incentives to promote end-user regular use. The experts reinforced end-user findings concerning the GIT session characteristics; the dichotomy of user engagement versus intervention integrity summarized these arguments. Nonetheless, these expert reviews of end-user critiques were informative to the refinement process of our therapeutic App. We recommend vetting end-user critiques by content experts as a standard practice for therapeutic App development to diminish the chance of compromising the intervention's efficacy.