Abstract
Objective
The purpose of this manuscript is to detail development and initial usability testing of an e-toolkit designed to provide skills and knowledge around self-management behaviors for individuals living with systemic lupus erythematosus.
Methods
Researchers worked with a steering committee of patients and providers to (1) develop a clickable prototype of an e-toolkit and (2) conduct alpha (individuals not affiliated with an academic clinic as patient or provider) and beta (individual patients with systemic lupus erythematosus as well as members of the clinic healthcare team and individuals who work in patient advocacy organizations) usability testing through semistructured interviews.
Results
During the review of the e-toolkit, the feedback provided by participants in both alpha and beta groups centered on two overarching themes: (1) improving user interface and materials and (2) integration of information and supports between toolkit and clinical personnel.
Conclusion
Digital approaches that are tailored to individual symptom variation and integrated with a clinical system have the opportunity to enhance ongoing clinical care. These findings support movement toward integrated, team-based care models, tailored digital resources, and use of expanded virtual interaction options to ensure on-going engagement between healthcare providers and systemic lupus erythematosus patients.
Keywords
Introduction
Systemic lupus erythematosus (SLE) is a chronic autoimmune condition with a paroxysmal and unpredictable disease course.1,2 SLE manifests with a wide range of symptoms such as joint pain, fatigue, and photosensitivity, leading to vital organ damage, hospitalization, or death. 3 SLE diagnosis is challenging because these symptoms vary considerably across different individuals and over time within each individual, and there are no diagnostic laboratory tests. These fluctuations in symptoms and diagnostic uncertainty create a persistent feeling of general uncertainty from the patient perspective that persists through the disease course. 4 In addition, SLE disproportionately affects women and non-white populations in the United States, with higher prevalence among African American women.5,6
The most common therapies used to control SLE disease activity are not uniformly efficacious and may cause side effects that limit their usage.3,7 In addition, individuals living with SLE often experience persistently impaired health related quality of life (HRQOL). Reductions in clinically measured disease activity have not consistently been associated with increases in HRQOL, 8 indicating that while there is some association, HRQOL and disease activity remain independent indicators of well-being among patients with SLE. Recent research identified potential psychosocial determinants of HRQOL, such as capacity to deal with uncertainty/ambiguity of the disease course, stress, social support, and depression/anxiety, as well as lifestyle/behavioral determinants such as nutrition, physical activity, and patient–provider communication.9–12
Interventions to address the determinants of HRQOL have been shown to improve knowledge, enhance clinical outcomes, and improve the use of medical resources.13,14 For instance, Williams et al. 15 adapted elements of the chronic disease self-management program (CDSMP) with peer mentoring elements to improve HRQOL among those living with lupus. Their results suggest that these types of interventions can lead to improvements such as reduced symptoms, increased self-efficacy, and increased perceived social support. 15 However, there is a question remaining if this type of involved self-management intervention with largely in-person components is the most practical approach for those living with SLE, given the persistence of unpredictable symptoms (flaring) and social determinants of health (e.g. transportation disparities and inability to secure childcare).16–22
Our preliminary interviews with SLE patients on this topic suggested that patients with SLE experience high levels of uncertainty because of the disease process, and this increases their need for informational and affirmational (validation) support. 23 In addition, patients indicated having difficulty managing the variability of symptoms and flares and maintaining relationships with others or attending in-person activities/support groups. These two findings suggest that interventions to improve HRQOL need to focus on providing social supports in formats that do not rely on physical attendance and can be tailored to individual symptom manifestations.
Recent studies suggest that digital interventions may be better than traditional in-person interventions or tools and resources distributed via mail because digital interventions offer opportunities for self-pacing and individualized feedback. 24 Previous studies showed that digital interventions can facilitate knowledge transfer.25,26 This is particularly important for SLE patients, because disease-related cognitive impairments (“brain fog”) may force patients to review information several times before it is retained. 27 Digital tools effectively improve patient–provider communication and disease self-management. 28
Digital healthcare tools may be particularly appropriate for patients with SLE as they can be tailored to individual readiness to change behaviors around self-management,29–31 responsive to individual symptom variation, and can be accessed on-demand. 14 While these benefits of digital healthcare tools are significant, there are some potential concerns about the utility of these types of digital healthcare tools for an urban, largely African American SLE patient population because of issues related to the digital divide (i.e. lack of access to devices and Wi-Fi, and comfort with technology).32,33 We conducted a study with 56 Washington University Lupus Clinic patients and found that 98.2% of participants had internet access and 80% currently used the internet for SLE information. 34 This finding suggests that these patients did not experience many of the traditional barriers in internet access as a population level understanding of the digital divide would suggest. In addition, qualitative data from this study indicated desire for enhanced digital tools such as personalized, dynamic information that is vetted by medical experts and the ability to exchange personal experiences and SLE knowledge with others. 34
Based on these findings, we developed a clickable prototype of an e-toolkit, designed to provide knowledge and skills around topics relevant to individuals living with SLE. The objective of this paper is to describe the development of this clickable prototype, which was developed in conjunction with individuals who either live with lupus or are involved with lupus management and is grounded in the transtheoretical model theory of behavior change, and present findings from our initial usability evaluation.
Methods
Development of the e-toolkit
We convened a patient–provider–advocacy organization steering committee to prioritize content areas (e.g. obtaining appraisal support, communication and listening skills, and ambiguity management) and intervention methods (e.g. professional vs. lay, existing vs. new, technology vs. in-person exchanges) to improve the HRQOL among patients with SLE. During the steering committee meetings, members of the research team (EAB, JL) first presented potential ideas for modules based on findings from previous work with the Washington University (WU) Lupus Clinic patient population and a review of the literature (e.g. communication about aspects of SLE, maintaining sources of social support). 23 Participants were asked to consider the suitability of these content areas and provide suggestions for additional areas. Next, the research team asked participants about specific modes of content delivery (e.g. virtual vs. in-person, static vs. dynamic/interactive). The research team synthesized meeting discussions to ensure feedback was accurately captured. Priorities of stakeholders suggested a need to focus on nutrition and patient–provider communication (content) and use of digital technologies, specifically a web-based platform (intervention methods), which was aligned with our previous work in this area.23,34,35
Based on the priorities and methods they identified, we developed a clickable prototype of an e-toolkit designed for individuals with SLE that includes knowledge, skill development, self- management tools, and “testimonials” in the areas of general nutrition, sodium reduction, and doctor–patient communication. The prototype of the e-toolkit was developed using a digital slide deck developed in Microsoft Powerpoint which simulated clicks to correspond with navigating a website. Members of the research team (JL and EAB) compiled a resource pool through searches of existing symptom management and lifestyle resources. The research team then adapted those resources for an SLE-specific context. If no resource existed, the research team would draft an idea for a potential resource for future development. These resources are available as a mixture of internal materials and outside links. Users are able to sign into to the e-toolkit and select from two methods for browsing content: (a) directly clicking on topic areas of interest or (b) completing a tailored assessment that will guide them through the material based on current experiences with SLE-related behaviors, as well as interest in and readiness for changing (stage of change) these behaviors (e.g. nutrition and pain level). We expect the overall usage of the tool to vary considerably across patients due to variability in SLE symptoms.
The e-toolkit is organized using a dashboard concept. The dashboard (Figure 1) is the first page encountered by the user and acts as the “homepage.” The dashboard contains blue buttons (top left of Figure 1), which are visible on every page of the e-toolkit and green buttons (middle of Figure 1) which are only visible on the home dashboard page. Table 1 documents the full range of functions accessible through these colored buttons.
Dashboard example.
Descriptions of dashboard button functions.
The pages accessed through the “assessment questions” (Figure 2) button present a series of questions to the user, answers to which feed into a tailoring algorithm that provides a customized recommendation of resources (Figure 3). Tailoring is a way to maximize the likelihood that the information and resources provided are relevant to the reader by customizing the message based on individual responses to questions.36–38 For purposes of this study, participants are asked general questions about key issues and behaviors (e.g. interactions with physicians and nutrition) and their readiness to engage in change in these behaviors. Based on participant responses, a computerized algorithm (for use with an app or website) was created (by LK) according to general categories of the transtheoretical model 31 with content that was divided broadly between (a) precontemplation/contemplation/preparation and (b) action/maintenance. The message content varied between these two categories, with precontemplation/contemplation/preparation (I believe behavior does not need to change/I do not intend to change behavior/I am planning to change behavior) receiving more knowledge-based resources and action/maintenance (I am actively changing behaviors) receiving more skill-building exercises.
Assessment question example.
Resources recommendation page.
Usability evaluation
We (EAB and JL) conducted both alpha and initial beta testing of the e-toolkit clickable prototype. Individuals for both groups were identified through purposeful sampling. Alpha participants were chosen based on their familiarity with online tools but were not affiliated with the WU Lupus Clinic as patients or providers. The purpose of this alpha testing was to identify any issues with e-toolkit prototype design as well as to test the flow of the guided session. 39 Changes after alpha testing were determined through debriefing by members of the research team (JL and EAB) after conducting each alpha test. 40 Based on the feedback from these individuals minor changes, such as adding a progress bar at the bottom of the screen, were made. No changes were made to the content or general navigation after alpha testing. The initial beta testing of the e-toolkit clickable prototype was then conducted with patients with SLE as well as members of the WU Lupus Clinic healthcare team (physicians and clinical and research staff) and individuals from patient advocacy organizations. Beta participants were identified through affiliation with WU Lupus Clinic. Patients were approached during regularly scheduled appointment times. Providers and patient advocacy groups were recruited via email/telephone contact.
The participants for both alpha and beta testing were informed of the purpose of the interview and asked for permission to tape record the session. Each session started with the presentation of a hypothetical patient. The participant was then guided through the website in a set sequence based on the hypothetical patient through the dashboard features, the assessment questions, and the resources provided based on the responses to the assessment questions. A set sequence was used in order to ensure that all pathways were reviewed. A hypothetical patient was provided at the beginning to reflect a realistic usage of the particular sequence and provide those who did not have lupus a sense of realistic background source explaining the tool usage.
A modification of cognitive interviewing and “thinking out loud” protocols was used. Participants were encouraged to provide concurrent thinking, or “thinking out loud” as they moved throughout the website as well as provide unprompted feedback.41,42 This was followed by prompted feedback, or retrospective probing, which was then solicited at the end of each toolkit section. This retrospective probing included general impressions, specific language changes, alternative methods of content delivery, and additional resource suggestions. Participants were asked to complete an online questionnaire to obtain basic demographic information.
Analysis
Descriptive statistics were generated for the demographic data. Data from the prompted and unprompted feedback from beta testing were transcribed verbatim and analyzed using focused coding techniques in which multiple members of the research team independently assigned codes to transcripts based on the interview topics, discussed disagreements, and came to consensus on the best way to address discrepancies. After initial coding, codes were then arranged into thematic clusters with summary paragraphs describing the key elements of each cluster. 40
Results
Participant demographics are provided in Table 2. There were six participants in the alpha group and nine participants in the beta group. The majority of alpha participants had no affiliation with lupus (83%) while all participants in the beta group had some affiliation with lupus.
Demographic characteristics of alpha and beta group participants.
During the review of the e-toolkit, the feedback provided by participants from the beta group centered on two overarching themes: (1) improving user interface and materials and (2) integration of information and supports between toolkit and clinical personnel.
Improving user interface and materials
Participants from the beta test provided a number of insights related to improving the user interface and materials including suggestions related to resource content (subjects and presentation of the resources to users), formatting (organization and layout of the toolkit), and language (alternative words to use and suggestions for assessment question answer choices).
Resource content
Overall, participants suggested that it would be helpful to develop an introductory paragraph that can be placed on the website landing page that lays out the intent of the website and emphasizes that working with the toolkit will give patients information and skills to reduce the negative impacts of SLE on their health and quality of life (Table 3, quote 1).
Improving user interface and materials.
Speaking to the breadth of content presented in the toolkit, participants appreciated both embedded material and outside links to resources as long as the material had been “approved” by their providers. There was a desire for a mixture of content that showed “the full picture” of an issue (Table 3, quote 2), while also having information summarized and simplified by including some embedded information that is created by the clinic, with both being vetted or approved by their healthcare team (Table 3, quote 3).
Participants noted that the resources did not have to be specific to SLE (Table 3, quote 4), and that they appreciated having both basic and more advanced information about the various topics (e.g. nutrition) so that patients can find information that is most pertinent to their needs (Table 3, quote 5).
Participants also emphasized the benefit of a variety of informational formats to match a variety of learning styles including text, video, and podcast (Table 3, quote 6). For text-based resources, participants preferred brief text with lots of spacing, supplemented with either videos or graphics (Table 3, quote 7). They also indicated that any text resources should be printable and allow changeable font sizes (Table 3, quote 8).
Similar to the request with text-based resources, participants suggested that video resources remain brief to retain attention (Table 3, quote 9). In addition, participants suggested using bullet points to highlight key information conveyed within video resources would be beneficial as it would ease remembering video content without having to re-watch the whole video (Table 3, quote 10).
Participants also indicated that they would like a way to identify previously viewed resources and manage the organization of those resources. One suggestion was to be able to organize their account into folders with time stamps so that they can track when they last used various elements of the system (Table 3, quotes 11–12).
Formatting
Participants had a number of suggestions for how to improve formatting, focused mainly on the assessment questions. For example, they suggested that there be open-ended questions or space to clarify their responses through written comments (Table 3, quote 13). They suggested shortening the number of questions on a page and providing definitions of certain terms (e.g. photosensitivity) (Table 3, quotes 14–15). Participants also indicated that it would be helpful to be able to print out their responses and/or otherwise share responses with their physician (Table 3, quote 16).
Language
In terms of the specific dashboard areas, participants suggested alternative language, such as “taking time to talk about lupus” as opposed to affinity groups (Table 3, quote 17). Participants also stated a desire to find both in-person support groups in their geographic area as well as online support groups, noting that online groups might be more appropriate for younger populations (Table 3, quotes 18–19).
When answering the questions about interactions with their physicians, participants indicated that it might be helpful to add a way to provide feedback about members of the healthcare team beyond the physician and being able to differentiate among these providers, such as through an open-ended question (Table 3, quote 20).
Integration of information and supports between toolkit and clinical personnel
In addition to the user experience of the toolkit, participants commented on their desire to have integration of information and supports offered through the toolkit and their clinical care. This entailed suggestions around tracking toolkit usage and privacy, consistency of information offered by the toolkit and clinical personnel, and building skills to facilitate communication with clinical personnel.
Tracking toolkit usage and privacy
Participants noted that the ability to track toolkit usage would be helpful. For instance, participants suggested that answers to the assessment questions could be stored and patients could choose to provide them to healthcare providers or clinic personnel (Table 4, quote 1).
Integration of information and supports between toolkit and clinical personnel.
This information could also be used by patients and healthcare providers to track the responses to various questions to see shifts in symptomatology for improved patient care and for future research studies (Table 4, quote 2).
Tracking could also assist with flagging responses that indicated potential concerns. This could then prompt a message to encourage the patient to seek medical care as needed and/or a message to providers to prompt them to be in touch with patients as needed. Tracking and flagging, possibly with a pop-up message, in this way was seen as useful as some people may need an “extra push” to go see a doctor (Table 4, quotes 3–5).
In their discussions about tracking and information sharing between the toolkit and healthcare clinic/providers, participants also voiced concerns about privacy and implications for clinical care. They emphasized wanting control over who had access to the information collected through the toolkit. Participants suggested that having this control, and reminding patients about it, would help patients be more honest in their responses. They also noted that it would be important for patients to be reminded that tracking features, and the assessment overall, would only be useful if they are honest in how they respond to the questions (Table 4, quotes 6–7).
One recommendation to address the issue of privacy was to create a sign-in through which users may customize information sharing privileges. For instance, in the diary/journal function, participants indicated that they wanted a variety of privacy controls to enable the user to determine what, when, and how (print or share digitally) they shared the information with others (providers or others they might invite to review their “account”). Some features of the toolkit (e.g. diary/journal) also may help patients increase the specificity with which they describe their symptoms which was seen as potentially enhancing the clinical appointment, particularly if healthcare providers are able to see the information ahead of time (Table 4, quotes 8–9).
They also indicated that an added sign-in for the e-toolkit would not only provide an extra level of privacy but would also make the tool more personal and enable them to track their experiences, such as with pain or different medications. They noted that providing templates might also help in this regard (Table 4, quotes 10–12).
Integration of information within the toolkit (as well as between toolkit functions and with electronic medical records)
Participants also commented on the consistency of information/resources across the different toolkit functions. For instance, participants noted with the calendar function that they would like information on lupus-related events both internal and external to the lupus clinic itself, including online or in-person workshops or opportunities to connect with others with lupus. They also suggested that these events should connect to the resources recommended after completing assessment questions (Table 4, quotes 13–14).
Participants also noted that it would be important to have this interactive toolkit connect to electronic medical record information and potentially with existing applications/websites that they may already be used to schedule appointments, ask questions, and refill medications (Table 4, quotes 15).
Facilitating communication with clinical personnel
In addition to integration of information, participants also spoke broadly about the toolkit as a way to facilitate communication with various clinical personnel. For instance, they recognized that while there would not be someone available all the time, having the chance to send private messages and/or view the scheduled times to speak in real time, potentially face to face with a healthcare expert via video, would be helpful (Table 4, quotes 16–18).
Participants saw this chat function as particularly useful for “minor questions,” to ask clinicians. Importantly, these clinicians may include not only healthcare providers (e.g. physicians and nurse practitioners) but also clinical support staff, social workers, health educator, or other lupus experts, so long as the person knows and is educated about lupus (Table 4, quotes 19–21).
Additionally, it was important that the “chat” option include speaking directly with the individual over the phone or video and that the chat option be responsive to family/significant others (Table 4, quotes 22–23).
Discussion and conclusion
Suggestions on improving user interface and materials centered on what content was available on the website, how to interact with the content, and the organization and presentation of content. The second category of suggestions involved having integration of information and supports offered through the toolkit and clinical care. Key suggestions in this category included balancing the ability to track toolkit usage with ability for the user to control privacy settings and information sharing, providing a connection to data from user's electronic medical record, and having ability to interact and communicate with clinical personnel outside of scheduled appointment times.
The novelty of our findings is the suitability of dynamic, interactive functions built into a web-based platform to deliver knowledge and skills to patients with SLE. While other patient-focused websites specific to SLE exist, these websites largely provide static, generalized information.43,44 While some recent efforts have sought to add self-management modules to patient-focused websites around SLE, these efforts have focused more on information (i.e. suggesting to patients that they should cooperate with physicians) rather than activities which seek to develop self-management skills (i.e. modules around how to improve communication with their physicians). Furthermore, these existing websites do not provide individualized or tailored guidance for using the resources, opportunities to track individualized patient experiences, or self-management tools and skill development resources. In other words, they do not harness the full range of innovative tools that digital platforms can currently provide.
Our e-toolkit prototype is the first to offer tailored guidance to which resources may be relevant to a patient at a given point in their illness course. This level of responsiveness aligns with what we understand about the SLE patient experience, which is governed by uncertainty/ambiguity of symptoms and unpredictable fluctuations in the disease course. 4
A key takeaway from these findings is that participants desire an e-toolkit, which enhanced clinical care received from a medical professional. For instance, participants liked the association between the e-toolkit and the clinic as it gave them confidence that the information was vetted by experts and consistent with the expectations of clinical care. In addition, participants appreciated the notion of interacting with providers (including physicians, support staff, nurses, social workers, health educators, and community health workers) through the e-toolkit more informally as opposed to having only periodic, primarily in-person formal clinic visits.
Our data also suggest participants want a certain level of control of the e-toolkit, including the ability to come back to information (my account).The “memory” of the e-toolkit would enable patients to revisit information that they may have received in the past but was not relevant to them at that moment, a useful feature for SLE patients living with constantly fluctuating symptoms.
This desire for control of the e-toolkit is notable and may also facilitate a greater feeling of empowerment by the patient in managing their SLE. The lack of definitiveness on the causes and treatments of SLE is recognized by physicians and researchers but presents unique challenges to patients in managing their illness. 4 The ability of patients to, for instance, document their symptoms through a guided e-toolkit such as the prototype evaluated here can help patients regain some control in a disease experience dictated by its uncertainty. In addition, healthcare providers who have access to patient information can be perceived as gatekeepers of patient information.45,46 The storage and access of this information through a user-friendly e-toolkit can help rebalance this discordance and empower patients to understand and control their own health information.
Participants also had concerns about the privacy of their data, specifically about who had access to their responses and usage of the toolkit. They desired a sense of control around their information. The simple solution would be dictated by current Health Insurance Portability and Accountability Act (HIPAA) regulations, treating this information as one would treat any information within the medical chart. Consent for participation in the e-toolkit could be integrated into the consent for treatment. However, our data still suggest that users would prefer greater access to their own information, meaning that patients would be able to determine what is visible to others as opposed to the electronic medical record (EMR), which generally has providers determining what is accessible to patients. 45
There are several limitations to this study. Recruitment for both alpha and beta groups was limited to the same geographic region. In addition, participants in both groups were generally highly educated and likely had previous comfort with using digital tools. Despite this limitation, our previous work suggests that the vast majority of the clinic population at the recruitment site do use the internet for lupus-related searches. 34 While our overall sample size (n = 15) was small, it was appropriate given the scope of our test (e.g. e-toolkit was not a final version). In addition, we intentionally sought a broad age range and racially diverse sample for beta participants. Lastly, the clickable prototype was not a live website/app and required some additional imagination from participants as they evaluated the e-toolkit.
The suitability of increasing integration of digital technologies, such as our e-toolkit, with clinical care fits into a larger movement toward integrated care models that incorporates direct care providers (e.g. physicians, nurse practitioners, and physicians’ assistants) and those without specific expertise in rheumatology (e.g. allied health professionals, social workers, and lay health advisors). Specifically, our data provide support for an integrated care model, 47 which centers the patient perspective in service delivery by providing “team-based care,” which addresses patient concerns across domains including biomedical care, social determinants of health (e.g. food and housing insecurity and transportation) and behavioral health, and which are enhanced by digital tools, such as the e-toolkit described here.48,49
In addition, patient encounters could be more productive as the e-toolkit provides ways for patients to continuously document their symptoms and share that information with their provider, which could lead to better provider–patient communication. Discordance in priorities between patients and their providers is well documented in rheumatic conditions and SLE specifically.50,51 Tools such as our e-toolkit can center patient priorities in treatment plans and also empower patients to advocate for themselves. The implementation of this toolkit would need to consider, though, the additional time required of healthcare providers to manage this new method of interacting with patients. Future iterations of this work may consider leveraging a network of community health workers who act as patient navigators and are integrated with a healthcare team to manage patient communication, respond to patient questions about using the toolkit, and address any barriers they may encounter in doing so. 52
Lastly, our findings point the suitability of expanding virtual interaction options beyond what has traditionally been considered as part of telehealth approaches to ensure ongoing engagement between healthcare providers and SLE patients.
Patients desire greater frequency of contact and more informal contacts with their healthcare providers and others with SLE expertise which can be facilitated by digital tools. While barriers exist in fully implementing these communications, such as integration with existing EMR systems and ensuring that appropriate payment models exist to reimburse such services, 53 it is imperative that we consider digital tools as ways to facilitate improved patient outcomes in order to overcome these barriers.
Footnotes
Acknowledgments
The authors would like to thank members of the patient–provider–advocacy steering committee and those who took part in interviews. The feedback from these individuals has made this work possible.
Contributorship
All authors contributed to study design and development. JL and EAB analyzed the data, with reviews from AHK and LK. JL and EAB drafted the manuscript. AHK assisted with participant recruitment and researched literature and conceived the study. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
Ethical approval
This study was approved by the Saint Louis University Institutional Review Board (protocol #30012, initially approved on 15 January 2019, last approved on 14 February 2020) and the Washington University Institutional Review Board (protocol #201910109, initially approved on 14 October 2019, last approved on 23 June 2020).
Funding
Mr Leung was supported by Rheumatology Research Foundation Medical Graduate Student Preceptorship Award and Lawren H. Daltroy Health Professional Preceptorship Award. Mr Leung, Dr Baker, and Dr Kim were supported by National Institutes of Health (NIH)/National Center for Advancing Translational Sciences (NCATS) grant UL1 TR002345. Dr Kim was also supported by NIH/National Institute of Arthritis and Musculoskeletal and Skin (NIAMS) P30 AR073752 and Rheumatology Research Foundation.
Guarantor
EA B.
