Design and evaluation of mobile scenario based learning in the self-management of chronic pain

Abstract

Chronic pain is a lifelong issue, being one of the main causes of disability, affecting a great number of people worldwide, many of which often avoid seeking medical advice from pain experts and/or demonstrate poor adherence to their therapeutic plan. One of the most important steps in achieving a manageable course of disease, is the ability of self-management. We aimed at applying a method of systematic patient education and self-management through the use of Virtual Patients (VPs), a well-established method for educating medical doctors and students but never before targeting patients. Two VPs scenarios were designed, tested and evaluated by patients with rheumatic disorders, achieving a SUS score of 88/100 “Best Imaginable”, alongside with positive reviews from the participants. The positive feedback from the patients supports the potential of VP educational paradigm to educate these patients and equip them with disease coping skills and strategies.

Introduction

Chronic pain with moderate to severe tension appears in 19% of adult Europeans, ¹ seriously affecting the quality of their social and professional lives. Although there are differences between EU countries, chronic pain is a major health problem in Europe that needs to be seriously confronted. Chronic pain is associated with many different diagnostic entities ranging from diseases such as for example, Osteoarthritis, back pain, inflammatory musculoskeletal conditions such as rheumatoid arthritis, ankylosing spondylitis and psoriatic arthritis but also neuropathic pain conditions such as diabetic polyneuropathy, post-stroke neuralgia or multiple sclerosis. ²

One of the most important steps in achieving a smooth course of a chronic pain condition is the ability to be managed from patients themselves. This will result in lower healthcare costs for both the healthcare system and the patients themselves, minimizing visits to healthcare providers and enhancing patients’ self-confidence in controlling their illness.^3–5

A prerequisite for achieving a satisfactory degree of self-management is that the patient is well informed about his/her condition, the various problems and situations that he/she may eventually be faced with, and also possible ways to resolve them. Although traditional methods of patient education through self-management information booklets and discussion with the medical practitioner have been established for many years in medical practice, new technologies now provide simple, easy and cost-effective solutions to patients, aiming at cultivating and developing problem solving skills for pain management (“Learning by Doing”). Some approaches implemented virtual reality technologies for patient education, especially for patients with diabetes,^6–10 while others focused on mobile applications visualizing patients outcomes during and after hospitalization with the addition of educative material on their treatment modules.^11–14

Projects for self-management that involved chronic pain conditions have also been developed for internet-based delivery systems, ¹⁵ mobile applications,^16–18 or home-exercise programs. ¹⁹ To this end, authors of this work are actively involved in the development of EPIONE™ mobile app in order to provide a truly patient-centered treatment experience that promotes the notions of evidence-based practices and precise and personalized medicine. EPIONE™ implements as part of its educational feature a contemporary, problem-based/case-based learning approach, as the one of Virtual Patients (VPs) promoting problem solving skills of patients, emotional coping strategies of relatives and family, and clinical competencies of non-pain clinical experts. A “virtual patient” is defined as an interactive, computer-based simulation of real scenarios in the form of a serious game, for the purpose of practicing, training or assessment. ²⁰ Virtual patients, unlike real patients, are inherently repeatable and offer few limitations with respect to time, place, and failures. Trainees have the opportunity to practice on any aspect of their condition that may be encountered later in daily life, even rare or highly risky cases. The reproducibility and capacity for standardized, validated assessments have made virtual patients an important and effective tool in modern medical education.

There is a worldwide trend to develop virtual patients and many academic institutions are working toward this. ²¹ Although traditionally virtual patients are used by physicians, paramedics and medical students,^22–27 we have set the goal to develop such scenarios for chronic pain patient use, to cultivate the special abilities needed to deal with a situation in real life. In particular, these virtual patients were created so as to be easy to use on both laptop and smaller portable devices (smartphone, tablet). For the trainee, being able to play a VP scenario on his/her mobile device is important mainly for two main reasons: (i) the global internet traffic which comes from mobile devices is increasing over the last years. Starting in 2015, the 52.7% of the global mobile phone population accessed the internet from their mobile phone – which is expected to grow in 63.4% in 2019, ²⁸ (ii) according to Nguyen et al., ²⁹ mobile application in chronic pain “complemented their role in patient management as additional sources of medical information of their patients”. Based on these facts, we chose the mobile platform as our main framework, for approaching patients with chronic diseases.

To our knowledge, our application is the first one to provide ubiquitous Virtual Patients for patient education in chronic pain management.

The structure of the paper is as follows: the Methods section includes a description of the mobile app developed, the VP scenario development process, the recruitment of the patient population that participated in the usability study and the evaluation methodology; the Results section includes the aggregated responses of twenty (20) chronic pain patients that interacted with scenarios with respect to the app usability and its value as an educational tool; the Discussion section provides analysis of the results and comments on the limitations and potential future work and finally, the Conclusion section provides an overall summary of the work performed.

Methods

Mobile app

Existing VP systems, such as Open Labyrinth (OL), ³⁰ are web-based and, thus, can be run using a mobile browser. But as these applications are not optimized for mobile viewing, the user experience (UX) suffers in small screens, as shown in Figure 1.

Figure 1.

Open Labyrinth scenario played in mobile device.

Hidden areas of the screen are clearly spotted, inconsistent buttons sizes etc. Considering that nowadays more than 50% of the users, navigate the web through a mobile device, one of the main actions was to develop a standalone mobile player for accessing VP content, to enhance the usability and the user experience. The mobile player was implemented as an independent progressive web application, ³¹ designed in a way that makes it easy to be integrated in the future with existing VP systems using a microservice mechanism or just via the XML (e.g. MedBiquitous Virtual Patient Data) export of the OL scenarios. ³²

A bottom-up approach was followed, designing the player not only to fit in small screens, but also to maximize the user experience, when played in a mobile device. The buttons were replaced by a radio button list, and a separate button to continue to the next node. This way we prevent accidental taps on the screen but also, we make the user interface more consistent, by having the exact same button size in all screens and let the text flow according to each answer size. Using coding techniques, the user is prevented from going forward and backward between nodes: if a user clicks the back button (on an Android device for example), the application routes the action to the main, home screen instead of the previous node. Thus, user transitions are better controlled between nodes in the virtual scenario, which might prevent some forms of dishonest or unexpected behaviors in the system. Modal dialogs for supporting information and pathway, let the trainee get extra information, without forcing to navigate or scroll. On top of the screen there is a progress indicator, that lets the user know if he/she reached any given (by the virtual scenario author) milestone.

From a developer’s point of view, special functions were added to allow the data exchange between the player and xAPI learning record stores. Excluding milestone top progress indicator, all other components are compatible with the VP MedBiquitous standard. ³³

In order to finalize and refine the usability and user experience several end user pilots took place in an iterative way. For instance, a usability evaluation of the mobile player was performed with an initial set of 15 medical students (beta testers).

Development process of the virtual patients

An author can develop VPs using a simple text editor (Figure 2). The formal data structure description of VPs is based on an extended Aiken format. ³⁴ VP files can contain links to images, videos, sounds and generally assets. The text files were processed using an online Json editor, ³⁵ rearranged in nodes containing all information related to a specific stage or time point through the course of the virtual patient. More specifically, the main elements (tags) included at each node formal description are: a text (“content”) where an episode of the scenario is elaborated, a progress percentage (“progress”) relevant to the scenario evolution, an image path (“images”/“src”) referring to a node-specific icon visualizing the VP context in an eye-catching way and a list with the different options (“buttons”/“label”) for each node, that the learner has to choose from, in order to form the story continuation (“target node”) based on his/her own preferences. The nodes were arranged with many crossovers, letting the patient to reach the final node and complete the scenario after choosing alternative paths. The aforementioned approach, can improve the patient’s engagement with the scenario to explore different choices as well in subsequent attempts to play. At any given time, the patient can review his/her previous choices, by pressing the “review pathway” button, while at the end of each scenario a log of the time spent during each node is displayed to the patient, with the option to either replay the scenario or choose to play another one from the available list. Some nodes have additional information and advice on disease management displayed to the user by pressing the “support-info” element/tag.

Figure 2.

Coding a VP case: using a simple text editor an author can easily embed text, images and other media types into the virtual patient mobile app user interface.

VPs content for chronic pain

The participatory design approach ³⁶ was followed for the development of the app contents. The participatory design was utilized as an approach aiming to actively involve all stakeholders (psychologists, patients, end users) in chronic pain management., In particular, the creation of the virtual patients’ scenarios was based on the insights and advice from a group of ten patients with rheumatic disorders who are members of EL.E.AN.A – the Hellenic League Against Rheumatism (https://www.arthritis.org.gr/). ³⁷ These patients were actively involved in a series of group meetings focusing on disease management through experiential learning. The meetings were coordinated by two psychologists experienced in dealing with patients suffering from rheumatic conditions and chronic pain. Since experiential learning is not a passive process but an active one which engages the learner, the patients involved were encouraged to analyze skills and attitudes on disease management through their personal experience. The use of experiential learning sessions was based on the principles of reflection: patients were engaged in exchanging direct experiences. The aim was to increase the ability of patients, as learners, to develop skills of self-management. Most commonly reported problems faced by patients in their daily lives, as well as, commonly suggested solutions expressed by patients and discussed with the two psychologists were all included in the designed scenarios. Therefore, this process essentially allowed the facilitation of knowledge created through the participation of patients and its transformation into experience. The virtual patients were then built around meaningful learning scenarios that reflect reality. These virtual experiences can recreate what happens in the everyday life of patients suffering from chronic pain giving them the chance to practice their skills in self-management. By the sequential steps of exploration, sharing, processing, generalizing, and application of this valuable experience, the training/learning scenarios that were created could then help patients reflect on the process, connect it with real world examples and apply it to another (real life) situation.

Two Virtual Patients scenarios were developed, including a female and a male main character, with a detailed description of their condition and some information on their background to make the scenarios more realistic and appealing to the patient trainees. The female character was a patient with rheumatoid arthritis and the male one was a patient with ankylosing spondylitis, as these are some of the most common autoimmune rheumatic conditions related with chronic pain in women and men accordingly. In each scenario, the virtual patient was faced with a different set of issues related to his/her occupation, daily life at home and personal relationships (Figure 3). These issues were suggested by the initial group of patients during the meetings and were supported by the available literature concerning chronic pain. Several nodes of both scenarios contained supporting information which consisted of advice on disease self-management. In certain nodes official, anonymous mental support from trustworthy sources was provided, for example the 24/7 support line of EL.E.AN.A.

Figure 3.

Indicative VP scenario nodes.

During the development of the above two Virtual Patients scenarios, care was taken to make them as generic as possible in order to relate to patients with chronic pain suffering from different conditions than the described ones and facilitate the development of new Virtual Patients scenarios.

Participants

The field testing of the Virtual Patients application took place at the premises of EL.E.AN.A regional chapters in various cities (Athens, Thessaloniki and Larissa). In total, twenty (20) patients, members of EL.E.AN.A, participated in the pilot, playing both scenarios and completing the evaluation successfully. All participants received identical training, instructions by the same instructors and followed the same protocol of activities during the testing sessions.

Mean age of all the participants was 49.4 years, all female. All of them were Greek native speakers, as Greek was the language used during the pilot. Rheumatoid Arthritis was the most common diagnosed rheumatic condition among the participants (10/20). Finally mean duration of the diagnosis was more than 10 years (Table 1).

Table 1.

Background and demographic data.

Gender	Female (100%)
Mean Age	49.4
Rheumatic Condition (patients may have more than one)	Rheumatoid Arthritis (10/20), Fibromyalgia (4/20), Ankylosing spondylitis (4/20), Sjögren Syndrome (3/20), Systemic Lupus Erythematosus (2/20) Psoriatic arthritis (2/20), Crohn’s arthropathy (2/20), Still’s disease (1/20), Crest syndrome (1/20)
Years since diagnosis	Mean value: 11,9
The patient lives alone	No (18/20), Yes (2/20)
Education (years spent in education after elementary school)	Mean Value (7.2)
Years of experience with smartphone/tablet	More than one year (13/20) 1 year or less (4/20) None (3/20)

Study design

Several session hours were set up to accommodate the daily schedule of the participants. Four (4) small groups of equal size were formed, where real-time feedback between the instructors and the participants could be provided and the same instructors were available through the entire duration of the study in case the participants requested any technical assistance. The questionnaires were administered to the participants for the anonymous collection of basic demographics followed by a brief PowerPoint demonstration of the application’s basic functions, with particular emphasis on navigating throughout the application environment. During this introduction, the participants were encouraged to ask clarifying questions regarding the application with the aim of ensuring that all participants would understand how to properly use it. Smartphones and tablets with the application were provided to the participants, although most of them preferred using the application within their own devices.

After the presentation, the participants played a short demo-scenario to become familiar with the application. The demo consisted of basic instructions on how to navigate through the application’s environment and how to choose the preferred option from the available ones, in order to progress the scenario. Finally, after completing the two scenarios, the participants completed questionnaires (similar to the ones used by Pecorelli et al., ¹² also see Figure 4) addressing the beneficial effects of VPs on future self-management activities in order to evaluate the application and both scenarios. Patients also had the ability to provide personal comments to improve the implementation of the application based on their own needs and preferences. The questions were chosen to cover the perceived effect of VPs on disease management, if the advice given through the application is in line with the patients’ doctors’ advice and how likely they are to use it in a regular basis, thus validating the application as a stable and reliable means of patient education and self-management. A usability evaluation was performed using the System Usability Scale (SUS). ³⁸ The answers to the questionnaires, except the open-ended questions (likes/dislikes about the app and recommended changes), were based on a 5-point Likert scale as follows: 1 = Completely disagree to 5 = Completely agree. Informed consent was obtained from all study participants.

Figure 4.

System Usability Score (SUS) questionnaire. Average SUS Score was 88/100, “Best Imaginable”. <25 = “Worst Imaginable”, 25.1–51.6 = “Poor”, 51.7–62.6 = “OK/Fair”, 62.7–72.5 = “Good”, 72.6–84 = “Excellent”, >84.1 = “Best Imaginable”.

Results

During the pilot testing, participants were able to go through the scenarios by themselves with relatively little technical support and using either their own devices or those provided to them. All participants were able to complete the two main scenarios, as well as, all questionnaires within an hour, including the time spent attending the PowerPoint presentation and finishing the demo scenario. Mean time required for completion was a little less than 5 min for each scenario, without any particular difficulties, thereby demonstrating a universally fluent use of the app, even by people without any prior smartphone experience. Specifically, the completion mean time for the first scenario was 286 s (SD 56 s) and 297 s (SD 138 s) for the second scenario respectively. The fastest completion time for the first scenario was 167 s and the slowest time was 367 s. Respective times for the second scenario were 102 s and 588 s. The average time spent on each node for the first scenario was 20.8 s and 19.7 s for the second one, including the process of reading the text, making their choice and moving forward to the next node, with no participant spending more than 75 s on any node of the two scenarios. Each participant visited an average of thirteen (13) nodes (out of nineteen (19) available in total) for the first scenario with a ratio of 21.8 s per node visited; respectively, for the second scenario, fifteen (15) nodes were visited (out of twenty-three (23) available) with a ratio of 19.7 s per node. Prior experience to smartphone and tablet use did not seem to affect the completion time of the scenarios. In particular, patients with less than a year experience with technology completed both scenarios faster than the relatively tech-experienced patients (i.e. those with more than a year experience with technology) (272 s vs 290 s for the first scenario and 218 s vs 316 s for the second scenario), indicating the ease of use of the app. There was a low degree of positive correlation between age and educational level and the completion times for either scenario (R2 < 0.24 for both parameters) and no correlation between the completion times of both scenarios and the SUS score provided by the participants (R2 < 0.0001).

All participants completed the SUS questionnaire resulting to an average SUS score of 88/100, “Best Imaginable” (Figure 4), as well as additional questions regarding their opinion on the effect of VPs in their disease management (Figure 5). Mean scores for the “Overall, how much did you enjoy the female/male scenario” were approximately the same for both scenarios.

Figure 5.

Self-management evaluation - mean scores are provided (1 = Completely disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Completely agree).

Finally, patients when asked about expressing their opinion about the app and the VP scenarios, most of them said that they liked both virtual patient scenarios and gave positive comments on the look and feel of the app, stating that there is nothing they would like to change. One participant said that the scenarios reflected a lot her reality, while another mentioned that “these scenarios look like as if I have written them”. Another participant highlighted the fact that “the app is easy to use even for people of older age”. However, there were some constructive comments with respect to the content and the presentation of the virtual patient. With respect to the presentation of the scenarios, one participant suggested that the scenarios were shorter than what she would have wanted them to be and that if someone wishes to get more analytical information then this should be provided via the “more information” section. Another participant commented that “I wish there were some more options to choose from. In some cases, the options provided did not fully covered my opinion on the matter”. Finally, three participants complained about not including more rheumatic conditions. However, it was explained to them that more scenarios are to be provided in the near future covering other cases as well.

Discussion

While the concept of patient education and management is not a new one, the use of Virtual Patients for chronic pain patient education has not been documented before in the available literature. Emphasis was put on the design of an easy to use mobile app following user-centered design methodology. The positive feedback regarding the ease of learning the application as a new user, with a total SUS score of 88, creates a solid ground for future upgrades and optimization to address a wider audience of users of all ages and common chronic conditions, especially when compared to other applications for chronic pain management, such as Leung et al. (Median SUS score = 73) ¹⁶ and Spyridonis et al. ³⁹ (similarly positive review on the amount of learning needed for their Virtual Reality application).

Overall, the patients agreed with most statements concerning the positive effect that the application may have on disease management. In addition, the general opinion of patients was that the required time for completing an educational case was little, thus considering the application as a potential everyday tool on the management of their disease.

The time required for the completion of a single scenario by the average trainee (5 min) is similar to the amount of time users of other applications are willing to spend, such as in the case of Leung et al., ¹⁶ where most users preferred to exercise alongside the application for not more than 10–30 min. The fact that the scenarios are available online at all times, enables users to access them whenever it is most convenient for them during the day, removing any time constraints arriving from potential scheduled sessions, such as other platforms like the one developed by Smarr et al. ¹⁵ Average time of 20.8 s spent on each node visited is a reasonable amount of time to navigate through the node and comprehend its content without becoming cumbersome for the user and completion times for the scenarios showed little to no correlation with demographic statistics (age, educational level and familiarity with mobile devices).

Of notice, is the fact that the development of a scenario requires minimum technical skills such as plain text editing, making it appeal to use by physicians who have expertise on chronic pain and possess a deep understanding of the patients’ needs. This is a notable advantage compared to the more engaging, yet complicated in development, virtual reality installments for patient education, like the one described in Reagan et al. ⁷ Furthermore, the ease of scenario creation enables the development of more scenarios, as we aim to cover more diseases and health problems that would be tailored to the individual difficulties that a specific disease may have caused to the patient using the VPs application.

On the other hand, our approach has certain limitations. The most notable limitation of the study was the small number of participants (20 participants), a number similar to the one during the pilot testing that Spyridonis et al. ³⁹ did (7 participants) and larger than the number of participants in the pilot study of the ProFibro app for fibromyalgia from Lee et al. ¹⁷ (10 participants). Another drawback was the gender representation, as the female to male ratio (all female). As this was a pilot study and the gained feedback was universally positive, there is a need for testing it in a wider scale and for a longer period of time, involving several groups of patients with chronic pain. The uneven gender ratio is common in studies dealing with autoimmune disorders, as the prevalence of rheumatic disorders, both in total as well as specific for most disorders of this type, is greatly inclined toward females regardless of nationality or race. ⁴⁰ This is due to multiple and complex aspects of the pathophysiology involved in autoimmunity. As we have already included a scenario with a male protagonist, testing the app with a wider audience would increase the number of male participants, thus addressing the more gender-specific needs. Additionally, the overall score for the evaluation of each scenario did not differ, thus pointing that the gender of the character in the scenario plays a relatively minor role.

Another limitation may be linked with the variety of diseases covered by the application. A specified condition (rheumatoid arthritis and ankylosing spondylitis) ailed each scenario character, in order to make the patient narrative more realistic instead of addressing the character as a generic patient experiencing chronic pain. However, a significant amount of effort was spent on addressing more generic patient issues that are nonspecific to a certain condition, but are, instead, universally considered debilitating, either physically or mentally, to the patients experiencing chronic pain. Those issues are considered to be caused by the disease itself or its complications and include physical pain, anxiety, depression and disability at the home or work environment. This may appear as a potential barrier that affects user adherence to the VPs application through decreased self-identification and projection on a protagonist with a diagnosis other than the trainee’s. Having said that, it has to be mentioned that the sets of experiments and trials of the app conducted by our team and presented in this paper did not seem to raise such an issue.

Applications like EPIONE enable users to take full advantage of the various available features offered by the application within their mobile devices, which means that they can have access to them from anywhere they are, for example, their workplace, outside areas or their homes. While the evaluation of our app was based on questionnaires completed by first-time patients on the beta version, our plan is to conduct future evaluations using the complete EPIONE app with a larger cohort of patients to address long-term effects on self-management, as well as, adherence to the application for everyday use.

Conclusion

Development of VPs for self-education and disease management is a new approach that may be used to tackle common problems, patients with chronic pain face in their everyday life, by giving them a more active role in decision-making that affects their physical and mental health.