Serious games for Parkinson’s Disease management as implemented in PROPHETIC platform

Abstract

In this paper, we describe the serious games, integrated into PROPHETIC which is an innovating personal healthcare service for a holistic remote management of Parkinson’s disease (PD) patients. The main objective of the three developed serious games is to allow health professionals to remotely monitor and appraise the overall physical status of their patients. The significant benefits for the patients, making use of this platform, is the improvement of their engagement, empowerment and, consequently, the provision of education about their condition and its management. The design of the serious games was based on the clinical needs derived from the literature and their primary target is to assess and record specific physical capabilities of the patient. All the games scores and the recorded parameters are gathered and also presented to the clinicians, offering them a precise overview of the patient’s motor status and the possibility to modify the therapeutic plan, if required.

Introduction

Parkinson’s disease ¹ (PD) is a movement disorder of neurodegenerative nature that affects the central nervous system. PD patients often suffer from movement—related symptoms such as tremor, freeze of gait, bradykinesia, rigidity, stiffness, and pain. At a later stage, after the aforementioned motor symptoms, several emotional, and behavioral symptoms may arise. The most common are sleep abnormalities, drooling, low mood, and even depression. ² Due to the wide variety and severity of the symptoms of this disease, it is very crucial to deploy technological capabilities as a valuable tool for assessment and patient-centered management. It is worth mentioning that technological tools should be used complementary to the classic therapies prescribed from health professional.

Serious games, as technological tool, are used to strengthen patients’ compliance to therapeutic plan in an entertaining and friendly way. Several investigations, based on serious gaming, concerning the special needs of PD patients are presented in this paper. The main objective of this research concerns the detailed presentation of the PROPHETIC ³ platform serious games. PROPHETIC concerns an “innovating Personal Healthcare Service for a holistic remote management and treatment of PD patients. One of the main objectives of PROPHETIC is the creation of a therapeutic proposal tailored to the end users of the platform, which is adapted to the needs and requirements of these users. The main purpose of PROPHETIC is to develop an infrastructure for remote, continuous, noninvasive acquisition and advanced processing of multi-parametric data, decision support for medical staff, and personalized friendly telecare provision based on serious gaming. PROPHETIC games differ gradually and are tailored to separate patient’s needs. They are deployed in order to help patients maintain a healthy status and improve their quality of life. In addition, they enable the doctor to keep track of the patient’s progress, through the game scores, and to modify the therapeutic treatment plan in cases where it is needed.

Related work

One of the most effective ways of helping PD patients maintain their quality of life is the employment of serious gaming. ⁴ Regarding neurorehabilitation, until now only virtual reality allows us to influence three key elements of the patient as repetition, feedback, and motivation. Repetition is important for the motor learning and for neuroplastic changes, which allow keeping in the long term what has been learned. In order for this to happen, repetition must be linked to sensorial feedback about the result of each one of the recurrences. In other ways, subject’s motivation is essential to carry out once and again the required activities for neurorehabilitation. That motivation is given by focussing the different activities that are part of the therapy like a videogame, so sessions are much more pleasant and attractive. ⁵

Therefore, several European Union (EU) funded research projects are based on serious gaming for patients’ rehabilitation.

CuPiD ⁶ is one of referred EU projects, and it concerns a “Closed-loop system for personalized and at-home rehabilitation of people with PD”. ⁷ CuPiD aims to provide technology-based and patient-—centered rehabilitation exercises for PD patients at home. In order to be effective, CuPiD offers remote monitoring and recording of patients’ activity. It also includes a computer interface that gives clinicians the ability to supervise patients’ progression during their exercises and modify them according to patient’s need. The most important CuPiD’s objective was to develop personalized rehabilitation programs and improve clinical guidelines with the help of existing technology. For that reason, in the project, inertial measurement sensors connect to an Android smartphone via Bluetooth standard protocols. A good Internet connection and connectivity to a game server are required for executing the training part.

REMPARK ⁸ is a “Personal Health Device for the Remote and Autonomous Management of PD” and is also an EU project. REMPARK aims to develop a Personal Health System that is characterized by detection, response, and treatment capabilities for better supervision and management of PD patients. The project’s main goal is to develop a wearable monitoring system so as patient motor status can be identified and evaluated in real time, and also develop a system to guide patient gait in everyday activities. ⁹

Another EU project is called “ACTIVA” ¹⁰ and targets to develop an “active therapy system based on physical exercise which allows muscular strengthening on PD patients. The main aim of the ACTIVA project is to provide the base for development of an interactive game of highly recreational and therapeutic value based on a combination of physical exercise and advanced visual and communication technologies in order to promote socialization and entertainment and to improve motor skills and muscular dexterity for people suffering from PD. The necessary technology used in the project ACTIVA consists of personal computers, webcam and Nintendo Wiimotes. Furthermore, interactive games are integrated in a PD information system which will allow therapists to manage users” rehabilitation.

Finally, Dias et al., ¹¹ present another interesting holistic approach for PD patients. This research team analyses “i-PROGNOSIS,” a unified platform consisting of various serious games in order to improve PD patients’ quality of life. “i-PROGNOSIS” is also an EU project and includes the “Personalized Game Suite” approach that combines different kind of serious games targeting a personalized suitable solution for PD patients. Each game category implemented, represents a different need and is part of the management plan of these patients. Specifically, the research team investigated serious games about patients gait mechanics, quality of their diet, emotional abnormalities due to their disease, and handwriting or speech mechanics. The first user acceptance evaluation showed a relatively high intention to use a PD early detection app by actual users of i-PROGNOSIS. In the case of i-PROGNOSIS, the characteristics of the respondents showed a rather good level of digital literacy regardless of the country considered. Variables such as age, gender, medical speciality, number of years in clinical practice and level of grade were particularly important for i-PROGNOSIS to better understand the requirements, preferences and concerns of the final users of the i-PROGNOSIS outputs and provide evidence to commercial and health service provider entities on the viability of the project innovations. ¹¹

Independent research teams developed game platforms of various settings targeting PD patients. Some of these efforts are presented here below:

Foletto et al. ¹² designed three worth mentioned game prototypes inspired by nature and farm life in order to trigger elders interest. The first game developed named “Pinchicken.” Three chicken nests appear on the screen of the game, with chickens and eggs that fall continuously on the ground. During this game, the user should pinch chicken eggs that show up on the screen and move them in the appropriate chicken nest, in order to succeed and earn points. The second game is called “Finger-Hero” and relies on the speed of user correspondence to stimuli. “Finger-Hero” screen has four lanes with flowers. Each flower has its own color (green, red, blue, yellow) and is associated with a thumb opposition gesture (index, middle, ring, and pinky). During the game a bee “flies” over the lanes, when the bee “stands” above a specific lane then the player should do the appropriate thumb opposition gesture so as to succeed. Finally, Foletto et al. developed “Grabduzeedo,” a game that is situated in a farm. The main purpose of this game is to prompt the user to control the movement of the virtual sheep by his hand, using the grab and release gestures. “Grabduzeedo” goal is to prompt the player to control the sheep pathway by closing his hand (grabbing the sheep), placing it to an appearing farm with fences and then opening his hand to release the sheep in the farm.

Additionally, Van der Meulen et al., ¹³ with the help of experts and end-users, created a simple concept game. The game is taking place in a candy factory where candy pieces end up in a basket via conveyor belts. The main goal of the game is that the players should “grab” the moving candies before they fall from the conveyor belt edge. The user controls the game by placing a virtual basket into the right position via a game controller. During the game’s first level, a tutorial familiarizes the player with the technology and some basic principles of the game. Moving to the next levels, the player is required to show speed and accuracy to his performance on the game.

Furthermore, Dauvergne et al. ¹⁴ developed a serious game called “Rhythm Workers.” The game executed via a tablet device and used in a program for home rehabilitation of PD patients, as a supplementary tool to existing rehabilitation programs for PD patients. The program lasted for 6 weeks and aimed to train the patient’s perceptual and sensorimotor rhythmic ability. “Rhythm Workers’ main purpose is to build block of flats whose aesthetic view and decoration depends on the user’s rhythmic correspondence to a finger—tapping task. During the game, the players are required to synchronize the tap of their finger (on the screen) to the rhythm of plenty of auditory stimuli produced by metronomes or music sequences in order to succeed and create aesthetically attractive buildings. The difficulty of the game differs gradually depending on the complication of music sequences.

Our proposed platform, which is analyzed in the following section, has a differential and a more holistic approach, focusing on the most common symptoms of PD in order to help the PD patients transcend the difficulties occurring in their everyday life.

Materials and methods

Game content design

The objectives of the PROPHETIC platform, are the integration, and development of an infrastructure and the execution of evaluation test phase, for the implementation of a comprehensive and holistic remote management and treatment plan for PD patients, by means of innovative technologies. Thereby, it relies on ICT and virtual reality gaming throughout the exploitation of haptic technologies and vision control and addresses older people living in the community. The main foreseen qualities of PROPHETIC are its affordability both in terms of cost and human factors, as well as its accuracy to assess indicators of rehabilitation/training strategy implementation and QoL (Quality of Life) improvement.

A key component of the PROPHETIC platform is the provision of serious gaming. These games are tailored to each patient’s needs and have a gradual level of difficulty. In order to meet the special needs of the patients, the medical experts studied and evaluated each case separately relying on both, known clinical guidelines and their own experience. They aimed to match the patients with games suitable for them and in the respective stage of difficulty. Serious games contribute to helping patients maintain a healthy status in an entertaining way. In addition, they enable the doctor to keep track of the patient’s ability and progress, through the game scores, and to modify the therapeutic treatment plan, as they deem appropriate. These games are also known as exergames as they combine exercise and video games (exercise + gaming) in order to improve patient compliance with the instructions of medical experts.

When the medical experts were asked which of the patients skills were considered to be particularly enhanced by PROPHETIC, all of them have highlighted those that are in some way related to physical skills, as they are the ones most affected by this type of disease, notably: bradykinesia, followed by muscular rigidity, balance disorders, and postural abnormalities, as well as, speech difficulties. Therefore, the experts coincide in stating that the type of exercises, which should be built into leisure proposals to ensure appropriate improvement in the most significant physical and motor skills, are related to balance, motor coordination, and rhythm, as well as, rate of reaction. All these skills are necessary to reach a level of independence that will enable a better QoL.

Medical experts advised PROPHETIC team on the design of the serious games based on the mitigation of the most common aforementioned PD symptoms. The combination of literature analysis and expert knowledge led to the development of the three presented games. Once the user’s requirements have been identified with the assistance of PD professionals, the PROPHETIC consortium has designed the game mechanics, which will be addressed to complement therapies implemented on patients. In this framework, a range of game typologies with a high therapeutic value based on new technologies, using advanced visualization, and interfaces interaction, was tailored in order to provide the PD patient with entertaining and efficient physical and speech therapy with the support of a TV and a basic equipment. This equipment includes a camera and a microphone. In particular, a 3D graphical avatar able to mimic human behavior will help the patient achieve the therapy exercises/games in an efficient way.

Thus, three serious games have been developed within the framework of the PROPHETIC project. Before the start of each game, the player—PD patient exemplified the exercise along with spoken and written instructions. All the following games have three levels of difficulty. Their description and technical characteristics are analysed in the next section.

The three developed games

GAME 1: “Shapes”—Physical game

The first game, is considered as a physical game which targets to help patient mitigate symptoms as bradykinesia, muscular rigidity, balance disorders, and postural abnormalities. In this game, the main objective is for the player to adopt the appropriate body posture, that fits with the approaching panel (wall with a hole with a certain shape), before it reaches them, and maintain it in a certain time (simulating passing through the panel). An avatar of the player is displayed in the middle of the screen to perform the exercises with them. The system will propose, in a random way, a position to be imitated by the player that he/she must maintain for 8 s in all levels of difficulty, so that the “silhouette wall” will have enough time to move on the screen and finally disappear. A game screenshot is presented in Figure 1. When the user has finished maintaining the position, the system based on the result that the player has obtained, will give to the patient a certain result feedback. The results that the system collects are the number of hits and errors, the speed of realization, the time of realization, the range of motion, and the heart rate. At the end of the game, a screen will appear that will show the correctness, the errors, and the time of accomplishment of the game. There are level levels of difficulty, easy, medium, and difficult, and each one will be composed of 10 specific postures that the system will present at random and in each game. More specifically, the first level of difficulty contains walls with simpler postures which move slowly toward the screen. As the levels of difficulty increase, the postures become more complicated and they are approaching move quickly, in order to test patient’s ability to react and make the proper poses.

Figure 1.

The “shapes” game.

GAME 2: “Piano”—Physical game

With regards to the second game, its aim is to trigger PD patients to deal with bradykinesia, muscular rigidity, balance disorders, and postural abnormalities, as it is also a physical game. During this game, the player should keep up with the music of the piano, by stepping over the virtual keys of the piano in front of their avatar as the musical notes that are approaching reach each one of the keys. The system shows you a giant piano that marks a certain rhythm with colored keys, as depicted in Figure 2. The keys light up and sound to the beat of the music. In the center of the screen will appear a giant piano that will be marking the rhythm through the illumination of some of its keys where the user has to go touching with the feet the keys illuminated to the rhythm. Upon completion of the game, the system, based on the scores that the player has obtained, will give a feedback of determined result. A screen appears that shows the correctness, the errors, and the time of accomplishment of the game. As the levels get gradually more difficult, the players have to go faster by touching the keys to keep up with the music. As previously, the first level of the game starts with a slow and easy-to-follow music tempo, that becomes quicker and compound as the levels of difficulty increase, demanding more immediate movements from the patient.

Figure 2.

The “piano” game.

GAME 3: “Syllable”—Speech game

The last game aims to aid patients front their speech difficulties and overcome them in an entertaining way. In this game, the player should pronounce correctly the different elements that will be appearing on the screen, syllable by syllable. The system shows a word or a phrase and indicates the syllable (with a more intense color and/or different color) that the player has to pronounce at each moment marking different degrees of speed. As it is presented in Figure 3, in the center of the screen will appear the different words or phrases that the user syllable to syllable should be reading at the pace that is marking the system. When the user has finished saying the phrase, the system based on the result that the player has obtained will give a certain result (score) as a feedback. This game has also three levels of difficulty, starting with small and commonly used words, and continues by increasing the complexity of the words and the time limit that the user has in order to pronounce it correctly. At the end of the game, the screen will appear that will show the correctness, the errors, and the time of accomplishment of the game. The system collects the results of the number hits and errors, the speed of realization and the time of realization.

Figure 3.

The “syllable” game.

In terms of evaluating the effectiveness of the PROPHETIC platform, participants were evaluated for their compliance with the proposed exercises and expert guidance. Their progress was evaluated based on the frequency with which they played the games and the scores they achieved. At the end of the whole process, they were asked if they felt that their QoL could be improved with the usage of the platform. All participants responded positively and thanked the experts for offering then the opportunity of experiencing PROPHETIC platform.

Results and discussion

The new technologies for healthcare interventions are recognized to have tremendous potential for all the stakeholders of the ecosystem. The main benefits for the direct stakeholders of the patient’s care, that is, the patient, the health providers and the informal carers, are analysed below.

The patient is the primary receiver as he constitutes the center of the technological health intervention. Thus, the utilization of the above-analysed serious games platform by the patient contributes to increasing their engagement. Patient engagement is a term used to refer to enhancing patients’ ability to fully involve in health and healthcare and to strengthening their influence on healthcare decisions. ¹⁵ This term is often linked to patient empowerment which is defined as “assisting them to discover and develop the inherent capacity to be responsible for their own health.” ¹⁶ The occupation of the patient with the serious games helps them maintain a healthy status in an entertaining way, record their progress in respect to their physical condition and, thus, be aware of their state. In this way, it bridges the gap between patient and provider (e.g. physiotherapist and neurologist) and revolutionizes the patient experience with care. It makes healthcare more accessible, timely, and convenient. Moreover, the game scores and rewards incentivise the patient’s adherence and retention of the prescribed technology intervention. Last but not least, the patient is reassured that their health provider is always up-to-date on their physical condition. This type of “connected health” allows patients to connect with their medical providers more quickly and conveniently than ever before. All the above have a direct impact on the way patients interact with and perceive their healthcare, increasing patient’s engagement and empowerment. These factors have been repeatedly associated with better health outcomes,^17–20. This means that the patients who are engaged in their healthcare decision-making process tend to be healthier and have better outcomes.

The stratification and total number of participants, during the evaluation test period, is outlined below:

• – Health professionals: The sample consisted of at least one representative from each of the specialties providing direct care to patients (e.g. neurologists, physiotherapists, and speech therapists). A total sample of 38 users was taken with different professional profiles (different users per country).

• – Patients: The sample of PD patients consisted of 10 users per country (30 users in total). The sample of patients with PD was stratified depending on age (60–65 years, 66–70 years, over 70 years), the level of deterioration (State 1, 2, or 3 according to the Hoehn and Yahr classification, ²¹ previous experience, or not in the use of New Technologies (Yes–No) and gender (male or female).

The involved health professionals (e.g. clinicians, caregivers) embrace this technological intervention in different ways. All the collected data from the serious games are constantly available to all the involved professionals and can be displayed through a GUI (Graphical User Interface), as presented in Figure 4. By using this, the health professional has authorized access to all of their patients’ records. These include diagrams about patient’s progress in the three different games, followed up with all the system’s collected data (e.g. number of hits/errors, the speed/time of patient’s realization). The results are presented by reverse chronological order, that is, most recent first. The health professional can go back to any measurements of previous dates, if needed. This enables them to keep track of the patient’s ability and progress and to modify the therapeutic treatment plan, if required. A significant benefit of the remote patient monitoring is the increase of the capacity of health professionals to treat more patients, as they can spend less time with each patient and handle them more efficiently. Furthermore, by means of this platform, health providers can connect with one another to share consults, expertise, and knowledge during patient care. As it was afore-mentioned the patient is aware of their own results, helping providers to take a collaborative approach to patient interaction that emphasizes shared decision-making, with benefits for both sides.

Figure 4.

A screenshot of health professionals’ GUI.

Subsequently, it is worth mentioning the beneficial outcomes of the technological interventions, such as serious gaming, for the informal carers. These usually include husband or wife, other family members, friends, and private duty carers. Family carers of people who need continuous looking after often experience physical and mental health morbidities, and burden. ²² The use of the serious games by the patient can give comfort to carers in the terms of update and information of the health professionals about their patients’ state. Also, they can have a clearer overview of the patient state and watch their progress. For some cases, the detection of any progress’ alterations could be easier to be noticed by the caregiver than by the patient, assisting health professional’s work. What is more, the provision of serious gaming could save valuable time and money from the carers that they will not have to transport their patients for their health sessions (e.g. physiotherapies, speech therapies).

The findings of the evaluation test have to be seen in light of some limitations. The test reviewed the normal operation of the integrated platform and its user acceptance at an initial stage. The results were very positive; however, a randomized controlled pilot study should be conducted to fully evaluate the integrated platform for the assisted living of PD patients and its efficiency on patients’ holistic management. The two PD patient groups (study and control group) should be systematically assessed at baseline and after some certain periods.

Last but not least, during PROPHETIC project, we have taken into account all the existing rules for the protection of the participants personal data. All experiments during the evaluation phase were conducted in compliance with EU ethical standards for experiments with human beings. More specifically, all actions have been pursued in accordance with the principles of the General Data Protection Regulation (GDPR). The GDPR is the new EU’s regulation replacing previous EU Data Protection Directive (DPD). GDPR designed so as to protect and warrant every subject’s data privacy and support clarity and liability in personal data usage. ²³ As the GDPR applies to every organization that processes personal data and is located in the EU, it is necessary to incorporate its principles throughout the design and implementation of PROPHETIC serious games. Aforementioned principles introduce new persons rights and procedures of importance regarding several activities related to serious games usage, such as data collection, storage, processing, and transfer. ²⁴ Subsequently, one of the main PROPHETIC’s concern is the processing of personal, biometric, and health data respecting the participants’ rights and freedoms.

Conclusions

To sum up, there is no treatment and therapeutic schema for PD although the great number of studies, which are being carried out, holds great hope for the future. As a result, until a treatment that will cure the disease is discovered, the symptoms can be suitably controlled with medicines prescribed and monitored by a neurologist. A combination of different drugs is commonly needed to achieve better control of the symptoms depending on each patient’s situation. For this reason, the neurologist tries to retrieve information for patient’s motor status for the previous time of the examination. This is usually difficult to be thoroughly evaluated, as it is difficult for the PD patients to describe their symptoms and they cannot assess exactly their reaction to the drug. Therefore, the clinician cannot receive proper information to define realistically the drug administration treatment. A remote, continuous monitoring system offers daily assistance to the clinician neurologist who tries through conflicting information, from the PD patients and their relatives, to determine the optimal treatment plan. The serious games described above constitute a module of an integrated system, called “PROPHETIC,” which is used by the PD patients in a simple, safe, painless and non-invasive way to record patient motor status for long-lasting terms. The serious games evaluate the patient’s physical condition through assessing limb control and mobility, balance, and reaction ability as well as enunciation and pronunciation potential. In this way, the clinician can have an accurate, long-term and objective view of patient’s motor status compared with drug and daily activities. With the PROPHETIC system the clinician can remotely receive exact information for the PD patient’s motor status on past days and define the optimal treatment plan.