Sage Journals: Discover world-class research

Abstract

Objectives

The core objectives of this study centre on enhancing the quality of life and well-being of individuals diagnosed with Parkinson’s and Alzheimer’s diseases. Our aim is to facilitate the monitoring of patient information, benefiting both caregivers and healthcare professionals.

Methods

As part of the PROCare4Life platform sensorial ecosystem, a web application with six engaging cognitive games focusing on developing cognitive training and stimulating brain activity are developed. A set of metrics calculated by the application feed machine learning predictive models to evaluate the cognitive status and evolution over time. Long-term analysis of the daily cognitive ability information is used to generate high-level outcomes and identify deviations for each patient from the multimodal fusion engine. And based on these results, a recommender system provides a set of personalized notifications.

Results

A 3-month pilot study that took place in five different countries shows the results obtained from 93 patients. An average of 22.4 games were completed per day and the recommender system generated a total of 260 game notifications, 37.7% of them were marked as read by the patients. The Cognitive State Score and the Deviations in Cognitive Abilities measurement, calculated by the multimodal fusion engine, when used in conjunction present a good overview of the patient’s current state and potential deviations.

Conclusion

The cognitive games application was well-received by elderly individuals who took part in the study. This tool can be valuable for caregivers and healthcare providers in assessing the cognitive function of patients through engaging in cognitive games.

Keywords

Alzheimer cognitive games cognitive skills dementia elderly healthcare quality of life Parkinson personalized notification

Introduction

Healthcare and social systems are key areas of focus for investment and research. Proof of this is the trend of spending on healthcare by the different countries in Europe.¹ The main motivation is due to the aging of the population,^2,3 thanks to the improvements in health services.⁴ However, an ageing population brings with it an increase in a set of diseases such as cardiovascular, diabetes, mental or physical and neurodegenerative disorders.^5,6

Dementias, such as Alzheimer’s disease (AD),⁷ and Parkinson’s disease (PD)⁸ are the most common chronic pathologies and the main causes of disability and dependence in older people worldwide. Both diseases profoundly affect the quality of life of people who suffer from them, and even their families and caregivers.

PD is a neurodegenerative disorder⁹ associated not only with motor symptoms such as slowness, movement, tremor, rigidity, gait disturbance or imbalance, but also associated with a wide range of non-motor complications such as cognitive impairment, mental and sleep disorders, pain, or other sensory disturbances that make difficult to carry out daily activities. PD mostly occurs in older people, and it is one of the fastest-increasing neurological disorders in terms of affected people with an estimation of 8.5 million people affected by 2019.¹⁰

On the other hand, dementia may be a progressive condition,¹¹ and is characterized by a deterioration of cognitive function and loss of the ability to process thought (memory, intellect, behaviour, and ability to perform daily activities). Dementia is often accompanied by impaired emotional control, social behaviour, and motivation. Although the forms of dementia are many and diverse, AD is the most common form, accounting for 60%–70% of dementia cases. Other common forms of dementia include vascular dementia and Lewy body dementia.¹² Annual cases, with numbers around 5% and 8% of the general population over 60 years of age suffering from dementia at some point in their lives, represent a projected reach of 82 million by 2030 and 152 million by 2050.¹³

The focus of this work is the Personalized Integrated Care Promoting Quality of Life for Older People (PROCare4Life) project.¹⁴ PROCare4Life aims at developing and testing a technology-based, integrated, scalable and interactive care platform, aimed at neurodegenerative patients such as Parkinson’s and Alzheimer’s, caregivers, and healthcare professionals. The study protocols for investigating user needs under PROCare4Life were made in Ahmed et al.¹⁵ and the results were presented in Ahmed et al.¹⁶

The main objective of the project platform is to improve the patient’s quality of life by providing relevant information such as the patient’s state of health, or the generation of personalized recommendations that can help improve their well-being. The platform has different types of interfaces: A web application for healthcare professionals and caregivers to manage and monitor patient information, a mobile application for patients to access and manage their data on the project platform, and a web application with six cognitive games.

The cognitive games application is used in the PROCare4Life project to assess the cognitive ability of patients and its evolution over time. Through the information collected during the gaming sessions, such as the user interaction with the application, click tracking, the level of difficulty of the games, the time taken to complete a game and the score obtained, a set of cognitive characteristics are obtained for each patient in terms of concentration, mathematical calculation, object and vocabulary recognition, short-term memory, and visual attention.

The application makes use of the recommendation system (RS) implemented in the project. Different types of recommendations are provided to patients with the purpose of improving the experience with the use of the application. Literature about RSs presents interactions on how to improve the general well-being of elderly people. Initial research was proposed in Azmi et al.¹⁷ using Collaborative Filtering, which group users with similar conditions to recommend interventions, previously used on similar users. In Hammer et al.,¹⁸ the work describes the development of a display with which users interacted, and from which they obtained feedback after answering a series of quizzes. Interesting work in, describing. More recent works extract additional behavioural information to propose RS. These works^19,20 were possible thanks to a data-driven mechanism to propose actions based on user preferences changes.

Related work

PD and AD affect each person differently, depending on the impact of the disease and the personality.²¹ In dementia disorder, there is no treatment that can cure the disease or reverse its progression. However, there are medications that can temporarily help and alleviate symptoms for certain periods of time.²²

Moreover, there are also numerous interventions that can be offered to support and improve the quality of life of people with dementia, their caregivers and family members. Initiatives such as the World Health Organization (WHO) guidelines for reducing the risk of cognitive decline,²³ aim to promote brain health and reduce the risk of Alzheimer’s and other dementia through brain stimulation and cognitive training. Dementia is preceded by cognitive decline²⁴ and some studies have shown that increasing cognitive activity can stimulate (or increase) the cognitive reserve and have a damping effect against a cognitive decline.²⁵ Music, art, occupational therapy, physical exercises, or brain games are leisure activities that allow patients with this type of disease to regain a good level of self-confidence and, above all, to maintain their autonomy and slow down the loss of abilities due to dementia.²⁶

Previous projects, such as the European project ICT4Life (ICT services for Life Improvement For the Elderly),²⁷ reveal that patients with this type of neurodegenerative disease identify useful and positive the technology to increase their freedom and independence. According to a study by the ICT4Life project,²⁸ the use of cognitive games was an effective methodology for cognitive training and brain activity stimulation. Accepted by patients, caregivers and healthcare professionals, the results revealed the use of these types of games as an opportunity to improve patients’ well-being, entertain and train cognitive skills.

In recent years, so-called serious games have emerged due to the aging of the population. Applied in the health and scientific sectors and even in training, they are educational games designed with a main objective other than pure fun or entertainment.²⁹

Serious games bring benefits and have a great potential to improve people’s motivation,³⁰ stimulate cognitive abilities and improve people’s daily lives.³¹ Some analyses^32–35 were made about the impact of video games on children and students for improving cognitive skills and learning. In elderly people, other works^36–38 also present performance improvements by using games to train cognitive abilities such as memory, attention or spatial processing,³⁹ and for Alzheimer’s patients for brain training and stimulation of cognitive abilities.⁴⁰

Carleton Word Search Game and Carleton Sudoku Game⁴¹ allow to track and measure the cognitive change in older people, Smart Thinker game⁴² allows older people to exercise the brain and achieve maximum cognitive performance, Ikiiki-pom game⁴³ and Seeker game⁴⁴ allow to train cognitive skills in a fun way. Protect Brain Training Study and Big Brain Academy games⁴⁵ were used in clinical trials to train and test reasoning, planning, concentration, identification, mathematics and memory.

Materials and methods

PROCare4Life setup

The initial concept of the PROCare4Life architecture is comprised of four elements which are observed in Figure 1: A low-level structure, collecting sensor and raw data from the patients; a high-level structure, processing and storing the information with AI services; interfaces for the users; and finally the backend components providing support functionalities to the platform.

Figure 1.

Personalized Integrated Care Promoting Quality of Life for Older People (PROCare4Life) architecture.

The low-level structure is built around the sensorial ecosystem, which is responsible for the data acquisition, pre-processing and management from the different sensors and devices of the platform. For the PROCare4Life platform implementation, the sensors and devices considered included a wristband, a smartphone (IMU sensors and GPS), binary sensors, a camera for skeleton tracking, and a tablet for the web access of the platform and the cognitive games.

As part of the PROCare4Life platform sensorial ecosystem, a set of cognitive games have been developed to collect data and measure patients’ cognitive skills. The cognitive games application is considered in the project as an essential system because it is the main tool used by patients with PD or AD to determine their cognitive status, stimulate brain activity and develop cognitive training.

The general flow can be observed in Figure 2. Using the information collected by the application during a game session, a set of data, metrics and statistics are obtained to determine the patient’s cognitive status and to evaluate its evolution over time in terms of cognitive skills. This exhaustive information such as user interaction including execution times, button click tracking, navigation into the application, game characteristics, times, scores and other relevant data are stored in the project’s databases, where each patient has their corresponding gaming sessions.

Figure 2.

Services interaction architecture.

The information stored in the databases is used to feed the machine learning predictive models implemented in the multimodal fusion engine. In this engine, the cognitive games results are used as daily cognitive ability information for the patients. A long-term analysis needs to be performed on this daily information to generate high-level outcomes from the game results. Furthermore, the long-term analysis results also indicate the deviation or the improvement from a baseline for each patient.

Based on the two sources of information, the data collected by the application and the data generated by the multimodal fusion engine, the RS generates a set of personalized recommendations for each patient, with the aim of providing them a better user experience with the application. The RS has all the necessary information to know the patient’s use of the application and the results obtained. Situations such as low usage of the application, always playing the same games, choosing inappropriate levels of difficulty or bad use and interaction with the application are detected by the RS. In this way, to help the patient the next time the patient uses the application, the RS generates and sends a set of notifications that best suits each patient. These recommendations are displayed, as notifications, directly in the cognitive games application.

Cognitive games

The cognitive games application is a web application developed in HTML (HyperText Markup Language), Cascading Style Sheets (CSSs) and JavaScript. To keep the web application on a single web page, AngularJS is used as the open-source JavaScript framework. Also, Bootstrap version 3.0 is used for the design of the application. These programming languages and versions are compatible with web browsers pre-installed on Smart TVs. The application can be used on Smart TVs regardless of the model and brand, or on tablets, smartphones or laptops.

The application has six different entertaining games that focus on stimulating visual attention, concentration, short-term memory, understanding, semantic memory, vocabulary, mathematical calculations, and object recognition. The six games, together with the different levels of difficulty and the different content categories, make the application accessible to all patients who participated in the PROCare4Life project, regardless of the severity of the disease, if they met the eligibility criteria discussed at the end of this section. Table 1 summarizes the objective and medical justification for each of them.

Table 1.

Cognitive games and measured cognitive abilities.

Game ID	Game name	Cognitive abilities	Description
Game 1	Concentration	Visual attention and short-term memory	Pair of cards
Game 2	Guess what?	Object recognition	Blurred image
Game 3	Guess the word	Vocabulary recognition	Hidden word
Game 4	Trivia	Concentration and mathematical calculations	General questions
Game 5	Bingo	Visual attention and object recognition	Objects and shapes
Game 6	Memory challenge	Short-term memory, visual attention and object recognition	Image memorisation

Game 1 – Concentration

The objective of this game is to find matching pairs of cards, from a pack of cards laid with the face hidden. For this game, there are four categories available: Colours, faces, fruits and sports, and three levels of difficulty, increasing the number of hidden pairs to find. In this way, in level 1 there are 8 cards or 4 pairs to find, in level 2 there are 12 cards (6 pairs), and in level 3 there are 16 cards (8 pairs).

During the game, the user selects two cards, which will be flipped over, showing their value. If the two cards match, a point will be counted, and the cards will remain face up. If the cards do not match, they will be turned face down again. The end-user will continue selecting pairs of cards that could be a match. The game ends when the patient finds all the matching pairs of cards.

In total, there are four different categories (colours, faces, fruits and sports). For each category, there are three levels of difficulty. Figure 3 represents the fruits category.

Figure 3.

Game 1. Example using the fruits category.

Game 2 – Guess what?

The objective of this game is to guess the hidden object as quickly as possible, while the image is getting progressively less blurred. For this game, there are 60 different sets of images of different topics (animals, places, technological devices, musical instruments, vehicles, nature, etc.), and in this game, there are no different levels of difficulty.

During the game, the patient is given a blurred image of the object to guess, four different possible answers to what the object is, and additional simple clues related to the object. The image becomes progressively less blurred, and therefore easier to see and guess, until it becomes completely sharp and clear. The patient must click the correct answer out of the four possibilities. There are 10 images per object, or in other words, 10 different levels of blurring (level 7 of blurring in Figure 4). The image is updated every 5 seconds.

Figure 4.

Game 2. Example using level of blurring 7.

A new object is reset if the patient clicks an answer (correct or not), or the patient has not provided an answer and the image was displayed completely sharp, after the 10 levels of blurring. The game ends after five ‘Guess what?’ objects.

Game 3 – Guess the word

The objective of this game is to guess the hidden word by suggesting letters, one by one, within the 10 attempts or lives. For this game, there are more than 50 different words available on different topics (objects, days of the week, colours, numbers, months of the year, adjectives, verbs, sports, foods, countries, professions, …), and there are not different levels of difficulty.

During the game, the patient can see how many letters are in the word and is given an additional simple clue related to the hidden word. The patient then proceeds to click on a letter that could be in the word. If the suggested letter occurs in the word, it is shown in all its correct positions. If the suggested letter does not occur in the word, the patient will lose one attempt or life. The patient then continues to click on possible letters included in the hidden word. The word is reset if the patient guesses the word or consumes all 10 lives. There are five hidden words to guess in each game. Figure 5 shows an example of this game. In this case, the hidden word was ‘POTATO’.

Figure 5.

Game 3. Example with Hidden Word game.

Game 4 – Trivia

The objective of this game is to get as many correct answers as possible from 10 questions about general knowledge, vocabulary, and mathematical operations. For this game, there are 60 different questions available on different topics, and there are no different levels of difficulty.

During the game, the patient is asked a question with four possible answers. The patient selects the answer that is considered correct and then the game proceeds to the next question. The game ends after 10 questions. Figure 6 shows an example of a question for the trivia game.

Figure 6.

Game 4. Example based on questions.

Game 5 – Bingo

The objective of this game is to match the items in the user’s box with the items being displayed progressively on the screen. There are four categories available: numbers, foods, animals, and flowers; and three levels of difficulty, which increase the number of items to be selected by the user. In level 1 there are 2 cards in the box, in level 2 there are 4 cards in the box and in level 3 there are 6 cards in the user’s box to match.

During the game, new items will be displayed on the screen every 5 seconds. In total, there are 30 different objects. Whenever one of these new items matches with the ones on the user’s box, the patient must mark it. The game ends when the user has marked correctly all the items in the user’s box or when all the available items (30 items) have been displayed.

This game allows to play four different categories (numbers, foods, animals in Figure 7, and flowers). Each of them has three levels of difficulty. The number of items to match increases with the level of difficulty.

Figure 7.

Game 5. Example using animals category.

Game 6 – Memory challenge

The objective of this game is to memorize a set of cards at each time interval and be able to correctly identify them in a larger group of random cards. There are three content categories available: Traffic signals, flags, and emergency signals; and three levels of difficulty, increasing the number of cards that the patient has to memorize. Both the number of cards to memorize and the group from which to check increase with the level of difficulty. In level 1 there is one card to memorize, to be selected from two possibilities, up to level 3 in which there are three cards to memorize and to be selected from a group of five.

For each content category, there are 30 different items. During the game, cards are displayed for 5 seconds and the patient must try to memorize them. When the time is up, those cards are flipped face down. Then, a larger set of cards is displayed and the user must mark those that are considered to have been shown initially. A new set of cards to memorize is reset when the user completes an attempt to click on a possible set of cards in the round (correct or not). The game ends after five rounds.

In this game, there are three different categories available to play (traffic signals, flags in Figure 8, and emergency signals). The number of items to memorize increases with the level of difficulty.

Figure 8.

Game 6. Example using flags category.

Data collection

The application is responsible for collecting all the relevant information that is interesting to determine the cognitive status of the patients and measure certain aspects related to user interaction and user navigation into the application. Every time a patient completes a game, the information collected is processed, sent to the project platform, and stored in databases. The data collection must be associated with a specific patient registered in the project platform, so authentication by the user in the application is mandatory before starting to play games.

The format used for sending the game summarization is the JavaScript Object Notation (JSON) format. To simplify the data collection, the game summarization JSON has the same structure in all six games and the type of information is always the same: Application information, game information and session information. Application information contains data related to the customization of the application such as the language, the style and the text size. Game information contains data related to the game completed by the patient such as the game identifier, the content category, the level of difficulty, the score, the time taken to complete the game or the interaction during the game. Session information contains data about the whole game session, understanding by game session the set of games played by the same patient during a continuous time interval. Session information is a summary of the most important information of all games completed in the session, as well as general information about the user’s interaction during the entire session.

User interaction data provides very useful information to understand how the user experience is, if the user knows how to use the application, knows how to play the games, or has difficulties understanding it. To measure this data, a set of identifiers have been defined for each object, element and button that are part of the application. In this way, each click event is associated with a specific action at a specific time, thanks to the implementation of different clocks in the application.

Among the large numbers of click-type identifiers contemplated in the application, there are two types that are very interesting to consider in order to know if the patient has problems when playing. The first type is the ‘undefined’ identifier. An ‘undefined’ click type indicates that the click has occurred in an area of the screen where there is no element, image or button, and where a priori the patient should not click when playing the games or interacting with the application. The second type is the ‘disabled’ identifier. A ‘disabled’ click type indicates that the click has occurred in an element, image or button that are part of the games, but which is not contemplated to be clicked to continue the progress of the game (for example, it may be the case of question boxes, images to decorate the games or elements that have been clicked before). During the game session, many clicks with these two types of identifiers could indicate a strange behaviour of the patient when playing or sensorimotor difficulties interacting with the application. In the same way, taking a long time to interact with the application or starting a game and not finishing it, can be relevant information to take into account in the analysis.

Data processing

The game summarization includes processed information, based on the data collection, which allows defining three different metrics to evaluate how well a patient has completed the game in terms of scores (matches, correct answers, attempts needed to complete the game…), times (time needed for the patient to finish the game) and interactions with the applications (clicks needed to complete the game, correct clicks, wrong clicks…). The metrics are normalized between 0 and 1, where 1 would be the maximum score. The numerical value of each of the three metrics is determined based on a comparison between the results obtained in the game and a set of pre-established thresholds. These thresholds, such as the approximate time, the number of attempts needed or the number of interactions the patient needs to complete a game correctly, were subjectively defined based on the feedback from caregivers who observed a controlled group of patients. PROCare4Life project ensured that at least two caregivers and five patients from the countries participating in the pilot study were involved in the definition of the thresholds, testing the different games and difficulty levels during the development and implementation phase of the application. This sample of selected patients would later participate in the study, according to the eligibility criteria discussed at the end of this section. Depending on whether the results are better or worse than the pre-established, and on the difference between the values, these numerical values of the metrics are obtained.

Combining these three metrics a global metric is obtained, a metric to determine how well a patient has completed a game considering all possible aspects. The metric is also normalized between 0 and 1, and different weights have been used to define it: A weight of 0.5 for the metric of the score, a weight of 0.25 for the metric of the time, and a weight of 0.25 for the metric of the interaction. The selection of these weights gives more importance to the score obtained in the game, and less importance to whether the patient spends more time, or additional clicks to complete the game. Patients knew the importance of taking the time to complete the game without having to do it as fast as possible. This fact considerably reduced the pressure that patients could feel during the game sessions.

Apart from the metrics mentioned above, data related to user interaction is also processed. The values of maximum time between clicks, the minimum time between clicks and the average time between clicks, are values that would indicate how fast or how slow the user interacts with the application. Interacting too fast or too slow with the application may be a symptom of incorrect use of the application and a lack of knowledge of how to play a game.

Multimodal fusion engine

The multimodal fusion engine aims to generate high-level representations of a plethora of sensorial data collected from the patients and analyse long-term changes in each patient’s condition. The engine utilizes the procedure described in Ghaleb et al.⁴⁶ to generate a single high-level value (CSS) which represents the cognitive abilities of a patient using data collected from the cognitive games. These abilities are concentration, math calculation, object recognition, short-term memory, visual attention, and vocabulary recognition. The engine then performs the deviation analysis to obtain the probability of observing deviations in cognitive abilities using other modalities that affect a patient’s cognitive state such as physical activities assessed using daily activity measurements (energy expenditure, inactivity, walking patterns, indoor mobility, heart rate) obtained from the wristband and binary sensors, motor functions assessed through the frequency of daily disease-related symptoms (bradykinesia, freezing, wandering, etc.) obtained from wristband and smartphone sensors, sleeping patterns using wristband sensors and medication intake with daily self-reports of the patients. The integration of these other modalities and the concept of long-term analysis follows the procedures described in Ghaleb et al.⁴⁶ Figure 9 presents the overall concept of the engine and illustrates the relationship between each game and the cognitive ability being measured.

Figure 9.

Multimodal fusion engine and its relation to the cognitive games.

The engine calculates the CSS weekly using the global metric value obtained from each game. For each cognitive ability, a performance value is calculated using all global metric values from the corresponding games to represent the state of a patient with six values each day. The values for the cognitive abilities that are represented in multiple games are calculated as the mean score of the related games. The daily game performances of the week CSS is being calculated for and the previous week’s values are concatenated to form a vector with 14 values. Then, for each ability, the long-term analysis score calculation procedure described in Ghaleb et al.,⁴⁶ which is the second-degree polynomial regression, is applied to obtain an interpolation value on the 14th day and the result is normalized with min-max scaling to obtain values between $0$ and $1$ , where $0$ represents the worst condition and $1$ represents the best condition. Finally, CSS is obtained with a weighted sum of all six values according to Equation (1) where $C$ represents a patient’s current concentration ability, $M C$ is the mathematical calculations ability, $O R$ is the object recognition ability, $S M$ is the short-term memory, $V A$ is visual attention, $V R$ is vocabulary recognition, $W_{a b i l i t y}$ is the weight for the corresponding cognitive ability, which are obtained from the expert opinion of clinicians specialized in PD and AD diseases in six different clinical organizations by taking the normalized mean of their responses to the impact of each ability on the representation of an overall cognitive state assessment in a 5-point Likert scale survey. The weight of an ability is calculated as the ratio between the sum of survey values for the ability and the sum of all survey values.

\begin{aligned} C S S & = C * W_{C} + M C * W_{M C} + O R * W_{O R} + \\ + S M * W_{S M} + V A * W_{V A} + V R * W_{V R} \end{aligned}

(1)

The obtained CSS is incorporated into a fusion system that detects probabilities of observing deviations from a patient’s regular routines. The fusion system aims to detect anomalies and deviations from everyday routines in physical activities, adherence to medication, motor functions, sleeping routines and cognitive abilities and benefits from a computational framework that utilizes the domain knowledge of clinical experts to support health and care professionals to better interpret a patient’s conditions.

The proposed approach utilizes Bayesian Networks (BNs)⁴⁷ which are probabilistic models that represent a set of variables and their conditional dependencies in a directed graph, to take advantage of their strong computational power that can combine deterministic and data-driven approaches. The structure of the BN was determined through extensive interviews and needs analysis with clinical experts on the relation between each modality and different categories of potential decline to be monitored by health and care professionals. The conditional dependencies are integrated through a linear approach for the discrete BN designed, where clinical experts can integrate their domain knowledge to personalize the impact of each modality (cognitive states, physical activities, motor functions, medication intake and sleeping patterns) on the deviations in cognitive abilities based on their knowledge of each patient. This is achieved through dedicated interfaces to indicate the impact of each high-level representation (modalities or scores) on the categories, where the decline in cognitive abilities (DCA) is one of these categories. The clinical experts can assign an impact value for each modality on DCA from $0$ to $1$ in $0.2$ intervals, where higher values represent higher contribution to the deviation of the cognitive ability. Then, these values are normalized to present a sum of $1$ over each impact value. Each modality is considered to have two states: High, where a high value is observed in the patient and Low, where a low value is observed. Finally, for each condition, the conditional probability is calculated using Equation (2), where $N$ is the number of modalities, $W_{i}$ is the normalized impact value of a modality, $M S$ is the states of the input modalities, and $S$ is the state, which can be either $0$ for Low and $1$ for High states. For example, P(Deviation=High $‖$ L,L,L,L,L) presents a high probability of observing deviations given that all modalities are observed to be Low.

P (D e v i a t i o n = S ‖ M S) = \sum_{i = 1}^{N} W_{i} \times S

(2)

In order to address the probabilistic nature of the BNs, the main set of probabilities needed is the probability of a patient obtaining low values from each modality based on a reference measurement. The proposed approach utilizes the score calculation procedure described in Ghaleb et al.⁴⁶ to obtain both a representation for the 14th day, which is the current week’s representation and on the seventh day to obtain the baseline reference value. The frequency of obtaining values in the measurement week (the second 7-day period) that are lower than the reference value (the score for the first 7-day period) is used to obtain the probability of observing low scores,

P (M o d a l i t y = L o w)

, over the week.⁴⁷ The probability of observing high values

P (M o d a l i t y = H i g h)

are calculated as

1 - P (M o d a l i t y = L o w)

The conditional probabilities and the probabilities obtained from the patients then utilized in the network with clinical experts’ impact assessments to generate the potential deviation probability for observing DCA, where higher values present a higher probability of a decline or a deviation and lower values are desired for a patient’s progression on the positive direction. This probability is computed as follows in Equation (3), where MS is the all potential combinations of the internal states, CA is cognitive abilities, PA is physical activities, MI is medication intake, MF is motor functions and SP is sleeping patterns:

\begin{aligned} P (D C A = H) & = \sum P (D C A = H ‖ M S) \\ = P (D C A = H ‖ L, L, L, L, L) \times P (C A = L) \\ \times P (P A = L) \times P (M I = L) \times \\ P (M F = L) \times P (S P = L) + \dots + \\ P (D C A = H ‖ H, H, H, H, H) \times P (C A = H) \\ \times P (P A = H) \times P (M I = H) \times \\ P (M F = H) \times P (S P = H) \end{aligned}

(3)

Cognitive skills recommendations

Cognitive skills recommendations (CSR) are a dynamic experience overall, with immediate recommendations being possible with the most recent games results and collected information, while longer-term actions are possible through the combination of information output from the multimodal fusion engine as well. This combination of information results in the production of patient-centred notifications.

The CSR on the other hand, has the advantage of being an area whose development can be considered to be able to be carried out in relative isolation from the rest of the components of the platform, as it uses a unique interface to interact with the patients and collect information. However, it also allows for a continuous increase in complexity as it advances from considering the game-processed data to being a part of essentially every other modality change that could be considered in the platform, as well as it being potentially impacted by other modalities as well.

The RS collects data from a whole week, analyzing these data to provide some feedback to patients about how better they could improve their skills using this section. The set of CSR contains notifications encouraging them to play more frequently, or even offering new motivations to test other games. Figures 10 and 11 represent the two possible receptors of notifications. These notifications are received for patients (in their respective language) in the PROCare4Life mobile application for multimodal data or in the same web application where they play for cognitive games data, appearing as a pop-up in the bottom right corner. The list of notifications is summarized in Table 2.

Figure 10.

Recommendation system (RS) notifications on mobile application.

Figure 11.

Recommendation system (RS) notifications on web application.

Table 2.

Recommender system notifications based on measured cognitive statistics.

Metrics	Priority	Actions
Cognitive Games	High	Use the cognitive game app more frequently./Start a different game.
Cognitive Games	Medium	Change game categories for games 3,4./Increase the level for game 3.
Cognitive Games	Low	You are doing great! Keep it up./Keep trying to improve your results.
Multimodal Fusion	General	Did you find the game difficult for you? You can adjust the difficulty level.
Multimodal Fusion	Weekly deviation	There is a 0.305 probability that decline in cognitive abilities will be observed.

Notifications are sent on a weekly basis. That is, patients will receive after each 7-day cycle, two notifications depending on their metrics and scores. These notifications are sampled from the whole list of CSR. By deciding the notifications to be sent by using game metrics, three levels of importance were defined, ordered by priority:

Emphasize aspects such as remembering to continue playing or starting other games.

Modify aspects of the games such as difficulty. At all times, the patient should select the category and difficulty level before starting a new game. This type of recommendation is useful to encourage the patient to try other configurations based on previous game sessions.

Motivational messages depending on the scores.

By contrast, notifications based on multimodal data fusion were selected based on the scale of the CSS score returned.

Events and notifications are handled by the RabbitMQ⁴⁸ configuration microservice. It is designed to be easily accessed by the RS, by exposing the REST HTTP RabbitMQ endpoints and can be called synchronously, to generate a notification or an event. The RabbitMQ config integrates the RabbitMQ broker server, which creates a queue and pushes notifications/events to the queue, using the notification unique identifier to access the appropriate UI client.

Study overview

Pilots and modules evolution

Evolution over time of the PROCare4Life modules has been guided by a phased approach, marked by three pilot waves aimed at codesigning and continually refining the system based on user feedback and preferences. These iterative waves placed a strong emphasis on user-centricity, with participants being under constant monitoring to ensure their positive perception of safety, comfort, and the system’s usefulness in enhancing their quality of life. Pilot 1 primarily assessed technical feasibility and usability, while Pilot 2 focused on characterizing and validating system metrics. Finally, Pilot 3 encompassed usability testing and evaluated the clinical impact of the PROCare4Life system in its final form.

Eligibility criteria

The pilot study was performed according to the inclusion and exclusion criteria, defined under the PROCare4Life guidelines. Inclusion criteria was defined as the following points:

People diagnosed with AD or PD.

65 years of age or older.

Willingness to enrol in the project.

Able to provide informed consent.

Living at home, rehabilitation centre, or day care centre.

Exclusion criteria filtered patients if they were ineligible for inclusion in the study, if they exhibited difficulties in learning or had limitations that hindered their capacity to utilize the PROCare4Life system or its essential technical components. Such limitations encompassed challenges related to speech, vision, language, hearing, or psychiatric impairments.

Population sample

A total of 93 patients (Female = 50, Male = 43, Mean Age = 77.7 and Std. Age = 9.5) were considered for the evaluation of the RS and cognitive games application. A subset of 78 patients (Female = 42, Male = 36, Mean Age = 77.9 and Std. Age = 9.3) were analysed for the multimodal fusion engine module, filtered by who completed the baseline and end-of-the-study questionnaires.

Statistical analysis

Several tests and statistical analysis have been carried out to validate the correct functioning of the different services and modules, as well as their complete integration into the project platform. Data are presented in the Results Section. We highlight analysis related to game notifications generated by the recommender and notifications marked as read by patients, the level of use of the cognitive games application, and information on the games completed by all patients during the months of the study. We also include data related to the customization of the application, the interaction of patients during the gaming sessions and the average scores obtained for each of the games. Finally, we present the results of the Multimodal Fusion Engine, giving an overview of the cognitive status and deviations of the patients, and the evolution during the months of the study.

Results representation

A set of visualization techniques are employed to present the outcomes of the study. RS notifications are presented through the utilization of histogram plots, providing a description of the evolution of notifications over time. For the cognitive games application, a combination of graphical representation is developed, including barplots, pie charts and box plots. These plots serve as a comprehensive view of the application’s performance and user engagement. Lastly, in the analysis of multimodal deviations, general plots are employed to track the evolution over the different weeks periods for an assesment of the cognitive status and potential deviations.

Results

The results section of this document presents the analysis of 3 months of data from the third and last pilot of the PROCare4Life project, for a controlled set of patients. This conducted pilot study was placed at six pilot sites across five countries (Germany, Spain, Portugal, Romania, and Italy). This pilot study was chosen to ensure that the latest versions, with all available functionalities of the cognitive games application, the multimodal fusion engine, and RS are deployed on the project platform at the end of the project.

The aim of this pilot study is to analyzse the use of the application, the functioning of the RS and the results, in terms of games, obtained by the set of patients who participated. And to be able to know and evaluate the effectiveness of the six cognitive games and the implemented RS in the improvement of cognitive ability and therefore, having a better quality of life.

Recommender system

Out of the 402 patients on the platform, a subset of 93 patients (23%) were willing to engage with the system and play cognitive games. Over the course of the trial period, these patients received a total of 260 notifications from the system, which were designed to encourage and motivate them to continue playing.

One of the most interesting findings of our study was the number of notifications that were marked as read by patients. In total, 98 of them (37.7%) were marked as read.

Figure 12 presents a histogram of the dates on which notifications were both sent and read as part of our pilot study on the cognitive game RS. The $x$ -axis represents the dates of the trial period, and the $y$ -axis shows the number of notifications sent and read on each date.

Figure 12.

Histogram of sent and read notifications.

The figure clearly illustrates two distinct waves in the number of notifications sent during the trial period, indicating the two-waves integration of patients into the platform. The initial wave exhibits a higher level of patient engagement, as evidenced by a greater number of notifications. As the trial period progressed, the number of notifications read declined. Several factors, such as patient fatigue and decreased motivation, likely contributed to this downward trend.

Cognitive games application

The results of the pilot study demonstrate a daily use of the cognitive games application. During this pilot, every day the application was used, with a daily average of 22.4 games completed among all patients. The day with the highest use of the application was the first day with 66 games completed. And the day with the lowest application usage was the 57th day with seven games completed. Figure 13(a) shows this distribution of the number of games completed among all patients during the study pilot.

Figure 13.

Cognitive game data exploration. (a) Daily game count histogram; (b) Sessions frequency per game; (c) Game count by category.

In addition, patients played all six available games, with game 1 being the most played game (by far) and game 5 being the least played game. The rest of the games were played equally. For those games that have available different categories and different levels of difficulty, patients have tested the games with different categories and levels, although there is a clear predominance of difficulty level 1 and the default selected categories in each of the games: Colours for game 1, numbers for game 5 and traffic for game 6. Figure 13(b) and 13(c) shows this information. Combining the data from these two graphs, the most played game is game 1 with category colours and level 1.

An important aspect that influences the use of the application is the design of the application and the set of customizable options available to patients. There are three options that can be modified: Language, text size and colour style. Although the language is conditioned by the country and the selection of the patients participating in this pilot study (Figure 14(a) shows the distribution per language), there is a clear preference for the default style and medium text size. The default style makes use of the corporate colours chosen in PROCare4Life project. Between the other two possible styles, dark style and light style, there is a higher acceptance for the light style (11.5% vs 1.0%), as can be seen in Figure 14(b).

Figure 14.

Application configuration. (a) Language; (b) style mode; (c) text size.

In terms of text size, although there are larger and smaller text size options available, it seems that the medium size is perfect for the use of the application. 0.1% of the patients have chosen a smaller text size, and only 5.5% of the patients have selected a larger text size, as can be observed in Figure 14(c).

By analyzing user interaction with the application, it is possible to assess whether patients have problems with its use, are unaware of how to play certain games or have misunderstandings or cognitive problems. In general, patients understand the six games and there does not seem to be an incorrect interaction when playing the different games. For this, the type of click helps to assess this behaviour: If the click belongs to the game progress (game click), if the click does not belong to the game progress (disabled click) or is an erroneous click in an area of the application where there is no button or element (undefined click). Most of the clicks are of the game type, which demonstrates the correct use of the application and understanding by the patients. However, as can be seen in Figure 15, game 6 has a large set of disabled clicks. This behaviour may be due to the characteristics of this game and the presence of elements and images that appear and disappear continuously. A similar analysis can be made with game 3 and the high number of undefined clicks. Patients tried to click again on letters they had previously selected in their final goal of guessing the hidden word.

Figure 15.

Application usability based on number of clicks.

Other interesting data that can be extracted from this study, related to the user’s interaction with the application, are the average time between consecutive clicks for each of the games, and the number of clicks (on average) needed to complete each of the games. Game 5, the bingo game, is the game with the longest elapsed time between two consecutive clicks. It makes sense because, in the end, it will depend on the instant the object appears on the screen. The same goes for game 2, the blurred objects game. The time between clicks will depend on when the patient correctly appreciates the object on the application. Figure 16(a) shows the average time between consecutive clicks for each of the games.

Figure 16.

Scores and number of clicks graphics. (a) Average time between clicks; (b) Number of clicks; (c) Global score per game.

On the other hand, game 1 and game 3 need a higher number of clicks than the rest of the games. Especially game 3, which is the hangman game, in which the user has to select the letters present in five different words. Other games, like game 2 and game 4, which have a fixed set of questions, only need an exact number of clicks to complete the games. This is consistent with the data obtained in Figure 16(b).

Finally, in terms of scores obtained for each of the games, among all patients, during the 3 months of this pilot study, the analysis of the data shows high scores in all of them. To perform the analysis, it is used the global metric obtained each time a game is completed. This metric determines how well a patient has completed a game considering three possible aspects: Game scores, time taken to complete the game, and interaction with the application. The global metric is normalized between 0 and 1, and in general terms, the results (on average) in each game are above 0.8 of the global metric value. As the graph in Figure 16(c) shows, the game with the lowest global metric would be game 3. This graph also shows anomalous data in game 2 and game 6. Specifically, particular cases where the value of the metric is below 0.6. It would be interesting to analyzse these types of data, associate them with the patients, and see why these low values occurred.

Combining the information from the global metrics between the six games, and representing this information per day, during the study pilot, it can be seen the average evolution of the metric (among all the games and all the patients) over time. This information is represented in Figure 17. The linear fit of all the data shows almost equal evolution in mean with the results in the passing of the days.

Figure 17.

Average global score over the analysis period.

Multimodal deviation analysis

Data pertaining to cognitive abilities, physical activities, motor functions, sleeping patterns, and medication intake were gathered from the patients. Additionally, patients completed questionnaires at the beginning (baseline) and end of the study to report their current cognitive ability levels. The patients in this analysis have all the data completed and saved in the system for the 90-day period (3 months). The evaluation of high-level representations (CSS) and deviations (DCA) was performed with 78 patients who completed the baseline and end-of-the-study questionnaires and the system collected information for all of the previously mentioned measurements. The details of the demographic of these patients are presented in Table 3. The assessment of AD and dementia patients involved the use of the Clinical Dementia Rating Scale,⁴⁹ while PD patients were evaluated using the Movement Disorders Society Unified Parkinson’s Disease Rating Scale.⁵⁰ The patients participated in the study over 3 months, the multimodal fusion engine should be able to provide an indication of the patients in terms of cognitive abilities and deviations by fusing all monitored measurements and the evaluation should be able to reflect the clinical assessments, which is determined by the experts as 3 months, hence, 12 weeks of scores and deviation probabilities were obtained from each patient and the analysis has been performed with weekly data.

Table 3.

Demographic details of the patients in multimodal evaluation.

	Male	Female	Age (Mean)	Age (SD)
AD	12	26	81.18	8.17
PD	24	16	72.95	8.67
Total	36	42	76.96	9.34

AD: Alzheimers disease; PD: Parkinsons disease.

The first evaluation is performed with the scores (CSS) obtained using the data from monitoring the game sessions of the patients and the questionnaires filled by them at the beginning and end of the study. Figure 18 presents the weekly means of CSS, the overall mean of CSS, the tendency of the 3-month period calculated through a linear fit and the regions obtained using the weekly mean values and weekly standard errors. The overall mean values are 0.1887, 0.1938 and 0.1851 for All patients, AD patients and PD patients, respectively. The linear fit for all patients reveals that the patients on average are expected to slightly improve over the 3 months period. This is reflected in the slight improvement in the positive difference between the mean end-of-study questionnaire results (0.3236) and the baseline results (0.3315). Furthermore, AD patients are showing an improvement over the study period which can be observed in the questionnaires as well (baseline= 0.4356, end-of-study=0.4437). On the other hand, a slight decline is expected according to the CSS measurement for the PD patients the questionnaires present a slight improvement, which might be attributed to the lower CSS scores observed from the PD patients with values less than 0.16.

Figure 18.

Cascading style sheet (CSS) from real patient data over 3 months. (a) All; (b) Alzheimer’s; (c) Parkinson’s.

The second evaluation is performed with the deviation results (DCA) obtained using the data from monitoring cognitive abilities, physical activities, motor functions, sleeping patterns, and medication intake. Figure 19 presents the weekly means of DCA probabilities, the overall mean of DCA, the tendency of the 3-month period calculated through a linear fit and the regions obtained using the weekly mean values and weekly standard errors. The overall mean values are 0.7999 (79.99%), 0.7938 (79.38%) and 0.8044 (80.44%) for All patients, AD patients and PD patients, respectively. Although these values are higher than expected for the patients participating in the study since they present a slight improvement, the low mean values obtained from the CSS score reveal the reason for obtaining such high probabilities. Consequently, these probabilities are utilized to alert the health and care professionals to the possible decline of the patients or potential for disorders. Furthermore, when the figures for AD and PD patients are compared the slope of the increase for the DCA of PD patients is higher than the AD patients which reflects the decline observed for their scores in Figure 18.

Figure 19.

Decline in cognitive abilities (DCA) from patient data. (a) All; (b) Alzheimer’s; (c) Parkinson’s.

Finally, we fitted a linear regression model to test if age, gender and diagnosis have an effect on the change from the baseline to the end of the study measurements for CSS, DCA and questionnaires. We used lm() function of the lme4 package for R using the formula lm(formula = end - bsl age + gender + diagnosis, data). The results showed that they don’t have any significant main effect on any of the measurements.

Another linear regression model has been fitted without the age, gender and diagnosis since they do not have any effects using the formula lm(formula = end - bsl 1, data). The analysis reveals that the change from the baseline to the end of the study for CSS and the questionnaires do not present statistically significant outcomes.

On the other hand, DCA results present a statistically slight increase ( $p$ = 0.000585). This shows that although the patients do not change significantly over 3 months according to the CSS and questionnaire results, the probability of observing deviations or disorders with the patients increases to account for the repeatedly observed low scores. DCA is revealed to be beneficial in terms of alerting the health and care professionals of situations where the patient is not showing improvements and might develop disorders or deteriorate.

Discussion

In terms of use of the application, it has been used every day, the six games available have been played, and in those games that have different categories and levels of difficulty, patients have tried those other options. However, the analysis of the data indicates a higher number of games played with the configuration that appears in the application by default. That is, always the same category and level of difficulty 1.

The default design chosen for the use of the application seems to be the right one. Since most use is made with the blue corporate colour style of the project and medium text size. As a second option, the light style predominates over the dark one and only in a small amount of the cases has it been decided to change the text to a larger size.

Regarding the understanding of how to complete each of the games, the user interaction data shows that in most cases, patients use correctly the application, not observing out-of-the-ordinary values. But for particular cases, such as low scores or incorrect interaction, it could be interesting to analyzse the reasons for these situations, with the intention of identifying the reason for this strange behaviour.

And that correct use of the application and understanding of how to complete the games translates into high scores, among all patients on each of the six games. The metric used, which takes into account aspects of scoring, times and interaction, normalized between 0 and 1, is higher than 0.8 (on average) in all games. The results indicate the possibility of increasing the level of difficulty of the games, always looking for the motivation and the challenge that this extra difficulty may imply for the patient.

As the notifications were received directly on the home screen of the web application, we expected that most of these notifications would be marked as read. It is possible that the low number of notifications marked as read in our study may be due to the fact that caregivers were not always present with patients and they may have forgotten to follow the instructions to both play cognitive games. This could have resulted in this number of unread notifications accumulating on patients’ devices, leading to a lower overall percentage of notifications being marked as read.

The evaluation of the multimodal deviation analysis revealed overall, the patients showed lower scores than expected. However, when compared with the questionnaires, consistent results were observed. The linear fit indicated a slight improvement in scores over the 3-month period, with AD patients showing greater improvement compared to PD patients. Furthermore, the high DCA probabilities and low CSS scores indicated that although there were slight improvements, patients’ cognitive abilities were still not at desired levels and this requires the attention of the health and care professionals. Comparing AD and PD patients, the DCA for PD patients was steeper than that observed in AD patients, which corresponded to the decline observed in their scores.

On the other hand, correlation analysis performed between questionnaires and probabilities and between questionnaires and scores for the difference between baseline and end of the study measurements does not reveal statistically significant results. Nevertheless, the increase in the probabilities in the presence of low scores indicates that health and care professionals need to be alerted to the current state of the patient, which might deteriorate if not interfered with.

During the assessment of the cognitive games application data, challenges and outcomes observed, are closely parallel those encountered with the integrated PROCare4Life solution.⁵¹ Several limitations have emerged, encompassing technical, implementation, and user-related aspects. A common thread among these limitations is the inherent difficulty in maintaining sustained user engagement. Users may sometimes perceive the solution as overwhelming due to the multitude of devices and feedback they receive. They may encounter issues that cannot be immediately resolved, leading to a potential drop in engagement. Additionally, users may not immediately grasp the benefits of certain recommendations offered by the games application. Consequently, these factors result in patients using the system, specifically the games application, less frequently than optimal, resulting in a reduction in the overall volume of collected information.

A great effort was undertaken by the PROCare4Life consortium to minimize the impact of these limitations and provide relevant outcomes and results from the solution. This applies to the games application as well as to the rest of the components and functionalities of the platform.

Conclusions

This document has presented some of the work done during the last years in the European project PROCare4Life, which focused on the use of tools for brain stimulation and cognitive training in elderly patients with AD or PD. Specifically, there are 3 fundamental elements that make up the complete system: a cognitive games application that collects data, the multimodal fusion engine that makes use of predictive machine learning models to assess the cognitive status of patients and the evolution over time, and the RS that generates notifications to help patient’s experience with the use of the application based on previous results. The data analysis of the study pilot demonstrates the usefulness and correct functioning of the above-mentioned elements. And in particular, the acceptance of the application and the six games implemented in the project for the 93 patients involved in the study, yielding interesting conclusions.

Application is used daily and patients engage with all the games across various categories and difficulty levels, but the analysis of the pilot study results reveals a clear inclination towards the default options provided by the application. The findings suggest that from a personalization perspective, the interaction and scores obtained from the diverse games indicate that patients do not encounter difficulties in utilizing the application. Therefore, the default text size and style configuration adequately meet their requirements. However, the uniformity in game selection, content categories, and difficulty levels among patients poses challenges in interpreting the results.

In order to solve this problem and as a future line, a mechanism to randomize the order in which the six games appear on the application’s home screen, coupled with an intelligent selection of game category and difficulty level for each patient, would significantly enhance the overall system. This proposed randomization mechanism could be integrated as a backend service in the application that could leverage patient’s identifier and previous gaming session results, which this mechanism would dynamically adapt the game configuration, encouraging exploration of different options over time. Proposing to the patients to play with higher levels of difficulty, a greater diversity in the values of the metrics and in the final scores would be obtained, which together with a wide variety of variables to enrich the predictive models of the multimodal system and the RS, would allow obtaining more robust insights into patients’ cognitive progression.

The number of notifications marked as read in our study highlights the need for further research on the cognitive game RS and how it can be optimized to better engage and motivate patients. By understanding the factors that contribute to low notification read rates, it may be possible to design more effective notifications and improve the overall effectiveness of the system in improving the quality of life of patients.

The CSS and DCA measurements present an overview of a patient’s current state and potential deviations. The usage of these measurements in the daily lives of health and care professionals for decision support is crucial since they can monitor and evaluate patients remotely and frequently on a weekly basis. The evaluation of the multimodal deviation analysis in longer periods than the 3-month study presented in this paper would be beneficial for observing the behaviour of the engine in the long term. Furthermore, replicating the study with other types of diseases and other age groups would contribute to the evaluation of the proposed multimodal approach and the accompanying decision support system.

Footnotes

Acknowledgements

The authors would like to thank the patients, caregivers, and professionals who agreed to participate in this pilot study. They would also like to thank the researchers and technical partners who supported data collection and data processing from the study sites; and all the consortium members of the Personalized Integrated Care Promoting Quality of Life for Older People (PROCare4Life) project.

Contributorship

All authors contributed substantially to the manuscript. All authors reviewed the manuscript and approved the final version to be submitted.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The study protocol was submitted for approval to the Research Ethics Committee of each participating clinical centre, specifically in Germany (number 020-37-MB), Italy (number 493-2020), Portugal (number 10-20), Romania (number 7/10.06.2020), and Spain (number 20/453-E). Once the study had been fully explained to the subject, written or digital informed consent was obtained prior to any study-related procedure, according to Good Clinical Practice and International Conference on Harmonization standards. Participation in this study was entirely voluntary. Participants had the right to withdraw from the study at any time, without giving reasons or experiencing any disadvantage in terms of the quality of care they would receive if they did not participate. After the withdrawal, their data were not considered for statistical purposes. No replacements were considered.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work has been supported by the EC Project PROCare4Life. This project has received funding from the European Union Horizon 2020 research and innovation programme under grant agreement No 875221.

Guarantor

AL and JM.

ORCID iDs

Álvaro Llorente

Alberto del Rio

David Jimenez

Yusuf Can Semerci

References

Health Systems Governance and Financing UHL. Global expenditure on health: public spending on the rise? Meeting report, World Health Organization, Geneva, 2021.

Union

. Ageing Europe – Looking at the lives of older people in the EU – 2020 edition. Publications Office of the European Union, 2020. ISBN 978-92-76-21520-2. DOI: https://doi.org/10.2785/628105.

Population structure and ageing - statistics explained. https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Population_structure_and_ageing#The_share_of_elderly_people_continues_to_increase (2022. accessed 27 April 2023).

Chłoń-Domińczak

Kotowska

Kurkiewicz

et al. Population ageing in Europe. Facts, implications and policies. EUROPEAN COMMISSION, Directorate-General for Research and Innovation, 2014. ISBN 978-92-79-35063-4. DOI: https://doi.org/10.13140/2.1.5039.6806.

Deary

Corley

Gow

et al. Age-associated cognitive decline. Br Med Bull 2009; 92: 135–152.

Lustig

Shah

Seidler

et al. Aging, training, and the brain: a review and future directions. Neuropsychol Rev 2009; 19: 504–522.

National Institute of Aging. Alzheimer’s disease fact sheet. https://www.nia.nih.gov/health/alzheimers-disease-fact-sheet (accessed 14 February 2029).

Reeve

Simcox

Turnbull

. Ageing and parkinson’s disease: why is advancing age the biggest risk factor? Ageing Res Rev 2014; 14: 19–30.

Mental Health and Substance Use. Parkinson disease: a public health approach: Technical brief. World Health Organization, 2022. ISBN 9789240050983.

10.

World Health Organization. Parkinson disease. https://www.who.int/news-room/fact-sheets/detail/parkinson-disease (accessed 18 January 2023).

11.

Alzheimer’s Association. Alzheimer’s disease facts and figures. Alzheimer’s & Dementia: J Alzheimer’s Assoc 2022; 18: 700–789.

12.

World Health Organization. Dementia. https://www.who.int/news-room/fact-sheets/detail/dementia (accessed 18 January 2023).

13.

Gauthier

Webster

Servaes

et al. World Alzheimer Report 2022: life after diagnosis: Navigating treatment, care and support. Alzheimer’s Disease International, 2022.

14.

Belmonte Hernández

Semerci

Proença

et al. Procare4life: an integrated care platform to improve the quality of life of parkinson’s and alzheimer’s patients. In PETRA ’22: The 15th International Conference on PErvasive Technologies Related to Assistive Environments. pp. 358–364. DOI: https://doi.org/10.1145/3529190.3534727.

15.

Ahmed

Marin

Bouça-Machado

et al. Investigating users’ and other stakeholders’ needs in the development of a personalized integrated care platform (PROCare4Life) for older people with dementia or Parkinson disease: protocol for a mixed methods study. JMIR Res Protoc 2021; 10: e22463.

16.

Ahmed

Marín

How

et al. End users’ and other stakeholders’ needs and requirements in the development of a personalized integrated care platform (PROCare4Life) for older people with dementia or Parkinson disease: mixed methods study. JMIR Formative Research 2022; 6: e39199.

17.

Azmi

Abdullah

Akmar

. A collaborative filtering recommender system model for recommending intervention to improve elderly well-being. Int J Adv Comput Sci Appl 2019; 10. DOI: https://doi.org/10.14569/IJACSA.2019.0100619.

18.

Hammer

Seiderer

André

et al. Design of a lifestyle recommender system for the elderly: requirement gatherings in germany and greece. In Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments. PETRA ’15, New York, NY, USA: Association for Computing Machinery. ISBN 9781450334525, p. 8. DOI: https://doi.org/10.1145/2769493.2769559.

19.

Beheshti

Yakhchi

Mousaeirad

et al. Towards cognitive recommender systems. Algorithms 2020; 13: 176.

20.

Gholizadeh Hamlabadi

Vahdati

Saghiri

et al. A framework for cognitive recommender systems in the internet of things (iot). In 2017 IEEE 4th International Conference on Knowledge-Based Engineering and Innovation (KBEI). pp. 0971–0976. DOI: https://doi.org/10.1109/KBEI.2017.8324939.

21.

Cipriani

Borin

Del Debbio

et al. Personality and dementia. J Nerv Ment Dis 2015; 203: 210–214.

22.

Kern

Cepeda

Lovestone

et al. Aiding the discovery of new treatments for dementia by uncovering unknown benefits of existing medications. Alzheimer’s & Dementia: Translational Research & Clinical Interventions 2019; 5: 862–870.

23.

Mental Health and Substance Use. Risk reduction of cognitive decline and dementia: WHO guidelines. World Health Organization, 2019.

24.

Stern

. Cognitive reserve in ageing and alzheimer’s disease. Lancet Neurol 2012; 11: 1006–1012.

25.

Stern

Munn

. Cognitive leisure activities and their role in preventing dementia: a systematic review. Int J Evid Based Healthc 2010; 8: 2–17.

26.

Wang

Xu W

JJP

. Leisure activities, cognition and dementia. Biochim Biophys Acta 2012; 1822: 482–491.

27.

Alvarez

Popa

Vretos

et al. Multimodal monitoring of parkinson’s and alzheimer’s patients using the ict4life platform. In 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). pp. 1–6. DOI: https://doi.org/10.1109/AVSS.2017.8078527.

28.

Lopez Velasco

Martín

Moreno

et al. Acceptance of cognitive games through smart tv applications in patients with parkinson’s disease. In PETRA ’18: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference. pp. 428–433. DOI: https://doi.org/10.1145/3197768.3201553.

29.

Klauser

Kötteritzsch

Niesenhaus

et al. Analysis and classification of serious games for elderly. In Message Understanding Conference. pp. 612–617.

30.

Tannous

Grébonval

Istrate

et al. Cognitive and functional rehabilitation using serious games and a system of systems approach. In 2018 13th Annual Conference on System of Systems Engineering (SoSE). pp. 189–194. DOI: https://doi.org/10.1109/SYSOSE.2018.8428731.

31.

Zaki

NAA

Wook

TSMT

Ahmad

. Analysis and classification of serious games for cognitive stimulation. In 2015 International Conference on Electrical Engineering and Informatics (ICEEI). pp. 612–617. DOI: https://doi.org/10.1109/ICEEI.2015.7352572.

32.

Chaarani

Ortigara

Yuan

et al. Association of video gaming with cognitive performance among children. JAMA Network Open 2022; 5: e2235721.

33.

Bediou

Adams

Mayer

et al. Meta-analysis of action video game impact on perceptual, attentional, and cognitive skills. Psychol Bull 2018; 144: 77–110.

34.

Jitjumlong

Chinbenchapol

Makasorn

et al. Development of game application for enhancement of children’s cognitive skills. In 2018 18th International Symposium on Communications and Information Technologies (ISCIT). pp. 304–307. DOI: https://doi.org/10.1109/ISCIT.2018.8587969.

35.

Eguia Gomez

Contreras Espinosa

Solano Albajes

. Video games as a cognitive resource. In 6th Iberian Conference on Information Systems and Technologies (CISTI 2011). pp. 1–6.

36.

Kioskli

Danassi

. Design, method and apparatus of a computerized cognitive gamified training program targeting to maintain and improve cognitive health in older adults. In The 14th PErvasive Technologies Related to Assistive Environments Conference. pp. 315–319. DOI: https://doi.org/10.1145/3453892.3461640.

37.

Clare

Bayer

Burns

et al. Goal-oriented cognitive rehabilitation in early-stage dementia: study protocol for a multi-centre single-blind randomised controlled trial (great). Trials 2013; 14: 1–15.

38.

Hill

Mowszowski

Naismith

et al. Computerized cognitive training in older adults with mild cognitive impairment or dementia: a systematic review and meta-analysis. American Journal of Psychiatry 2017; 174: 329–340.

39.

Bengoa

Badiola Martínez

Saiz

et al. Development and integration of serious games with focus in the training of different cognitive abilities in elderly people to improve their quality of life. In 2020 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT). pp. 1–6. DOI: https://doi.org/10.1109/ISSPIT51521.2020.9408775.

40.

Imbeault

Bouchard

Bouzouane

. Serious games in cognitive training for alzheimer’s patients. In 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH). pp. 1–8. DOI: https://doi.org/10.1109/SeGAH.2011.6165447.

41.

Wallace

Goubran

Knoefel

et al. Design of games for measurement of cognitive impairment. In IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI). pp. 117–120. DOI: https://doi.org/10.1109/BHI.2014.6864318.

42.

Chi

Agama

Prodanoff

. Developing serious games to promote cognitive abilities for the elderly. In 2017 IEEE 5th International Conference on Serious Games and Applications for Health (SeGAH). pp. 1–8. DOI: https://doi.org/10.1109/SeGAH.2017.7939279.

43.

Ogomori

Nagamachi

Ishihara

et al. Requirements for a cognitive training game for elderly or disabled people. In 2011 International Conference on Biometrics and Kansei Engineering. pp. 150–154. DOI: https://doi.org/10.1109/ICBAKE.2011.30.

44.

Toma

Dascalu

Trausan-Matu

. Seeker: a serious game for improving cognitive abilities. In 2015 14th RoEduNet International Conference - Networking in Education and Research (RoEduNet NER). pp. 73–79. DOI: https://doi.org/10.1109/RoEduNet.2015.7311831.

45.

Craven

Fabricatore

. Game features of cognitive training. In 2016 International Conference on Interactive Technologies and Games (ITAG). pp. 42–49. DOI: https://doi.org/10.1109/iTAG.2016.14.

46.

Ghaleb

Semerci

Asteriadis

. Modelling behaviours of people living with neurodegenerative conditions. In Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments. pp. 351–357. DOI: https://doi.org/10.1145/3529190.3534721.

47.

Wiegerinck

Burgers

Kappen

. Bayesian networks, introduction and practical applications. Handbook on Neural Information Processing 2013; 401–431. DOI: https://doi.org/10.1007/978-3-642-36657-4_12.

48.

RabbitMQ. What is rabbitmq? https://www.rabbitmq.com/ (accessed 20 December 2022).

49.

Goetz

Tilley

Shaftman

et al. Movement disorder society-sponsored revision of the unified parkinson’s disease rating scale (MDS-UPDRS): scale presentation and clinimetric testing results. Movement Disord: Off J Movement Disord Soc 2008; 23: 2129–2170.

50.

Morris

. The clinical dementia rating (cdr): current version and scoring rules. Neurology 1993; 43: 2412–2414.

51.

Gangas

Judica

Marin

et al. Procare4life lessons learned. Open Res Eur 2023; 3: 147.

Assessment of cognitive games to improve the quality of life of Parkinson’s and Alzheimer’s patients

Abstract

Objectives

Methods

Results

Conclusion

Keywords

Introduction

Related work

Materials and methods

PROCare4Life setup

Cognitive games

Game 1 – Concentration

Game 2 – Guess what?

Game 3 – Guess the word

Game 4 – Trivia

Game 5 – Bingo

Game 6 – Memory challenge

Data collection

Data processing

Multimodal fusion engine

Cognitive skills recommendations

Study overview

Pilots and modules evolution

Eligibility criteria

Population sample

Statistical analysis

Results representation

Results

Recommender system

Cognitive games application

Multimodal deviation analysis

Discussion

Conclusions

Footnotes

Acknowledgements

Contributorship

Declaration of conflicting interests

Ethical approval

Funding

Guarantor

ORCID iDs

References