Conception and usability pilot study of a new digital cognitive screening tool on tablet: AlzVR project

Abstract

Introduction

Alzheimer's disease (AD) is the first cause of dementia worldwide without any current curative treatment. Facing an increasing prevalence and its associated costs, AD represents a public health challenge. Usual diagnostic methods still rely on extended interviews and paper tests provided by an exterior examiner. We aim to create a novel, quick cognitive-screening tool on a numerical tablet.

Methods

This pilot program, built and edited with Unity^®, runs on Android^® for the Samsung Galaxy Tab S7 FE^®. Composed of seven tasks inspired by the Mini-Mental Status Examination and the Montréal Cognitive Assessment, it browses several cognitive functions. The architectural design of this tablet application is distinguished by its multifaceted capabilities, encompassing not only seamless offline functionality but also a mechanism to ensure the singularity of data amalgamated from diverse sites. Additionally, a paramount emphasis is placed on safeguarding the confidentiality of patient information in the healthcare domain. Furthermore, the application empowers individual site managers to access and peruse specific datasets, enhancing their operational efficacy and decision-making processes. We performed a preliminary usability assessment among young, healthy subjects.

Results

Twenty-four participants were included with a final F-SUS ‘excellent’ score. Participants perceived the tool as simple to use and achieved the test in a mean time of 142 s. No technical errors occurred.

Conclusion

These preliminary results suggest that our new assessment on a numerical tablet might be usable and acceptable for a short cognitive screening but requires further studies among older populations.

Keywords

Numerical tablet screening Alzheimer's disease usability primary care

Introduction

Alzheimer's disease (AD) is the first cause of neurodegenerative decline, affecting millions of people worldwide with a considerable cost for countries.¹ In AD, patients progressively lose their cognitive abilities, and behavioural troubles can occur. Without any current and efficient treatment, loss of memory and autonomy becomes an essential burden for caregivers and families. Alzheimer's disease management is a global and public challenge for health systems that face a constantly increasing prevalence in ageing populations. Dubois et al. recently defined AD as a clinical amnestic syndrome (memory impairment) associated with the presence of amyloid markers in human fluids.² Assessing cognition is central in diagnosing AD, and numerous memory tests exist for precise or global evaluation. Mini-Mental Status Examination (MMSE)³ and the Montréal Cognitive Assessment (MoCA)⁴ are widely used in primary screening because they are short (less than 10 min) and have good sensibility and specificity. Both tests share several questions and explore approximately the same cognitive functions, even though MoCA evaluates frontal deficits more precisely. They both browse several cognitive functions quickly and can be repeated through time in the medical following of patients. More recently, the MoCA seems to have a higher sensitivity (Se) than the MMSE in differentiating healthy subjects from demented patients, whereas MMSE still performs a higher specificity (Sp).⁵ Nevertheless, there are good correlations between the two tests.^6,7 Some other tests were developed for a very short screening, such as the Mini-Cog, which combines the three-word test (immediate and delayed recall) with the clock drawing task.⁸ The Codex test⁹ provides a more precise evaluation by integrating questions about spatial orientation. Both tests share questions from MMSE and MoCA. All these assessments are beneficial because they can briefly browse cognitive functions with good performances, especially memory impairment. For further explorations, there are labelled memory consultations where a patient can undergo a global neuropsychological evaluation. However, these specialised centres are primarily available in hospitals, and appointments can be long, delaying diagnosis and social measures. Moreover, the diagnostic process still requires paper tests involving an exterior examiner.

Besides these classical evaluations, numerous authors have developed new screening tools on numerical tablets that show good correlations with usual tests. Although several recent systematic reviews have been published,^10–12 only one meta-analysis focused on digital drawing tasks.¹³ In all these studies, numerical tests could run either on numerical tablets or on simple computers and touch screens. All assessment could be self or hetero-administered. Unfortunately, few of these programs were available in French,^14,15 limiting their use in francophone patients.

Finally, they have not exceeded the experimentation stage. Although many applications are already available on commercial platforms such as Apple iTunes or Google Play Store,¹⁶ health practitioners do not use them daily. Thus, the usability of numerical tablets has been globally demonstrated among large populations,¹⁷ and there is better accessibility to new technologies, with most patients owning a tablet or a smartphone.

Getting a specialised memory consultation can take a long time, and there is a need to facilitate early screening in primary care with general practitioners (GPs) even if some barriers remain.¹⁸ Indeed, GPs have a heavy workload and do not always have time to perform an efficient screening. There is also a false and persistent idea that cognitive decline is expected in ageing, delaying medical diagnosis. Recent literature shows that digital assessment can be performed in primary care with promising results.^19–21 General practitioner would highly benefit from using these new technologies, especially autonomous programs that do not require an exterior examiner. By providing early cognitive screening in primary care, they could anticipate and optimise patients’ journeys (precocious imagery, biology) before addressing them to specialised consultations. It would also benefit caregivers by identifying daily burdens and accompanying them in social procedures. Targeting primary care networks, a digital tablet assessment should be short, autonomous, reliable and understandable for patients with cognitive tasks reproducing classical questions from usual paper tests.

We have developed the AlzVR project, which proposes a multimodal digital program for cognitive screening. Previous papers presented our immersive assessment using Oculus Quest^®.^22,23 We now aim to explore a new modality by making a short auto-questionnaire (as Codex or Mini-Cog) on a touchscreen-based application.

Materials and methods

We constructed our program using Unity^® (v.2021.3.11) for Android^® (tablet). Three scenes compose AlzVR: the welcome menu, playing scene and F-SUS questionnaire.

Most of the figures presented in this paper have been translated from their French version for a better understanding.

Welcome menu

When launching the application, there are three possibilities (Figure 1):

Supervised experience: medical questionnaire and cognitive assessment

Quick experience: cognitive assessment only

Results: results visualisation

The game consists of three main scenes:

The ‘Menu’ scene includes the main menu, medical questionnaires and results consultation.

The ‘InGame’ scene contains the tutorial and all the user's tasks.

The ‘Survey’ scene collects user feedback, which the administrator can only consult.

Medical questionnaire

The supervised experience includes a primary medical questionnaire (Figure 2) to collect socio-demographic items (name, age, type of residence) and medical background (diagnostic, previous cognitive tests, treatments and sensory loss).

Figure 1.

Welcome menu view.

Figure 2.

Medical questionnaire.

Anonymization

After the last validation of the medical questionnaire, the program generates an automatic anonymised number composed of date and hour until seconds without integrating the name's initials. A typical anonymous number looks like YYYYMMDDHHMMSS. This process safeguards the confidentiality of patient information (personal information) and allows a further blinded analysis. In ‘quick experience’, an ‘A’ precedes all anonymised numbers, such as A-YYYYMMDDHHMMSS. In ‘supervised experience’, the letter of an eventual medical centre can be automatically added before the number.

Playing scene

Experiences architecture

The main module of the ‘InGame’ scene, the GameManager, references the list of nine tasks performed. Although each task has a different objective, each has textual and audio instruction and then proposes none, one or several answers in the form of images or text. Therefore, the ‘JExperience’ parent class groups all the attributes and methods common to all the tasks. However, the specific features of each task have necessitated the creation of new classes (‘JExpMonoChoice’, ‘JExpChoiceTown’, ‘JExpImages’) inherited from ‘JExperience’ (Figure 3).

Figure 3.

Main classes’ diagram.

General aspect

The visual aspect should be simple without any cognitive surcharge. All scenes appear in a uniform blue background. All the pictures implemented into the scenes are royalty-free.

In all experiences, the user must select answers by touching one or several buttons. These buttons are big, allowing for an easy touch. A maximum of eight buttons are on the screen, ensuring good visibility.

Answer modality

Since the user selects all the answers, a confirmation screen appears with a button ‘Yes’ and ‘No’. This step avoids inattentive answers and validates the choice (Figure 4). A ‘Yes’ leads to the next question, and ‘No’ allows a new chance to answer.

Figure 4.

Answer modalities.

Each exercise lasts 30 s maximum. The next question automatically occurs if the user does not answer within the time (counted as TimeOut). The choice of a ‘No’ in the confirmation step reinitialises the time, but only three attempts are allowed.

In every case (success or failure), a message ‘Congratulations’ congratulates the user (Figure 5). This message provides a cheerful ambience and can reduce further false results of stress or fear.

Figure 5.

Congratulations message.

Training task

Although we suppose that many older adults have already used a smartphone or a tablet, we have decided to evaluate numerical abilities with two training tasks before the cognitive questionnaire. The user needs to touch shapes on the screen. With these two short exercises, we can ensure a good understanding of the tablet's functioning by the user (Figure 6). A failure in the training tasks leads to the assessment's stopping, and the test cannot continue.

Figure 6.

Training task.

Cognitive questionnaire

If the training tests are successful, the cognitive assessment begins and comprises seven tasks. We wanted a varied assessment browsing multiple cognitive fields and our cognitive tasks are inspired by Codex, Mini-Cog (three words, clock drawing task, spatial orientation), MMSE or MoCA questions. Our tasks do not require anexterior examiner, stylus or voice recognition and are presented in Table 1.

Table 1.

Numerical cognitive tasks.

Paper test	Cognitive function explored	Numerical cognitive task
MoCA, MMSE	Auditory memory and attention	Three words task (immediate recall)
MoCA	Memory and attention	Clock recognition
MoCA, MMSE	Auditory memory and attention	Three words tasks (delayed recall)
MoCA, MMSE	Spatial orientation	Flags
MoCA, MMSE	Spatial orientation	Town
MoCA, MMSE	Temporal orientation	Season
MoCA, MMSE	Temporal orientation	Year
MoCA	Abstraction	Abstraction

MMSE: Mini-Mental Status Examination; MoCA: Montréal Cognitive Assessment.

The first task is the « three words » test. In the MMSE or MoCA, the examiner orally delivers the three words, and the patient must repeat them (immediately and with a delayed recall). To get a self-questionnaire, we kept the oral deliverance by the program (sound only) but replaced the oral repetition with a choice of three images among eight (Figure 7). There is still an immediate and delayed recall. The three words belong to different semantic fields (animal, vehicle and vegetable).

Figure 7.

Three words test.

The clock recognition (Figure 8) task is inspired by the clock drawing test (CDT),⁸ where the patients draw a circle, number and needles indicating a precise hour (e.g., 11h10). We created a novel task proposing three different clocks: the good one (10h30), the symmetric clock (05h50) and a false clock (Figure 8). The oral instruction delivers the hour to choose (‘select the clock indicating …’), and the patient selects on the screen. There are two series of clocks followed by the three words delayed recall.

Figure 8.

Clock recognition test.

To explore spatial and temporal orientation, we selected a simple format with an oral question (what is the current season? Select the country's flag where we are) and several pictures as answers. The country is represented as a flag for spatial orientation, limiting written instructions (Figure 9). Names of town are presented as classical French signs (Figure 10). The answer proposals for the town change depending on the site assessment.

Figure 9.

Country test.

Figure 10.

Town test.

Temporal orientation tasks are relatively similar, as they select the current season (Figure 11) and present the name of the season and a typical image. Considering varying dates for season changes, we left a 48-h margin for the answer.

Figure 11.

Season test.

In the year test, we introduced several confusing dates (minus one year, minus one century). All dates finish by the same number as the current year (Figure 12).

Figure 12.

Year test.

In the MoCA test, abstraction's ability is tested on the similarity between two words (e.g., an orange and a banana are fruits). We chose a fruit series to fill with a third picture in our abstraction test. A confusing element is among the four choices (one image from the three-word test) (Figure 13).

Figure 13.

Abstraction test.

Results menu

After the cognitive questionnaire, the results’ menu allows a simple visualisation of the patient's score. A password protects this section and only uses an anonymised number (ID patient). Three possibilities exist: ‘X’ (failure), ‘V’ (success) and ‘?’ (Timeout).

F-SUS questionnaire

After the cognitive tasks, we implemented the French translation of the System Usability Scale (Figure 14)²⁴ and the F-SUS questionnaire²⁵. It evaluates global satisfaction through ten questions and five degrees of response from 1 (strongly disagree) to 5 (strongly agree). F-SUS results do not appear in the results menu and are directly stored.

Figure 14.

F-SUS questionnaire view.

Data storage

AlzVR exports and stores all data in a CSV format, allowing easy exploitation. We planned separate storage for personal information from other results (experiences and F-SUS). Thus, a blinded analysis is possible using only results files containing anonymised data (Figure 15).

Figure 15.

Process of data storage and anonymisation.

Preliminary usability assessment

Study population

We carried out an experimental, qualitative study in IBISC Laboratory (University of Évry-Paris Saclay, Department of Sciences and Technologies) among volunteers (staff and students) to assess preliminary usability using ISO 9241-11 norm²⁶ and the Nielsen method.²⁷ The tablet was a Samsung Galaxy Tab S7 FE^® (screen of 315.0 mm, 2560 × 1600) running on Android 11^® (user interface One UI 3.1).

The inclusion criteria were age >18 years old and French language understanding. The exclusion criteria were age < 18 years old, no understanding of the French language, and visual or hearing loss with no equipment.

We recruited participants through mailing lists of university and advertisements in the locals.

Stages

Participants successively and anonymously achieved several stages:

Pre-questionnaire: fill in an online questionnaire to collect socio-demographic data (age, profession, sex) and numerical habits (smartphone and tablets)

Quick experience

F-SUS questionnaire

Post-questionnaire: An online questionnaire will be used to collect free comments about the program.

Data collection and exploitation

During the tests, we collected the following parameters: answer (success, fail), number of trials and response time (ms).

We chose the total F-SUS score calculated on the author's recommendation^24,25 as the primary endpoint to assess usability with a goal of 85.5%, considered ‘excellent’.

All data were blinded, collected and analysed using the anonymised numbers of participants.

Results

Population

We included 24 participants between 27 September 2022 and 12 October 2022. Table 2 presents their socio-demographics, and Table 3 their numerical habits.

Table 2.

Socio-demographic characteristics of the population.

		Population (n = 24)
Gender (F/M)		10/14
Age^a (years) m(sd)[min-max]		41.88 (13.11)[23–66]
Profession	2	Student (under PhD)
	2	Engineer
	3	PhD student
	3	Technician
	5	Researcher^b
	9	Administrative Staff

m: mean; sd: standard deviation; min: minimum; max: maximum.

Professor or associate professor.

Table 3.

Numerical habits.

Question^a
Have you ever used a smartphone? (%)	100
If yes, for how many years? m(sd)	12.37 (4.8)
If yes, during 2022, how often? (%)	100	Everyday
Have you ever used a numerical tablet? (%)	96	Yes
Have you ever used a numerical tablet? (%)	4	No
If yes, for how many years? m(sd)	8.25 (3.13)
If yes, during 2022, how often? (%)	21.7	Everyday
	13.1	Once/week
	17.4	Once/month
	47.8	Once/year

m: mean; sd: standard deviation.

Success rate

Cognitive tasks were completed by 100% of participants. We observed a success rate for the questions of 97.4% (187 correct answers out of 192). The two tests that presented failures were the clock task (2 failures) and the season (3 failures).

Time of completion

The average test administration time (excluding training tasks) was 141.47 (± 18.77) seconds, and details of task completion times are presented in Table 4.

Table 4.

Tasks completion times.

Completion time^am(sd)[min-max]	Cognitive task
31.98 (2.62) [25.16–38.60]	Three words task (immediate recall)
36.43 (5.49) [25.86–51.05]	Clock recognition (2 series)
17.41 (2.83) [11.04–22.65]	Three words task (delayed recall)
12.23 (1.87) [8.66–16.29]	Flags
10.91 (2.39) [6.95–16.45]	Town
10.41 (2.12) [6.38–14.68]	Year
11.73 (3.97) [6.75–23.68]	Season
10.39 (2.67) [6.23–15.19]	Abstraction
141.47 (18.77) [97.64–183.58]	Total

m: mean; sd: standard deviation; min: minimum; max: maximum.

F-SUS questionnaire

Ninety-six percent of participants completed the F-SUS questionnaire (one person left the application before completing it), and the results for each question are presented in Table 5.

Table 5.

F-SUS questionnaire results.

Question	Result^a m(sd)
I think that I would like to use this system frequently.	2.70 (1.46)
I found the system unnecessarily complex.	1.52 (0.71)
I thought the system was easy to use.	4.91 (0.28)
I think that I would need the support of a technical person to be able to use this system.	1 (0)
I found the various functions in this system were well integrated.	4.65 (0.87)
I thought there was too much inconsistency in this system.	1.43 (0.97)
I would imagine that most people would learn to use this system very quickly.	4.96 (0.20)
I found the system very cumbersome to use.	1.65 (1.34)
I felt very confident using the system.	4.83 (0.38)
I needed to learn a lot of things before I could get going with this system	1.74 (1.42)

m: mean; sd: standard deviation.

The overall score on the F-SUS questionnaire was 89.24%, considered ‘excellent’.

General remarks

In the post-questionnaire, we collected general opinions about the computer program. Users overwhelmingly found the program to be easy to use. The negative remarks reported were a lack of fluidity in the oral instructions and the tests being judged too simple.

Discussion

Numerous existing paper tests assess cognition for a global screening or precisely for a specific function.²⁸ At the same time, several authors studied the possibility of numerical tablet use in evaluating cognitive decline and performing training tasks in healthy and cognitively impaired patients.^29,30 Despite these numerous and efficient digital tests,¹⁰ cognitive evaluations still rely on paper tests and need an examiner. Facing an increasing prevalence of AD in the future decades,¹ there is a need to get simple, quick and performing tools to help practitioners in cognitive decline screening. During our conception, we chose to create a new tool in the French language inspired by two primary used and recommended tests^31,32: MMSE, MoCA and the CDT (integrated into the MoCA). In usual tests, the patient answers most of the questions orally to the examiner. We chose not to use speech recognition because of its limitations.³³ Incorrect speech interpretation would have led to false results. Excluding oral answers does not allow a global language evaluation as in MMSE or MoCA.

Our first task might present some limits. Indeed, in the traditional three-word tests, the examiner orally delivers the words, and the patient repeats them. There is no picture of written instruction. A direct digital transposition would have needed an automatic speech or human analysis. So, we decided to switch to a picture test with an immediate and delayed recall. Although we kept an identical first step (oral instruction), remembering pictures or words may require some different memory abilities.

Clock drawing test is also hugely used in daily practice and belongs to quick screening tools such as Codex⁹ or Minicog.³⁴ Müller et al. have proposed a digital clock drawing task using a stylus,³⁵ showing good correlations with paper tests. This transposition still requires exterior human validation or automatic image analysis, as proposed by Park et al.³⁶ We wanted a simple and short task with no exterior analysis, so we switched from a drawing task to a recognition picture task. Inoue et al.³⁷ proposed a similar task with a clock timer recognition among a demented population, which showed promising results. Drawing a clock and placing needles (as in the CDT) requires visuospatial abilities and executive functions. Nevertheless, there were technical limitations to producing a self-administered questionnaire with few written instructions, no exterior validation and simple orders. These limitations may lead to potential bias in cognitive evaluation with an underestimation of executive functions as MMSE or MoCA do.

Our assessment does not evaluate writing abilities because we did not want to use a stylus or further human validation. Thus, it is known that dysgraphia is a symptom of AD.³⁸ Despite exploring several questions and different cognitive fields, our new assessment has limitations that need to be followed and may need upgrades in future versions.

As a first evaluation, we performed a short usability assessment of a healthy population (without cognitive decline) among university users.

Completion time should be short, and the mean time observed in our study (142 s) is a good result. Moreover, usability reached the global score of 89.24%, surpassing the initial objective of 85.5% and close to 90.9% (« best imaginable »).

The participants globally perceived the test as easy to use, corresponding to F-SUS scores (questions 3, 5, 7 and 8). It was a positive evaluation because participants did not know about cognitive tests and thus discovered them for the first time. These results are preliminary satisfying data, but there is a considerable limitation about the population. Indeed, our participants were young (41.88 years old), healthy and used to touch screens. This mean age is widely below the age of AD patients (>60 years)³⁹, which can explain the observed good results. They may not be transposable into an older population with cognitive decline and poor use of tablets. The questions were negatively perceived as « too simple » or « too slow » due to the young age of our participants.

Although the participants were healthy, we noted errors in the clock recognition task, probably due to the needle shapes signalled in complimentary remarks. However, recent findings showed that students had more and more difficulties reading traditional clocks,⁴⁰ and our two failed users were 24 years old. These difficulties appear in the mean task time of realisation (Table 4); indeed, it is the task with the most significant difference between the minimal and maximal time of realisation. Season errors may be explained by the recent season changes (summer/autumn) before the beginning of the study (Sept 27). We also found an extensive range in task time realisation.

When extracting the results from the tablet, we reported no errors in the CSV files. Data were easily exploitable and well anonymised.

These first results are satisfying, but AlzVR remains a pilot questionnaire. Although classical cognitive tasks inspire the conception, we cannot extrapolate the results of a young population into an older one with cognitive impairment. AlzVR should be tested in an older population for usability assessment and accuracy of discrimination between healthy and demented subjects. However, this pilot study belongs to the global research COGNUM-AlzVR. It aims to evaluate the efficiency and relevance of two numerical tablet programs for cognitive assessment in AD and mild cognitive impairment patients. The Committee for the Protection of People of Ile de France approved the multicentric project in 2022, and the study began in April 2023 (NCT06032611). We intend to include 150 participants (150 patients with AD, 150 patients with MCI and 150 healthy subjects) in two French hospitals. Participants undergo both classical and digital assessments. We extracted some preliminary results about AlzVR. Delayed recall appears to be a discriminant test,⁴¹ and AlzVR shows good feasibility and acceptability among the patients.⁴²

Conclusion

We have developed a new digital cognitive screening tool with preliminary good feedback among a young and healthy population. The application could also be transposed onto smartphones to enhance its diffusion and utilisation. Definitive accuracy, usability and efficacy will be appreciated at the end of the ongoing COGNUM-AlzVR study.

Footnotes

Acknowledgements

The authors thank all participants and the Génopole (Evry-Courcouronnes, France) for their partnership with IBISC Laboratory.

Contributorship

Conceptualization, FM, GL, JD, FD and SO; methodology, FM, GL, JD and FD; software, GL, JD and FD; validation, FM and GL; formal analysis, FM; investigation, FM, GL and FD; resources, GL and FD; data curation, GL; writing – original draft preparation, FM and FD; writing – review and editing, FM, GL, FD and SO; visualisation, FM and GL; supervision, GL and SO; project administration, GL and SO; funding acquisition, none. All authors have read and agreed to the published version of the manuscript.

Data availability statement

The anonymised data supporting this study's findings are available on simple request from the corresponding author, SO. The original data are not publicly available because they contain information that could compromise the privacy of research participants.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

Ethical approval

This work has been carried out in accordance with the Declaration of Helsinki of the World Medical Association, revised in 2013 for experiments involving humans. Data exploitation was anonymous using an automatic number of participation generated from the date and hour of completion. The local University Paris-Saclay ethics committee approved all documents and protocols on 2022/07/07 (Approval registration number CER-2022-433). All subjects involved in the study gave their written informed consent. Participation was free with no remuneration.

Guarantor

SO.

ORCID iD

Florian Maronnat

Guillaume Loup

Frederic Davesne

Samir Otmane

References

International

Guerchet

Prince

, et al. Numbers of people with dementia worldwide: an update to the estimates in the World Alzheimer Report 2015. 2020 [cited 2022 Jul 25], https://www.alzint.org/resource/numbers-of-people-with-dementia-worldwide/.

Dubois

Villain

Frisoni

, et al. Clinical diagnosis of Alzheimer’s disease: recommendations of the International Working Group. Lancet Neurol 2021; 20: 484–496.

Folstein

McHugh

. 'Mini-mental state'. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12: 189–198.

Nasreddine

Phillips

Bédirian

, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005; 53: 695–699.

Ciesielska

Sokołowski

Mazur

, et al. Is the Montreal Cognitive Assessment (MoCA) test better suited than the Mini-Mental State Examination (MMSE) in mild cognitive impairment (MCI) detection among people aged over 60? Meta-analysis. Psychiatr Pol 2016; 50: 1039–1052.

Bergeron

Flynn

Verret

, et al. Multicenter validation of an MMSE-MoCA conversion table. J Am Geriatr Soc 2017; 65: 1067–1072.

Chua

SIL

Tan

Wong

, et al. Virtual reality for screening of cognitive function in older persons: comparative study. J Med Internet Res 2019; 21: e14821.

Sunderland

Hill

Mellow

, et al. Clock drawing in Alzheimer’s disease. a novel measure of dementia severity. J Am Geriatr Soc 1989; 37: 725–729.

Belmin

Pariel-Madjlessi

Surun

, et al. The cognitive disorders examination (Codex) is a reliable 3-min test for detection of dementia in the elderly (validation study on 323 subjects). Presse Med 2007; 36: 1183–1190.

10.

Chan

JYC

Yau

STY

Kwok

TCY

, et al. Diagnostic performance of digital cognitive tests for the identification of MCI and dementia: a systematic review. Ageing Res Rev 2021; 72: 101506.

11.

Amanzadeh

Hamedan

Mahdavi

, et al. Digital cognitive tests for dementia screening: a systematic review. 2022.

12.

Tsoy

Zygouris

Possin

. Current state of self-administered brief computerized cognitive assessments for detection of cognitive disorders in older adults: a systematic review. J Prev Alzheimers Dis 2021; 8: 267–276.

13.

Chan

JYC

Bat

BKK

Wong

, et al. Evaluation of digital drawing tests and paper-and-pencil drawing tests for the screening of mild cognitive impairment and dementia: a systematic review and meta-analysis of diagnostic studies. Neuropsychol Rev 2022; 32: 566–576.

14.

Rai

Boyle

Brosnan

, et al. Digital biomarkers based individualized prognosis for people at risk of dementia: the AltoidaML multi-site external validation study. Adv Exp Med Biol 2020; 1194((Rai L.; Boyle R.; Brosnan L.; Rice H.; Farina F.; Whelan R.) Trinity College Institute of Neuroscience, Trinity College Dublin, Dublin, Ireland): 157–171.

15.

Vidal

De Rotrou

, et al.

Can a tablet-based cancellation test identify cognitive impairment in older adults?

PLoS ONE [Internet] 2017; 12, https://www.embase.com/search/results?subaction=viewrecord&id=L617460519&from=export .

16.

Thabtah

Peebles

Retzler

, et al. Dementia medical screening using mobile applications: a systematic review with a new mapping model. J Biomed Inform 2020; 111.

17.

Kortum

Sorber

. Measuring the usability of Mobile applications for phones and tablets. Int J Hum Comput Interact 2015; 31: 518–529.

18.

Siddiqui

Nyahoda

Traber

, et al. Screening for cognitive impairment in primary care: rationale and tools. Mo Med 2023; 120: 431–439.

19.

Kalafatis

Modarres

Apostolou

, et al. The use of a computerized cognitive assessment to improve the efficiency of primary care referrals to memory services: protocol for the accelerating dementia pathway technologies (ADePT) study. JMIR Res Protoc 2022; 11: e34475.

20.

Giaquinto

Battista

Angelelli

. Touchscreen cognitive tools for mild cognitive impairment and dementia used in primary care across diverse cultural and literacy populations: a systematic review. J Alzheimers Dis 2022; 90: 1359–1380.

21.

Libon

Matusz

Cosentino

, et al. Using digital assessment technology to detect neuropsychological problems in primary care settings. Front Psychol 2023; 14: 1280593.

22.

Maronnat

Seguin

Djemal

. Cognitive tasks modelization and description in VR environment for Alzheimer's disease state identification. In: 2020 Tenth International Conference on Image Processing Theory, Tools and Applications (IPTA), 2020, pp.1–7.

23.

Maronnat

Davesne

Otmane

. Cognitive assessment in virtual environments: how to choose the Natural User Interfaces? In: Laval Virtual VRIC ConVRgence Proc 2022 [Internet] 2022; 1, https://hal.archives-ouvertes.fr/hal-03622384 .

24.

Brooke

. SUS: a 'Quick and dirty' usability scale. In: Usability evaluation in industry. Earlay, UK: CRC Press, 1996, pp.189–194.

25.

Gronier

Baudet

. Psychometric evaluation of the F-SUS: creation and validation of the French version of the system usability scale. Int J Hum Comput Interact 2021; 37: 1571–1582.

26.

International Organization for Standardization I. ISO 9241-11:2018 [Internet]. 2018 [cited 2023 Feb 25. https://www.iso.org/standard/63500.html.

27.

Valentin

Lemarchand

. La construction des échantillons dans la conception ergonomique de produits logiciels pour le grand public. Quel quantitatif pour les études qualitatives ? Trav Hum 2010; 73: 261–290.

28.

De Roeck

De Deyn

Dierckx

, et al. Brief cognitive screening instruments for early detection of Alzheimer’s disease: a systematic review. Alzheimers Res Ther 2019; 11: 21.

29.

Koo

Vizer

. Mobile technology for cognitive assessment of older adults: a scoping review. Innov Aging 2019; 3: igy038.

30.

Wilson

Byrne

Rodgers

, et al. A systematic review of smartphone and tablet use by older adults with and without cognitive impairment. Innov Aging 2022; 6: igac002.

31.

Janssen

Koekkoek

Moll van Charante

, et al. How to choose the most appropriate cognitive test to evaluate cognitive complaints in primary care. BMC Fam Pract 2017; 18: 101.

32.

Pinto

TCC

Machado

Bulgacov

, et al.

Is the Montreal Cognitive Assessment (MoCA) screening superior to the Mini-Mental State Examination (MMSE) in the detection of mild cognitive impairment (MCI) and Alzheimer’s disease (AD) in the elderly?

Int Psychogeriatr 2019; 31: 491–504.

33.

Basak

Agrawal

Jena

, et al. Challenges and Limitations in Speech Recognition Technology: a Critical Review of Speech Signal Processing Algorithms, Tools and Systems. Comput Model Eng Sci 2023; 135: 1054–1089.

34.

Borson

Scanlan

Chen

, et al. The Mini-Cog as a screen for dementia: validation in a population-based sample. J Am Geriatr Soc 2003; 51: 1451–1454.

35.

Müller

Herde

Preische

, et al. Diagnostic value of digital clock drawing test in comparison with CERAD neuropsychological battery total score for discrimination of patients in the early course of Alzheimer’s disease from healthy individuals. Sci Rep 2019; 9: 3543.

36.

Park

Automatic

. Qualitative scoring of the clock drawing test (CDT) based on U-Net, CNN and Mobile sensor data. Sensors 2021; 21: 5239.

37.

Inoue

Okayama

Horie

, et al. A computerized testing system for screening dementia in community-based settings. In: Proceedings of the International Conference on Health Informatics. Yonago, Japan, 2009, pp. 401–404. https://www.scitepress.org/papers/2009/11238/11238.pdf .

38.

Onofri

Mercuri

Archer

, et al. Legal medical consideration of Alzheimer’s disease patients’ dysgraphia and cognitive dysfunction: a 6 month follow up. Clin Interv Aging 2016; 11: 279–284.

39.

National Institute on Aging. What Are the Signs of Alzheimer's Disease? [Internet]. [cited 2023 Mar 6], https://www.nia.nih.gov/health/what-are-signs-alzheimers-disease.

40.

BBC news. Young can 'only read digital clocks’. BBC News [Internet]. 2018 Apr 24 [cited 2023 Mar 6], https://www.bbc.com/news/education-43882847.

41.

Um Din

Maronnat

Loup

, et al. Cognitive testings on a touch-screen tablet for older adults: a feasibility study. Gerontechnology 2024; 23: 1–1.

42.

Um Din

Maronnat

Le Boterff

, et al. Contribution of a tablet-based tests to the diagnosis of neurocognitive disorders in older adults: a feasibility study. Stud Health Technol Inform 2024; 316: 1163–1164.