Validity and diagnostic performance of a virtual reality-based supermarket application “MEMOSHOP” for assessing episodic memory in normal and pathological aging

Abstract

Objective

MEMOSHOP is a virtual reality (VR)-based supermarket application for assessing episodic memory in aging. The aim of this study was to examine its construct validity against the gold standard paper-and-pencil neuropsychological test for clinical memory assessment in mild cognitive impairment (MCI) in older adults.

Methods

Patients with isolated subjective cognitive complaints (SCCs) or MCI were recruited in the Bordeaux Memory Clinic (MEMENTO cohort). Cognitively normal elderly controls were also recruited. MEMOSHOP allows a near-ecological evaluation of episodic memory during a usual daily life activity, i.e. shopping at the supermarket. MEMOSHOP and the gold standard Free and Cued Selective Reminding Test (FCSRT: French adaptation) were administered to all participants to assess episodic memory. Non-parametric tests and receiver operating characteristic curves were computed to compare their performances.

Results

Twenty-nine patients (21 females, age = 71 years ±7) and 29 matched controls were evaluated. The performance trends observed with MEMOSHOP and FCSRT on free and cued recall were associated (p < .01) and comparable (p < .0001), without any participants' groups interaction. Although easier than FCSRT in free recall for participants, MEMOSHOP demonstrated better diagnostic performance based on cued recall in isolated SCCs/MCI patients (p < .05).

Conclusion

MEMOSHOP demonstrated its reliability and validity for VR-based episodic memory assessment in the early stage of MCI and is potentially of interest for use in memory clinic settings.

Keywords

Virtual reality supermarket episodic memory cognition aging mild cognitive impairment

Introduction

The aging of the population in developed countries has progressively increased the risk of cognitive decline and neurodegenerative diseases.¹ As cognitive impairment impacts autonomy,² the early assessment and management of cognitive decline would help in identifying individuals at risk and attempting interventions.^3–5 In this setting, the assessment of the memory impairment that occurs in the very early stages of Alzheimer's disease is central for diagnostic purposes. Mild cognitive impairment (MCI) is the condition in which individuals with cognitive complaints present a slight decline in cognitive functioning as evidenced by neuropsychological evaluation, but without any major repercussions on daily life.⁶

The transfer of new technologies to health settings is opening new perspectives of research on cognitive evaluation and monitoring. Neuropsychological assessment could be improved by implementing computer technology.^7–9 The advantages are numerous such as compensating for the lack of specialized centers, removing mechanical and redundant tasks, automating demanding aspects of tasks (e.g. scoring), and facilitating data storage and patient follow-up. Computerized neuropsychological batteries offer ease and standardization of administration compared to traditional paper-and-pencil tests.^8,10,11 In this perspective, virtual reality (VR), a computer-based technology that allows users to interact with a three-dimensional multisensory simulated environment, reinforces the ecological aspect of assessment by evaluating cognitive impairment under controlled conditions.^12–14

Episodic memory decline with age and is one of the earliest deficits in Alzheimer's disease. The episodic memory system allows individuals to acquire and recollect personally experienced events associated with a specific spatio-temporal context.¹⁵ Episodic memory involves processes used to acquire (i.e. encoding), maintain over time (i.e. long-term storage), and subsequently access or recall stored information (i.e. retrieval with recall or recognition). It can fail at any stage, leading to forgetting or having false memories. The clinical procedure developed by Grober and Buschke^16,17 to evaluate episodic memory deficits makes it possible to maximize learning by inducing deep semantic processing¹⁸ and by controlling encoding and retrieval components on the basis of the encoding specificity principle.¹⁹

The Free and Cued Selective Reminding Test (FCSRT) is a verbal memory test. In the study phase which ensures encoding, the examiner asks the subject to point to and name 16 stimuli (e.g. currant) presented on four different cards containing four words in response to a semantic category cue (e.g. fruit). The card is then removed and immediate recall is examined. The search is performed again for items not retrieved by cued recall. The memory part consists of three recall trials, with a non-verbal interference task before (i.e. counting backwards for 20 s). First, participants need to spontaneously recall as many items as possible in a 2-min period (“free recall”). Then, the non-retrieved items are recalled with the help of the specific semantic cue for each item (“cued recall). If the participant fails to retrieve the item with the category cue, the examiner verbally reminds the subject of the item. After 20 min, the same procedure is used to test delayed recall (i.e. free delayed recall followed by cued delayed recall).

This test is a simple and reliable test for evaluating episodic memory in aging with sensitivity and specificity in identifying patients with Alzheimer's disease. It helps distinguish between simple difficulties in retrieval of stored information (retrieval performances improved with cuing) and genuine encoding deficits characterizing typical Alzheimer's disease (retrieval performance poorly improved by cuing). It demonstrates the specific dysfunctional mechanisms involved during the successive stages of memory functioning: encoding, storage, and retrieval in memory. A French adaptation of the FCSRT (“Rappel libre/Rappel indicé à 16 items RL/RI-16”) has been developed²⁰ and validated in older adults population.²¹

The paper-and-pencil FCSRT based on learning words is considered essential for the neuropsychological examination of elderly people consulting in memory clinics. VR scenarios designed to assess episodic memory in aging have already been developed.^13,22–29 To our knowledge, however, no scenario based on the clinical procedure of the FCSRT exists. Previous studies have used VR applications featuring shopping tasks for cognitive screening or assessment in aging based on memory, attention, spatial navigation, and/or executive functions^{14,22–24,30,31} and provided promising results. Shopping at the supermarket is a usual activity of daily life largely involving memory processes. Therefore, we developed MEMOSHOP, a VR application that immerses the user in a supermarket environment and is based on the standard clinical procedure for assessing episodic memory in aging. The task reflects cognition in a lifelike scenario to capture episodic memory deficiencies. It also assesses the dysfunctional components of episodic memory and their memory-based functional repercussions on this activity of daily life.

A recent study shows that individuals with subjective cognitive complaints (SCCs) may be at higher risk of developing MCI or dementia compared to individuals who do not report complaints.³² SCC may be a signature of preclinical stage Alzheimer's disease. The aim of this study was to examine the construct validity of MEMOSHOP against the gold standard neuropsychological test for the clinical assessment of episodic memory deficits in patients with isolated SCCs or MCI.

Methods

Population

The patients were recruited from the MEMENTO cohort at the University Hospital of Bordeaux. MEMENTO (deterMinants and Evolution of AlzheiMer's disEase aNd relaTed disOrders) which is a 5-year prospective large cohort of patients with either isolated SCCs or recently diagnosed MCI while not demented attending an outpatient memory clinics (CMRR—Center Mémoire de Ressource et de Recherche) of public hospitals in France. Patients followed at University Hospital of Bordeaux were invited to participate in this study (SCOAL study).³³

The inclusion criteria were as follows:

- Having at least MCI defined by a performance of more than 1.5 standard deviation from the mean (defined according to age, gender, and level of education) in one or more cognitive domains (assessed on a battery of neuropsychological tests exploring memory, language, praxis, vision, executive functions),³⁴ the deficit being identified for the first time by tests performed <6 months before the inclusion date.

- Or presenting an isolated cognitive complaint (assessed through visual analog scales) without any of objective cognitive deficit and was over 60 years old.

- Having a clinical dementia rating [CDR]³⁵ scale score ≤0.5 and being non-demented (DSM-IV).

Elderly controls considered as cognitively healthy and matched on age, gender, and level of education were recruited after clinical and neuropsychological examination. To verify the absence of objective cognitive deficit, the participants were administered the Mini-Mental State Examination (MMSE),^36,37 the French adaptation of the FCSRT,^38,39 the Trail-Making Test (TMT),⁴⁰ and the CDR scale³⁵ by a trained neuropsychologist.

All participants provided written informed consent and the study was approved by the local ethical committee (consultative committee for the protection of persons participating in biomedical research, Comité de Protection des Personnes [CPP] Sud-Ouest et Outre Mer III). The study was registered as a clinical trial (ClinicalTrials.gov identifier: NCT01650454).

Design

This study is part of a larger SCOAL study which is a 1-year follow-up case-control cross-sectional study. This study was carried out at the Bordeaux University Hospital, France.

The participants were administered the reference paper-and-pencil test FCSRT and the MEMOSHOP to assess episodic memory.

FCSRT

The FCSRT^38,39 is a verbal memory test evaluating the ability to learn and recall a list of 16 written words presented with a semantic cue (i.e. semantic category) to control for memory encoding and retrieval conditions. This task involves word-list learning with free and cued recall, and delayed recall.

The learning phase is followed by three trials of recall (each preceded by a counting backwards interference task for 20 s), each consisting in retrieving the words first spontaneously (i.e. “free recall”) and then with the help of a semantic cue (i.e. “cued recall”) for those items not retrieved in free recall. Twenty-minute interference non-verbal tasks separate the free and cued recall from delayed recall phase (i.e. free recall followed by cued recall).

MEMOSHOP, a Vr-based supermarket application

Before the VR test, the participants completed a questionnaire on their shopping habits.³⁰ Then, they were administered the VR application MEMOSHOP. The administration lasted approximately 60 min. MEMOSHOP was developed using Unity 3D. It was designed and developed in an interdisciplinary collaboration (researchers, clinicians, and developers of VR-based scenario) with the Bordeaux-based company, Immersion. As part of shopping at the supermarket, MEMOSHOP was designed to assess the ability to perform the task of memorizing products on a shopping list. As mentioned earlier, this original paradigm is based on the methodology of the free and cued reminding test³⁸ adapted from the Grober and Buschke test.¹⁷ MEMOSHOP simulates a textured three-dimensional supermarket virtual environment representing four aisles displaying products which are commonly sold in a real-life supermarket. The environment is projected onto a large 3-fold touchscreen on a floor stand (Figure 1). No shelf-labelling is present in order to eliminate any source of interference with the words of the memorization task.

Figure 1.

Photo of supermarket virtual environment projected onto large touchscreen on floor stand.

The subject is seated comfortably in front of the screen showing a shopping trolley. Movement along the aisles is managed automatically by displaying four products at each stop in each of the four aisles (Figure 2). Each product is presented visually on a list with pictures side by side at the bottom of the screen with the name of the item below the picture. Only a few virtual humans are present in the supermarket such as other customers to limit distractibility. Only soft acoustic ambiance music is heard during the task to limit noise in the environment. Visual and auditory sources of interference are thus limited. The shelves are organized classically and display drinks, grocery products, cold cuts, textiles, fruits and vegetables, bakery products, and household items. Four check-out counters are also represented.

Figure 2.

Screenshots of four products displayed at each stop of trolley in the four aisles of the supermarket virtual environment.

The participant enters the supermarket behind the trolley as if pushing it, so he/she experiences the virtual environment from a first-person perspective. Task-related instructions are given verbally by the experimenter as the task progresses. In the learning phase, the patient is instructed to perform a memorization task by purchasing a list of 16 products one by one. The task is to select and name aloud the picture of the correct product (e.g. mirabelle) among four pictures (e.g. mirabelle, caramel, tea towel, and plaster) located at the bottom of the screen with their name written below. The patient selects the product by pointing to the touchscreen after hearing a self-question spoken by a synthetic voice (e.g. what do I need to make a pie?) (Appendix 1). These questions promote semantic encoding. The 16 products to be learned are presented four at a time. After all four products have been identified correctly, the pictures are removed, and immediate cued recall of these four products is tested by presenting the self-questions again. The patient is reminded of any product he/she failed to retrieve by presenting the cue and the item again. This encoding phase allows the memorization of the 16 familiar products to be controlled. Once immediate recall of a group of four products has been completed, the next set of products is presented for learning. If the product selected is the correct one regarding the self-question, the product goes into the trolley. Selection is considered incorrect if the participant chooses a product that does not correspond to the logical answer to the self-question.

The learning phase is followed by three recall trials, each preceded by 20 s of counting backwards. Each recall trial consists of two parts: first, a 2-min period to retrieve as many products as possible spontaneously (i.e. “free recall”); then semantic cues (questions) (i.e. “cued recall”) are provided for those products not retrieved in free recall. If the subject fails to retrieve the product with the semantic cue, they are reminded of it by receiving a cue about the product again. Twenty-minute interference non-verbal tasks separate the free and cued recall, and a delayed recall phase (again free followed by cued recall). In addition, a 2-min period to retrieve as many products as possible spontaneously (i.e. “free delayed recall”) followed by a semantic cue (questions) (i.e. “cued delayed recall”) are provided for those products not retrieved in free recall. The recall phases are performed in front of the avatar of a check-out counter. Participants are instructed to recall as many products as possible in the order in which they came to mind.

The experimenter has a specific separate interface on a laptop that allows him/her to run the task, monitor its progress and to facilitate the transcription of oral responses by selecting pictorial representations of recalls. The 16 semantically associated distractors (i.e. distractors that match the targets by taxonomic category) and the 16 neutral distractors are selected, excluding stereotypical items from categories (e.g. apple for fruit, to reduce the likelihood of simply guessing the correct item based on its semantic category), and according to the length and frequency of usage in the French language based on the Brulex frequency dictionary⁴¹ (Appendix 1).

The outcome measures in MEMOSHOP are the same as those in the FCSRT: total free recall (Sum of free recall from the three trials; range 0–48), total cued recall (Sum of cued recall from the three trials, range 0–48), free delayed recall (range 0–16) and total delayed recall (sum of free delayed recall + cued delayed recall, range 0–16). Finally, correct responses, repetitions and intrusions during free recall or cued recall for both tests are analyzed.

Statistical analyses

Descriptive analyses

Quantitative variables were expressed as Mean ± Standard Deviation (SD), and qualitative variables were expressed as relative frequency.

Comparative analyses

Univariate analyses with t-test comparisons for continuous variables or Chi² test for categorical variables were used to compare demographical and neuropsychological characteristics, and shopping habits in participants with isolated cognitive complaints or MCI, with those characterized as cognitively normal. Non-parametric correlations (Kendall's rank correlations, two-sided) were computed on correct responses to verify the association between MEMOSHOP and FCSRT scores.

Variables (correct responses in free and cued recall, and repetitions and intrusions in free recall) were analyzed with non-parametric tests (Mann–Whitney U test for independent samples test, Friedman test for repeated measures and Wilcoxon-matched pairs signed-ranks test). The Mann–Whitney U test was used to compare the two independent groups (SCC/MCI versus control groups) on recalls in the MEMOSHOP and in the FCSRT.

The Friedman test was used to compare recalls in the MEMOSHOP and in the FCSRT.

The Wilcoxon-signed rank test was used to compare MEMOSHOP versus FCSRT performance by recalls.

Receiver operating characteristic (ROC) curves⁴² were computed to compare the diagnostic performances of MEMOSHOP in comparison with FCSRT in patients with isolated SCCs or MCI, and controls. The ROC curve is a graphic representation of the relation between the sensitivity and the specificity of a test, calculated for all possible thresholds. It reflects the capacity of a test to correctly classify subjects into clinically relevant subgroups (e.g. diseased/healthy). ROC curves make it possible to complete the area under the ROC curve (AUC) with 95 percent confidence intervals [CI] and sensitivity/specificity. Sensitivity refers to the probability that a test result is positive when the disorder is present (i.e. true positive). Specificity refers to the probability that a test result is negative when the disorder is not present (i.e. true negative).

The alpha risk threshold was set at p = .05. All statistical analyses were performed using the SPSS statistical software package (PASWR Statistics 18).

Results

Population

Twenty-nine patients with isolated SCCs or MCI enrolled from the MEMENTO cohort (71.0 ± 7, range: 58–85 years; 21 females; 12.4 ± 3.8 years of educational level) and 29 matched controls (68.1 ± 4.4, range: 58–77 years; 21 females; 11.8 ± 4.2 years of educational level) examined as cognitively normal were recruited. Concerning cognitive status of the patients, six patients had amnestic MCI (1 single-domain amnestic MCI), 14 patients had non-amnestic MCI (11 single-domain non-amnestic MCI) and nine patients had isolated cognitive complaints.

Regarding the baseline neuropsychological evaluation, the CDR score was higher in patients with isolated cognitive complaints or MCI than in controls (0.40 ± 0.21 vs. 0.01 ± 0.09, p < .001). By contrast, patients with isolated SCC or MCI did not differ from cognitively normal controls on MMSE score (28.1 ± 1.5 vs. 28.2 ± 1.6, NS). No difference was observed between both groups on TMT or FCSRT, except for delayed recall (15.6 ± 1.2 vs. 12.3 ± 2.5, p < .001) (Table 1).

Table 1.

Neuropsychological characteristics (Mean ± SD) in patients with isolated subjective cognitive complaints or mild cognitive impairment, and in cognitively normal controls.

	Patients with isolated cognitive complaints or mild cognitive impairment (n = 29)	Controls (n = 29)	p-value
Neuropsychological scores
Mini-mental state examination (MMSE)	28.1 ± 1.5	28.2 ± 1.6	ns
≥24	100%	100%	ns
Free and cued selective reminding test (FCSRT)
Free recall	29.1 ± 6.8	31.5 ± 6.9	ns
Total recall	45.3 ± 5.4	46.1 ± 2.5	ns
Delayed recall	15.6 ± 1.2	12.3 ± 2.5	<.001
Trail-making test (TMT)
TMT-B (RTs)	96.9 ± 40.8	107.1 ± 79.0	ns
TMT-B (correct)	22.5 ± 4.3	22.4 ± 2.4	ns
TMT interference index (RTs)	0.5 ± 0.1	0.4 ± 0.2	ns
Clinical dementia rating (CDR)	0.40 ± 0.21	0.01 ± 0.09	<.001

Abbreviations. SD = Standard Deviation; n = Effectives.

All participants fully completed the MEMOSHOP and FCSRT. Regarding self-reported shopping habits (Table 2), patients with isolated SCCs or MCI were as accustomed to shopping at the supermarket and autonomously as much as the controls (96.6% vs. 96.6% and 89.7% vs. 89.7%). Patients with isolated SCCs or MCI tended to go shopping less frequently [Chi² (2) = 5.44, p = .06]. They also went less frequently to supermarket chains (79.3% vs. 100%) [Chi² (1) = 6.69, p < .01] than controls.

Table 2.

Percentages (and frequencies) of responses in patients with isolated cognitive complaints or mild cognitive impairment and in controls on the questionnaire on shopping habits (French translation in italics).

	Patients with isolated cognitive complaints or mild cognitive impairment (n = 29)	Controls (n = 29)	Chi² test p-value
Questionnaire on shopping habits
Avez-vous l’habitude de faire vos courses?
Do you usually go shopping?
Yes	96.6% (28)	96.6% (28)	NS
No	3.4% (1)	3.4% (1)
A quelle fréquence allez-vous dans un supermarché?
How often do you go to a supermarket?
Several times a week	37.9% (11)	62.1% (18)	p = .06
Once a week	41.4% (12)	34.5% (10)
Less than once a week	20.7% (6)	3.4% (1)
Avez-vous l’habitude de faire vos courses dans un supermarché?
Do you usually do your shopping in a supermarket?
Yes	79.3% (23)	100% (29)	p < .01
No	20.7% (6)	0% (0)
Avez-vous l’habitude de faire vos courses seul?
Do you usually do your shopping alone?
Yes	89.7% (26)	89.7% (26)	NS
No	10.3% (3)	10.3% (3)

Construct validity analyses

Table 3 summarizes the results of correlations on “Correct responses” scores during free recall and cued recalls between MEMOSHOP and FCSRT. Figure 3 represents the number of correct responses for MEMOSHOP versus FCSRT in patients in the function of free recalls.

Figure 3.

Number of correct responses for MEMOSHOP versus FCSRT in patients with isolated cognitive complaints or mild cognitive impairment (SCC) versus controls in function of free recalls (free recalls 1, 2, 3 and free delayed recall). *p < .05. **p < .01.

Figure 4.

Receiver Operating Characteristic (ROC) curves of MEMOSHOP and FCSRT for correct responses on Total cued recall (Sum of cued recall from the three trials). Memoshop: AUC = 0.686 (95% CI [0.549–0.823]), p < .05. FCSRT: AUC = 0.554 (95% CI [0.403–0.704]), NS. AUC: area under the curve.

Table 3.

Association between MEMOSHOP and FCSRT scores with Kendall's rank correlations on correct responses in participants.

MEMOSHOP versus FCSRT scores	Kendall's Tau	N	p-value
Free recall 1	= 0.263	58	<.01
Free recall 2	= 0.504	58	<.01
Free recall 3	= 0.512	58	<.01
Cued recall 1	= 0.089	58	NS
Cued recall 2	= 0.337	55	<.01
Cued recall 3	= 0.321	48	<.01
Free delayed recall	= 0.447	58	<.01
Cued delayed recall	= 0.379	42	<.01
Total free recall	= 0.478	58	<.01
Total cued recall	= 0.350	58	<.01

Correct responses

For free recall, patients with isolated cognitive complaints or MCI obtained lower number of correct responses than controls for recall 1 (Mann–Whitney U test, Z = −2.55, p < .05, η²= 0.11), recall 3 (Mann–Whitney U test, Z = −2.81, p < .01, η²= 0.14) and delayed recall (Mann–Whitney U test, Z = −2.36, p < .05, η²= 0.10) in the MEMOSHOP, and for recall 3 (Mann–Whitney U test, Z = −2.02, p < 05, η²= 0.07) in FCSRT (Figure 3).

Participants obtained higher number of correct responses from recall 1 to delayed recall in the MEMOSHOP (Friedman test, Chi²(3) = 103.98, p < .001, η²= 0.19) and in the FCSRT (Friedman test, Chi²(3) = 90.34, p < .001, η²= 0.19).

Participants obtained higher number of correct responses in MEMOSHOP compared to FCSRT for recall 1 (Wilcoxon test, Z = 3.82, p < .001, η²= 0.25), for recall 2 (Wilcoxon test, Z = 5.68, p < .001, η²= 0.56), for recall 3 (Wilcoxon test, Z = 5.31, p < .001, η²= 0.49), and for delayed recall (Wilcoxon test, Z = 6.49, p < .001, η²= 0.73).

For cued recall, patients with isolated cognitive complaints or MCI obtained higher number of correct responses than controls only for recall 1 (Mann–Whitney U test, Z = 1.99, p < .05, η²= 0.07) in the MEMOSHOP.

Participants obtained lower number of correct responses from recall 1 to delayed recall in the MEMOSHOP (Friedman test, Chi²(3) = 54.72, p < .001, η²= 0.22) and in the FCSRT (Friedman test, Chi²(3) = 66.06, p < .001, η²= 0.20).

Participants obtained lower number of correct responses in MEMOSHOP compared to FCSRT for recall 1 (Wilcoxon test, Z = 2.57, p < .05, η²= 0.11), for recall 2 (Wilcoxon test, Z = 4.67, p < .001, η²= 0.38), for recall 3 (Wilcoxon test, Z = 3.87, p < .001, η²= 0.26), and for delayed recall (Wilcoxon test, Z = 5.67, p < .001, η²= 0.55).

Repetitions

For free recall, no difference was observed between patients with isolated cognitive complaints or MCI and controls on number of repetitions in recalls of the MEMOSHOP or FCSRT.

Participants obtained lower number of repetitions at recall 1 compared to other recalls in the MEMOSHOP (Friedman test, Chi²(3) = 12.67, p < .01, η²= 0.12) and in the FCSRT (Friedman test, Chi²(3) = 9.91, p < .05, η²= 0.13).

No difference was observed on number of repetitions in MEMOSHOP compared to FCSRT for recall 1, for recall 2, for recall 3, and for delayed recall.

Intrusions

For free recall, no difference was observed between patients with isolated cognitive complaints or MCI and controls on number of intrusions in recalls of the MEMOSHOP or FCSRT.

No difference was observed for number of intrusions from recall 1 to delayed recall in the MEMOSHOP and in the FCSRT.

No difference was observed on number of intrusions in MEMOSHOP compared to FCSRT for recall 1, for recall 2, for recall 3, and for delayed recall.

Diagnostic performances (ROC curves)

For correct responses on Total free recall (Sum of free recalls 1, 2, and 3), ROC analyses showed an Area Under the Curve (AUC) of 0.301 (95% CI [0.167–0.435]), p < .01 for MEMOSHOP, and 0.420 (95% CI [0.271–0.568]), NS for FCSRT.

For correct responses on total cued recall (sum of cued recall from the three trials) (Figure 4), ROC analyses showed an AUC of 0.686 (95% CI [0.549–0.823]), p < .05 for MEMOSHOP and 0.554 (95% CI [0.403–0.704]), NS for FCSRT. ROC curves of correct responses on total cued recall for MEMOSHOP and FCSRT are shown in Figure 4.

For correct responses on free delayed recall, ROC analyses showed an AUC of 0.319 (95% CI [0.179–0.459]), p < .05 for MEMOSHOP and 0.431 (95% CI [0.283–0.579]), NS for FCSRT.

For correct responses on total delayed recall (Sum of free delayed recall + cued delayed recall), ROC analyses showed an AUC of 0.398 (95% CI [0.252–0.545]), NS for MEMOSHOP and 0.548 (95% CI [0.398–0.697]), NS for FCSRT.

Discussion

MEMOSHOP is an innovative VR device designed for clinical use in the assessment of episodic memory in patients suffering from cognitive impairment in conditions close to everyday life (i.e. shopping at the supermarket).

First, results show its feasibility to examine episodic memory in older patients with memory impairment in preclinical and early disease. MEMOSHOP demonstrated association and adequate convergent validity compared to the gold standard neuropsychological test (FCSRT) to evaluate episodic memory in patients with isolated SCCs or MCI. Patients with isolated SCCs or MCI had lower performance than controls on the successive free recalls on both the MEMOSHOP and FCSRT and benefitted as much as controls from the interspersed cued recalls. Both tests show that patients with isolated SCCs or MCI present a retrieval deficit but preserved encoding and storage components, as expected in normal aging. Both tasks showed a benefit of cued recall over the course of the successive free recalls (i.e. existence of a learning curve for the number of recalled words). Both documented the efficient functioning of appropriate retrieval operations in patients with subtle episodic memory impairment, as also evidenced by the low repetition and intrusion rate.

Higher performances were globally observed with MEMOSHOP than with FCSRT. It is well known that the more in-depth information is processed, the stronger and more lasting is the memory trace (i.e. information encoding).¹⁸ In MEMOSHOP, double encoding (i.e. verbal and imagery) takes place during the learning phase. According to the dual coding theory,⁴³ information which undergoes double coding (e.g. verbal and image) facilitates recall better than information than has only been coded verbally. Performance is explained in terms of dynamic associative processes that operate in a rich network of modality-specific verbal and non-verbal (or imagery) representations.

Second, ROC curves showed a better diagnostic performance for isolated SCCs/MCI patients for the MEMOSHOP compared to FCSRT based on the performance of cued recall. Although the trends with the FCSRT and MEMOSHOP are identical, the latter seems easier to perform in terms of number of words recalled. Furthermore, MEMOSHOP better differentiates patients with isolated cognitive complaints or MCI from healthy controls based on cued recall performance. In fact, it provides a more precise overview of impaired or relatively preserved memory capacities than the FCSRT. As MEMOSHOP is easier to perform than FCSRT, impaired performances are more noticeable and allow the early dysfunctional mechanisms during storage and retrieval to be identified. This is crucial since memory is fundamental for activities of daily living. However, this result should be interpreted with caution given the not excluded lack of statistical power in this study.

Results also show that shopping habits seem to evolve early with memory impairment, patients with isolated memory complaints or MCI seeming to prefer shopping in smaller convenience stores to supermarket chains compared to cognitively normal controls. This putative relationship between memory abilities and adaptive behavioral/functional habits underlines the added value of assessing the elderly by using more natural/ecological tasks than paper-and-pencil ones. Further studies are required to verify whether MEMOSHOP measurements better correlate with measures of activities of daily living and functional problems in the elderly than the FCSRT. Moreover, in addition to the advantages of computerized batteries compared to traditional paper-and-pencil neuropsychological tests,^7,8,10,11 VR applications allow cognitive processes to be tested in an attractive, interactive, and meaningful environment closer to real-life settings.^12–14

MEMOSHOP could have practical applications in memory clinics. Dementia remains largely underdiagnosed,⁴⁴ and it is critical to diagnose it early so that action can be taken to limit its advance.^3–5 Efficient sensitive digital tools based on standardized tests will be needed for longitudinal evaluations of large elderly populations in the future. Demonstrating the validity of MEMOSHOP for assessing memory in patients with early-stage MCI is just the first step. It could be integrated into neuropsychological practice to standardize cognitive healthcare and to improve the early detection, assessment, management, and follow-up of cognitive decline in older adults in memory clinics. Further studies are needed to test the discriminant validity of MEMOSHOP for evaluation of individual's cognitive status such as Alzheimer's disease.

While MEMOSHOP could become an innovative tool for assessing memory decline and episodic memory in memory clinics, its use could be extended to the cognitive rehabilitation of memory strategies in everyday life in patients suffering from memory complaints. It probably has the potential to be used in cognitive and behavioral interventions, allowing the transfer of acquired strategies and improving functioning in daily life in older adults with memory deficiencies.

To overcome the limitations of our study due to its small sample size and conceivable lack of statistical power, additional studies with larger sample sizes, a follow-up period and specific-memory deficits are required. Future research will also be needed to calibrate the device in terms of the number of items to be encoded in order to come closer to real-life conditions.

Conclusions

In conclusion, MEMOSHOP is a reliable tool for evaluating episodic memory in older adults suffering from slight cognitive impairment. It has ecological relevance since it assesses memory by focusing on a familiar activity of daily living. Its ease of use and the present findings show that it holds promise for assessing older people at risk of cognitive decline.

Footnotes

Acknowledgments

We thank the following clinical research assistants from CHU de Bordeaux, F-33000 Bordeaux, France: Muriel Bacarisse, Cédric Valtat, Aurélien Boiseau and Victor Bibène for selecting participants, collecting, and monitoring data, and for providing administrative, technical and logistic assistance. We thank Emmanuel Orvain from Immersion and Xue Chen Cao from University of Bordeaux for their participation in the development of the virtual environment. We thank: CIC 1401-EC, Bordeaux University Hospital, Inserm and the University of Bordeaux for supporting this work.

Contributorship

PS, HA, JFD, JT, and PP participated in the study concept and design. PS, HA, JO, JBR, and CC participated in the conception and development of the VR tool. PS, JT, PP, and HA participated in data acquisition. PS, JT, PP, HA, and JFD participated in data analysis and interpretation. PS, JT, PP, and HA participated in writing the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors report no disclosures relevant to the manuscript. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Ethical approval

The ethics committee “Comité de Protection des Personnes [CPP] Sud-Ouest et Outre Mer III” approved this study. The study was registered as a clinical trial (ClinicalTrials.gov identifier: NCT01650454).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported by a grant from ANR (Agence Nationale de la Recherche) SCOAL – AAP MALZ 2011, by the grant EQUIPEX PHENOVIRT ANR-10-EQPX-12–01 funded by the French Ministry of Research (Equipements d’Excellence, Investissements d’Avenir), and by the grant LABEX BRAIN ANR-10-LABX-43 and LABEX BRAIN ANR-10-IDEX-03-02. The MEMENTO cohort was sponsored by the Fondation Plan Alzheimer.

Guarantor

PS.

ORCID iD

Patricia Sagaspe

Appendix 1.

Versions (version 1 and 2) of MEMOSHOP with 16 target products of the shopping list for memorization task (with Brulex frequencies between brackets and French translation in italics), with 16 semantically associated distractors (corresponding to both versions) and 16 neutral distractors for recognition task (48 items).

Version 1	16 target products	16 semantically associated distractors	16 neutral distractors
Version 2	16 semantically associated distractors	16 target products	16 neutral distractors
De quoi ai-je besoin pour le goûter?	boudoir (212)	madeleine (195)	cadenas (140)
What do I need for afternoon snack?	boudoir	madeleine	lock
De quoi ai-je besoin pour boire avec le dessert?	champagne (1076)	cidre (285)	guitare (433)
What do I need to drink with dessert?	champagne	cider	guitar
De quoi ai-je besoin pour relever mes plats?	poivre (382)	moutarde (280)	hameçon (170)
What do I need to spice up my meal?	pepper	mustard	fishhook
De quoi ai-je besoin pour faire mon repas de dimanche?	gigot (280)	escalope (68)	verveine (140)
What do I need to prepare my meal for Sunday?	leg	escalope	verbena

De quoi ai-je besoin pour me protéger du mauvais temps?	parapluie (1025)	écharpe (957)	pendule (1540)
What do I need to protect myself against bad weather?	umbrella	scarf	pendulum
De quoi ai-je besoin pour mon petit déjeuner?	confiture (655)	biscotte (55)	biberon (178)
What do I need for my breakfast?	jam	rusk	baby bottle
De quoi ai-je besoin pour préparer ma soupe?	poireau (174)	oignon (604)	ventouse (182)
What do I need to prepare my soup?	leek	onion	suction cup
De quoi ai-je besoin pour marcher confortablement?	semelle (995)	pantoufle (625)	robinet (587)
What do I need for comfortable walking?	shoe sole	slipper	tap

De quoi ai-je besoin pour pouvoir proposer comme friandise aux enfants?	caramel (144)	réglisse (72)	camembert (63)
What do I need to be able to offer candy to kids?	caramel	licorice	camembert
De quoi ai-je besoin pour faire une tarte?	mirabelle (89)	framboise (208)	thermos (106)
What do I need to make a pie?	mirabelle	raspberry	thermos
De quoi ai-je besoin pour compléter ma trousse de premiers secours?	pansement (604)	coton (1118)	bouquin (923)
What do I need to complete my first aid kit?	plaster	cotton	book
De quoi ai-je besoin pour renouveler mon linge de cuisine?	torchon (378)	tablier (1591)	épingle (1395)
What do I need to renew my kitchen linen?	tea towel	apron	pin

De quoi ai-je besoin pour ranger mes papiers?	agraphe (195)	trombone (93)	truelle (72)
What do I need to store my personal documents?	staple	paper clip	trowel
De quoi ai-je besoin pour changer l'éclairage de ma cuisine?	ampoule (731)	néon (63)	paillasson (195)
What do I need to change the light in my kitchen?	bulb	neon	doormat
De quoi ai-je besoin pour faire cuire mes aliments?	casserole (735)	marmite (629)	guirlande (553)
What do I need to cook my food?	pan	cooking pot	garland
De quoi ai-je besoin pour faire mon ménage?	éponge (638)	balai (723)	entonnoir (276)
What do I need to clean up?	sponge	broom	funnel

References

Niu

Alvarez-Alvarez

Guillen-Grima

, et al. Prevalence and incidence of Alzheimer's disease in Europe: a meta-analysis. Neurologia 2017; 32: 523–532.

Barberger-Gateau

Fabrigoule

Rouch

, et al. Neuropsychological correlates of self-reported performance in instrumental activities of daily living and prediction of dementia. J Gerontol B Psychol Sci Soc Sci 1999; 54: P293–P303.

Ladenbauer

Kulzow

de Boor

, et al. Promoting sleep oscillations and their functional coupling by transcranial stimulation enhances memory consolidation in mild cognitive impairment. J Neurosci 2017; 37: 7111–7124.

Kivipelto

Mangialasche

Ngandu

. Lifestyle interventions to prevent cognitive impairment, dementia and Alzheimer disease. Nat Rev Neurol 2018; 14: 653–666.

Rodakowski

Saghafi

Butters

, et al. Non-pharmacological interventions for adults with mild cognitive impairment and early stage dementia: an updated scoping review. Mol Aspects Med 2015; 43–44: 38–53.

Portet

Ousset

Visser

, et al. Mild cognitive impairment (MCI) in medical practice: a critical review of the concept and new diagnostic procedure. Report of the MCI working group of the European consortium on Alzheimer's disease. J Neurol Neurosurg Psychiatry 2006; 77: 714–718.

Parsey

Schmitter-Edgecombe

. Applications of technology in neuropsychological assessment. Clin Neuropsychol 2013; 27: 1328–1361.

Zygouris

Tsolaki

. Computerized cognitive testing for older adults: a review. Am J Alzheimers Dis Other Demen 2015; 30: 13–28.

Sagaspe

Lespinet-Najib

Poulette

, et al. Feasibility of the cognitive assessment of nursing-home residents with mild-to-moderate cognitive impairment using the intelligent voice-guided digital assistant THALIE: a pilot study. Gerontechnology 2020; 19: 1–9.

10.

Schatz

Browndyke

. Applications of computer-based neuropsychological assessment. J Head Trauma Rehabil 2002; 17: 395–410.

11.

Fillit

Simon

Doniger

, et al. Practicality of a computerized system for cognitive assessment in the elderly. Alzheimers Dement 2008; 4: 14–21.

12.

Liu

Tan

Chen

, et al. A review of the application of virtual reality technology in the diagnosis and treatment of cognitive impairment. Front Aging Neurosci 2019; 11: 280.

13.

La Corte

Sperduti

Abichou

, et al. Episodic memory assessment and remediation in normal and pathological aging using virtual reality: a mini review. Front Psychol 2019; 10: 173.

14.

Zygouris

Ntovas

Giakoumis

, et al. A preliminary study on the feasibility of using a virtual reality cognitive training application for remote detection of mild cognitive impairment. J Alzheimers Dis 2017; 56: 619–627.

15.

Tulving

. Episodic and semantic memory. In Organ mem 1. New York, London: Academic Press, 1972, pp. 381–403.

16.

Buschke

. Cued recall in amnesia. J Clin Neuropsychol 1984; 6: 433–440.

17.

Grober

Buschke

. Genuine memory deficits in dementia. Dev Psychol 1987; 3: 13–36.

18.

Craik

FIM

Lockhart

. Levels of processing: a framework for memory research. J Verbal Learning Verbal Behav 1972; 11: 671–684.

19.

Tulving

Thomson

. Encoding specificity and retrieval processes in episodic memory. Psychol Rev 1973; 80: 352–373.

20.

Van der Linden

Coyette

Poitrenaud

, et al. L’épreuve de rappel libre/rappel indicé à 16 items (RL/RI-16). In: Van der Liden

Adam

Agniel

Baisset Mouly

, et al. (eds), L’évaluation des troubles de la mémoire: Présentation de quatre tests de mémoire épisodique (avec leur étalonnage). Marseille: Solal. 2004.

21.

Amieva

Carcaillon

Rouze L’Alzit-Schuermans

, et al. Cued and uncued memory tests: norms in elderly adults from the 3 cities epidemiological study. Rev Neurol (Paris) 2007; 163: 205–221.

22.

Ouellet

Boller

Corriveau-Lecavalier

, et al. The virtual shop: a new immersive virtual reality environment and scenario for the assessment of everyday memory. J Neurosci Methods 2018; 303: 126–135.

23.

Corriveau Lecavalier

Ouellet

Boller

, et al. Use of immersive virtual reality to assess episodic memory: a validation study in older adults. Neuropsychol Rehabil 2020; 30: 462–480.

24.

Parsons

Barnett

. Validity of a newly developed measure of memory: feasibility study of the virtual environment grocery store. J Alzheimers Dis 2017; 59: 1227–1235.

25.

Sauzeon

Arvind Pala

Larrue

, et al. The use of virtual reality for episodic memory assessment: effects of active navigation. Exp Psychol 2011; 59: 99–108.

26.

Plancher

Tirard

Gyselinck

, et al. Using virtual reality to characterize episodic memory profiles in amnestic mild cognitive impairment and Alzheimer's disease: influence of active and passive encoding. Neuropsychologia 2012; 50: 592–602.

27.

Allain

Foloppe

Besnard

, et al. Detecting everyday action deficits in Alzheimer’s disease using a nonimmersive virtual reality kitchen. J Int Neuropsychol Soc 2014; 20: 468–477.

28.

Rizzo

Gambino

Sardo

, et al. Being in the past and perform the future in a virtual world: vR applications to assess and enhance episodic and prospective memory in normal and pathological aging. Front Hum Neurosci 2020; 14: 297.

29.

Abichou

La Corte

Piolino

. Does virtual reality have a future for the study of episodic memory in aging? Geriatr Psychol Neuropsychiatr Vieil 2017; 15: 65–74.

30.

Werner

Rabinowitz

Klinger

, et al. Use of the virtual action planning supermarket for the diagnosis of mild cognitive impairment: a preliminary study. Dement Geriatr Cogn Disord 2009; 27: 301–309.

31.

Zygouris

Giakoumis

Votis

, et al. Can a virtual reality cognitive training application fulfill a dual role? Using the virtual supermarket cognitive training application as a screening tool for mild cognitive impairment. J Alzheimers Dis 2015; 44: 1333–1347.

32.

Dauphinot

Bouteloup

Mangin

, et al. Subjective cognitive and non-cognitive complaints and brain MRI biomarkers in the MEMENTO cohort. Alzheimers Dement (Amst) 2020; 12: e12051.

33.

Taillard

Sagaspe

Berthomier

, et al. Non-REM sleep characteristics predict early cognitive impairment in an aging population. Front Neurol 2019; 10: 197.

34.

Petersen

. Mild cognitive impairment as a diagnostic entity. J Intern Med 2004; 256: 183–194.

35.

Morris

. The clinical dementia rating (CDR): current version and scoring rules. Neurology 1993; 43: 2412–2414.

36.

Folstein

McHugh

. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12: 189–198.

37.

Hugonot-Diener

. MMS Version consensuelle GRECO. In: La consultation en gériatrie. Paris: Masson, 2001, pp. 13–20.

38.

Van der Linden

Adam

Agniel

Membres du

GREMEM

(eds). L’évaluation des troubles de la mémoire. Marseille: Solal, 2004.

39.

Grober

Buschke

Crystal

, et al. Screening for dementia by memory testing. Neurology 1988; 38: 900–903.

40.

Tombaugh

. Trail making test A and B: normative data stratified by age and education. Arch Clin Neuropsychol 2004; 19: 203–214.

41.

Content

Mousty

Radeau

MBRULEX

. Une base de données lexicales informatisée pour le français écrit et parlé. BRULEX: a computerized lexical data base for the French language. L’Année Psychologique 1990; 90: 551–566.

42.

Zweig

Campbell

. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem 1993; 39: 561–577.

43.

Paivio

. Mental representations: A dual coding approach. Oxford: Oxford University Press, 1986.

44.

Waldemar

Phung

Burns

, et al. Access to diagnostic evaluation and treatment for dementia in Europe. Int J Geriatr Psychiatry 2007; 22: 47–54.