Sage Journals: Discover world-class research

Abstract

Objectives

This work has developed a modified mental state assessment tool for impact analysis of therapeutic interventions for patients with cognitive impairment. This work includes a pilot study to validate the proposed tool and assess the impact of virtual reality-based interventions on patient well-being, which includes assessment of cognitive ability and mood.

Methods

The suggested tool’s robustness and reliability are assessed in care home facilities with elderly residents over the age of 55. Because of the repetitive nature of the pilot study, test-retest strategy for Cronbach’s alpha coefficient is employed to validate the internal consistency of the proposed tool over time. Qualitative and quantitative analyses are performed on the collected data to draw inferences on the impact of virtual reality-based interventions on patients with cognitive impairments.

Results

The Cronbach’s alpha coefficient value shows that the proposed tool’s resilience is comparable to that of its pre-intervention counterparts. The Cronbach’s alpha coefficient values are determined for Pre-virtual reality and Post-virtual reality interventions, which include 116 virtual reality sessions for 52-participant, and three cohorts of virtual reality sessions for 21 participants. These values for a majority of the interventions remained within the acceptable range of 0.6–0.8.

Conclusions

The proposed modified mental state assessment tool is observed to be a reliable tool for investigating the impact of virtual reality-based interventions on patients with cognitive impairments. One of the notable significance of the proposed tool is that this allows for resource allocation for such interventions to be tailored to the needs of the patient, leading to greater therapeutic efficacy and resource efficiency.

Keywords

Digital health virtual reality mini-mental state examination virtual reality for social care multi-sensory immersive environment

Introduction

Cognitive impairment is a condition in which an individual struggles with activities that involve remembering, learning and focusing to make decisions in everyday life. In a study involving more than 1.3 m adults within the age group 65–99 years, it was observed that the number of new cases demonstrating cognitive decline in the year 2009 was 1 in 1000, which increased to 3 in every 1000 adults by the year 2018.¹ The cost implications of cognitive impairment can be estimated from the fact that the care costs associated with only dementia patients, who form a fraction of total numbers with cognitive impairment, is as much as 5bn.² Although the primary risk factor for cognitive impairment is age, other risk factors such as brain injury, stroke, diabetes, heart disease, physical inactivity, family history and chronic conditions like Parkinson’s, are also significant contributors.³

It is a well-known fact that finding ways to relate can have a significant impact on people demonstrating cognitive decline, which is the reason why sensory stimulation therapy is known to be extremely successful in such cases.⁴ One of the methods used for sensory stimulation include the use of everyday sounds, objects and environments to awaken senses and generate a positive response. Among emerging technologies, recent advances in multi-sensory interventions are rapidly gaining traction in exposure therapy, presenting an unprecedented opportunity to support isolated population (due to frailty, social exclusion and limited mobility). Technologies such as virtual reality (VR) immerse the users into fascinating virtual environments enabling multi-sensory stimulation. However, in order to bridge the gap between technological development and clinical adoption, robust assessment and impact analysis tools are required. Moreover, reliable impact assessment can assist in treatment planning and enhance resource efficiency as it shall provide information to the therapist about the current stage of impairment and patient’s response to intervention.

Impact analysis of these technologies is restricted to mood analysis and self-reporting tools.^5,6 There are evident limitations associated with the use of such assessment tools, which include bias as a result of participant’s recent experiences. For instance, a participant may have had a doctor’s visit prior to coming for the intervention session. In this case, the participant’s mood will be skewed on the basis of his or her experience at the medical facility. Therefore, there is a need for a comprehensive assessment tool that takes into account the participant’s mental state, which shall be assessed using a score determined from participant’s responses to a set of questions. These questions must assess the participant’s mental state in a multi-dimensional fashion taking into account cognitive functions such as attention and recall. Such a tool, named mini-mental state examination (MMSE), is conventionally used for cognitive screening in elderly.

This work modifies the standard MMSE to provide questionnaires that can be used prior and post intervention. Cumulatively, this set of questionnaires is referred to as modified mental state assessment tool, which is a combination of pre and post-multi-sensory intervention questionnaires. Evaluation of the proposed tool is performed using a pilot study, which uses Nature Treks VR for intervention. The participant group chosen for the study are elderly in care home settings. The data collected during the pilot study is subject to multiple statistical instruments for validating the proposed tool. Therefore, the key contributions of this paper are as follows:

Proposed a Mental State Assessment Tool by making appropriate modifications to the existing MMSE tool to suit the context in terms of demographic location of the study and its use for impact analysis of technology-based therapeutic interventions in patients with cognitive impairments.

In order to analyse the impact of the proposed tool, a pilot study was designed, where participants, chosen from care homes based in Shropshire, West Midlands, UK, were exposed to virtual environments from Nature Treks VR. The proposed tool was used to assess the impact of the exposure. Moreover, additional observations from caregivers and researchers were also recorded to cross-validate the results.

An extensive experimental analysis was conducted and the findings reveal that the tool is robust with the Cronbach’s alpha coefficient for both the pre- and post-intervention questionnaires above the benchmark value of 0.7. Moreover, quantitative and qualitative analyses of the collected data validate the effectiveness of the proposed tool.

This paper proposes a modified mental state assessment tool, which is a combination of pre- and post-intervention questionnaires. Both these questionnaires are modified versions of MMSE. Details of these modifications are provided in Proposed Mental State Assessment Tool. In order to evaluate the proposed tool, a pilot study was designed, as illustrated in Design of Pilot Study. Quantitative and qualitative data collected from the pilot study was analysed to prove the robustness, usability and effectiveness of the proposed tool. The tools used in this study for this purpose are elaborated in Measures for Evaluation of Proposed Mental State Assessment Tool.

Quantitative analysis was performed to establish correlation between VR sessions and participant well-being by analyzing the changes in MMSE and mood scores prior and after the VR sessions. The formulated null and alternate hypotheses for analysis of changes in MMSE scores are as follows -

Null Hypothesis $(H_{0})$ : There is no difference in mean pre- and post-intervention MMSE scores. Alternate Hypothesis $(H_{1})$ : There is a difference in mean pre- and post-intervention MMSE scores.

The formulated null and alternate hypotheses for analysis of changes in mood scores are as follows -

Null Hypothesis $(H_{0})$ : There is no difference in mean pre- and post-intervention mood scores. Alternate Hypothesis $(H_{1})$ : There is a difference in mean pre- and post-intervention mood scores.

The organization of this paper is as follows: Literature Review presents an in-depth analysis outlining the existing state-of-the-art multi-sensory technologies used for therapeutic interventions in the elderly population. Moreover, Literature Review discusses the assessment tool utilized in the literature to measure the impact of such interventions, particularly in care home settings. Research Design and Methods provides details of the experimental methodology, pilot study design and tools used for impact assessment of interventions. Results and Discussion provides the experimental results along with details of the analytical methods used for quantitative and qualitative analysis performed for cross-validation of results. Besides this, it also provides a discussion on the limitations of the study. Finally, Conclusion summarizes the key findings and discusses the future direction.

Background

This section is divided into the following subsections, each of which provides more information on different aspects, such as the state-of-the-art in technological support in care home settings, VR in a care home setting, tools for assessing the impact of VR and MMSE tools for elderly care.

State-of-the-art technological support for elderly care

Digital technologies have the ability to monitor and promote mental health across all age groups, and this recognition is growing.⁷ For instance, adopting digital tools for ecological momentary assessment or experience sampling of mood^8,9 as well as recording GPS data to track mental health using technology¹⁰ are becoming more and more common. Additionally, an increasing number of mobile apps, such as Mood Mission,¹¹ Happier,¹² and HeadSpace,¹³ advise and enable activities to maintain a high mood. Digital technologies are also being used more frequently by older persons, particularly for leisure activities and staying in touch with loved ones. Less research has been done on their use of technology to support mental health, though. Studies that have looked into this subject have their limitations. For instance, Sauve et al.¹⁴ evaluated just one game, created by the authors themselves, limiting generalizability to other digital tools, but they did find a significant improvement in participants’ self-reported mood after playing an online educational game on the theme of well-being. The results of a single case study that looked at online cognitive behavioural therapy with a therapist are also somewhat restricted, but they did show user satisfaction and favourable reactions to the use of technology in this context.¹⁵ Given these insufficient findings, additional study including bigger cohorts of senior citizens is required to broaden the spectrum of opinions voiced.

Understanding potential obstacles to older individuals using digital tools to improve mental health is also necessary. For instance, many senior citizens view technology as pricey¹⁶ and frequently favour the “old-fashioned” methods of doing things. Older persons may find it difficult to recall how to utilize new technologies, according to earlier research.¹⁷ The opinions of senior citizens about the employment of digital technologies for promoting mental health, on the other hand, are not well understood. Relating the use of technology for mental health, use of virtual reality (VR) has been a common practice. With the advantage of VR, researchers are trying to investigate the level of cognitive development of the elderly patients who suffer from dementia, mental impairment and other psychological disorders.¹⁸

VR at care home settings

In 2018, Rose et al.¹⁹ presented a scoping review on the feasibility of VR exposure among people living with dementia and observed mixed impact, especially in terms of acceptability. VR technology has seen a great improvement since then although it is constantly evolving. Despite the popular belief of resistance towards the use of new technology among elderly population, care home residences have shown a great interest in VR.²⁰ The reported articles in the literature found improvement in mood when older adults are exposed to VR experience.^21,22

Thach et al.²¹ performed an in-depth systematic review highlighting the impact of VR experience on elderly and recommended personalized VR exposure according to the attributes of the elderly. For instance, elderly living with dementia can benefit from guidance and assistance provided by a trained care giver for an enhanced social experience.^23,24 In contrast, healthy elderly participants appreciate more independence while using a VR headset.²¹

Yeo et al.²⁵ performed a comparative study using high definition television, 360 video, and computer generated VR, investigating the preferred mode of hardware components in up-lifting mood. The study conducted in a lab-based environment showed promising results in reducing boredom through exposure to virtual nature. Although the study reported equal effectiveness despite the difference in the hardware settings, the findings may not be applicable to elderly population.

There have been a very few randomized control trials (RCTs) evaluating the impact of VR therapy. The doctoral research by Sultana investigated the potential of VR for mental stimulation to reduce behavioural and psychological symptoms of dementia (BPSD) among elderly living in the long-term care facility.²⁶ Although Sultana designed a cluster RCT protocol, the study did not perform a trial. The pilot study proved to be feasible in reducing BPSD among target population.

The pilot RCT conducted by Oliveira et al.²⁷ indicated VR experience to be effective for neurocognitive stimulation among elderly with dementia associated with Alzheimer’s Disease. The preliminary results suggested an improvement in overall cognitive function for the experimental group, with an effect size corresponding to a large effect in global cognition, which suggests that this approach is effective for neurocognitive stimulation in older adults with dementia, contributing to maintaining cognitive function in AD.

The ease-of-use has been identified as one of the most crucial aspects concerning acceptability and continuation of usage of the technology.²⁸ A number of studies recorded complaints from the participants on weights of the device.^23,29 Dementia patients also expressed concerns that they may look silly in the VR headsets.³⁰ The overall discomfort and usability requires improvement for a wider acceptability among the target population, as emphasized by Thach et al.²¹ The systematic review conducted by Dermody et al.³¹ emphasized on the importance of further understanding of the design considerations, making it more suitable for elderly demographics with auditory and vision difficulties. All the studies who used VR for treating mental health or investigating psychological condition of the patients/participants, used some tools to measure the improvement of the condition as pre- or post-intervention approach. Measuring techniques and assessments of the tool plays a crucial role to justify the studies and thereby it is important on focus on the common practice of the research tools on investigating the impact of VR on mental well-being.

VR impact assessment tools

It is well-established that nature has a positive impact on well-being.³² The reported articles mentioned in the previous section demonstrates potential of VR-based interventions as well. However, it is imperative to understand how to measure such outcomes for scientific validation, which is carried out in this section.

Pre/Post-intervention surveys

Most studies carried out surveys before and after interventions on different aspects such as mood, memory, well-being etc.,^6,5 however, limited number of studies also reported their observations during the experiment or intervention such as Appel et al.⁵ collected information on participants’ comfort while using the VR devices. Appel et al. performed both observational and questionnaire-based evaluation on several factors such as mood and engagement during the VR exposure to older adults.

Qualitative assessments

Kalantari et al. conducted a semi-structured interview and used Naturalistic Inquiry method while analysing the data to understand participants’ experience with nature-based VR exposure.

Abeele et al.³³ conducted short open-ended interviews and performed Laddering analysis using Means-End theory to explore accessibility, usability and user experience. As Abeele et al. provided more attention towards the design considerations, the impact of VR exposure was not analysed. However, there was an indication of elderly participants enjoying the natural elements in the virtual environment when they were exposed for the first time.

Appel et al.⁵ recorded observations on verbal and non-verbal expressions to understand participants’ interest and enjoyment and analysed such observations using both structured and unstructured data and approach.

Measuring emotional states

A number of reported articles explore the impact of VR exposure on mood, however, the measuring techniques varied accross the board.

Kalantari et al.⁶ performed elaborate measurements on the psychological aspects such as technological acceptance using a Likert scale-based tool, level of immersion by measuring spatial presence, frustrations of using VR technology, scored through NASA Task Load Index and cybersickness using the Simulator Sickness Questionnaire. This background information can provide valuable insights while analysing the potential benefit such as impact on the mood, which Kalantari et al. measured using Multidimensional Mood State Questionnaire.

The doctoral research of Strong used visual analogue sub-scales to reflect on participant’s mood, which was further substantiated using Cronbach’s alphas.³⁴ Strong analysed the change in mood from the baseline, as well as with varying environment, i.e., 360^o forest and beach, and observed age-related impacts using linear regression. In another 360^o based study. Yu et al.³⁵ measured the feelings of restoration. Appel et al.⁵ measured the level of enjoyment and reduction in anxiety level using a modified version of the State-Trait Anxiety Inventory questionnaire.

Chan et al.³⁶ gathered insights on degree of presence in the virtual natural world using igroup presence questionnaire, followed by measurement of degree of complexity and sense of being enclosed as compared to the urban environments. Using the positive and negative effect schedule, the study showed that connectivity with nature in a virtual forest can reduce self-reported stress, which was further supported by quantitative measurement of heart rate and heart rate variability using portable electrocardiograms.

Memory and cognition tests

Several studies used VR tools for cognitive training.^37,38,19,39 However, due to the difference in VR technology, this section focuses mostly on measuring the impact on memory and cognition through nature-based VR exposure.

Unlike task-based or reminiscence therapy, where a range of instruments are used pre- and post-interventions, the impact on memory and cognition has not been well-studied for nature-based VR studies on elderly residents at long-term care facilities. While exploring the feasibility of VR technology on elderly with memory and cognition challenges, Appel et al.⁵ considered patient history assessed through standardized tests of cognition such as MMSE, Montreal Cognitive Assessment (MoCA) and Cognitive Performance Scale (CPS), however, have not carried out any such tests post interventions. Similarly, Kalantari et al.⁶ used MoCA prior to the intervention just to distinguish the participants based on their cognition capabilities, however, measuring changes in the cognition status was not considered in their study.

VR is still an emerging technology and VR exposure to older adults attract researchers from a range of disciplines. This resulted in lack of a standardized tool to assess the impact of the intervention as evident from the aforementioned articles. Moreover, further studies are required to capture the immediate and long term effect on brain functionality. Since MMSE is well established tool to measure the levels or improvements in cognitive ability of the elderly patients, in the following it is briefly mentioned about the studies that adopted and adapted the tool in various ways.

Use of MMSE tool in related studies

This sub-section highlights on studies that used MMSE for measuring the cognitive development of different age group of people, mostly on the elderly, and also focuses on the impact of VR on dementia patients. This section evaluates the range of use and compares MMSE with other tools, analysing the impact of adaptation of MMSE and other assessment tools on the findings as well as the effectiveness of VR on patient’s cognitive progress and stability.

Adoption of MMSE is evidenced in several studies that utilized the tool for detecting mental or cognitive development of elderly patients.^40–44 These studies correlated results from the modified version of MMSE with stages of dementia identified by the clinical dementia rating, the Functional Assessment Staging Test, modified MMSE (3MS), MoCA and the Diagnostic and Statistical Manual of Mental Disorders.^45–52 The goal of the studies was to contrast two approaches to arriving at equivalent scores across MMSE versions.

There are some studies that adapted or modified MMSE questionnaire i.e.^53–57 It is also evident that there are some studies that focused on comparing MMSE with a modified version of MMSE to test the feasibility of the questionnaire on the outcome of cognition and health-related disabilities.^58,41 These changes aim to increase the validity and reliability of the MMSE by sampling a wider range of cognitive functions and covering a wider range of difficulty levels.⁵⁹ Orientation, immediate memory, attention/concentration, delayed recall, and language are the five areas of cognitive functioning that are assessed using MMSE. For the purpose of examining correlations between these components in terms of factorial structure, the component elements of the MMSE have been the subject of numerous studies. In a study, the components/scales to identify cognitive functioning were modified i.e., adding Visuospatial functions (1 point) and modifying the language (8 point) were added in the questionnaire.⁵³ Each subscale was included in the analysis as a variable to determine the reliability of modified MMSE and Cronbach alpha coefficient was determined. Moreover, the study followed Spearman correlation analysis to investigate the correlation between each subscale of the MMSE. The other studies that have compared MMSE with 3MS have shown that 3MS has more psychometric properties to identify and share more variance with neuropsychological tests than MMSE.^41,55

Research on dementia has also begun to use virtual reality (VR), particularly in the field of cognitive training. The explanation is that VR exercises a variety of psychophysical perception mechanisms, including visual, tactile, and kinesthetic perceptual sensations. It has been applied to cognitive training for patients with dementia, brain injury, post-stroke intervention, and musculoskeletal healing.

There is a study that utilized MMSE to measure improvement in physical, memory, and brain stimulation. The results demonstrate that the participants have a low focus on decision-making. Twenty-one randomly selected patients, aged between 59 and 70, participated in the study. The study was designed using a mixed-methods approach with pre- and post-assessment measures.⁶⁰

In this study, it was investigated to examine the relationship between virtual reality (VR) intervention programmes and cognitive, and physical functions. In total, 68 (aged between 65 and 85) people with mild cognitive impairment were enrolled in a randomized controlled experiment (MCI). Medical assessments through a clinical interview conducted by a dementia specialist served as the basis for the MCI diagnosis. Neuropsychologists used standardized techniques to conduct cognitive evaluations, such as the MMSE. The intervention group having (number = 34) displayed significantly enhanced executive function and brain function than the control group (number = 34). The outcome revealed that there was a noticeable improvement in gait speed and mobility before and after the follow-up. In comparison to controls, the VR-based training programme helped MCI patients with their cognitive and physical function.⁶¹

To evaluate the feasibility of the system, Cognitive Assessment by VIrtual REality for testing the cognitive function of participants across different age groups from 35 to 84, using different questionnaires such as MoCA, abbreviated mental test, MMSE were used in a study.⁶² Likewise, the previous study and many more studies used the MMSE questionnaire to assess the effect of VR on the condition of cognitive development of the participants.^63–66 Some of the studies focused on the effect size to show the impact of VR on the patients.⁶⁷

Unlike the previous research works on VR for dementia patients or measurement of different levels of cognitive development using MMSE, this study focuses on the modification of MMSE for post-intervention assessment. Moreover, to justify the validity of the questionnaire, pilot study is conducted where correlation and cross validation through participants comments and close observation report is maintained.

Proposed mental state assessment tool

The proposed mental assessment tool is a combination of pre- and post-intervention questionnaires, both of which are modified versions of MMSE. The questionnaire used for pre- and post-intervention assessments is an adaptation of the standard MMSE questionnaire. Specific adjustments made to the questionnaire are discussed in this section. Besides this, minor changes to questions were made to ensure that the pre- and post-intervention questionnaires are not exactly same and repetitive for the participants. For instance, the designs to be copied were changed. Also, minor modifications such as changing the objects to be identified and words to be spelled backwards, were also made. However, it was ensured that the difficulty level remained the same. The ordering of questions within categories was changed to reduce repetitiveness.

MMSE is divided into five major components namely Orientation, Registration, Attention and Calculation, Recall and Language. The maximum score that can be attained in MMSE is 30 points. Studies such as Shigemori et al.⁶⁸ performed factor analysis of MMSE using promax rotation to determine 11 subsets, which are divided into three factors namely immediate memory, orientation and delayed recall, and working memory. These three factors and the 11 subsets belonging to these factors are detailed below. The mapping of the five MMSE components to the 3 factors and 11 subsets identified by Shigemori et al.⁶⁸ are illustrated in Figure 1.

Figure 1.

Different components, functions and factors of mini-mental state examination (MMSE).

The first factor deals with writing, immediate memory, verbal repetition and naming. The second factor included delayed recall and spatial and temporal orientation. The third factor included verbal comprehension, attention and concentration, sentence reading and constructional praxis. The abilities that are evaluated in the first factor include verbal skills. Thus, this factor is a reflection of consolidated knowledge, which directly relates to implicit memory. Existing literature^69,70 on dementia population suggests that linguistic abilities exhibit lower sensitivity to cognitive deterioration.

The abilities included in second factor are related to semantic and episodic memory, which possess strong association with deterioration of cognitive function, particularly in early stages of dementia. Thus, Shigemori et al.⁶⁸ states that variations in these subsets are most reliable indicators for detection of cognitive deterioration. The third factor includes abilities related to working memory such as the individual’s ability to handle and learn new information. This factor is also a reliable indicator for cognitive deterioration. Since the scope of this work is limited to VR-based therapeutic interventions that rely on the interaction of participant with a virtual environment, out of factor two and three, variations to factor two are only feasible. Within this categorical factor, variations in questions related to recall were made for the post-intervention questionnaire because spatial and temporal orientation of the participant can reliably be tested using the same questions. The structure of the modified mental state assessment tool is described in Figure 2.

Figure 2.

Structure of the modified mental state assessment tool.

Orientation

Orientation deals with an individual’s sense of general awareness. This component entails two functions namely spatial and temporal orientation, which deal with the individual’s sense of awareness with regard to space and time. MMSE includes a question to test both these functions individually. For the former, the corresponding question checks if the participant knows the date, day and season. However, for the latter, the corresponding question checks if the participant knows the location of their current residence.

For the pre-intervention questionnaire, minor contextual adaptations to the standard MMSE questionnaire were made. For example, the second question for Orientation asks for different components of the address. In the United Kingdom, postcode and county are used for pincode and state. Thus, appropriate adaptations of this kind were made. It is noteworthy that most of the residents were suffering from dementia and one of the objectives of the study was to investigate the memorability of the residents. Thus, the same pre-intervention questionnaire was used across cohorts. The post-intervention used questions similar to those used in pre-intervention questionnaire.

Registration

This component of MMSE tests the functioning of the participant’s immediate memory. Thus, registration constitutes a short list of objects or prompts that the participant is expected to repeat. For this purpose, simple lists of objects such as clock, sofa and table were used. Similar questions were used for both pre- and post-intervention questionnaires.

Attention and calculation

The objective of keeping questions related to testing of arithmetic ability in the questionnaire is to assess the attentiveness and concentration levels of the participant. The standard MMSE questionnaire requires the participant to spell the word ‘world’ backwards. Both the pre- and post-intervention questionnaires had questions similar to the standard MMSE questionnaire. However, to reduce repetitiveness in the pre- and post-intervention questionnaires, different words with similar difficulty levels in terms of number of alphabets were used.

Recall

Recall is a test of the participant’s ability to remember the list of prompts or objects that they were told before. Thus, this component corresponds to the function of delayed recall. The standard MMSE requires the participant to repeat or recall the list of prompts or objects that they were told as part of the registration question. For the pre-intervention questionnaire, this question is kept unchanged. However, this question was replaced for post-intervention questionnaire.

One of the core objectives of sensory stimulation therapy is to improve engagement and recover memory threshold. The latter is tested by the questionnaires as the participants answer the same set of questions across cohorts. In order to test participant engagement, modifications were made to include elements from the VR experience. Thus, recall was tested on the basis of any three elements that the participant remembers from the VR experience. The objective of this modification is to test user engagement and direct impact of VR on mental state.

Language

This component includes seven functions and cumulatively tests language and cognitive skills of the participant. The first of these functions is naming and requires the participant to identify an object and name it. For instance, objects such as a pencil or watch is shown to the participant and he or she is asked to name it. Verbal repetition requires the participant to repeat the sentence spoken to them as-it-is. The standard MMSE questionnaire suggests the use of the sentence “No ifs, ands, or buts. On the other hand, verbal comprehension requires the participant to listen to verbal commands and follow the instructions.

The participant is required to read and write a sentence for testing the functionalities of “read a sentence and “write. Finally, constructional praxis is a function that requires participants to articulate different parts to create a unified structure. To test this function, the participant is required to copy a design. There were specific challenges with regard to resident participation in this component as many participants could not see, hear and write clearly due to medical reasons. The modified tool uses the standard testing procedure for pre- as well as post-intervention questionnaires. However, minor modifications were introduced in the post-intervention questionnaire to reduce repetitiveness, keeping the complexity unchanged.

Methods

In order to evaluate the robustness, reliability and efficacy of the proposed tool, a pilot study was designed. The different aspects of pilot study design are discussed in this section.

Participants

For the purpose of the study, seven residential homes were identified within Midlands, United Kingdom. The inclusion-exclusion criteria for participant selection considered two factors namely age and the presence of cognitive disorders. Therefore, residents aged over aged over 55 years with diverse medical conditions such as dementia, mobility issues and vision or hearing impairments were included in the study. Additionally, the study also includes a proportion of normal participants from the staff and there was no age or health condition requirement for the participants to qualify for participation in the study. The demographic information of the participants is summarized in Table 1. Images from the care homes taken during the study are provided in Figure 3. Participants have provided their consent to use these images for publication.

Figure 3.

Screenshots of Nature Treks virtual reality (VR) environments.

Table 1.

Demographic information.

Variable	$N$ (%)
Sample	52
Gender
Male	19
Female	33
Age
Under 55	8
55–64	3
65–74	10
75–84	12
85+	19

Resident participants lie within the age group 55 to 85+ with 47.37% of the participants lying within age range of 55–74 and 52.63% of the participants being 75+. It is worth mentioning that 11 out of 19 resident participants were on regular anxiety medication. The study was divided into two phases. As part of the first phase, taster sessions were organized in the selected care-homes. On the basis of the number of registered participants from each care-home, four care-homes were shortlisted for the second phase.

In the second phase, three cohorts were organized, which are referred to as C1, C2, and C3. As part of the taster sessions, residents as well as staff were registered to maintain a healthy ratio between normal and abnormal participants. Moreover, NHS quality burnout is a critical problem and VR has demonstrated significant benefits for staff wellbeing.⁷¹ The sample size used for analysis includes 116 VR sessions conducted for a total of 52 participants, out of which 41 were residents (elderly participants with cognitive disorders) and 11 were staff members (normal participants). The number of sessions, participants and residents in the taster session and across three cohorts are described in Table 2. For all analytical purposes, each session is considered an independent session. However, to analyse the impact of repeated VR exposure, three sets of non-periodic data are used for each of the participants, wherever available, to generate a Cohort-level analysis.

Table 2.

Total sessions and participants.

	Taster session	C1 Session	C2 Session	C3 Session	Overall
Virtual reality (VR) sessions	35	25	28	28	116
All participants	35	25	28	28	52
Resident participants	27	22	25	24	41

Virtual environment for intervention

As mentioned previously, sensory stimulation therapy is one of the most effective therapeutic interventions used for cognitive decline and impairment. Virtual reality enables multi-dimensional sensory stimulation and has found applications in different clinical pathways such as pain, disability and anxiety management. This study uses Nature Treks VR, which is a fully immersive, interactive VR product that has 15 different nature-themed environments with different highlights.⁷² The screenshots from the different environments are shown in Figure 4. There are several advanced elements within these environments, which can be activated or deactivated using orbs. For instance, the Green Meadows environment has orbs for growing flowers, butterflies and changing day to night or night to day. These orbs allow interactivity and facilitates user engagement. Moreover, participants can use these orbs to customize the environment according to their personal preferences and suit their relaxation needs. Screenshots of these controls are illustrated in Figure 5.

Figure 4.

Orbs or Controls in Nature Treks virtual reality (VR) environments.

Figure 5.

Images taken during the study.

Nature Treks VR-based intervention is chosen because a majority of the patients suffer from memory-related disorders including dementia. Although, products such as Nature Treks VR might be repetitive for an average user, they remain engaging for the chosen population across multiple sessions. Moreover, this environment provides them an opportunity to work on their memory-related issues by repetitively exposing them to the same set of information. Additionally, this allows therapists and caregivers to identify the memory thresholds at individual levels.

Study procedure

The major construct investigated in the study include well-being, which is further divided into cognitive ability and mood that are captured using the MMSE and modified MMSE scores, and mood scores, respectively. In addition, factors such as anxiety and mood swings are also considered in the study. The experimental methodology used for the study is illustrated in Figure 6. For every participant, a session is divided into four stages namely, registration, pre-intervention questionnaire, VR session and post-intervention questionnaire. As a rule, participant registration is performed the first time a resident registers to participate for the study. Thus, it includes seeking formal consent to participate and collecting basic data from the participant and the participant’s carer. The objective of this stage is to inform the participant about the study so that he or she can make an informed decision about their participation. Secondly, it also involves collecting basic and background information about the participant so that the rest of the stages can be appropriately modified to suit the needs of the participant. For instance, participants who wear glasses can be provided additional insert for the headset to ensure a comfortable experience.

Figure 6.

Block diagram of pilot study design.

The second stage of the session involves filling the pre-intervention questionnaire for the participant. As part of this stage, the researcher, with assistance from the carer, asks the set out questions and on the basis of responses, grades the participant. Additionally, the researcher also documents any observations, both from carer perspective and researcher perspective. In the next stage, the participant is briefed about the headset and VR experience. The participant is free to choose any of the available VR experiences, keeping the experience time flexible between 10 and 20 minutes, in a normal scenario. Finally, after the VR experience, post-intervention questionnaire is filled in the same manner as pre-intervention questionnaire.

Measurement tools

This study uses a modified version of MMSE with an auxiliary assessment of participant mood. For the purpose of mood assessment, a self-reporting tool that quantifies mood on a scale of 1–5, was used. This paper introduces modifications to the standard MMSE questionnaire for use as a tool to assess instantaneous cognitive function before and after a VR experience. The pilot study was designed to validate the modifications made to the questionnaires. Thus, this study uses several tools to get comprehensive information about the participant’s medical history and VR experience. Details about the tools used for this purpose are described in this section.

Consent to Participate: Before starting the experiment, all the participants were asked to provide their written consent for participation. In addition, the form also included questions about personal information and medical history. Additionally, the form also includes information about data ownership, which lies solely with Greener Games, and that the study complies with GDPR guidelines. The form used for this purpose is provided as Supplementaal File 1 (Research Consent Form).

Carer Questionnaire: In order to gain information about the participant, the carer was provided with a questionnaire, which includes information about participant background and medical condition. The form used for this purpose is provided as Supplementaal File 2 (Carer Questionnaire).

Pre-VR Questionnaire: The questionnaire used for pre-intervention mental state assessment is provided as Supplementaal File 3 (Pre-VR Questionnaire). It is worth mentioning that some of the residents had specific impairments; as a result of which they were unable to read, write, see or recognize objects. Therefore, questions were omitted on need basis, giving the participant a zero for the specific question. Besides this, a self-reporting mood assessment tool that requires the participant to rate their mood on a scale of 1 to 5, 1 being sad and 5 being happy, was used for mood assessment.

Post-VR Questionnaire: As mentioned previously, the post-intervention questionnaire changes the intra-category ordering of questions to reduce repetitiveness. Moreover, questions that test participant engagement with VR are included. The questionnaire used for post-intervention mental state assessment is provided as Supplementaal File 4 (Post-VR Questionnaire). The instructions for carrying out pre- and post-intervention questionnaires are available as Supplementaal File 5 (Instructions). Moreover, the reading commands and designs used for executing the questionnaire are available as Supplementaal Files 6 (Command I), 7 (Command II), 8 (Design I), 9 (Design II).

Measures for evaluation of proposed mental state assessment tool

Questionnaire development is a well-established process and involves several phases, which include expert advise and pilot study.⁷³ The proposed tool is evaluated on the basis of analysis of the data collected during pilot study. Firstly, the internal consistency and reliability of proposed questionnaires need to be tested to prove the robustness of the tool and its potential usage for impact analysis of therapeutic interventions in patients with cognitive impairment. Secondly, quantitative and qualitative analyses needs to be performed to correlate and cross-validate the results obtained using the tool.

This research paper modifies existing MMSE questionnaire to provide a tool for pre- and post-intervention instantaneous cognitive state testing. Since, this questionnaire is adapted, only test to ensure internal consistency and reliability of the questionnaire is performed. In order to evaluate the internal consistency and reliability of the proposed tool, Cronbach’s alpha coefficient⁷⁴ was used.

Quantitative analysis was performed to establish correlation between the use of VR-based intervention and participant well-being. Participant well-being was assessed on the basis of MMSE and mood scores. Therefore, changes in MMSE and mood scores are analysed by hypothesis test using Paired $T$ -test and Wilcoxon Signed Rank $T$ -test. Finally, cohort-level analysis was performed by analysing the mean MMSE and mood scores across the three cohorts of VR sessions. Besides this, quantitative analysis of the collected data was performed to determine the trends for impact of VR-based therapeutic intervention. Furthermore, qualitative analysis was performed using Lincoln and Guba’s Naturalistic Inquiry method.⁷⁵ The participant comments received and researcher observations with regard to participant response documented during the study were analyzed. This method involves separating content and categorizing them into segments to summarize the results.

Dataset development

Data collected during the pilot study, in the form of a set of four questionnaires filled for every first-time participant and two questionnaires filled for repeat participants, was collated and segregated into quantitative and qualitative data for analytical purposes. Every participant was assigned a registration ID to maintain data anonymity. The dataset included the score obtained by a participant for every question in the questionnaire along with the mood score. The cumulative scores obtained in the pre- and post-intervention MMSE questionnaire is referred to as pre- and post-intervention MMSE score, respectively. Moreover, the score obtained using the self-reporting mood assessment tool is referred to as mood score.

This dataset was developed using data from participant responses to pre and post-intervention questionnaires. Since, independent as well as cohort-level analysis is performed for the collected data, repeat participation was also taken into account. Mood assessment was not performed during the taster sessions and thus, the number of records vary for MMSE and mood score analyses. For cohort-level analysis, a few records from taster sessions were used to satify the requirement of three VR sessions. The mood scores, which were unavailable for such records, were imputed with mean mood score value for the participant from other two sessions.

Results

The pre-intervention and post-intervention questionnaires are modified versions of MMSE, with slight adaptations made to the standard MMSE to contextualize the questionnaire for use in VR trials. Therefore, in order to validate the proposed tool, a pilot study was performed. Cronbach’s alpha was calculated for the pre- and post-intervention response data collected to prove the internal consistency and reliability of the questionnaire. Additionally, the results of the pilot study were analysed and counter-validated with researcher and carer observations to conclusively prove the efficacy of the proposed questionnaires. This section provides a detailed account of the results obtained from the study.

Internal consistency and reliability of the proposed tool

This research paper modifies existing MMSE questionnaire to provide a tool for pre- and post-intervention instantaneous cognitive state testing. Since, this questionnaire is adapted, only test to ensure internal consistency and reliability of the questionnaire is performed. For this purpose, Cronbach’s alpha coefficient was calculated for the questionnaire responses received during the pilot study. This test is performed using the test-retest strategy to determine the internal consistency of the questionnaire over multiple instances of the test.

The study included 116 independent VR sessions for 52 participants. An overall Pre-VR and Post-VR Cronbach’s alpha coefficient value is determined for these sessions. This total number of VR sessions also included three cohorts of VR sessions performed for the same set of 21 participants. The Cronbach’s alpha coefficient for Pre-VR and Post-VR sessions of these three cohorts are also determined. These results are summarized in Table 3. The Cronbach’s alpha coefficient for all the instances lie within the acceptable range of 0.6–0.8,⁷⁶ with the exception of one instance in the comparison of both Pre-VR and Post-VR tools, which are 0.588 and 0.587, respectively. Thus, within the scope of this pilot study, the questionnaires can be considered acceptable. Improving these scores by modifying the questionnaires shall be attempted in future work.

Table 3.

Cronbach’s alpha coefficient values.

	Pre-virtual reality (VR) tool	Post-VR tool
Overall	0.674	0.641
Cohort 1	0.642	0.625
Cohort 2	0.639	0.587
Cohort 3	0.588	0.692

Quantitative analysis for correlation

Changes in MMSE scores

In order to test the statistical significance of collected data with respect to MMSE scores, test for normal distribution, Paired-T test and Wilcoxon Signed-Rank test were performed. The descriptive statistics for the dataset are provided in Table 4. The range, mean, standard deviation and negative values of skewness and kurtosis indicate that the data is skewed to the left with lighter tails than normal distribution, indicating that the data is not normally distributed.

Table 4.

Statistics for paired samples—mini-mental state examination (MMSE) scores.

Data	Points	Min-Max	Mean	Std	Skewness	Kurtosis
Pre-intervention MMSE score	116	[4,30]	20.71	5.79	$-$ 0.15	$-$ 0.56
Post VR MMSE score	116	[6,30]	22.45	5.92	$-$ 0.45	$-$ 0.45

Shapiro-Wilk test was performed to test for normal distribution. The $p$ -value for pre-intervention data, post-intervention data and Change (difference between post-intervention and pre-intervention data) are 0.006698, 0.000039, 0.000664, respectively. Since the $p$ -values are low, satisfying the requirement of $p - v a l u e < 0.05$ , there is sufficient evidence to state that none of the datasets come from normal distribution.

The results of the Paired $T$ -Test provide values of Statistic = $- 5.43339885$ and $p$ -value = $3.13373222 \times 10^{- 7}$ . The value of statistic is negative because the second dataset mean is greater than that of first dataset. A low $p$ -value, which satisfies the requirement of p-value < 0.05, suggests that there is enough evidence to reject the null hypothesis and accept the alternate hypothesis. In other words, there is a statistically significant change (or increase) in MMSE scores post-VR intervention.

Having said that, in order for the paired sample $t$ -test results to be trusted, the dependent variable should be approximately distributed. Moreover, there should not be any significant outliers. Both the variables and their difference violate the assumption of normality. However, there are no outliers in data. The data distribution histograms and box plots are provided in Figure 7 and Figure 8, respectively. Thus, the results of the Paired $T$ -test cannot be completely trusted. As a result, an appropriate alternative in the form of the Wilcoxon signed-rank Test was performed.

Figure 7.

Data distribution of mini-mental state examination (MMSE) scores. (a) Pre-intervention score; (b) Post-intervention scores; (c) change in scores.

Figure 8.

Box plots for mini-mental state examination (MMSE) scores. (a) Pre-intervention score; (b) post-intervention scores; (c) change in scores.

The results obtained for the Wilcoxon Signed-Rank Test provide the values of Statistic = 989.5 and $p$ -value = $3.218679891914893 \times 10^{- 7}$ . A low $p$ -value, which satisfies the requirement of $p - v a l u e < 0.05$ , suggests that there is enough evidence to reject the null hypothesis and accept the alternate hypothesis. In other words, there is a statistically significant change (or increase) in MMSE scores post-VR intervention.

Changes in mood scores

In order to test the statistical significance of collected data with respect to mood scores, test for normal distribution, Paired-T test and Wilcoxon Signed-Rank test was performed. The descriptive statistics for the dataset are provided in Table 5. The range, mean, standard deviation and negative values of skewness and kurtosis indicate that the data is skewed to the left with lighter tails than normal distribution for pre-intervention mood scores and more data around the tails for post-intervention mood scores. These values cumulatively indicate that the data is not normally distributed.

Table 5.

Statistics for paired samples – mood scores.

Data	Points	Min-Max	Mean	Std	Skewness	Kurtosis
Pre-intervention mood score	79	[2,5]	3.88	0.89	$-$ 0.19	$-$ 0.94
Post virtual reality (VR) mood score	79	[2,5]	4.42	0.76	$-$ 1.26	0.99

Please note that the data points for mood scores are lower (79) as against 116 data points for MMSE scores because mood score data was not collected during taster sessions and for two data points, the mood scores were unavailable (unsure/no data).

Shapiro-Wilk test was performed to test for normal distribution. The $p$ -value for pre-intervention data, post-intervention data and Change (difference between post-intervention and pre-intervention data) are $3.73 \times 10^{- 7}$ , $2.78 \times 10^{- 10}$ , $5.77 \times 10^{- 7}$ , respectively. Since the $p$ -values are low, satisfying the requirement of $p - v a l u e < 0.05$ , there is sufficient evidence to state that none of the datasets come from normal distribution.

The results of the Paired $T$ -Test provide the values of Statistic = $-$ 5.63 and $p$ -value = $2.73 \times 10^{- 7}$ . The value of statistic is negative because the second dataset mean is greater than that of first dataset. A low $p$ -value, which satisfies the requirement of $p - v a l u e < 0.05$ , suggests that there is enough evidence to reject the null hypothesis and accept the alternate hypothesis. In other words, there is a statistically significant change (or increase) in Mood scores post-VR intervention.

Having said that, in order for the paired sample $t$ -test results to be trusted, the dependent variable should be approximately normally distributed and should not contain any significant outliers. Both the variables and their difference violate the assumption of normality. Moreover, box plots show outliers in data. The data distribution histograms and box plots are provided in Figures 9 and 10, respectively. Thus, the results of the Paired $T$ -test cannot be completely trusted. Therefore, an appropriate alternative in the form of Wilcoxon signed-rank test was performed.

Figure 9.

Data distribution of mood scores. (a) Pre-intervention score; (b) post-intervention scores; (c) change in scores.

Figure 10.

Box plots for mood scores. (a) Pre-intervention score; (b) post-intervention scores; (c) change in scores.

The results obtained for the Wilcoxon Signed-Rank Test provide the values of Statistic = 100.5 and $p$ -value = $1.97 \times 10^{- 6}$ . A low $p$ -value, which satisfies the requirement of $p - v a l u e < 0.05$ , suggests that there is enough evidence to reject the null hypothesis and accept the alternate hypothesis. In other words, there is a statistically significant change (or increase) in Mood scores post VR intervention.

Independent sessions and Cohort-level analysis

Data analysis was performed for a dataset with 116 data points. The mean total pre- and post-intervention MMSE scores are 20.72 and 22.45, respectively. Thus, the percentage increase in mean MMSE score is 8.35%. Out of the total 52 participants, 21 participants were present for at least three VR sessions, out of which 19 were residents. For cohort-level analysis, set of three VR sessions attended by a resident were considered. The mood scores for some sessions, which were part of the taster sessions cohort) were not available. Any such sessions that were included in the analysis were imputed with mean value mood scores to maintain data consistency. The changes in MMSE and mood scores across three cohorts are illustrated in Figure 12.

Qualitative analysis for cross validation

The general consensus on the VR experience indicated that the experience was relaxing and a majority of the participants liked the experience. There were initial apprehensions among participants on the use of this technology and its impact on their health. To counter these apprehensions, the participants were briefed about the technology and objective of the experience. Besides this, they were briefed on a one-to-one basis about the use of the headset. Written instructions were additionally provided. Having said that, some participants still found it difficult to operate the headset and navigate through the experience. To help them, the experience was expert-assisted who continuously interacted with them throughout the experience and navigated them as per their preferences.

Most of the participants acknowledged the realism of the VR experience with a few participants commenting, “It feels like to be back in Africa,” “Feels like I’m abroad,” and “It is amazing how they fit all this into these goggles.” Most participants expressed a sense of full immersion and detachment from the physical world. Some of the comments received in this regard include “Makes me want to go there.” “Takes you away for a little while,” and “It’s a different world.” To ensure that there were no known obstacles to immersion such as physical reminders of the physical world in the form of connecting cables brushing up against the participant’s body, wireless headsets were used. Moreover, all participants were provided a stationary experience because of mobility issues and their lack of experience with this technology.

With that said, the negative comments around the VR experience focused on it being complicated for participants who wished to take control and tiring for participants who had a sore neck because of wearing the headset for a stipulated amount of time. These comments indicated that technological design of the interface and hardware should be adapted for use by vulnerable population segments such as differently abled, elderly and children. Interestingly, a common complaint from the participants was that the experience was not long enough and they wished they could stay in the experience for a longer time.

A majority of the participants appreciated the experience and showed willingness to try it again. Observations reported by researchers before, during and after the VR sessions indicate an overall improvement in activity levels post-VR sessions and enhanced willingness to participate across the three cohorts of VR sessions. In fact, positive comments and recommendations from residents who tried the experience before their fellow residents attracted attention and interest from others, leading to many first-time participants across the different cohorts as well. For many participants, the VR experience was all about revisiting times they spent with loved ones with one participant saying, “This reminds me of so many travel memories with my husband.” Interestingly, one of the participants remembered a holiday she spent with her brother who passed away the previous year and saw her brother in the experience. She called the experience “weird” and “creepy.” A few participants also posited that this experience is good for people with mobility issues stating that, “Good for somebody who doesn’t go out.”

From the caregiver’s perspective, there were some interesting observations. Caregivers in one of the residential homes shared their observation that they had noticed considerable reduction in mood swings among residents with a persistent anxiety problem. This observation was also noted for bedridden residents. An intriguing observation made during the study was that people with vision issues confirmed by the caregiver and initial cognitive assessment test, could see specific environments and objects inside the VR experience. This observation is under consideration for future study and investigation.

Discussion

A paired sample $t$ -test was used to analyze the MMSE and mood scores, before and after the VR session, to test if the intervention had a significant effect on the MMSE total scores and mood scores. However, due to violation of normality assumption, an alternative Wilcoxon signed-rank Test was also performed to gain reliable evidence for statistical significance of the results.

Therefore, it can conclusively stated that the MMSE score before the intervention was lower ( $20.71 \pm 5.79$ units) compared to the MMSE score post-intervention ( $22.45 \pm 5.92$ units) and there was a statistically significant increase in MMSE score after VR session. The impact of intervention on normalized MMSE components namely, Registration, Orientation, Language, Recall and Attention & Calculation are provided in the spider diagrams (Figure 11).

Figure 11.

Spider diagrams for normalized mini-mental state examination (MMSE) components. (a) Results for C1; (b) results for C2; (c) results for C3; (d) overall results.

Figure 12.

Comparison of results for C1-C2-C3 session cohorts. (a) Average mini-mental state examination (MMSE) scores; (b) average mood scores.

Moreover, it can be conclusively stated that the mood score before the intervention was lower ( $3.88 \pm 0.89$ units) compared to the Mood score post intervention ( $4.42 \pm 0.76$ units) and there was a statistically significant increase in Mood score after VR session.

With respect to the cohort-level analysis, MMSE scores demonstrate improvement. Moreover, similar improvements in mood scores are indicative of the demonstrable impact of the intervention. Therefore, trends in mood scores validate the results obtained using the proposed tool, in the form of MMSE scores.

The results of qualitative analysis indicate that the participants and caregivers were convinced about the impact of VR on cognitive function and mood, with some specific benefits to people with anxiety-related issues. Therefore, caregiver comments and participant responses validate that nature-based VR has a positive impact on the wellbeing of the participant with short-term as well as long-term benefits.

Although, the tool is evaluated for a VR-based intervention in the care home setting, it is expected to be applicable for impact assessment of therapeutic interventions for any form of cognitive decline across different age groups. For instance, cognitive impairment due to accidental damage, congenital disability, learning disability and geriatric issues, are potential cases where this tool can be adopted. Having said that, minor adjustments to the questionnaires may be required to fit the context. It is important to state that this study suffers from some inherent limitations, which are listed below.

Limitations of data collection phase: One of the most significant challenges of this study was to maintain repeat participation. Similar challenges have been identified by other researches that have conducted similar studies.^77,78 As is evident from the results, there was reduced repeated presence with reasons such as variable duty shifts for staff, hospital appointments of residents, scheduling of other care home activities such as outdoor visits for residents and personal reasons such as visitors or unwell health, being most prominent. In addition, there are some limitations related to data entry and compilation and associated human error that may be introduced in the process.

Weaknesses of MMSE as a tool: The use of MMSE introduces inherent bias against participants with poor education with many questions based on testing of language and mathematical elements. Besides this, there is also an inherent bias against people who possess any form of visual impairment. Having said that, the impact analysis method used in the study compares the MMSE scores of the same participant pre- and post-intervention, thus tackling biases and offsets that may appear because of the participant’s limited ability or capacity. Other limitations of this tool include its inability to translate across cultures and dependence on language proficiency.⁷⁹

Lack of content: The most critical limitation of any VR-based intervention is lack of personalization and diversity in content.⁸⁰ Every participant has different preferences, which are dependent on the participant’s past experiences and personality. Therefore, catering to a group of participants with varied backgrounds is challenging. Development of automated personality assessment and AI-based content personalization shall be attempted in the future to address this limitation.

Hardware limitations of VR headsets: The chosen population of participants have more physical vulnerabilities than the average population. Therefore, wearing the headset for a session was a challenge for many participants because of headset size and usability of the interface. Moreover, people with acute hearing issues and implanted hearing aids with high frequency settings could not tolerate the sounds within the headset and the volume had to be muted. Similar challenges and need for adaptation were also identified in another study.⁸¹ This opens opportunities for development of user-friendly headsets and VR applications that can be independently used and managed by vulnerable populations such as children, diabled and elderly.

Conclusion

The work presented in this paper has successfully proposed a Modified Mental State Assessment Tool that can be used to assess the pre- and post-intervention of technology-based therapeutic interventions for elderly people that has been developed by modifying the standard MMSE. The pilot study conducted and the extensive experimental analysis that followed, validate the effectiveness of the proposed assessment tool in care home settings, using Nature Treks VR environment as intervention tool. The pilot study conducted and the extensive experimental analysis that followed, validate the effectiveness of the proposed tool. The results of the study indicate that the Cronbach’s alpha coefficient for Pre-VR and Post-VR interventions, which include 116 VR sessions for 52-participant, and three cohorts of VR sessions for 21 participants, remained within the acceptable range of 0.6–0.8. Additionally, both quantitative and qualitative observations from the caregivers and researchers provided further evidence of the tool’s efficacy.

This work provides a strong foundation for further research in the area of therapeutic interventions with variety of digital technology-based interventions for elderly people, especially those with cognitive impairments, and will help improve their quality of life. The proposed tool can also be used as a baseline for designing more effective therapeutic interventions for elderly people in similar care home settings. This tool will additionally facilitate treatment planning and optimization, which in turn will aid planning and resource efficiency.

In conclusion, this work has successfully demonstrated the effectiveness of the proposed Mental State Assessment Tool in assessing the impact of technology-based therapeutic interventions for elderly people in care home settings. The results of the study provide a strong foundation for further research in this direction and the proposed tool can be used as a baseline for designing more effective therapeutic interventions for elderly people.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-docx-1-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Supplemental Material

sj-doc-2-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-doc-2-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Supplemental Material

sj-docx-3-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-docx-3-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Supplemental Material

sj-docx-4-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-docx-4-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Supplemental Material

sj-docx-5-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-docx-5-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Supplemental Material

sj-docx-6-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-docx-6-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Supplemental Material

sj-docx-7-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-docx-7-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Supplemental Material

sj-docx-8-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-docx-8-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Supplemental Material

sj-docx-9-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-docx-9-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Supplemental Material

sj-docx-10-dhj-10.1177_20552076231203800 - Supplemental material for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment

Supplemental material, sj-docx-10-dhj-10.1177_20552076231203800 for A modified mental state assessment tool for impact analysis of virtual reality-based therapeutic interventions in patients with cognitive impairment by Samiya Khan, Muhammad Kamran Naeem, Marzia Hoque Tania, Nadia Refat, Md Arafatur Rahman and Mohammad Patwary in DIGITAL HEALTH

Footnotes

Acknowledgements

The authors of this paper extend their gratitude to Ria Kalograni and John Carline for providing access to Nature Treks VR and all the other resources required to successfully carry out VR sessions for the care home residents. Additionally, we would also like to thank Debbie Price for providing resources and access to the Coverage Care facilities in Shropshire for conducting this study. We would also like to extend our heartfelt gratitude to the care homes’ staff and residents for cooperating with us during the study.

Authors’ contribution

All authors have made a substantial contribution in developing the concept, designing the pilot study and acquiring, analysing and interpreting data. The article was collectively drafted and revised.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Ethical approval

The authors confirm that all the necessary ethical approvals were taken to carry out this research. Moreover, informed consent was taken from the participants before registering them for this study.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Guarantor

ORCID iD

Samiya Khan

Supplemental material

Supplemental material for this article is available online.

References

UCL. Cases of cognitive decline in older people more than doubles in ten years. Available at: https://www.ucl.ac.uk/news/2022/mar/cases-cognitive-decline-older-people-more-doubles-ten-years (2022, accessed 11 February 2023).

Hub DS.: Cost by sector in the uk. Available at: https://www.dementiastatistics.org/statistics/cost-by-sector-in-the-uk-2 (nd, accessed 11 February 2023).

CDC. Cognitive impairment: A call for action, now! - centers for disease control and prevention. Available at: https://www.cdc.gov/aging/pdf/cognitive_impairment/cogimp_poilicy_final.pdf (2011, Accessed 11 February 2023).

Collier

. The use of multi-sensory stimulation to improve functional performance in older people with dementia: A randomised single blind trial. PhD Thesis, University of Southampton, 2007.

Appel

Bogler

et al. Older adults with cognitive and/or physical impairments can benefit from immersive virtual reality experiences: A feasibility study. Front Med (Lausanne) 2020; 6: 329.

Kalantari

Bill Xu

Mostafavi

et al. Using a nature- based virtual reality environment for improving mood states and cognitive engagement in older adults: A mixed- method feasibility study. Inno Aging 2022; 6. DOI: 10.1093/geroni/igac015.

Cotton

Irwin

Wilkins

et al. The future’s digital: Mental health and technology. NHS Confederation, Liverpool ACC, 2014.

Brown

Astell

. Assessing mood in older adults: A conceptual review of methods and approaches. Int Psychogeriatr 2012; 24: 1197–1206.

Proudfoot

Parker

Pavlovic

et al. Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress. J Med Internet Res 2010; 12: e1475.

10.

Mohr

Zhang

Schueller

. Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annu Rev Clin Psychol 2017; 13: 23.

11.

Bakker

Rickard

. Engagement with a cognitive behavioural therapy mobile phone app predicts changes in mental health and wellbeing: Moodmission. Aust Psychol 2019; 54: 245–260.

12.

Huberty

Eckert

Larkey

et al. Experiences of using a consumer-based mobile meditation app to improve fatigue in myeloproliferative patients: Qualitative study. JMIR Cancer 2019; 5: e14292.

13.

Militello

Sobolev

Okeke

et al. Digital prompts to increase engagement with the headspace app and for stress regulation among parents: Feasibility study. JMIR Formative Res 2022; 6: e30606.

14.

Sauve

Renaud

Kaufman

et al. Can digital games help seniors improve their quality of life? In International Conference on Computer Supported Education. Springer, pp. 179–192.

15.

Pugh

Hadjistavropoulos

Klein

et al. A case study illustrating therapist-assisted internet cognitive behavior therapy for depression. Cogn Behav Pract 2014; 21: 64–77.

16.

Greenhalgh

Wherton

Sugarhood

et al. What matters to older people with assisted living needs? a phenomenological analysis of the use and non-use of telehealth and telecare. Soc Sci Med 2013; 93: 86–94.

17.

Peek

Luijkx

Rijnaard

et al. Older adults’ reasons for using technology while aging in place. Gerontology 2016; 62: 226–237.

18.

Baragash

Aldowah

Ghazal

. Virtual and augmented reality applications to improve older adults’ quality of life: A systematic mapping review and future directions. DIGITAL HEALTH 2022; 8: 20552076221132099. DOI: 10.1177/20552076221132099.

19.

Rose

Stewart

Jenkins

et al. A scoping review exploring the feasibility of virtual reality technology use with individuals living with dementia. A scoping review exploring the feasibility of virtual reality technology use with individuals living with dementia 2018.

20.

Hayden

Chaze

Kamath

et al. Implementa- tion of a virtual reality recreation program in long-term care. J Rehabil Assist Technol Eng 2022; 9: 20556683211070994.

21.

Thach

Lederman

Waycott

. How older adults respond to the use of virtual reality for enrichment: A systematic review. In 32nd Australian Conference on Human- Computer Interaction. OzCHI ‘20, New York, NY, USA: Association for Computing Machinery. ISBN 9781450389754, p. 303–313. DOI:10.1145/3441000.3441003.

22.

Zhai

Dilawar

Yousef

et al. Virtual reality therapy for depression and mood in long-term care facilities. Geriatrics 2021; 6: 58.

23.

Brimelow

Dawe

Dissanayaka

. Preliminary research: Virtual reality in residential aged care to reduce apathy and improve mood. Cyberpsychol, Behav, Soc Network 2020; 23: 165–170.

24.

Baker

Waycott

Robertson

et al. Evaluating the use of interactive virtual reality technology with older adults living in residential aged care. Inf Process Manag 2020; 57: 102105.

25.

Yeo

White

Alcock

et al. What is the best way of delivering virtual nature for improving mood? an experimental comparison of high definition tv, 360 video, and computer generated virtual reality. J Environ Psychol 2020; 72: 101500.

26.

Sultana

. Reducing behavioural and psychological symptoms of dementia in older adult long term care home residents with virtual reality therapy. PhD Thesis, University of Western Ontario, 2020.

27.

Oliveira

Gamito

Souto

et al. Virtual reality-based cognitive stimulation on people with mild to moderate dementia due to alzheimer’s disease: A pilot randomized controlled trial. Int J Environ Res Public Health 2021; 18: 5290.

28.

Yeh

Pai

Jeng

. The factors affecting older adults’ intention toward ongoing participation in virtual reality leisure activities. Int J Environ Res Public Health 2019; 16. DOI: 10.3390/ijerph16030333.

29.

Brown

. An exploration of virtual reality use and application among older adult populations. Gerontol Geriatr Med 2019; 5. DOI: 10.1177/2333721419885287.

30.

Hodge

Balaam

Hastings

et al. Exploring the design of tailored virtual reality experiences for people with dementia. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. pp. 1–13.

31.

Dermody

Whitehead

Wilson

et al. The role of virtual reality in improving health outcomes for community-dwelling older adults: Systematic review. J Med Internet Res 2020; 22: e17331.

32.

Houwelingen-Snippe

Ben Allouch

Van Rompay

et al. Virtual reality representations of nature to improve well-being amongst older adults: A rapid review. J Technol Behav Sci 2021; 6: 464–485.

33.

Abeele

Schraepen

Huygelier

et al. Immersive virtual reality for older adults: Empirically grounded design guidelines. ACM Trans Access Comput 2021; 14. DOI: 10.1145/3470743.

34.

Strong

. 360o Videos of Natural Scenery Delivered Via Virtual Reality Devices and its Effect on Mood. PhD Thesis, University of Tennessee, Knoxville, 2021.

35.

Lee

et al. Restorative effects of virtual natural settings on middle-aged and elderly adults. Urban For Urban Gree 2020; 56: 126863.

36.

Chan

SHM

Qiu

Esposito

et al. Nature in virtual reality improves mood and reduces stress: Evidence from young adults and senior citizens. Virtual Real 2021;.

37.

Bauer

ACM

Andringa

. The potential of immersive virtual reality for cognitive training in elderly. Gerontology 2020; 66: 614–623.

38.

Maeng

Hong

Kim

et al. Effects of virtual reality- based cognitive training in the elderly with and without mild cognitive impairment. Psychiatry Investig 2021; 18: 619.

39.

Gamito

Oliveira

Morais

et al. Cognitive stimulation of elderly individuals with instrumental virtual reality- based activities of daily life: Pre-post treatment study. Cyberpsychol, Behav, Soc Network 2019; 22: 69–75.

40.

Chua

Choo

RWM

NHL

et al. Mapping modified mini- mental state examination (mmse) scores to dementia stages in a multi-ethnic asian population. Int Psychogeriatr 2019; 31: 147–151.

41.

Van Patten

Britton

Tremont

. Comparing the mini- mental state examination and the modified mini-mental state examination in the detection of mild cognitive impairment in older adults. Int psychogeriatr 2019; 31: 693–701.

42.

Pinto

Machado

Bulgacov

et al. Is the montreal cognitive assessment (moca) screening superior to the mini- mental state examination (mmse) in the detection of mild cognitive impairment (mci) and alzheimer’s disease (ad) in the elderly? Int Psychogeriatr 2019; 31: 491–504.

43.

Qiao

Chen

Zhu

. Ranking convolutional neural network for alzheimer’s disease mini-mental state examination prediction at multiple time-points. Comput Methods Programs Biomed 2022; 213: 106503.

44.

Yamashita

Kuwashiro

Ishikawa

et al. Identification of predictors for mini-mental state examination and revised hasegawa’s dementia scale scores using mr-based brain morphometry. Eur J Radiol Open 2021; 8: 100359.

45.

Yang

Yan

Jin

et al. Estimation of diagnostic performance of dementia screening tests: Mini-mental state examination, mini-cog, clock drawing test and ascertain dementia 8 questionnaire. Aging Ment Health 2018; 22: 948–952.

46.

Mancuso

Demeyere

Abbruzzese

et al. Using the oxford cognitive screen to detect cognitive impairment in stroke patients: A comparison with the mini-mental state examination. Front Neurol 2018; 9: 101.

47.

Palsetia

Rao

Tiwari

et al. The clock drawing test versus mini-mental status examination as a screening tool for dementia: A clinical comparison. Indian J Psychol Med 2018; 40: 1–10.

48.

Larner

. Mini-mental state examination: Diagnostic test accuracy study in primary care referrals. Neurodegener Dis Manag 2018; 8: 301–305.

49.

Cebi

Babacan

Oktem Tanor

et al. Discrimination ability of the short test of mental status (stms) compared to the mini mental state examination (mmse) in the spectrum of normal cognition, mild cognitive impairment, and probable alzheimer’s disease dementia: The turkish standardization study. J Clin Exp Neuropsychol 2020; 42: 450–458.

50.

Arevalo-Rodriguez

Smailagic

Roque-Figuls

et al. Mini-mental state examination (mmse) for the early detection of dementia in people with mild cognitive impairment (mci). Cochrane Database Syst Rev 2021; 1.

51.

Dahbour

Hashim

Alhyasat

et al. Mini- mental state examination (mmse) scores in elderly jordanian population. Cerebr Circ-Cognition Behav 2021; 2: 100016.

52.

Suda

Muraga

Ishiwata

et al. Early cognitive assessment following acute stroke: Feasibility and comparison between mini-mental state examination and montreal cognitive assessment. J Stroke Cerebrovasc Dis 2020; 29: 104688.

53.

Yildiz

Ozcelik

Kolukisa

et al. Validity and reliability studies of modified mini mental state examination (mmse-i) for turkish illiterate patients with diagnosis of alzheimer disease. Turk J Psychiat 2016; 27: 41–46.

54.

Pierce

Chen

et al. Conversion between the modified mini-mental state examination (3mse) and the mini- mental state examination (mmse). Alzheimer’s Dement: Diagn, Asses Dis Monitor 2021; 13: e12161.

55.

Caman

Karahan

Unal

et al. Adaptation of the modified mini-mental state examination (3ms) and determination of its normative values in turkey. Dement Geriatr Cogn Disord 2019; 47: 315–322.

56.

Kumar

Himanshu

Tandon

et al. Prevalence of hiv associated neurocognitive disorder using modified mini mental state examination and its correlation with cd4 counts and anti-retroviral therapy. J Assoc Physicians India 2019; 67: 47–51. 57.

57.

Ryan

Woods

Britt

et al. Normative performance of healthy older individuals on the modified mini-mental state (3ms) examination according to ethno-racial group, gender, age, and education level. Clin Neuropsychol 2019; 33: 779–797.

58.

Mwene-Batu

Bisimwa

Baguma

et al. Long-term effects of severe acute malnutrition during childhood on adult cognitive, academic and behavioural development in african fragile countries: The lwiro cohort study in democratic republic of the congo. PLoS ONE 2020; 15: e0244486.

59.

Spencer

Wendell

Giggey

et al. Psychometric limitations of the mini-mental state examination among nondemented older adults: An evaluation of neurocognitive and magnetic resonance imaging correlates. Exp Aging Res 2013; 39: 382–397.

60.

Park

Liao

Kim

et al. Feasibility and tolerability of a culture-based virtual reality (vr) training program in patients with mild cognitive impairment: A randomized controlled pilot study. Int J Environ Res Public Health 2020; 17: 3030.

61.

Thapa

Park

Yang

et al. The effect of a virtual reality- based intervention program on cognition in older adults with mild cognitive impairment: A randomized control trial. J Clin Med 2020; 9: 1283.

62.

Lim

Wong

Teh

et al. A fully-immersive and automated virtual reality system to assess the six domains of cognition: Protocol for a feasibility study. Front Aging Neurosci 2021; 12: 604670.

63.

Moreno

Wall

Thangavelu

et al. A systematic review of the use of virtual reality and its effects on cognition in individuals with neurocognitive disorders. Alzheimer’s Dement: Transl Res Clin Interv 2019; 5: 834–850.

64.

Mrakic-Sposta

Di Santo

Franchini

et al. Effects of combined physical and cognitive virtual reality-based training on cognitive impairment and oxidative stress in mci patients: A pilot study. Front Aging Neurosci 2018; 10: 282.

65.

Optale

Urgesi

Busato

et al. Controlling memory impairment in elderly adults using virtual reality memory training: A randomized controlled pilot study. Neurorehabil Neural Repair 2010; 24: 348–357.

66.

Park

Yim

. A new approach to improve cognition, muscle strength, and postural balance in community-dwelling elderly with a 3-d virtual reality kayak program. Tohoku J Exp Med 2016; 238: 1–8.

67.

Davis

Ohman

Weisbeck

. Salient cues and wayfinding in alzheimer’s disease within a virtual senior residence. Environ Behav 2017; 49: 1038–1065.

68.

Shigemori

Ohgi

Okuyama

et al. The factorial structure of the mini mental state examination (mmse) in japanese dementia patients. BMC Geriatr 2010; 10: 1–7.

69.

Feher

Mahurin

Doody

et al. Establishing the limits of the mini-mental state: Examination of ‘subtests’. Arch Neurol 1992; 49: 87–92.

70.

Tombaugh

McIntyre

. The mini-mental state examination: A comprehensive review. J Am Geriatr Soc 1992; 40: 922–935.

71.

Adhyaru

Kemp

. Virtual reality as a tool to promote wellbeing in the workplace. DIGITAL HEALTH 2022; 8. DOI: 10.1177/20552076221084473.

72.

Games

. Nature Treks VR. Available at: https://www.greenergames.net/nature-treks (2023, accessed 11 February, 2023).

73.

Park

Geum

Choi

et al. Validation of a questionnaire for patient awareness and the need for a community-based outpatient antimicrobial stewardship program (o-asp): A pilot study. Antibiotics 2021; 10: 441.

74.

Streiner

. Starting at the beginning: An introduction to coefficient alpha and internal consistency. J Pers Assess 2003; 80: 99–103.

75.

Lincoln

Guba

. Naturalistic inquiry. Beverly Hills, CA: Sage Publications, 1985.

76.

Raharjanti

et al. Translation, validity and reliability of decision style scale in forensic psychiatric setting in Indonesia. Heliyon 2022; 8: e09810.

77.

Collingridge Moore

Payne

Van den Block

. et al.: Research, recruitment and observational data collection in care homes: Lessons from the PACE study. BMC Res Notes 2019; 12: 508.

78.

Maas

Kelley

Park

, et al. Issues in conducting research in nursing homes. West J Nurs Res 2002; 24: 373–389.

79.

Mini-mental state examination. Available at: https://oxfordmedicaleducation.com/geriatrics/mini-mental-state-examination-mmse/ (2023, accessed 7 March 2023).

80.

D’Cunha

Nguyen

Naumovski

, et al. A mini-review of virtual reality-based interventions to promote well-being for people living with dementia and mild cognitive impairment. Gerontology 2019 May 20; 65: 430–440.

81.

Stamm

Dahms

Müller-Werdan

. Virtual reality in pain therapy: A requirements analysis for older adults with chronic back pain. J Neuro Eng Rehabil 2020; 17.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB

0.02 MB

0.05 MB

0.03 MB

0.01 MB

0.05 MB