Sage Journals: Discover world-class research

Abstract

Introduction

Increasingly, geriatric institutions are adopting video calling services to promote social interactions between residents and their loved ones. However, a gap persists between the technologies used and the needs and abilities of older users. Interactive Televisions (iTVs) could be part of the solution as they provide a familiar and accessible way to strengthen the relationship between residents and their environment (family, friends, professionals).

Methods

User tests were conducted to evaluate the ability of older adults (OAs) from different place of living to use iTV. A total of 32 user tests were carried out with OAs living at home, in residential facilities (RFs), and in nursing homes (NHs) between November 2022 and June 2023.

Results

The quantitative analysis revealed mixed opinions on the iTV’s usability. OAs living at home tended to produce less errors and needed less help to successfully use video calling and messaging functionalities of the iTV than others. However, participants in NHs required more repetitions to benefit from a learning effect, unlike participants in RFs and at home.

Conclusions

This study proposed another approach to user testing, based on cognitive psychology methods. Based on the analysis of the critical stages, ergonomic recommendations were identified.

Keywords

Interactive television older adults user testing usability

Introduction

Information and Communication Technologies (ICTs) are increasingly present in various care settings such as hospitals or geriatric institutions (e.g., nursing home). Aceto et al.¹ have summarized novel healthcare applications of ICTs between 2011 and 2016. They identified four major roles played by the ICT ecosystem: communication, sensing, processing, and actuation. For several years now, care homes and nursing homes for older adults (OAs) have been equipping themselves with ICTs for purposes of entertainment or cognitive stimulation purposes (e.g., Tovertafle,² Lumeen.³ During the Covid-19 pandemic and the resulting confinements, residents in these settings found themselves totally or partially deprived of opportunities for social contact (e.g., interruption of activities, decrease in visits from relatives).^4,5 This forced isolation had a dramatic impact on the physical and emotional well-being of residents who had or had not contracted Covid-19, with an increase in depressive symptoms and weight loss, reduced cognitive function and incontinence.⁶ Although there is a lack of evidence on the impact of ICTs on the loneliness of OAs,^7,8 these technologies could present a real opportunity for geriatric institutions to strengthen: (1) personal relationships (e.g., resident-family relationships), (2) links with the community (e.g., sporting, religious or recreational activities), and (3) social participation (i.e., a person’s access to information and resources, but also their ability to contribute - even in a modest way - to society).⁹ For this reason, and to promote personal relationships between residents and their loved ones, geriatric facilities have turned towards video calls,¹⁰ mainly using tablets, computers, or smartphones.¹¹ However, the long-term use of these ICTs does not solely depend on their usefulness, especially with a population as heterogeneous as OAs, and considering the range of residential institutions that accommodate different profiles of OAs. Residential facilities (RFs) are usually designed to accommodate autonomous or semi-autonomous OAs, while nursing homes are medical structures for OAs facing a loss of autonomy or suffering from medical problems requiring constant assistance, in most cases presenting as well varying degrees of neurocognitive disorders. In France, RFs represent a stepping-stone between home care and NH care, as residents’ clinical characteristics are probably different compared to other countries.¹² Thus, depending on their place of living (home, nursing home, residential facility), OAs may present very different cognitive and functional characteristics that can influence technology use.^12,13

With the development of SmartTVs, or Interactive Televisions (iTVs), entertainment (e.g., games), information (e.g., Internet search engine) and social networking functionalities (e.g., social networks) are now accessible directly on the television screen.^14,15 Television has the advantage of being already accepted as the main source of information and entertainment for OAs.¹⁵ The use of a familiar device that is already part of the daily lives of OAs, and thus less anxiety-provoking than other ICTs,^15–19 could therefore make it easier for this population group to accept new technology-based services. However, adding new functionalities to a device primarily designed for entertainment may lead OAs to adopt a more active attitude when using the remote control2²⁰ (e.g., navigating the menu, choosing between several services, etc.). Thus, new ergonomic issues could arise from both the TV interface (e.g., bad usability of the search functionality and user confusion when faced with the many services available on the iTV),^21,22 and the remote control (e.g., unclear buttons’ labels and icons, and small buttons).¹⁵ When designing an interface for OAs, it is important to consider the specific characteristics of this population (e.g., it takes longer to learn how to use new applications or devices, it takes longer to complete tasks, they use different search strategies, they are less successful at tasks requiring working-memory, they are more distracted, they have more difficulty managing errors).²³

To observe and identify more precisely the impact of these specific characteristics on activities of daily living, some researchers opted for activity analysis i.e., the evaluation of the ability to carry out an activity, based on the observation of the types of errors produced when carrying out a task. Schwartz et al. proposed a taxonomy of errors based on the following two main categories: omission and commission errors (e.g., objects substitution, action-addition, sequence errors like anticipation-omission, performance of a step in reverse order or perseveration).^24–26 Based on this model, Anselme et al. added initiation errors (i.e., not starting the activity, for any reasons).²⁷ Finally, Giovannetti et al. considered that action-addition errors were conceptually different from commission and omission errors.²⁸ More recently, Quillion-Dupré et al.²⁹ applied this methodology to the analysis of technology use (e.g., fixed phone, smartphone, and tablet) by young and healthy older adults. Based on this literature, they chose to examine the following four types of error: execution, omission, action addition and initiation error. They also identified the types of help needed to successfully complete each task, using a hierarchically organized help grid comprising verbal and physical aids, and inspired by the Profinteg tool.²⁷ Quillion-Dupré et al. found that OAs produced more omission, execution and initiation errors than young adults. According to Giovannetti et al.,²⁸ error profiles are influenced more by the context (the nature and difficulty of the task) than by the characteristics or cognitive disorders of the participants.

Several authors have carried out task analyses with OAs to study usability of iTVs offering games and other entertainment functionalities,¹⁵ home automation,³⁰ or medication and medical appointment reminders.¹⁸ However, few studies have been carried out with OAs living in geriatric institutions. The aim of this research was to assess the ability of OAs living in different types of housing configuration to use an iTV. To that end, we looked at the number and nature of the difficulties encountered by the participants, their level of autonomy when using the iTV, and the perceived usability of the iTV. The second objective was to put forward recommendations for usable iTV systems for OAs.

Methods and materials

Method

The method used for this study was inspired by User Centered Design, combining qualitative and quantitative methods for user testing.³¹ This approach seeks to evaluate the use of a product or service in real-life conditions, according to pre-established usage scenarios. By combining observation and an interview with the participant, the researcher can identify the difficulties encountered by the person during the use of the product. In a traditional user test, the participant must complete the task alone and the researchers remain observers. However, in this study, one of the objectives was to evaluate the level of assistance needed by participants and the kind of aid required to succeed in each task. We therefore drew inspiration from the aid grid used by Quillon-Dupré et al.²⁹ to differentiate the levels and types of help required for each stage.

Materials

The iTV

The iTV selected for these user tests is a system developed by Technosens (Grenoble, France). Since 2011, this system has been installed in geriatric facilities, directly in residents’ rooms. Its main aim is to foster social connectedness within geriatric institutions, and within the triad of resident/family/staff members. It consists of a box connected to a TV screen which offers communication (e.g., messages, photos, video calls), entertainment (e.g., TV channels, radio) and information services (e.g., news from the institution). All these functions can be accessed via a single remote control (Figure 1). The operating principle is as follows: when the TV is switched on, the user is taken directly to his or her TV channels. From there, they can either display the TV’s main menu (Figure 2(a)) or use shortcut buttons to access other functionalities. Notifications of video calls and messages are received directly on the TV channels (Figure 2(b)).

Figure 1.

The e-lioTV system (box, camera, and remote control).

Figure 2.

The e-lioTV interface: (a) Main menu; (b) TV program with a message notification.

The scenarios

The iTV used for this study offers a total of six functions. To limit the length of the user tests, given the participants’ fatigue, only two functions were tested: video calls and messages. These features, which are common on computers and smartphones, are not usual on televisions. Moreover, they could play a major role in preventing loneliness among OAs in geriatric institutions.

Three scenarios were designed to guide participants:

1. Answering a video call. This scenario involves answering an incoming video call (Figure 3(a)).

2. Checking a message. This scenario involves reading a message and viewing the related photos (Figure 3(b)).

3. Making a video call. This scenario involves launching a video call from a directory on the iTV (Figure 3(c)).

Figure 3.

Steps required to complete each scenario assessed: (a) Scenario 1: Answering a video call; (b) Scenario 2: Checking a message; (c) Scenario 3: Making a video call.

Each scenario was then broken down into sub-tasks, corresponding to the smallest possible task units, enabling the researcher to identify the difficulties encountered at each stage of the use of these functionalities, but also to identify the critical stages, i.e., the steps requiring the most help in general. It is important to note that these scenarios were always presented in the same order: from the easiest or involving the fewest steps, to the most difficult or involving the most numerous steps.

Performance assessment grid

To evaluate the participants’ performance, we used the evaluation grid used by Quillion-Dupré et al.,²⁹ which was inspired by the Profinteg tool²⁷ (see Figure 4). From the videos and transcripts, we noted the number and type of errors made (Table 1), a short description of the difficulty encountered, and the number and type of help required to succeed at each subtask (Table 2). The order and nature of aids given were pre-established to give a response from the least to the most informative. Thus, for the same subtask, level 0 help was given two times before moving on to level 1 help. Note that help was only given if the participant got stuck, asked for help, or made a mistake.

Figure 4.

An extract from the performance assessment grid for the first scenario.

Table 1.

Taxonomy of errors.

Type of error	Description
Initiation	The participant does not start the activity spontaneously after 15 s (for whatever reason) or he/she says, “I don’t know”
Omission	The participant does not perform a step (including forgetting the content) (e.g., does not read the message). To be considered an omission, the error must not result from another type of error
Erroneous execution	Incorrect performance of the task, not in the right way, or not at the right time (e.g., press the “microphone” button to activate the camera), including perseveration: a step is performed more than once (e.g., scrolling through the photos in a loop)
Action-addition	Performs an action that is not considered a necessary step to complete the task (e.g., deactivates the camera before hanging up)

Table 2.

Taxonomy of the hierarchical aids.

Type of aid	Description
General (AVG)	Encourage the participant to continue or point out that they are making a mistake without specifying its nature (e.g., “careful, you’re making a mistake”, “Hm hm”)
Specific (AVS)	Verbal indication of the nature of the error but no indication of what to do or how to do it (e.g., “You’ve forgotten something”)
Total (AVT)	Detailed explanation of what the participant must do (e.g., press the “Ok” button)
Gestural (AG)	Associating a finger gesture with a full verbal aid (e.g., “Press…” while pointing to the button in question)
Physical (AP)	Performing the step in the participant’s place (e.g., the assessor presses the power button on the remote control)

Questionnaires

Before the beginning of the user test, we administered the Mini-Mental State Examination (MMSE)³² to each participant. A total of four questionnaires were then submitted to assess the socio-demographic characteristics and technological habits of the participants (e.g., frequency of technology use), the perceived difficulty of each scenario (Single Ease Question (SEQ), see Figure 5), and the overall perceived usability (System Usability Scale (SUS)³³).

Figure 5.

Single ease question (SEQ).

Participants and recruitment

Participants in the study lived either in nursing homes (NHs), residential facilities (RFs) or at home, and were not e-lioTV users. The inclusion criteria were as follows: (1) To be over 60 years; (2) To be literate; (3) To be able to use a television remote control by themselves. Exclusion criteria included: (1) having an MMSE score of less than 16; (2) having visual, hearing or motor problems that could prevent the use of a TV remote control, unless deficits were compensated for by an assistive technology. To help the recruitment of suitable participants, for NHs and RFs residents, professionals working in these institutions draw up a preliminary list of willing residents who met the above criteria.

In total, 38 participants were approached (G_RF: n = 10; G_NH: n = 12; G_Home: n = 16) between November 2022 and June 2023. Among the participants in the G_RF, one dropped out at the last scenario, and among those from the G_NH, four dropped out before the end of the three scenarios, and one person was excluded from the group because he was illiterate. A total of 32 participants were included, including 23 women and 9 men.

The demographic characteristics of the three groups are presented in Table 3. The G_NH had a significantly higher average age than the G_RF (U = 12.5, p < .05) and the G_Home (U = 18, p < .001). In terms of education, the G_Home had on average significantly higher levels of education than the G_RF (U = 8, p < .001) and the G_NH (U = 3.5, p < .001). There were no significant differences between the three groups for the MMSE score, as only people without moderate or major cognitive impairment were included in this study. However, it is interesting to note that the MMSE scores obtained in this study are in line with the literature^12,34: on average, participants living at home obtained a higher MMSE score than those living in RF, and those living in NH.

Table 3.

Socio-demographic characteristics of participants.

	Age	Education (years)	MMSE
	m (SD)	m (SD)	m (SD)
	Min - Max	Min - Max	Min - Max
Nursing home n = 7	86 (7.75) 70–93	9.43 (1.81) 7–12	24.9 (1.61) 21–29
Residential facility n = 9	78.2 (9.92) 62–89	10.4 (2.3) 7–15	26.2 (2.82) 21–29
Home n = 16	76.8 (6.61) 62–89	16.2 (2.46) 10–20	27.9 (1.61) 24–30
Total n = 32	79.2 (8.47) 62–93	13.1 (3.86) 7–20	27.5 (2.18) 21–30

To explore participants’ technological experience, we assessed the frequency of mobile phone use by functionality (e.g. calling, answering a call, sending a message, checking a message), as well as the overall frequency of computer and tablet use. We then categorized their expertise with the technology as follows: a person is considered Expert if they use the feature/technology “Several times a day” or “Every day or almost every day”; a person is considered Non-Expert if they do not use the feature/technology or if they use it “Every week” or “Every month”. According to Table 4, most participants in the G_RF and the G_NH were not experts in the use of computers, unlike participants in the G_Home. In addition, some of the G_RF used their mobile phones more frequently (phone calls and messages) than the G_NH.

Table 4.

Technological expertise of participants.

	Level of expertise	Mobile phone use				Computer use	Tablet use
	Level of expertise	Make a call	Answer a call	Send a message	Receive a message	Computer use	Tablet use
Nursing home n = 7	Expert	1	2	0	0	1	0
Nursing home n = 7	Non-expert	6	5	7	7	6	7
Residential facility n = 9	Expert	5	8	4	5	0	1
Residential facility n = 9	Non-expert	4	1	5	4	9	8
Home n = 16	Expert	14	15	15	15	16	5
Home n = 16	Non-expert	2	1	1	1	0	11
Total n = 32	Expert	20	25	19	20	17	6
Total n = 32	Non-expert	12	7	13	12	15	26

Procedure

The tests were carried out in individual sessions. On the day of the appointment, the participant was reminded of the objectives of the study, and then presented with the overall functioning of the iTV (e.g., the different components of the system such as the box, the camera, and the remote control). Once written consent had been obtained, and before starting the test, we assessed the overall cognitive level (MMSE score) of each participant.

The test consisted of three phases. The aim of the first phase was to examine the socio-demographic characteristics and the technological expertise of each participant. The second phase involved using the iTV in the three scenarios described in the previous section. After each scenario, participants were asked to rate the level of difficulty they had experienced completing the task (SEQ questionnaire). A short discussion was also held to review the most difficult actions encountered when completing the tasks and to gather any recommendations or opinions on the iTV. The aim of the third phase was to examine the usability of the iTV using the SUS questionnaire. If necessary, the researcher asked the participant to expand on certain answers. Finally, at the participant’s request, the researcher could provide further explanations of the iTV’s target audience, and the rest of the functionalities not used as part of this test (e.g., radio, news of the institution, games). A summary of the procedure applied is available in Figure 6.

Figure 6.

Diagram of the procedure adopted for this research.

Ethical approval for this study was obtained from the Research and Ethics Committee from Université Paris Cité in November 2021 (No. 00012021-91). Validation of the data management procedures and related compliance with the General Data Protection Regulation (GDRP) was obtained after the Data Protection Office and registered in the general register of Greater Paris University Hospitals (AP-HP) in February 2022 (No. 20220228123925).

Quantitative data analysis

The participants’ ability to use the iTV was assessed based on the number and types of errors and aids required, as well as the score obtained on the SUS questionnaire. To compare the performance between each scenario, we used an error and aid index corresponding to the number of errors/aids divided by the number of steps in the scenario.

In this study, the dependent variables were the error and aid indices, the SUS score and the SEQ scores. Of these variables, only the SUS score followed a normal distribution and showed equality of variance. Intergroup differences (G_RF, G_Home, G_NH) were then calculated using the non-parametric Kruskal-Wallis tests (error and aid indices, SEQ scores) and the Student test (SUS score). Intra-group differences (scenarios 1, 2 and 3) were analyzed using Friedman’s ANOVA, and correlations were calculated using Spearman tests.

Results

Level of difficulty and autonomy of participants while using the iTV

The inter-group analysis, using the Kruskal-Wallis test, revealed that the average error (χ² (2) = 20.5, p < .001) and aid indices (χ² (2) = 22, p < .001) were significantly different between the three groups. While the average error index curve increases for the G_NH, for the G_RF and for the G_Home it tends to decrease (Figure 7(a)). The pairwise comparisons revealed that participants in the G_NH and the G_RF produced significantly more errors than those in the G_Home for scenario 1 and 3 (G_NH: S1, p < .05; S2, p < .01; S3, p < .001) (G_RF: S1, p < .01; S2, ns.; S3, p < .01). However, no significant differences were observed between the G_RF and the G_Home for the scenario 2. Concerning the average aid indices (Figure 7(b)), the G_NH and the G_RF needed significantly more help than the G_Home for all the scenarios (G_NH: S1, p < .01; S2, p < .01; S3, p < .001) (G_RF: S1, p < .001; S2, p < .001; S3, p < .001). Finally, although there were no significant differences between the G_NH and the G_RF in the errors produced and aids required, it is interesting to note a difference in trends across the three scenarios. Participants in the G_NH required increasing aid, while those in the G_RF became more independent for the tasks.

Figure 7.

Average (a) error and (b) aid index by scenario.

Intra-group (scenario 1, 2, and 3) descriptive analyses, considering the number of steps, revealed a slight decrease in errors and aids as the scenarios progressed for participants in the G_RF and the G_Home. Regarding mean error indices, the Friedman ANOVA conducted for the three scenarios showed that the differences observed were significant only for the G_Home (F(2) = 6.92, p < .05), with pairwise comparisons confirming a significant reduction in errors between scenario 2 and 3 (p < .01). About the average aid indices, the differences observed were significant only for the G_RF (F(2) = 11, p < .01), with pairwise comparisons confirming a significant reduction in aids between scenario 1 and 2 (p < .001).

After each scenario, participants were asked to rate the difficulty of the tasks using a 7-point Likert scale. When we look at Figure 8, SEQ scores of the G_RF do not differ significantly from those of the G_Home, unlike those of the G_NH, which are significantly lower for scenario 2 (S1, ns.; S2, p < .01; S3, ns.). A correlation analysis for all the participants, using the Spearman coefficient, showed a significant negative correlation between the average error indices and the SEQ scores obtained in scenarios 1 (ρ = −0.365, p < .05), 2 (ρ = −0.586, p < .001) and 3 (ρ = −0.551, p < .01). In other words, the more errors participants made during the scenarios, the more likely they were to consider the tasks difficult.

Figure 8.

Perceived difficulty of each scenario (SEQ scores).

Nature of the difficulties encountered by participants with the iTV

The participants produced a greater number of execution errors on average, regardless of the scenario or group (see Figure 9). The major difference observed between the three groups was in the production of initiation errors: for the G_RF and the G_NH, they represent the second most common type of error produced for all scenarios, whereas they are negligible for the G_Home. Exploring the inter-group differences by type of errors, significant differences were found between the three groups for initiation (χ² (2) = 19.2, p < .001), execution (χ² (2) = 12, p < .01) and omission errors (χ² (2) = 10.2, p < .01). Indeed, the pairwise comparisons confirmed the institutionalized participants produced significantly more initiation errors than the G_Home (G_NH and G_RF: p < .01). However, it was the G_RF participants who produced significantly more omission errors than the G_Home (p < .01), and the G_NH participants who produced significantly more execution errors than the G_Home (p < .01). No significant differences were found between the G_NH and the G_RF for all types of errors. Finally, while initiation error indices were significantly different between the three groups for all scenarios (S1, χ² (2) = 9.8, p < .01; S2, χ² (2) = 17.99, p < .001; S3, χ² (2) = 19.85, p < .001), the execution error indices were only significantly different for scenarios 2 and 3 (S1, χ² (2) = 4.67, ns.; S2, χ² (2) = 9.23, p < .05; S3, χ² (2) = 17.59, p < .001), whereas the omission error indices were significantly different for scenario 2 (S1, χ² (2) = 5.01, ns.; S2, χ² (2) = 6.96, p < .05; S3, χ² (2) = 3.9, ns.).

Figure 9.

Types and number of errors by scenario and by group of participants.

Intra-group descriptive analyses of the mean error indices, considering the number of steps, revealed a slight decrease in execution errors as the scenarios progressed for participants in the G_RF and the G_Home, whereas they slightly increase for the G_NH. Regarding mean execution error indices, the Friedman ANOVA conducted for the three scenarios showed that the differences observed were significant only for the G_RF (F(2) = 6.89, p < .05), with pairwise comparisons confirming a significant reduction in errors between scenario 1 and 3 (p < .01). Considering other types of errors, no significant differences were found between the scenarios for the three groups.

A correlation analysis for all the participants, using the Spearman coefficient, showed significant negative correlations between the SEQ scores and the execution (S1, ρ = −0.458, p < .01; S2, ρ = −0.556, p < .001; S3, ρ = −0.496, p < .01), initiation (S1, ns.; S2, ρ = −0.510, p < .01; S3, ρ = −0.429, p < .05), and omission error indices (S1, ns.; S2, ns.; S3, ρ = −0.579, p < .001). No correlations between the addition error indices and the SEQ scores were found for all the scenarios.

In the following sub-sections, we will explore more precisely the nature of the difficulties encountered at each stage, for each scenario. Since the aim was to identify the critical stages in the use of the video calling and messaging functions, no distinction has been made between people living at home, in NH or in RF. Thus, the following figures include all participants, regardless of group.

Scenario 1

Regarding the first scenario, participants made more errors during step 2: picking up the video call. Apart from potential ergonomic issues, this was the first step to be performed on the iTV after switching on the TV, so it’s also possible that participants were surprised. However, most of the errors produced were execution errors, meaning that participants tended to press the wrong button during this step (Figure 10).

Figure 10.

Number and type of errors produced by all participants in scenario 1, step 2.

Based on the observations, the main problems encountered by participants were:

- Not paying attention to the notification (initiation errors: staring at the TV program)

- Searching for a button related to the sound cue (telephone ringtone) (execution errors: pressing the bell, or camera buttons)

- Not understanding the notification or the action to be taken (execution errors: pressing a TV channel button; initiation errors: asking for help).

Scenario 2

Regarding the second scenario, participants made more errors during step 5: scrolling over the second photo. A large majority of the errors produced concerned execution errors, meaning that participants were aware that they had to perform an action (go and see the second photo), but had great difficulty in finding the right button on the remote control (Figure 11).

Figure 11.

Number and type of errors produced by all participants in scenario 2, step 5.

Based on the observations, the main problems encountered by participants were:

- Searching for a button related to the photos or the photo number (execution error: pressing the channel number 2 button).

- Not understanding the action to be taken (initiation error: starring at the photo and asking for help)

- Focusing only on the guidance that says “back” (omission error: returning to the message without having seen all the photos).

These observations, combined with the evaluation of the types of errors produced, highlighted the lack of meaningful information on the interface, as well as the use of codes (e.g., arrow buttons) that may not yet be universally accepted by OAs.

Scenario 3

Regarding the third scenario, participants made more errors during step 3: choosing the contact to call. Participants produced almost as many initiation errors as execution errors, meaning that they were almost as confused by the action to be performed as they were misled by the interface (Figure 12).

Figure 12.

Number and type of errors produced by all participants in scenario 3, step3.

Based on their observations, the main problems encountered by participants were:

- Difficulties in identifying button indications from descriptive information in the interface (initiation errors: reading the call history, ignoring the guidance)

- Unclear navigation in the contact list (initiation errors: searching for the contact’s name directly on the remote control; execution errors: pressing the repertory button; omission errors: initiating the call without having selected the contact).

Perceived usability of the iTV

Bangor et al.³⁵ developed a grading scale in which SUS scores below 50.9 out of 100 were poor, between 50.9 and 71.4 were ok, and above 71.4 were good and acceptable. According to Figure 13 and the Student’s t-test, the SUS scores obtained by the G_NH and the G_RF were significantly lower than the one obtained by participants living at home (t (30) = 6.09, p < .001). . In other words, participants living in NH considered the iTV’s usability to be poor (m = 47.9/100), i.e. not acceptable for a device intended for people living in institutions. Nevertheless, the usability of the iTV was correct for participants living in RF (m = 55.8/100), and good for participants living at home (m = 76.6/100). If we consider only the G_RF and the G_NH, no significant difference was observed concerning the overall usability of the system.

Figure 13.

Perceived usability of the iTV system (SUS scores).

If we take a closer look at the scores obtained for each statement on the SUS scale (Figure 14), it is possible to identify the dimensions on which the three groups diverge. Examining the inter-group differences in terms of scores per item, significant differences were found between the three groups for the perceived complexity (χ² (2) = 15.5, p < .001), the perceived ease of use (χ² (2) = 6.4, p < .05), the need for technical support (χ² (2) = 20.6, p < .001), the perceived learnability (χ² (2) = 14.3, p < .001), and the efforts required to become familiar with the iTV system (χ² (2) = 18.6, p < .001).

Figure 14.

Average scores by statement of the SUS scale and by group of participants.

The opinions on the iTV’s perceived complexity (statement (2) and ease of use (statement (3) of institutionalized participants (G_NH and G_RF), and particularly those living in NH, were more mixed than other participants.. Moreover, institutionalized participants reported a greater need for technical support with the iTV system (statement 4). It is also interesting to note that the perceived learnability (statement (7) may depend on the respondent’s context, as the G_NH found the iTV significantly more difficult to learn than the G_Home. Finally, participants in RF and NH thought it would require significantly more effort to become familiar with the iTV system (statement 10) than participants at home.

Discussion

This study aimed to evaluate the ability of OAs living at home (n = 16), and in geriatric institutions (nursing home: n = 7; and residential facility: n = 9) to use an iTV providing social functionalities, such as video calls and messaging. The ability to use the iTV was measured using the number and type of errors and aids produced and needed by the participants, as well as the perceived usability. Except for the participants living in NHs, all the OAs tended to make fewer mistakes, and to become more autonomous as the scenarios progressed. Finally, only participants living in NHs considered the usability of the iTV to be poor, with all others considering it to be either acceptable or even good.

OAs’ abilities to use an iTV

The correlation analysis, using the Spearman coefficient, indicated a negative correlation between the number of errors and the perceived difficulty of each scenario. A closer look at the relationship between the perceived difficulty of the task, and the type of errors produced revealed that blocking (i.e., initiation errors) during the first scenario did not seem to affect the participants’ experience, as if it was normal to hesitate at this point in the interaction. However, the more the participants pressed the wrong buttons (i.e., execution errors), the more likely they were to consider the tasks difficult. Finally, during the last scenario, forgetting some steps (i.e., omission errors) could be more frustrating for participants, affecting the perceived ease of use of the iTV.

Regarding the perceived usability of the iTV, differences between the groups were identified for the SUS score as well as the following dimensions: the perceived complexity, the perceived ease of use, the need for technical support, the perceived learnability, and the efforts required to become familiar with the iTV system. These differences could be linked to the technological expertise (Table 4), as only a few institutionalized participants, mostly living in RF, were experts in using smartphones or computers. Depending on their habits, some actions may seem difficult to carry out, such as navigating using the arrows on the remote control. In addition, participants living at home seemed more comfortable and confident than the others with learning to use a new technology, influencing perceived workload of using the iTV. Finally, the perceived learnability statement was phrased to imagine how other people would learn to use the iTV. The responses could then be influenced by the participants’ immediate environment, and therefore the configuration of their living environment. Thus, participants living in NH may think of their next-door neighbors, perceived as frail, while participants living at home may think of their friends who are still independent (“Given the state of the house… [no, they couldn’t learn to use the device] Always having to repeat, and “I can’t remember”, and “why we’re doing this”” (NH-6)).

The three groups (home, NH, and RF) performed differently in terms of number and type of errors and aids. Each type of error was committed at least once by each group, even though participants living in institutions produced more initiation, omission, and execution errors than the G_Home. Indeed, institutionalized OAs are usually frailer, or more at risk of frailty, and present a greater cognitive decline than OAs remaining in the community,^12,34 these characteristics may influence their performance and perceived usability when discovering a technological innovation. Moreover, the number of initiation and execution errors of the NH and RF groups only differed in the third scenario. Indeed, Boucaud-Maitre et al. noted that RFs in France represent a stepping-stone between home care and nursing home care in terms of health trajectories of OAs.¹² This is in line with our results where participants living in RF seemed to constitute an intermediate group between the home and the NH groups. Finally, only the home and RF groups saw their number of aids and errors decrease over the three scenarios. Since the third scenario had two steps in common with the first one (e.g. turning on the camera and hanging up the call), only the G_NH seemed to need additional repetitions to benefit from a learning effect. Moreover, Wandke et al.³⁶ have noticed that if the first attempts are unsuccessful, OAs tend to adopt avoidance strategies. These results show how important it is to develop interactive technologies that are easy for everyone to learn, and to provide appropriate support for people living in NHs, using a method based on error-free learning and spaced recovery.³⁷ This method consists of avoiding incorrect responses when learning new information, while reducing the number of cues provided until the desired response is given without assistance. The information is then repeated and recalled at increasingly long intervals until the desired response is memorized.

Ergonomic evaluation of the iTV

The second part of this study consisted in identifying the critical stages in the handling of the iTV. This stage is based entirely on the match between three elements: the participants’ conceptual model (i.e., how they imagine the iTV will work), the designer’s conceptual model, and the system image (i.e., what users perceive of the product’s physical structure).³⁸ Most of the difficulties encountered can be explained by a mismatch between the user’s mental model and that of the iTV’s designer. The very principle of this iTV is to centralize on a TV screen functionalities usually available on a mobile phone or computer. The design of the interface and the signifiers used therefore have a major influence on the user’s reasoning. For example, in scenario 1, based on the contextual elements provided by the researcher and the iTV (visual: remote control, TV screen; audio: researcher’s explanation and ringtone), the participants had difficulties understanding how a video call worked on this TV. Some participants were frustrated at not being able to find the button usually used on a telephone (“I had to navigate around and forget what I normally do on my own devices. That’s the problem. Normally, to pick up the phone, I press this [phone button]” (H-2)), while others were stuck, not understanding how to pick up with a remote control (“Pick up, what does that mean? Pick up what?” (NH-8)). This problem seems to be inherent in the principle of diverting the use of TV. In an already complex and rich technological environment, it is vital to think about compatibility with everyday technologies. Turning on the TV or picking up the phone are actions that have become almost automatic for many OAs. The challenge then lies in identifying these automatisms, as the OAs have a wide range of experience of technologies. Another source of misunderstanding for the participants was the principle of having indications on the TV screen. When participants picked up the remote control, they focused solely on it, ignoring the indications on the TV screen. This operating principle could be part of the information to be given during initial training.

Conducting user tests with OAs

The method adopted for this study was inspired by a User Centered Design method named user testing. However, to evaluate the level of assistance required to successfully perform each task, aids were provided along with the difficulties encountered. One of the advantages of this approach was to limit the number of dropouts during the test. Aging is accompanied by perceptual, physiological, and cognitive changes,³⁹ and is characterized by greater fatigability (i.e., a phenotype characterized by the relationship between self-reported fatigue and the level of activity with which the fatigue is associated) than younger adults.⁴⁰ Despite the measures taken to limit participant fatigue, five participants abandoned before the end of the three scenarios. In the literature, it is recommended to adopt some strategies when conducting user tests with OAs, such as creating a safe environment, proposing a small number of tasks to be carried out, or establishing a climate of trust with the participants. All these recommendations have a major influence on the format and duration of the test.⁴¹

Future work

This article presents a quantitative analysis of the study, focusing solely on the iTV’s ergonomic issues and perceived usability. Although the ergonomic qualities of the technology are crucial for assessing its adoption, other factors may also be at play. According to the Technology Acceptance Model (TAM), the acceptance of a technology i.e., the extent to which a product will actually be used or not, depends on two main factors: perceived usefulness and perceived ease of use.⁴² Venkatesh et al.⁴³ have also integrated to the model contextual factors (e.g., organization, physical environment), as well as individual factors (e.g., user characteristics, technologies). Thus, future work will consist of a qualitative analysis of the user tests, focusing on the factors determining the acceptability of the iTV among OAs.

Limitations of the study

The first limitation of this study was the kind of recruitment of OAs living at home. Due to technical and recruitment constraints, the user tests with the G_Home were all conducted in the Paris region, while those with OAs living in nursing homes were conducted in rural areas. This recruitment bias may explain the association observed between the number of years of study and the place of living (χ² (1) = 21.2, p < .001).

The second limitation concerns the MMSE score used to assess participants’ global cognitive functions. Although this score may be correlated with the number of errors produced while using a technology,^28,44 no correlation has been found in this study. MMSE scores do not provide information on the precise state of executive functions, which are particularly vulnerable with ageing and key in technology use. In this case, the assessment of executive functions could then provide further explanations for performance on iTV.

Recommendations for the design of an iTV for older adults

Using the telephone metaphor for video calls

In psychology, the principle of perpetual habit states that individuals prefer to rely on their memory and habits when doing familiar tasks, rather than spending time learning new procedures. OAs are used to the telephone functioning i.e., picking up by clicking on the green phone, and hanging up with the red phone. Thus, changing this familiar action can trigger confusion, anxiety and even anger. To facilitate the use of video calls on the iTV, it might be preferable to rely on existing mental models using familiar telephone symbols for example (e.g., green telephone button).

Differentiating the call and message notifications from the TV channel banner

TV channel banners are usually displaying information about the channel and its current program. As a result, participants of the user tests tended to ignore a part of the information displayed on the iTV notification. Designers could change its appearance and wording to differentiate it from the TV channel banner, and thus draw users’ attention to important information. For example, designers could use a different background color, enlarge the notification, and include the user’s first name to encourage action. This will help users to feel more involved and committed to responding to iTV requests (e.g., opening/answering or closing the notification).

Indicating the role of the arrows on the screen

Older adults usually use the same buttons on their remote control, ignoring the purpose of others, such as the arrows. However, these buttons are essential for navigating the iTV interface and scrolling through photos. Designers should then indicate at each step how to use the arrows. On the photo screen, an arrow symbol could be displayed to the right of the photo all the time, except for the last photo.

Ensure that the guidance is unique for a given screen

On the iTV, the buttons are systematically indicated on the left of the screen in a dedicated insert. However, OAs may find it difficult to remember to look at the screen to find the right button, and to switch back and forth between the remote control and the screen. For all these reasons, designers should ensure that guidance information does not change on the same screen: all the information needed to interact with the iTV should be displayed from the outset.

Conclusion

This study aimed to evaluate objectively the ability of OAs with different levels of dependency to use an iTV providing social functionalities, such as video calls and messaging. Intergroups differences were found, as participants living in NHs produced more errors and needed more aids than more independent participants during the test. However, participants living in geriatric institutions produced more initiation errors than those living at home, thus they needed more confirmation aids from the researcher. The detailed analysis of each scenario allowed to highlight the critical steps when using the video calling and messaging functionalities for the first time, as well as the types of errors most frequently produced. Concerning the usability of the device, participants living in institutions considered the iTV significantly less usable than those living at home. However, a difference should be noted between the two types of institution. While participants in NH considered the iTV’s usability to be poor, those in RF considered it to be acceptable. Finally, regardless of the type of institution, institutionalized participants reported needing additional support, as well as more time and effort to master the iTV than participants living at home.

To conclude, combining a user centered approach with a cognitive psychology method have enabled us to gain a better understanding of the difficulties encountered by OAs when using iTV for the first time. While some OAs only seemed to need confirmation aids to succeed at each step (e.g., general, or specific verbal aid), others required more extensive support from the researcher (e.g., total verbal aid, or even pointing). Based on the analysis of critical stages, four recommendations have been formulated to improve the iTV ergonomics.

Footnotes

Acknowledgements

This article was part of the PhD thesis (CIFRE) of B.N., which aims to study the development and use of an interactive television system in elderly care institutions. We sincerely thank the Technosens team, especially Cindy CHASSANG, Mélissa NEYRAND, Cyril THEVENET, and Yoann ROCHE for their contributions during the evaluation stage. Finally, we thank the geriatric institutions for agreeing to welcome the user tests and, of course, all participants for their contribution to this study.

Author contributions

Conceptualization, B.N. and A.-S.R.; methodology, B.N., A.-S.R. and M.P.; formal analysis, B.N. and A.-S.R.; investigation, B.N. and S.D.; writing—original draft preparation, B.N., A.-S.R., S.D., and M.P.; writing—review and editing, B.N., A.-S.R. and M.P.; project administration, B.N. All authors have read and agreed to the published version of the manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Technosens® (Grenoble), Association Nationale de la Recherche et de la Technologie (ANRT) (funding number 2021/0344), and the APC was funded by Assistance Publique-Hôpitaux de Paris (AP-HP).

Guarantor

BN.

ORCID iDs

Bérangéré Naudé

Maribel Pino

References

Aceto

Persico

Pescapé

. The role of information and communication technologies in healthcare: taxonomies, perspectives, and challenges. J Netw Comput Appl 2018; 107: 125–154. DOI: 10.1016/j.jnca.2018.02.008.

Tovertafel for seniors with dementia . Tover n.d. https://www.tover.care/uk/tovertafel/seniors (accessed January 9, 2024).

Lumeen, la réalité virtuelle sociale et thérapeutique n.d. https://www.lumeen.com/cas-dusage/etablissements-personnes-agees (accessed January 9, 2024).

Bouillon-Minois

J-B

Lahaye

Dutheil

. Coronavirus and quarantine: will we sacrifice our elderly to protect them? Arch Gerontol Geriatr 2020; 90: 104118. DOI: 10.1016/j.archger.2020.104118.

Freedman

Kasper

. Changes in older adults’ social contact during the COVID-19 pandemic. J Gerontol B Psychol Sci Soc Sci 2022; 77(7): e160–e166. DOI: 10.1093/geronb/gbab166.

Levere

Rowan

Wysocki

. The adverse effects of the COVID-19 pandemic on nursing home resident well-being. J Am Med Dir Assoc 2021; 22: 948–954.e2. DOI: 10.1016/j.jamda.2021.03.010.

Noone

McSharry

Smalle

, et al. Video calls for reducing social isolation and loneliness in older people: a rapid review. Cochrane Database Syst Rev 2020; 5(5): CD013632. DOI: 10.1002/14651858.CD013632.

Malinga

Schmidt

B-M

Wiysonge

. Cochrane corner: video calls for reducing social isolation and loneliness in older people. Pan Afr Med J 2020; 35: 78. DOI: 10.11604/pamj.supp.2020.35.24283.

Waycott

Vetere

Ozanne

. Building social connections: a framework for enriching older adults’ social connectedness through information and communication technologies. In: Neves

Vetere

(eds). Ageing and Digital Technology. Singapore: Springer, 2019, pp. 65–82. DOI: 10.1007/978-981-13-3693-5_5.

10.

Robič

Rotar Pavlič

. COVID-19 and care for the elderly in long-term care facilities: the role of information communication technology. Acta Med Acad 2021; 50: 414–422. DOI: 10.5644/ama2006-124.363.

11.

Naudé

Rigaud

A-S

Pino

. Video calls for older adults: a narrative review of experiments involving older adults in elderly care institutions. Front Public Health 2021; 9: 751150. DOI: 10.3389/fpubh.2021.751150.

12.

Boucaud-Maitre

Meillon

Letenneur

, et al. Health trajectories of elderly living in French senior housing: a longitudinal perspective. Sci Rep 2023; 13: 5471. DOI: 10.1038/s41598-023-32429-3.

13.

Cotten

Yost

Berkowsky

, et al. Designing technology training for older adults in continuing care retirement communities. Boca Raton, FL: CRC Press, 2017. DOI: 10.1201/9781315382463.

14.

Ahmed

Bures

. Testing of smart tv applications: key ingredients, challenges and proposed solutions. In: Arai

Bhatia

Kapoor

(eds). Proceedings of the Future Technologies Conference (FTC) 2018. Cham: Springer International Publishing, 2019, vol 880, pp. 241–256. DOI: 10.1007/978-3-030-02686-8_20.

15.

Epelde

Valencia

Carrasco

, et al. Providing universally accessible interactive services through TV sets: implementation and validation with elderly users. Multimed Tool Appl 2013; 67: 497–528. DOI: 10.1007/s11042-011-0949-0.

16.

Coelho

Guerreiro

Duarte

. Designing tv interaction for the elderly: a case study of the design for all approach. In: A multimodal End-2-End Approach to Accessible Computing. In Biswas

Duarte

Langdon

Almeida

Jung

(Eds.),London, UK: Springer, 2013, pp. 49–69. DOI: 10.1007/978-1-4471-5082-4.

17.

Mitchell

Nicolle

Maguire

, et al. Web-based interactive TV services for older users. Gerontechnology 2007; 6: 20–32. DOI: 10.4017/gt.2007.06.01.003.00.

18.

Santana-Mancilla

Anido-Rifón

. The technology acceptance of a TV platform for the elderly living alone or in public nursing homes. IJERPH 2017; 14: 617. DOI: 10.3390/ijerph14060617.

19.

Zamir

Hennessy

Taylor

, et al. Video-calls to reduce loneliness and social isolation within care environments for older people: an implementation study using collaborative action research. BMC Geriatr 2018; 18: 62. DOI: 10.1186/s12877-018-0746-y.

20.

Costa

Anido-Rifon

Fernandez-Iglesias

. An open architecture to support social and health services in a smart tv environment. IEEE J Biomed Health Inform 2017; 21: 549–560. DOI: 10.1109/JBHI.2016.2525725.

21.

Dou

Qin

Wang

, et al. Identification of usability problems and requirements of elderly Chinese users for smart TV interactions. Behav Inf Technol 2019; 38: 664–677. DOI: 10.1080/0144929X.2018.1551423.

22.

Andreadis

Zambon

Parlangeli

. TV as an experience conveyer for better acceptance of ICT services by older adults. Univers Access Inf Soc 2021; 20: 359–374. DOI: 10.1007/s10209-020-00731-w.

23.

Johnson

Finn

. Designing user interfaces for an aging population: towards universal design. Burlington, NJ: Morgan Kaufmann, 2017.

24.

Schwartz

Buxbaum

Montgomery

, et al. Naturalistic action production following right hemisphere stroke. Neuropsychologia 1999; 37: 51–66. DOI: 10.1016/s0028-3932(98)00066-9.

25.

Schwartz

Montgomery

Buxbaum

, et al. Naturalistic action impairment in closed head injury. Neuropsychology 1998; 12: 13–28. DOI: 10.1037//0894-4105.12.1.13.

26.

Schwartz

Montgomery

Fitzpatrick-DeSalme

, et al. Analysis of a disorder of everyday action. Cogn Neuropsychol 1995; 12: 863–892. DOI: 10.1080/02643299508251406.

27.

Anselme

Poncelet

Bouwens

, et al. Profinteg: a tool for real-life assessment of activities of daily living in patients with cognitive impairment. Psychol Belg 2013; 53: 3. DOI: 10.5334/pb-53-1-3.

28.

Giovannetti

Libon

Buxbaum

, et al. Naturalistic action impairments in dementia. Neuropsychologia, 2002; 40(8).

29.

Quillion-Dupré

Monfort

Rialle

. Abilities to use technological communication tools in aging: contribution of a structured performance-based evaluation. In: Zhou

Salvendy

(eds). Human aspects of IT for the aged population. Design for aging. Cham: Springer International Publishing, 2015, pp. 496–508. DOI: 10.1007/978-3-319-20892-3_48.

30.

Enríquez

Villamarín

Acosta

. Design and implementation of a home automation system with an interactive application for digital television based on the Ginga middleware. In: Abásolo

Silva

González

(eds). Applications and usability of interactive TV: jAUTI 2018. Cham: Springer, 2019.

31.

Abras

Maloney-Krichmar

Preece

. User-centered design. In: Encyclopedia of human-computer interaction, Bainbridge. Thousand Oaks, CA: Sage Publications, 2004.

32.

Folstein

McHugh

. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12: 189–198. DOI: 10.1016/0022-3956(75)90026-6.

33.

Brooke

. SUS: a « quick and dirty » usability scale. SUS: a « quick and dirty » usability scale, Padstow. Boca Raton, FL: CRC Press, 1996.

34.

González-Colaço Harmand

Meillon

Rullier

, et al. Cognitive decline after entering a nursing home: a 22-year follow-up study of institutionalized and noninstitutionalized elderly people. J Am Med Dir Assoc 2014; 15: 504–508. DOI: 10.1016/j.jamda.2014.02.006.

35.

Bangor

Kortum

Miller

. Determining what individual SUS scores mean: adding an adjective rating scale. Journal of Usability Studies. 2009; 4(3).

36.

Wandke

Sengpiel

Sönksen

. Myths about older people’s use of information and communication technology. Gerontology 2012; 58: 564–570. DOI: 10.1159/000339104.

37.

Czaja

Sharit

. Designing training and instructional programs for older adults. Boca Raton, FL: CRC Press, 2012.

38.

Norman

. The design of everyday things. New-York, NY: Basic Book, 2013.

39.

Matthews

Davies

Stammers

, et al. Human performance: cognition, stress, and individual differences. London, UK: Psychology Press, 2000.

40.

Eldadah

. Fatigue and fatigability in older adults. PM&R 2010; 2: 406–413. DOI: 10.1016/j.pmrj.2010.03.022.

41.

Franz

Neves

. Usability is ageless: conducting usability tests with older adults. In: Neves

Vetere

(eds). Ageing and digital technology. Singapore: Springer, 2019, pp. 99–114. DOI: 10.1007/978-981-13-3693-5_7.

42.

Davis

. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 1989; 13: 319. DOI: 10.2307/249008.

43.

Venkatesh

Thong

. Unified theory of acceptance and use of technology: a synthesis and the road ahead. JAIS 2016; 17: 328–376. DOI: 10.17705/1jais.00428.

44.

Quillion-Dupré

. Usage de la tablette tactile par les personnes âgées: Une approche écologique de l’évaluation et de l’aide à l’apprentissage. 2018, p. 458.

Assessment of an interactive television that supports social connectedness among older adults living at home and in geriatric institutions: Usability and task performance analysis

Abstract

Introduction

Methods

Results

Conclusions

Keywords

Introduction

Methods and materials

Method

Materials

The iTV

The scenarios

Performance assessment grid

Questionnaires

Participants and recruitment

Procedure

Quantitative data analysis

Results

Level of difficulty and autonomy of participants while using the iTV

Nature of the difficulties encountered by participants with the iTV

Scenario 1

Scenario 2

Scenario 3

Perceived usability of the iTV

Discussion

OAs’ abilities to use an iTV

Ergonomic evaluation of the iTV

Conducting user tests with OAs

Future work

Limitations of the study

Recommendations for the design of an iTV for older adults

Using the telephone metaphor for video calls

Differentiating the call and message notifications from the TV channel banner

Indicating the role of the arrows on the screen

Ensure that the guidance is unique for a given screen

Conclusion

Footnotes

Acknowledgements

Author contributions

Declaration of conflicting interests

Funding

Guarantor

ORCID iDs

References