Sage Journals: Discover world-class research

Abstract

Background:

Digital solutions can support healthier ageing and better functioning throughout life; the AGAPE project aimed to co-create and implement a digital ecosystem of services to promote Active and Healthy Ageing, with the involvement of Older adults (OAs) and their formal and informal caregivers. The project’s co-creation methodology followed five phases: Pre-selection of devices; Focus groups and questionnaires; Workshops and Living Labs; Pre-Pilot; Pilot. Aim: The aim of this paper is to describe the first four phases of the project, including the results of the usability study (phases 3–4).

Design and methods:

Co-creation followed the YOUSE Toolbox of Active and Assisted Living projects for user-centred design. Phases 1–4 of the study were conducted in parallel in Italy, Portugal, Romania, to integrate feedback from all sites. To assess the usability and the level of stress related to technology usage (phases 3 and 4), the System Usability Scale (SUS) and the Technostress (TS) were administered, respectively.

Results:

Two hundred and eighty-two participants (Italy n = 103; Portugal n = 86; Romania n = 93) were involved. Participants appreciated the solution, its services and the health self-monitoring opportunities. Feedback from phases 1 to 4 led wearables choice, interfaces design and coaching strategies. SUS scores showed a significant reduction during Living Lab and Pre-Pilot (p < 0.001); TS did not significantly change. Both scores differed between user groups and countries.

Conclusions:

This paper presents a reproducible and measurable co-creation approach that has been successfully applied to develop different digital health services for AHA and rehabilitation.

Keywords

active and healthy ageing digital ecosystem co-creation user involvement

Introduction

The ageing pace of the population is significantly faster than in the past. Older Adults (OAs), defined as people aged 65 years or over, are a vulnerable population: various physiological, psychological, and socioeconomic factors may contribute to impairments, limitations and restrictions among the OAs.¹ The United Nations (UN) General Assembly declared 2021–2030 the ‘UN Decade for Healthy Aging’, asking WHO to lead its implementation, to reduce health inequalities and improve the life of older people, their families and communities.² Healthcare systems and funders need to invest in solutions that address both the social and behavioural determinants of health, while promoting healthy ageing.^3,4 Active and Healthy Ageing (AHA) is an approach aimed at improving the quality of life for older adults, emphasizing active participation, physical and mental health and independence.⁵ Social participation, maintaining healthy habits and institutional support are key factors.⁵ Rudnicka et al.,⁶ following the WHO’s perspective, defined healthy ageing as a process to optimize functional abilities and reduce inequalities.

Integrated policies and targeted interventions are essential to promoting a dignified, active, and healthy ageing process worldwide.^5,6 Actually, the preferred term now is Active and Helthier Ageing,⁷ meaning that such interventions should be aimed not only to healthy independent older persons, but also to those with morbid conditions and/or some kind of disability, to preserve and possibly improve their functioning and quality of life throughout their whole lifespan. Digital health solutions and smart technologies can provide clinical monitoring, telerehabilitation and support healthy ageing⁸ and OAs’ wellbeing.⁹ Older people using technology are empowered to keep in touch with community, family and peers and this is crucial to prevent social isolation and depression.¹⁰ Despite their cost-effectiveness and ability to promote healthier behaviours, the adoption rates of home-based digital solutions remain low.¹¹ OAs are increasingly motivated in using technology in everyday life¹² but age-related changes in physical and cognitive functioning may be barriers to the successful use of the technologies designed to help.¹³ An important barrier to the adoption of digital solutions for AHA is the lack of Information and Communication Technologies (ICT) knowledge and skills in older populations.¹⁴ Improving e-education and e-Health Literacy seems mandatory to ensure technology adoption and a better quality of life nowadays,¹⁵ as much as involving older adults in the co-creation of inclusive technological solutions that are both useful and user-friendly.

Within the AHA framework, the Active aGeing And PErsonalized services (AGAPE) project¹⁶ was designed to co-create and implement an ICT-based ecosystem of services to promote to promote OAs socialization, physical activity and digital literacy, by specifically targeting their innovation (digital) adoption, involving partners from Romania, Italy and Portugal to run three different pilot sites.

A key feature, developed in the AGAPE project to reach this purpose, is to provide users with expert coaching to adopt the co-developed digital solutions. The coach, an e-literate health professional, interacts with the users to customize the services, provide personalized assistance, content of knowledge, training and support.

The other key feature of AGAPE is the choice to develop the solution by a co-creation approach in all phases of the project, from the selection of the devices to the pre pilot phase, aimed at identifying potential usability issues before actual implementation. Co-creation involves collaborative efforts among various stakeholders to develop effective solutions¹⁷ and implies continuous communication, testing and changes being done to meet the needs of end-users. Everyone brings their own unique views, experiences, needs to be implemented, tested and optimized iteratively, to develop an effective and user-friendly solution.

The purpose of this paper is to present the co-creation methodology used in AGAPE to promote AHA, by involving OAs, their formal and informal caregivers, from the selection of devices to the results of the solution usability study.

Methodology

Sample, time and location

The AGAPE co-creation phases 1–4 took place during years 2022 and 2023, applying the same timeline, in three European regions: Bucharest (Romania), Coimbra (Portugal) and Mugello, Tuscany (Italy). To explore the cultural diversity in terms of cultural specifics and multiplicity of living context, an urban area in Romania, a rural area in Italy and day care urban centres in Portugal were selected. The study phases were conducted in parallel to incorporate feedback form all three sites from one phase to the next.

Representative samples of three categories of end-users were recruited, including:

- Older adults (OAs) without full-blown disease; aged 65 years and beyond; ability to provide written informed consent.

- Informal caregivers (ICs; i.e. family members of older participants) without cognitive impairment.

- Formal caregivers (FCs; healthcare and social care workers).

Participants were recruited by direct contact of usual clients of the centres involved in the research (local Health Society in Mugello, Elias Hospital in Bucharest, and Caritas in Coimbra) who were eligible and agreed to participate, their informal caregivers and healthcare and social care workers involved with them as formal caregivers, and included people with various background regarding education level, work experience domains, hobbies, level of digital skills and propension to use – from total lack of knowledge, resources and experience (especially Portuguese OAs) to higher levels of adoption and knowledge (Italian and Romanian OAs and potential Coaches in all pilot sites). con

The AGAPE project last phase implied a pilot study to deploy the co-created solution in the three study sites, as a premise for future testing of its effectiveness in promoting AHA. Thus, to carry out all project’s phases, a convenience sample was chosen, based on the researchers’ estimates of how many users they could possibly access within the project’s given resources and time-frame. All participants provided written informed consent.

Methodology and phases

The co-creation methodology applied in each phase of the present study is based on the YOUSE Toolbox of Active and Assisted Living (AAL)¹⁸: is a set of tools designed to actively integrate end user into the development of technological innovations projects for user-centred design purposes. It provides methods and resources to ensure that solutions are personalized and meet the real needs of users. It is multidimensional and defined to allow reproducibility,¹⁸ combining three key approaches: the Value-led approach¹⁹; the User-centred design²⁰; and the Design thinking.^21,22

The co-creation methodology of the project followed five phases with the gradual involvement of all user categories (Figure 1): (1) Pre-selection of devices, (2) Focus groups and questionnaires, (3) Workshops and Living Labs, (4) Pre-Pilot, (5) Pilot.

Figure 1.

Phases of project’s co-creation methodology.

To assess the usability and the level of stress related to technology usage, respectively System Usability Scale (SUS)²³ and Technostress (TS)²⁴ were administered during phases 3 and 4. Phase 5 Pilot is ongoing. These scales were administered to assess the impact of technologies on different groups of end users. The administration of the tests was authorized by the respective copyright holders.

The pilot study is registered on ClinicalTrials.gov as ‘AGAPE’ (ID NCT06859489).

Pre-selection of devices

A panel of experts from healthcare (medical doctors, physiotherapists, social workers, psychologists. . .), social care and technological development professionals assembled and decided upon wearables more likely to be included in the platform, based on their degree of user friendliness, robustness and safety in use. All devices proposed were made in China and CE-marked, available on the market at affordable prices for the target population. This phase involved all the partners of the project.

Focus groups and questionnaires

Focus groups were conducted by trained moderators according to pre-designed interview scenarios. The moderator was accompanied by a facilitator, in charge of providing materials and assisting participants in filling written informed consent forms and questionnaires, taking pictures and filling tables. The feedback was divided into three sections: socio demographic data (age, formal education, living environment, health condition), technology usage and attitude towards the solution proposed (health literacy, coaching strategies, market value). Closed and open-ended questions were used. The questionnaires were provided in online forms in order to reach as many responses as possible. The two data collection procedures took place in parallel by collecting data from different samples. Only qualitative data were collected in this phase.

The objectives of the phase were: to explore the values, needs and expectations of users; to discuss the importance of providing care based on the value of the results generated; to gather perspectives on optimal ICT based solutions and services. Answers were used to refine the solution envisaged by the consortium.

Workshops and living labs – hands-on sessions

The Workshops included two sub-phases: the first included workshops on the concept and services; the second subphase included workshops with demo of devices and mock-ups. In the first sub-phase a general introduction and open discussion on the approach and the solution was performed: participants’ demographic data (age, gender, education, living area, occupational status) were recorded; their confidence in using technology, interest and motivation, level of acceptance of the new services (perceived utility, accessibility, services designed, parameters monitored, coaching methods) were collected through open questions.

In the second phase, the presenter presented the wearables and mock-ups of the applications, and performed a demonstration of wearables in a controlled environment. Open discussions about the potentialities and criticalities of each device were carried out. Anonymous structured questionnaires were applied to the participants, in order to assess: perceived usefulness and user-friendliness of each tool; adequacy of devices for purpose; intention to receive personalized training and information about Active and Healthy Ageing.

Living Labs (LL) included hands-on testing of wearables, services and platform, system’s usability and the end users’ acceptance were evaluated. The overall goal of Living lab sessions was assessing the functionality, usability and perceived usefulness of the project’s platform and services and the user experience (including opinion, potentialities, and critical aspects of each tool) in a controlled environment, in order to refine the solution. From this phase, all data recorded from devices was managed entirely by the consortium’s technology partners in compliance with European privacy regulations.

A session of 2 h divided in three parts was scheduled for each end-user group. The first part included a presentation of the project, platform, objectives, services and technological components to participants. The second part included a brief presentation of technological devices and apps, and the setup of the apps on users’ phones. The third part included testing apps and devices (hands-on session). At the end of the session, each end user filled out (anonymously) SUS²³ and TS,²⁴ along with demographic data. A user interview was also performed, along with free discussion, to provide feedback to the developers on the overall status, satisfaction, weak points and concerns to be fixed. At this stage the technologies presented to the participants were in development.

Pre-pilot

In the Pre-Pilot (PP) phase, the solution was tested for 1 month by OAs at their homes and, in parallel, by their caregivers (formal and informal). Devices were provided to end-users by the consortium free of charge, not requiring any kind of financial contribution for participation. A sample of 10 OAs, 10 ICs and 2 FCs was expected for each pilot. The following objectives were identified:

- to preliminary assess, in an ecological home-based setting, the long-term feasibility of use, usefulness and usability of the platform and services;

- to preliminary assess the potential impact of the solution on users’ behaviour, as well as on their quality of life;

- to test the sensibility of the assessment battery prepared for the pilot itself.

End users were informed in regards of their participation and signed the written informed consent form, which also included information on the collection and management of personal data. Then, they participated in a project’s platform presentation and initially tested the technological solutions proposed. Following questionnaires were administered at the beginning and at the end of the pre-pilot month, including: socio-demographic questionnaire, SUS,²³ TS,²⁴ EQ-5D-5L (in older adults),²⁵ CarerQol-7D (in caregivers)²⁶ and UCLA Loneliness Scale.²⁷ During the pre-pilot phase, OAs connected and used solutions in different configurations, adapted to each pilot site and to the needs and limitations of each participant, being supported by Coaches (instructed and trained in-person and monitored remotely). Coaches were selected among e-literate professional caregivers and they were trained about the solution, as well as digital literacy, AHA, digital coaching. Sessions with engineers, psychologists, medical doctors and communication experts were organized in order to prepare the Coaches for their work. Soft skills like persuasiveness, understanding, communication, empathetic listening, friendliness and patience, were considered as key aspects of coaches’ profile, as the coach must be able to develop and maintain a special partnership with the people they coach. Coaches were also informed on qualitative feedback and needs emerged in the previous projects’ phases, to adapt their coaching strategies to emerged users’ needs.

Data analysis

Data were analysed by an independent investigator using descriptive statistics. The analysis focussed on describing the characteristics of the participants and examining the distribution and correlations between the SUS (System Usability Scale, which measures perceived usability of a system) and Technostress (which assesses stress related to the use of technology) questionnaires across different phases of co-creation, specifically during the Workshops, Living Labs and Prepilot phases. The sample recruited in the WS, LL and PP co-creation phases was merged.

A one-way analysis of variance (ANOVA) was employed to compare the avg scores of SUS and TS across the three stages of the study (WS, LL and PP), in rapport with Users (FCs-ICs-OAs) and Country (IT-PT-RO). The assumption of homogeneity of variances was assessed using Levene’s and Bartlett’s tests. Post-hoc pairwise comparisons were conducted using Tukey’s Honestly Significant Difference (HSD) test to identify specific group differences.

A paired-sample t-test was conducted to compare the mean differences in SUS and TS scores before and after usage of project’s solutions in the PP phase of the study. The assumption of normality for the differences between paired observations was assessed using the Shapiro-Wilk test. Additionally, the paired-sample t-test assumes that the paired differences are independent and have a symmetric distribution around the mean. In cases where the assumption of normality was violated, non-parametric alternatives such as the Wilcoxon signed-rank test were considered. The significance level was set at α = 0.05 for all statistical tests. Correlation analyses were performed to explore the associations between variables. Spearman’s rank correlation coefficient (ρ) was calculated to assess the monotonic relationship between SUS and TS scores, considering non-normal distributions or ordinal data. Kendall’s tau-b coefficient (τ) was employed to evaluate the strength and direction of association between characteristics of the sample (gender, education, country, user) and SUS and TS scores while accounting for tied ranks. Pearson’s correlation coefficient (r) was computed to measure the linear relationship between age and scores of SUS and TS when both variables were normally distributed. The significant level was set at p ≤ 0.05. Analyses were performed with JASP Team (2024) - JASP (Version 0.18.3) [Computer Software].

Results

The first four stages of co-creation took place in all three pilot sites, in the organizational environment (first three stages) and in older participants’ home environment (1-month pre-pilot). Quantitative and qualitative results are shortly reported. Details are available from authors, on demand.

Participants characteristics

During the Pre-Selection of Devices phase, the technological partners presented age-friendly technologies to the social and healthcare experts panel (20 persons).

During the Focus Groups and Questionnaires phase, a total of 157 end users were involved (Table 1). Specifically, 109 participants filled in the initial Questionnaire. During Focus Groups, eight sessions were held (three in Portugal, three in Romania and two in Italy), including 48 participants.

Table 1.

Respondents to questionnaires and participants in focus groups.

Site	Questionnaires				Focus groups
	OAs	FCs	ICs	Total	OAs	FCs	ICs	Total
IT	19	10	6	35	10	- - -	- - -	10
RO	34	4	8	46	- - -	12	- - -	12
PT	11	12	5	28	26	- - -	- - -	26
Total	65	26	19	109	36	12	0	48

FCs: formal caregivers; ICs: informal caregivers; IT: Italy; OAs: older adults; PT: Portugal; RO: Romania.

From the Workshops & Living Labs phase, a total number of 62 participants was reached, including 28 OAs, 29 FCs and 5 ICs. Workshops subphase included seven sessions (two for OAs, one for all users, two for FCs, two for potential coaches). Living Labs subphase included six sessions (two for OAs, three for FCs, one for potential coaches).

The Pre-Pilot phase involved 63 participants (46 F, 17 M), including 36 OAs (22 F, 17 M), 14 FCs (all F) and 13 ICs (10 F, 3 M).

Quantitative results

SUS and Technostress results are reported in Tables 2 and 3.

Table 2.

System usability scale (SUS) and technostress (TS) scores from workshops and living labs.

User categories	Workshops			Living labs
	N	SUS, mean (SD)	TS, mean (SD)	N	SUS, mean (SD)	TS, mean (SD)
IT OAs	6	67.5 (17.75)	28.83 (11.16)	12	58.75 (14.29)	28.67 (4.59)
IT FCs	7	76.25 (4.79)	27 (7.7)	6	74.17 (9.44)	25.5 (7.15)
IT ICs	- - -	- - -	- - -	3	78.33 (12.33)	33 (7)
RO OAs	5	64.4 (18.07)	20 (3.65)	- - -	- - -	- - -
RO FCs	5	96 (5.48)	4.4 (3.43)	6	80.42 (16.39)	11 (10.1)
RO ICs	2	92.5 (10.61)	1 (1.41)	- - -	- - -	- - -
PT OAs	- - -	- - -	- - -	5	46 (6.52)	21 (7.58)
PT FCs	- - -	- - -	- - -	5	66.5 (18.92)	15.6 (6.15)
PT ICs	- - -	- - -	- - -	- - -	- - -	- - -
All	25	74.1 (18.5)	27 (12.76)	37	53.5 (25.96)	21.7 (9.98)

FCs: formal caregivers; ICs: informal caregivers; IT: Italy; OAs: older adults; PT: Portugal; RO: Romania.

Table 3.

System usability scale (SUS) and technostress (TS) scores from the pre-pilot phase.

User categories		System usability scale (SUS)			Technostress (TS)
	N	Mean (SD) pre	Mean (SD) post	p *	Mean (SD) pre	Mean (SD) post	p *
IT OAs	11	69.81 (18.38)	59.44 (18.23)		15 (9.04)	17 (8.18)
IT FCs	2	73.33 (12.58)	- - -		8.66 (3.21)	- - -
IT ICs	11	68.88 (11.33)	- - -		11.77 (7.46)	- - -
PT OAs	10	68.75 (14.58)	49.69 (11.13)		11.62 (5.73)	17.5 (7.37)
PT FCs	10	65.23 (24.99)	- - -		10.73 (7.16)	- - -
PT ICs	2	- - -	- - -		- - -	- - -
RO OAs	15	57.32 (15.05)	56.43 (11.99)		22 (12.95)	16.07 (7.88)
RO FCs	2	51.25 (19.44)	55 (7.07)		35 (14.14)	25.5 (4.95)
RO ICs	- - -	- - -	- - -		- - -	- - -
All	63	62.7 (16.49)	56.3 (12.18)	1896 (0068)	19.4 (11.79)	17.8 (7.14)	0925 (0363)

FCs: formal caregivers; ICs: informal caregivers; IT: Italy; OAs: older adults; PT: Portugal; RO: Romania.

Paired t-test.

SUS and TS (pre and post-administration) are reported in Table 3.

At present, 282 end-users participated in the co-creation activities overall. The statistical analysis was performed over a merged sample that includes Workshops, Living Labs and Pre-Pilot participants who filled all the questionnaires (n = 107). The sample had a mean age of 58.7 (SD = 20.6) and was composed of 63.5% women. The educational level was distributed as follows: Primary school 21.6%; Secondary school 21.6%; higher education 56.8%.

A one-way ANOVA test was performed on the merged WS-LL-PP sample (n = 107) by analysing the results of the SUS and TS scales. Homoscedasticity was confirmed by both Levene’s test (SUS: 2.018, p = 0.138; TS: 0.819, p = 0.444) and Bartlett’s test (SUS: 5.06, p = 0.080; TS: 1.70, p = 0.444). Results indicate that SUS scores differ depending on the administration phase (df = 2; F = 6.25; p = 0.004). Through Tukey’s Post-Hoc Test, a significant difference is shown between SUS scores in the WS and LL stages, specifically with a reduction in the score at the LL stage (test = −20.57; p =< 0.001; Figure 2). No difference is shown between WS-PP and LL-PP. Tukey’s Post-Hoc Test was performed also for TS, but no significant differences can be observed. A paired-sample t test was performed on the older sample that participated in the Pre-Pilot phase (n = 31), in which SUS and TS scales were administered pre and post-usage of project’s solutions: no statistically significant differences were appreciated (Table 3). Correlations performed on the merged sample between SUS, TS and demographic variables show significant negative correlations between SUS and TS, while a positive correlation is found between SUS and Education (Table 4). A one-way ANOVA test was performed to investigate whether the results of SUS and TS differed in Users and Country categories. Analyses show that TS scores differ between Country (df = 2; F = 13.19; p =< 0.001), with a significantly higher score recorded in Italian participants compared to Portuguese (mean difference = 10.4; p =< 0.001) and Romanian (mean difference = 7.42; p = 0.008) participants. TS scores also differ between Users (df = 2; F = 5.497; p = 0.014), with a significantly lower score recorded in Formal caregivers compared to Informal caregivers (mean difference = −12.9; p = 0.012). No significant differences in SUS scores are appreciated by controlling Users and Country.

Figure 2.

Mean (95% CI) of system usability scale (SUS) scores collected at different phases.

Table 4.

Correlations between system usability scale (SUS), technostress (TS) and sociodemographic variables.

		SUS (Spearman’s Rho)	TS (Spearman’s Rho)	Age (Pearson’s r)	Gender (Kendall’s Tau B)	Education (Kendall’s Tau B)
SUS	Correlation		−0.329	−0.057	−0.016	0.198
SUS	p value		<0.001	0.584	0.847	0.015
TS	Correlation	−0.329		0.122	0.005	−0.135
TS	p value	<0.001		0.239	0.955	0.098

Qualitative results

The changes to the study that occurred during the different co-creation phases basically concerned the proposed devices (e.g. elimination of the Smart T-shirt) and the functionality of the different apps/web apps, which were implemented as feedback was collected from potential end users. Below is a detailed description of the results achieved in the different phases, which led to the co-designed AGAPE solution.

Pre-selection of technologies phase

Non-invasive customizable technologies supporting pseudo anonymized data transmission for remote monitoring were chosen to be presented to end-users in the study in order to decide their inclusion in the ecosystem developed in the project: Smart Band (bracelets with inertial and physiological sensors accounting number of steps, sleep, calories consumed, heart rate, blood oxygenation, body temperature), Smart T-Shirt (with inertial sensors able to record movement data for gait and respiration analysis), Smart Socks (with pressure and inertial sensors for gait analysis). Regarding communication, the consortium chose to focus on digital services: a chat service, an online dashboard for FCs, an app for OAs.

Focus groups and questionnaires phase – qualitative results

The one-time questionnaire was a more individual approach and proved an efficient, low-cost approach to gather information regarding the attitude towards digital health from a broad range of stakeholders. Group discussions created more opening and stimulated idea and view development regarding the concept of the proposed solution. During focus groups, perceptions, opinions, beliefs and attitudes were shared in a comfortable environment. During this phase, the technologies chosen by the consortium were presented to potential users, applying a preset script, use cases, user scenarios and free conversation. Finally, the consortium chose to develop an application to collect data regarding the activity level and vital parameters from wearables to provide them in a friendly way to both older users (on smartphones) and coaches (via a monitoring webapp), coaching materials and techniques aiming to improve digital adoption, adapted for older users, to adopt an age-friendly smartphone interface app and a special secured chat app and to integrate these services in a unique platform.

Workshops subphase – qualitative results

Participants from all three categories discussed and appreciated the bottom-up approach for identifying user needs. The integration of all end-user categories within the ecosystem was appreciated as well-planned. Participants unanimously appreciate the coach and personalized coaching strategies. The involvement of the coach was seen as a valuable innovation, providing personalized support for behavioural change, though concerns arise about users feeling the coach as too controlling. Participants appreciate both individual and group coaching strategies, suggesting gamification and competition to enhance engagement. Suggestions included coaches to address organizational needs, as well. Smart Socks and Smart Bands were favoured for their ease of use, but concerns about hygiene and aesthetics arise about Smart Socks. Smart Socks were deemed easy to use, though some participants had concerns about aesthetics, comfort, use in hot weather and maintenance. The Smart Band received most positive feedback for its ease of use and valuable data. The Smart T-shirt presents limited variables and wearability issues, the textile was considered too tight, uncomfortable and not respirable enough. The app for OAs was appreciated for the clarity of the simplified visual interface that was adapted to the user’s digital skill level, also adding more images. The dashboard for the formal caregiver was appreciated for functionality and graphics, being considered very useful.

Drawbacks noticed by professional caregivers included: devices might run out of power or break, might not be accurate and device-platform synchronizations and device-apps connectivity might fail.

Combining the information collected during Focus Groups and Questionnaires phase, the identification of different scenarios was possible. Needs identified in OAs were about improving the use of technologies, both in health and social care, without them becoming intrusive. ICs report the need to stay in touch with their older relatives even at a distance, and would like their loved ones to learn how to use technologies safely. FCs required technology that was nonintrusive, simple, and helpful for their work and contact with patients. This information enabled the consortium to design the digital solutions that would support users.

Living labs subphase – qualitative results

Testing in hands-on sessions brought more in-depth feedback regarding the technologies.

A new device was introduced as an alternative to Smart Socks: Smart Strap, consisting of an elastic strap including an inertial sensor-node able to gather, record and transmit data.

Most participants thought Smart Socks can be useful for gait analysis in clinical settings but less usable for daily home-use. Socks have a thick fabric and can be used only occasionally. The Smart Strap, a viable alternative to Smart Socks, from hygiene respects, appeared useful and friendly, but some OAs have shown puzzlement over the device’s aesthetics because it is similar to anklets used for delinquents. The Smart band was considered friendly, apt to daily use, but is inconvenient to sleep with, it is difficult to dismantle in order to be plugged in to get powered; its screen was considered small, a larger screen being suggested. The simplified interface of the smartphone app was appreciated as friendly thanks to its colours and easy identifiable buttons and functions. The large font and the well-defined icons make it easy to use, even for someone not familiar with touch screens. The idea of monitoring the OAs’ progress is a good idea to promote safe physical activity, but the actual lack of automation for some parameters was perceived as a flaw and the assistant app for smartphones can address more goals. The dashboard is considered useful and easy to use, but needing to enter patient’s data manually could be a drawback. The visual interface was appreciated as well structured and easy to navigate.

The final wearable choice was made dictated by participants’ feedback. Smart Socks and Smart Band were chosen for the Italian pilot; Portuguese and Romanian pilots will also test the Smart strap. The apps were confirmed and implemented following feedback, to be deployed in all pilot sites for the pre-pilot.

Pre-pilot phase – qualitative results

Pre-Pilot provided more detailed, but also differentiated qualitative feedback. The Smart Band was appreciated in all sites for its manageability, simplicity, aesthetics, comfort and familiarity. The Smart Socks were appreciated especially for the type of data that can be collected. Opinions from OAs and ICs differed: some appreciated the originality of the device, while others expressed concerns related to aesthetics and hygiene. The apps developed for activity and vital signs self-monitoring, communication between users, data transmission and monitoring were highly accepted by all users from all pilot sites: they resulted as instinctive, simple, ergonomic and accessible to all.

Opinions from OAs: Smart Socks were received more positively by men but less positively by women: many older women do not really use socks in their daily lives and would need to change their dressing style. One further inconvenience was that for recording steps, the users need to always carry their phone on themselves. The wearables also had to be charged every 2 days. The Smart Band was well received. OAs also liked the app dedicated to them. Users who were comfortable with smartphones did not need the simpler visual interface.

Opinions from ICs: The technologies can help their loved ones keep their vitals monitored remotely, to prevent relapse and improve lifestyles, self-monitoring encouraging them to set a goal each day a little bit higher. The Coach was appreciated as enabling the OAs to learn the use of technologies in a simple but effective and safe way. ICs reported an elevation in their loved ones’ interest and time using the technologies, as well as improved mood and engagement in activities of daily living and social activities of their older family members. The assessment kit designed to evaluate the impact of the solution on participants was considered valuable and relevant by Pre-Pilot participants.

Opinions from FCs: Their patients usually like to walk and meet face-to-face, instead of using video calls. FCs appreciate the solution as important if implemented and sustained, for all older users, but especially for rural areas that are miles away from the nearest hospitals. They perceived that this informed and assisted self-monitoring could reduce the workload on FCs, preventing sedentariness, improving physical activity level, supporting behaviour changes in older users, decreasing the risk of disability. FCs tested the Monitor webapp and described it as simple and intuitive, very useful. They appreciated the proposed monitoring features.

FCs recognized various potential benefits including facilitating access to health information, reducing isolation and inactivity risks, improving efficiency in activities of daily living (ADLs), accessing health data and enhancing communication with healthcare teams and families. All participants emphasized the importance of increasing older generations’ openness to technology through awareness and perceived need, suggesting detailed and friendly user guides and personalized teaching as solutions. While some users found the devices not so innovative but still useful, especially with guidance and training, they gave positive feedback on the digital apps tested.

Discussion

Studies show the benefits of using person-centred digital health solutions to promote AHA, but the adoption rates of these technologies beyond the clinical studies remains low.¹¹ Co-creation methods are applied widely to create more user-friendly solutions, and seem to provide relevant feedback to developers, helping them optimize their solutions, involving mainly primary users in usability testing and co-creation, as well as social and healthcare workers.²⁸ In contrast to what has been reported for similar research,²⁹ in our approach, primary users and secondary users were involved in co-creation from the early stages of concept design, to record and implement the real needs and expectations of users in the solution. The methodology of co-creation, based on the Toolbox of AAL projects for user-centred design purposes, was multidimensional and it was defined to allow reproducibility.¹⁸

The research team explained to the participants the decision to use the different questionnaires to collect feedback. All categories of end-users, who participated in the different phases, were pleased to be involved in the development of smart assistive technologies aimed at improving the quality of life of older adults. They were also eager to express their ideas and suggestions as well as their concerns.

During Workshops & Living Labs phase, OAs attitudes towards technology varied; although some were familiar with technological tools, many reported difficulties due to the complexity of devices and platforms, as reported also in literature.³⁰ To overcome this problem, the support of a professional coach has been proposed. Able to evaluate objectively the capabilities and the needs of the older users in regards of digital literacy and skills, as well as their need of assistance, the Coach has to adapt interventions, raising to meet the actual momentary requirements. The Coach has also to be able to monitor the progress of the OAs in reaching personalized objectives, adapting behaviour along with improvement of health literacy and self-awareness and improving social participation. During the different phases, the Coach was seen by users as the major strength. The best strategy to engage OAs resulted in the relation one-to-one, but also group sessions were appreciated for generating emulation, active and dynamic relationships between peers and enhancing the sense of belonging. All participants were highly receptive to the concept of a personalized coach measured by individual digital literacy levels. They appreciated the inclusivity of the solution, welcoming all individuals regardless of their degree of familiarity with digital technologies. There was unanimous agreement that the solution offered a proactive approach to reducing the digital divide, with teaching, training and coaching strategies being well-received. Hands-on personalized teaching and video remote live communication with coaches were particularly favoured.

The ecosystem and service package tested is the first Internet of Things-based AAL solution generating goals adapted to the digital level of the user, including the human Coach in the loop as a decision support system. The solution provides users with capability, opportunity and motivation to improve, to support behavioural change, including in regards of lifestyle and in engagement in using smart assistive technologies.³¹ The renegotiation of the attitude towards technologies³² is essential, as the user comfort related to the solution appeared as the main factors influencing the user behaviour. This can support long-term use of assistive technologies and, possibly, translate to higher propension to innovation adoption overall.³³ There are attempts to include e-coaches in ICT based solutions for AHA³⁴ but a professional human coach in live interventions can bring more customized and adaptable training and instruction.³⁵

As to usability, the System Usability Scale (SUS) is an assessment tool commonly used also for rehabilitation technologies,³⁶ with lower scores associated with worse experience and higher frustration.³⁷ Usability was highly variable both in Workshops and Living Labs, highlighting the sample’s unevenness in the use of technology. Thus, multiple levels of e-literacy were identified and different services were chosen for the various OAs. At baseline, usability and technostress were inversely correlated, as expected. Between WS and LL, and between stages T0 and T1 of the Pre-Pilot, the usability decreased: this result is essential for co-creation purposes, because it expresses the not-yet-mature digital solutions and the various technological problems encountered, including different bugs encountered during the testing. In fact, unlike the WS phase, in which participants saw a preview of the proposed technologies, during the later phases they were able to test them. On this occasion, the non-maturity of the technologies and the need for further refinement emerged. However, the related technostress did not increase in T1, possibly attributable to the effectiveness of the Coaches’ support. Technostress also differed between Country and Users, indicating a significant variation of stress levels between these groups. Specifically, the Italian population reported a higher score than other countries involved in the project. This was probably due to the sample coming from rural areas, barely exposed to the use of technology in community life, while the Romanian site participants in pre-pilot were chosen between younger and more tech-savvy urban older adults. Digital divide between rural and urban older adults is no news.³⁸ FCs reported a higher Technostress score than other users, probably due to their not so high digital literacy associated with continued exposure to new technology and digital services, considered generally as a supplementary burden before being perceived as aids.³⁹

The main limitations of this research are related to the number and heterogeneity of the sample and the selection of participants. The choice to recruit subjects in different countries and in different areas (urban or rural) was dictated by the need to investigate needs and expectations of different users with different social and cultural conditions. While the small sample size can be considered adequate for a pilot study, the absence of random or consecutive recruitment does not ensure that the sample enrolled in the different areas was representative of the local population with similar characteristics and may have therefore introduced a selection bias. Achieving a larger and better distributed sample is recommended in future research. Another limitation of our study is that we did not perform a formal assessment of data saturation, however according to recent research,⁴⁰ 9–17 interviews or 4–8 focus group discussions can be sufficient to reach saturation, especially for studies with homogenous populations and narrowly defined objectives, such as AGAPE, targeting community living older persons with well-defined inclusion and exclusion criteria and their formal and informal caregivers and aiming to innovation adoption.

Conclusion

Referring to theoretical frameworks already deployed in other European projects on AHA, this paper presents a measurable and reproducible co-creation approach to develop different services for AHA. This approach was used to design a user-friendly digital health ecosystem for OAs and their caregivers, through a mixed methodology, including quantitative data (SUS and TS) and qualitative data (collection of needs and feedback). Each phase contributed with its own preliminary results to the changes and decisions made by the consortium, to refine the solution provided during the ongoing pilot study from the choice of the devices to the app interface features and the coaching strategies. The mixed methodology was applied to allow the collection of as much and relevant qualitative and quantitative information as possible from the participants, to support the iterative development of an innovative platform of services for safe and informed AHA. Beyond the scope of the AGAPE project, our findings highlight the broader relevance of co-creation in digital health, to promote inclusive and adaptable digital interventions and to empower end-users, including older persons, in the iterative development of accessible and engaging solutions. The reproducible nature of our approach forsters its scalability and adaptability to different cultural and socio-economic contexts, offering valuable insights for future AHA initiatives across Europe and beyond.

Footnotes

ORCID iDs

Chiara Pedrini

Maria Anna Szczepanska

Samuele Pinna

Diego Longo

Mihai Berteanu

Author contributions

Conceptualization, F.C., C.P. and M.A.S.; methodology, C.P., F.C., D.L. and M.B.; formal analysis, D.L and M.B.; investigation, C.P., M.A.S., A.P., S.T., S.P. and I.C.; data curation, I.C., C.P. and A.P; writing — original draft preparation, C.P., M.A.S. and I.C.; writing — review and editing, I.C., E.T. and F.C.; supervision, M.B. and F.C., project administration, E.T. and P.D. All authors have read and agreed to the published version of the manuscript.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported and funded by AAL Program and the Italian Ministry of Health - Ricerca Corrente.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Langmann

Vulnerability, ageism, and health: is it helpful to label older adults as a vulnerable group in health care?

Med Health Care Philos [Internet] 2023; 26(1): 133–142.

Amuthavalli Thiyagarajan

Mikton

Harwood

, et al. The UN Decade of healthy ageing: strengthening measurement for monitoring health and wellbeing of older people. Age Ageing 2022; 51(7): afac147.

Liotta

Canhao

Cenko

, et al. Active ageing in Europe: Adding healthy life to years. Front Med 2018; 5: 301606.

Mulhausen

Healthcare systems and healthcare funding supporting healthy aging. In: Coll

(ed.) Healthy aging. Cham: Springer, 2019, pp. 367–375.

Ayala

Rodríguez-Blázquez

Calderón-Larrañaga

, et al. Influence of active and healthy ageing on quality of life changes: insights from the comparison of three European countries. Int J Environ Res Public Health 2021; 18(8): 4152.

Rudnicka

Napierała

Podfigurna

, et al. The World Health Organization (WHO) approach to healthy ageing. Maturitas 2020; 139: 6–11.

Santoni

Angleman

, et al. Temporal trends in impairments of physical function among older adults during 2001–16 in Sweden: towards a healthier ageing. Age Ageing 2018; 47(5): 698–704.

Helbostad

Vereijken

Becker

, et al. Mobile health applications to promote active and healthy ageing. Sensors 2017; 17(3): 622.

Velciu

Spiru

Dan Marzan

, et al. How technology-based interventions can sustain ageing well in the new decade through the user-driven approach. Sustainability 2023; 15(13): 10330.

10.

Gunnes

Løe

Kalseth

Exploring the impact of information and communication technologies on loneliness and social isolation in community-dwelling older adults: a scoping review of reviews. BMC Geriatr 2024; 24: 215.

11.

Xie

Kalun Or

. Acceptance of mHealth by elderly adults: a path analysis. Proc Hum Factors Ergon Soc Annu Meet 2020; 64: 755–759.

12.

Tyler

De George-Walker

Simic

Motivation matters: older adults and information communication technologies. Stud Educ Adults 2020; 52(2): 175–194.

13.

Tomczyk

Mascia

Gierszewski

, et al. Barriers to digital inclusion among older people: a intergenerational reflection on the need to develop digital competences for the group with the highest level of digital exclusion. Innoeduca 2023; 9(1): 5–26.

14.

Finkelstein

Brennan-Ing

Older adults’ experiences with using information and communication technology and tech support services in New York City: findings and recommendations for post-pandemic digital pedagogy for older adults. Front Psychol 2023; 14: 1129512.

15.

Vercruyssen

Schirmer

Geerts

, et al. How “basic” is basic digital literacy for older adults? Insights from digital skills instructors. Front Educ 2023; 8: 1231701.

16.

Joymangul

Ciobanu

Agnoloni

, et al. Empowering active and healthy ageing: integrating IoT and wearable technologies for personalised interventions. Appl Sci 2024; 14(11): 4789.

17.

Rill

Hämäläinen

MM.

The art of co-creation: a guidebook for practitioners. Cham: Springer, 2018. DOI: 10.1007/978-981-10-8500-0.

18.

Active and Assisted Living Programme. AAL programme: Toolbox for project planning [Internet]. Brussels: AAL Association. 2015. [cited 2025 Jun 6]. https://www.aal-europe.eu/wp-content/uploads/2015/02/AALA_ToolboxA5_online.pdf

19.

Mittelmark

Bauer

. Salutogenesis as a theory, as an orientation and as the sense of coherence. In: Mittelmark

Bauer

Vaandrager

, et al. (eds) The handbook of salutogenesis [Internet]. 2nd ed. Cham: Springer; 2022.

20.

Hallewell Haslwanter

Neureiter

Garschall

User-centered design in AAL: usage, knowledge of and perceived suitability of methods. Univers Access Inf Soc 2020; 19(1): 57–67.

21.

Dam

RF.

The 5 stages in the design thinking process. Interaction design foundation - IxDF. 2024. https://www.interaction-design.org/literature/article/5-stages-in-the-design-thinking-process (accessed 12 June 2024).

22.

Brooks

Jr.

The design of design: Essays from a computer scientist. Pearson Education, 2010.

23.

Brooke

SUS—A quick and dirty usability scale. In: Jordan

Thomas

Weerdmeester

, et al. (eds) Usability evaluation in industry. London: Taylor & Francis, 1996, pp. 189–194.

24.

Tarafdar

Ragu-Nathan

Technostress questionnaire [Internet]. PsycTESTS Dataset. Washington (DC): American Psychological Association, 2010.

25.

Herdman

Gudex

Lloyd

, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011; 20(10): 1727–1736.

26.

Brouwer

van Exel

van Gorp

, et al. The carerqol instrument: a new instrument to measure care-related quality of life of informal caregivers for use in economic evaluations. Qual Life Res 2006; 15(6): 1005–1021.

27.

Russell

DW.

UCLA loneliness scale (version 3): reliability, validity, and factor structure. J Pers Assess 1996; 66(1): 20–40.

28.

Garschall

Neureiter

Hallewell Haslwanter

, et al. Investigating user-centered design practices in Austrian AAL projects. In: Hochgatterer

Wichert

(eds) Innovative Lösungen für eine alternde Gesellschaft. Konferenzbeiträge der SMARTER LIVES. vol. 16. Lengerich: Pabst Science Publishers, 2017, pp. 125–137.

29.

Bastardo

Martins

Pavão

, et al. Methodological quality of user-centered usability evaluation of ambient assisted living solutions: a systematic literature review. Int J Environ Res Public Health 2021; 18(21): 11507.

30.

Zhang

Older people’s attitudes towards emerging technologies: a systematic literature review. Public Underst Sci 2023; 32(8): 948–968.

31.

Michie

van Stralen

West

The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 2011; 6: 42.

32.

Kebede

Ozolins

Holst

, et al. Digital engagement of older adults: scoping review. J Med Internet Res 2022; 24(12): e40192.

33.

Muñoz Esquivel

Gillespie

Kelly

, et al. Factors influencing continued wearable device use in older adult populations: quantitative study. JMIR Aging 2023; 6: e36807.

34.

Lentferink

Oldenhuis

de Groot

, et al. Key components in eHealth interventions combining self-tracking and persuasive eCoaching to promote a healthier lifestyle: a scoping review. J Med Internet Res 2017; 19: e277.

35.

Leedahl

Brasher

Estus

, et al. Implementing an interdisciplinary intergenerational program using the Cyber Seniors® reverse mentoring model within higher education. Gerontol Geriatr Educ 2019; 40(1): 71–89.

36.

Zanatta

Giardini

Pierobon

, et al. A systematic review on the usability of robotic and virtual reality devices in neuromotor rehabilitation: patients’ and healthcare professionals’ perspective. BMC Health Serv Res 2022; 22: 523.

37.

ICTskills4All. Empowering old adult citizens for a digital world. Intelectual Output 3. 2021. https://all-digital.org/wp-content/uploads/2018/09/IO3_final-EN.pdf

38.

Lee

Choi

Kim

, et al. Rural and non-rural digital divide persists in older adults: internet access, usage, and perception. Innov Aging 2020; 4(Suppl 1): 412–413.

39.

Golz

Peter

Müller

, et al. Technostress and digital competence among health professionals in Swiss Psychiatric Hospitals: cross-sectional study. JMIR Ment Health 2021; 8(11): e31408.

40.

Hennink

Kaiser

BN.

Sample sizes for saturation in qualitative research: a systematic review of empirical tests. Soc Sci Med 2022; 292: 114523.

Developing an ecosystem of advanced digital services for healthy ageing by a co-creation methodology: preliminary results of the AGAPE project

Abstract

Background:

Design and methods:

Results:

Conclusions:

Keywords

Introduction

Methodology

Sample, time and location

Methodology and phases

Pre-selection of devices

Focus groups and questionnaires

Workshops and living labs – hands-on sessions

Pre-pilot

Data analysis

Results

Participants characteristics

Quantitative results

Qualitative results

Pre-selection of technologies phase

Focus groups and questionnaires phase – qualitative results

Workshops subphase – qualitative results

Living labs subphase – qualitative results

Pre-pilot phase – qualitative results

Discussion

Conclusion

Footnotes

ORCID iDs

Author contributions

Funding

Declaration of conflicting interests

References