Abstract
Objective
Technologically assisted support systems and social support in everyday life can help senior citizens live longer independently in their homes. The purpose of this process evaluation is to investigate an innovative care model integrating monitoring technology with social support services, aimed at enabling senior citizens to live independently and extend their longevity in their homes.
Methods
Data collection of this mixed-method study was conducted through three distinct sources: expert interviews with employees of the participating social service, focus groups with seniors participating in the intervention, and involved consortium partners in the project. Following Kuckartz's methodology, we employed a structural qualitative content analysis using MAXQDA software. Additionally, a portion of the standardized survey administered post-intervention to participants was analyzed using descriptive statistics.
Results
The focus groups identified key challenges related to technical implementation such as false alarms and the failure of sensors as well as communication between invested parties. However, significant potential was noted in the practical execution of the intervention and social care. Interview participants emphasized the need for improved technical implementation. Results from the questionnaires indicate a generally positive perception of the intervention, particularly regarding its social dimensions.
Conclusions
Surveying individuals who implement and utilize assistive technology can yield valuable insights into its effectiveness. Additionally, it is crucial to comprehensively and in detail capture the experiences of those involved in testing new care models. Future research on assistive technologies for older adults should integrate both technical and social support components, while also addressing secure data protection measures and the paradox of reassurance.
● Older adult, frail ● Aged ● Independent living ● Qualitative research ● Assistive technology
Introduction
Background
As in most high-income countries, society is aging in Germany. 1 The baby boomer generation, born between 1955 and 1965, is approaching retirement, entering a phase of life characterized by an increased likelihood of declining health. This demographic shift places significant demands on healthcare systems to meet the evolving needs of this aging population. 2 In Germany in particular, the social systems are called upon to continue to meet the needs of the aging population. The vast majority of people wish to grow old in their homes. 3 For the majority of individuals, this desire is fulfilled: 80% of caregiving occurs at home with family members, and only a small percentage of seniors transition into care facilities. 4 Politically, this wish has long been reflected in the principle of "outpatient before inpatient" according to Section 43, para. 1 SGB XI. At the same time, this principle is reaching its limits against the backdrop of a high number of people with health burdens. 5 The future high demand for care and nursing services is met with an increasing shortage of nursing and medical professionals, as well as a significant physical, time-related, and psychological burden for family caregivers. 6 Furthermore, the individual care sectors in Germany are not sufficiently interconnected to ensure the necessary and appropriate care for older people with health impairments at the neighborhood level. 7 Even in old age, individuals continue to pursue self-acceptance, life satisfaction, and an active lifestyle. 8
However, these aspirations for a healthy aging process often confront a reality marked by declining health and a growing risk of disease and injury that are often interrelated. Falls are among the most significant factors impacting the quality of life in older adults, 9 with the frequency of falls increasing with age. 10 The likelihood of fall-related injuries leading to serious consequences also increases with age. 11 In 10%–20% of cases, falls result in bone fractures and head injuries 12 and can even lead to increased mortality.12,13 Falls not only cause a loss of confidence and fear of future falls, 14 but also chronic pain, reduced independence, and a diminished quality of life. 15 Additionally, they impose significant economic costs, which are often underestimated by policymakers.. 16
Assistive technologies
Information and communication technologies offer promising opportunities to improve healthcare, 17 particularly in integrated concepts that allow seniors to live safely and independently in private homes and communities. 18 These concepts are currently discussed under the labels ambient assisted living, welfare technologies, or smart homes, and they include a network of connected sensors and actors in the healthcare system that monitor and contact the older adults in the community as passive and supportive infrastructure.19,20 A review of 54 studies 21 found technological systems supportive in the domains of falls, home support, and dementia care, with benefits ranging between home safety, cognitive and psychosocial status, reduction of caregiver burden, and improved caregiver functioning. Barriers to these benefits included an unwillingness to accept assistive technologies, poor user experience, and concerns about user privacy. A systematic review including randomized controlled trials surveyed 12 studies and found that all studies reported positive effects on participant's health status in applications varying between robots, computers, smartphones, telephones, or telemonitors within a duration between 4 weeks and 6 months. 22 The use of sensors for home monitoring enables a comprehensive picture of a person's health, level of activity, and how they function in the apartment. 23 This approach could enable the analysis of mobility patterns, the identification of risk situations, and the prevention of falls. Previous study results provide evidence that the use of technologies improves the sense of security for seniors when it comes to accidental falls and delays in institutional care. 24 By leveraging technology, older adults experience enhanced quality of life, greater autonomy, improved self-management, and a reduction in hospital stays. 25 However, there is limited knowledge regarding the challenges and potential of implementing such interventions. This article aims to present initial findings from the recent implementation of a technological intervention in Germany.
The project virtual assisted living (German: Virtuell Betreutes Wohnen)
The project presented here is called virtual assisted living (VAL, German: Virtuell Betreutes Wohnen [VBW]) and is funded by the Innovation Fund of the Federal Joint Committee as a new form of care that addresses the aforementioned challenges. The VAL project ran from October 2019 to September 2023 in Berlin, Germany, and it consisted of a matched-control intervention study with accompanying interviews and focus group discussions. VAL utilized a novel intervention consisting of a home emergency call system with automatic fall detection, motion sensors in rooms and on the refrigerator, and contacts to a social service provider. The goal of the project was to promote long-term independence in old age and a longer stay in one's home. This was linked to improving quality of life, preventing risks (e.g. falls), monitoring movement behavior, providing acute support services when needed, reducing care efforts, and lowering healthcare costs, especially with regard to avoidable emergency and hospital care. In order to achieve this goal, it was necessary to analyze mobility patterns, identify risk situations, prevent falls, and quickly provide help in acute crises. We used a convenience sample of N = 181 older adults ≥75 years old based on an underlying sample size calculation using G*Power version 3.1.9.2 26 with dependent t-tests anticipating a difference in hospitalization rates of 7% (14% vs. 7%) between the intervention group (IG) and the matched control group (CG), which translates to a medium effect size (Cohen's d = 0.24) using established values for two-sided significance levels (α = 0.05) and power (1 − β = 0.80), an allocation ratio of 1:4 between IG and matched CG, and a dropout rate of 13%. VAL was a preventive care concept for older people in their neighborhoods. By connecting with local social service providers and their neighborhood assistants, individually desired health services and actors were networked and coordinated for people in their homes. The sensor data from the apartments were displayed on a dashboard. The sensors provided discrete, non-camera-based, passive, all-day monitoring of the seniors’ activity patterns. Data were collected in order to identify unusual activity patterns and react promptly by comparing them with a personalized behavioral constellation. Additionally, risk scores were to be transmitted by the technology provider, identifying falls and subsequent hospital admissions based on various parameters. The social service provider ensured social support, including the availability of neighborhood assistants in the event of queries or problems from the project participants (PP). The social service providers gave personal advice (by telephone or at home) on topics relating to old age (e.g. advice on care levels and living wills) and offered additional services (e.g. tablet training), arranged services (e.g. domestic help, transportation services), establishing a corresponding partner network and established the support of voluntary structures for the holistic social care of PP in everyday life (e.g. volunteer companion calls). The project consortium was composed of three health insurance companies, a technology company, a social service provider, a housing company, and a research institute responsible for scientific evaluation. The accompanying scientific evaluation used a mix of research methods consisting of qualitative elements (focus groups, semi-structured expert interviews) and quantitative data (standardized pre- and post-surveys to measure subjective attitudes, analysis of routine data from participating health insurance companies). This article refers to the data (see Table 1) collected at the end of the project: the findings include qualitative results from interviews and focus groups conducted with participating seniors and the project consortium, as well as partial results from standardized surveys administered to the participating seniors.
Overview of data for evaluation.
IG: intervention group.
Methods
In the project evaluation, data collection was carried out using three different sources. Table 1 provides an overview. These three data sources reflect different perspectives and represent different levels of the healthcare system, including the seniors and those facilitating their care. In the VAL project, these were the older people living at home (PP) as well as the social service provider and cost bearer (health insurance companies) and other parties involved (housing company and technology developer). At the level of the qualitative individual interviews and in the survey, individual and subjective views were elicited. In the focus groups, collective data were created through interactive discussions. Both methods produced detailed reports on the experiences and lessons learned from the project. The survey provided a simple and precise evaluation of the intervention which supplemented the qualitative research methods. The combination of methods provides a comprehensive evaluative picture.
Social service provider interviews
Two employees and the leading manager of the participating social service were chosen to be interviewed and sent a request by e-mail. The interviews were conducted to shed light on the work required for the new form of care being tested, as well as the overall study process from the perspective of neighborhood management. This allowed for the collection of necessary, specific, relevant practical, and actionable knowledge. 27 The interviews took place in the summer of 2022, lasting between 30 and 90 min each. Two were conducted and recorded in audio form in the workspace of the evaluating institute and one at the workplace of the interviewees. The interviewer took field notes. Only the interviewer and the interviewee were present in each case. An untested semi-structured interview guide was used (Appendix 1). Individual interviews investigated how the project was experienced and integrated into everyday work. There were sub-questions about the user experience of the technology and the participants’ assessment of the benefits of the project. The interviewees were also asked about the work in the consortium and their assessment of the future in terms of technology. All interviews were conducted successfully without the need for repetition. Additionally, two focus groups were conducted, which were audio recorded, and field notes were taken. One focus group included seniors participating in the intervention, while the other consisted of consortium partners (CPs) involved in the project.
Focus groups
The focus group with intervention participants (N = 5) took place in October 2022 in the workplace of the local social service providers in Berlin and lasted 80 min. Five participants were suggested by the participating social service provider based on the following criteria: different age groups, physical and mental ability to participate in a group discussion, and the use of different service options during the project (Table 2 for participant characteristics in the Results section). No one refused to participate. All five proposed participants were informed in a phone call, invited by mail, and received compensation after participation. A focus group discussion guide (Appendix 2) was used. It was not pre-tested. The main themes for the focus group discussion were reasons for participating in the intervention study, positive and negative experiences, and suggestions for improving such a project.
Demographics of the focus group with the PP.
PP: project participants.
The focus group with CPs (N = 6) took place in February 2023 on the premises of the consortium management and lasted 90 min. We aimed to involve representatives from all project partners. Representatives from two participating health insurance companies (a representative of the third health insurance company was unable to attend), two employees of the social service provider, as well as one representative each from the participating housing company and technology developer participated. The main topics in the focus group guidelines were expectations, assessment of the project, experience of project implementation, benefits of the project, and criticism of the project. Appendix 3 provides the focus group guidelines. They were not pre-tested. Only the researcher and the focus group participants were present and no focus group discussion had to be repeated. Focus group discussions provided a forum for exchange regarding a certain topic. Using specific stimuli, a group process was initiated to explore different perspectives on the topic and thus went beyond individual interviews. Focus groups are suitable for revealing the experiences, needs, motives, attitudes, and behavioral patterns of healthcare service users. Assessing the perceived effectiveness of the intervention from the perspective of different actors is central. Focus groups are used as a discursive outcome evaluation of the success and quality of an intervention. 28 They enable mutual stimulation of the participants and represent a collective body of knowledge, making them suitable for jointly proposing improvements.
The evaluation by the participants themselves represents an important contribution to the perspective of the older people who have tested the new form of care within the project. At the same time, there was a need to evaluate the experiences of the CPs in order to ascertain existing challenges and positive aspects as well as listen to their suggestions.
Surveys
As a third data source, in addition to expert interviews and focus groups, we utilized a section of the standardized survey administered at the end of the intervention to the remaining N = 165 participants who completed the entire 12 months of the intervention as per the protocol. These questions were specifically designed to evaluate the new form of care. A standardized questionnaire was only used for the written evaluation at the end of the intervention. It was not necessary to request permission for a questionnaire because evaluative questions were developed by the research team itself as they related to the various components of the intervention. At the end of the 12-month intervention, all participants were called individually and asked to complete the questionnaire. The questionnaire was completed either over the phone, collaboratively, or sent by mail to the participants, along with a prepaid return envelope. The following evaluation questions were answered using a school grading system from 1 (very good) to 5 (insufficient) and the option to choose “I can't judge”:
evaluation of the overall care offering (“How do you rate the care offering overall?”) evaluation of individual components of the care offering: general emergency call system, automatic fall detection, motion sensors, contact person at the social service provider (“How do you evaluate the individual components of the care offer?”) safety provided by the care offering contribution of the care offering to continued living in their own home functionality of the emergency call system and sensors accessibility of contact persons through the care offering
All evaluation questions can be found in Appendix 4.
Data analysis
The evaluation of the three individual interviews and the two focus groups were analyzed jointly due to similarities in theme and methodology, which facilitated a higher level of anonymization. This is particularly important because the content of the three individual interviews can be traced back quite clearly to the three employees of the social organization by insiders. Based on the approach of Kuckartz, 29 the first and second authors conducted a structural qualitative content analysis using the software MAXQDA. 30 In this process, text segments from all interviews were initially deductively assigned to the three main categories: challenges, potentials (positive aspects), and future wishes/requirements. They were then inductively subdivided and systematized into further subcategories. The subcategories are partly interdependent, and their separation is artificial for heuristic reasons. Subsequently, both authors refined the subcategories following a two-person review process. Finally, statements were quantified and placed in a hierarchy. The coding tree evolving from the content analysis should be viewed in relationship to the subcategories found in Table 2. Because it would not have been possible in terms of organizational handling, the participants did not provide feedback on the findings. For the evaluative survey questions, the questionnaires were transferred into Research Electronic Data Capture Software, and descriptive statistics were used to analyze their results.
Ethical aspects
The study VAL (German: VBW) was approved by the Ethics Committee of Charité—Universitätsmedizin Berlin with application number EA1/032/20 on May 7, 2020, and registered in the German Clinical Trial Register (DRKS-ID: DRKS00023171). The principles of ethical standards for medical research involving human subjects, as outlined in the Declaration of Helsinki, 31 were followed. Furthermore, there is a comprehensive data protection concept including commitment declarations from the employees. Additionally, for the results presented here: the individual interviews and focus groups were anonymized with pseudonyms after transcription, and the standardized survey data were evaluated anonymously. All participants gave informed consent. The consent forms are available for the focus groups and interviews, and the participants gave their consent to take part in the intervention with the health insurance company (data transmission) and the social service (participation in the study).
Results
The results will be analyzed here first in terms of qualitative information provided by the mention of common topics in focus groups and interviews, and afterward in terms of quantitative information derived from surveys.
Results of the focus groups and interviews
Tables 2 and 3 provide an overview of the characteristics of the participants of the focus group.
Participants of the focus group with project consortium.
To ensure a clear presentation of the qualitative results evaluation, we will present the findings in a hierarchy organized by the frequency of mention in descending order within the main categories: challenges, potentials (positive aspects), and future wishes/future requirements. “Challenges” are difficulties, obstacles, or problems that require effort and determination to overcome, to improve the outcome. “Potentials” are positive aspects and opportunities that can be leveraged to achieve a goal or advance and “future wishes” refer to hopes or aspirations that a person has for the future. These are future requirements. Table 4 shows, in addition to the category system, an overview of the distribution of statements (CPs or PP) as well as the number of mentions in parentheses. Statements made only by CPs are marked with an a and statements made exclusively by intervention participants are marked with b. This analysis includes only responses with at least four mentions.
Qualitative evaluation from focus groups and individual interviews—overview of main and subcategories.
Note. Number of mentions in parentheses. CPs: consortium partners.
Exclusively CPs.
Exclusively participants.
Challenges
Regarding the challenges, the most mentions were related to implementation/technology. A total of 51 statements referred to problems with the technical implementation.
False alarms
The most frequent mentions,
17
mostly from CPs, were related to false alarms of the automatic fall sensor integrated into the emergency call necklace. One CP reported: Usually it was due to an accidentally triggered alarm, false alarm, which mostly, that is, with the participants who have now actually dropped out of the project, were automatically triggered by the fall sensor. This does not mean that the participants accidentally touched it or anything like that, but they may have made a too fast movement (CP 5, 48).
This also happened at night, as the interviewees suspect, due to air pressure under the duvet, or a too-fast movement of the participants. The exact cause remained unclear. To further reduce false alarms, three CPs suggested attaching the alarm to the wrist in the future. Several participants also reported false alarms, as shown in the following quote: I somehow caused several false alarms. [laughs] The first false alarm was when I woke up in the middle of the night, it was around half past one (…) And I have a relatively low toilet. And since I’m not very tall, I probably got up there vigorously. Anyway, when I was in bed, my phone rang. I have one right by my bed. I picked it up right away. Yes, a nice gentleman called (…) and asked if I was okay. That didn't just happen once (…). So afterwards, of course, whenever I wandered around in my nightgown, went to the toilet (…) I held onto it. Then it didn't happen anymore. At some point it happened when I turned in bed, I was awake but I turned and somehow it happened (PP 4, 23).
Another intervention participant reported that the alarm went off when she moved into or out of the bathtub.
Sensor failures
Furthermore, some CPs criticized sensor dashboard failures (15 mentions). On one hand, it was found that the sensors were defective and had to be replaced: "We then quickly realized that these door sensors, refrigerator door, apartment door, somehow didn't work row by row" (CP 3, 38). On the other hand, sometimes the sensors did not provide any or little data as shown in this quote from the same interviewee: And with the dashboard stuff we still have some systems that don't bring any data [sighs], where we now really go there for the fourth or fifth time partially exchange the device for a second time and simply (…) we are not clear what is actually the problem (CP 3, 48).
Fundamental issues
Fundamental unspecific challenges with the technology were mentioned eleven times, mainly by CPs: "Exactly. However, I must also say that the technology would need to be revised a bit more to put it that way [laughs]" (CP 7, 58).
Evaluation of risk scores
The last point listed under challenges in technology concerned evaluating risk scores (six mentions). Evaluating scores appeared time-consuming and posed transmission and synchronization difficulties due to a lack of delineation of the interpretation framework. Some CPs thus found using risk scores to be less beneficial as evidenced by this quote: So from our perspective now with regard to risk score how I am currently experiencing it say that it is rather an additional effort and that there is no benefit yet and we cannot assess it. The problem with risk score is also that we do not know (…) which factors are included in this whole mechanism (CP 5, 56).
In terms of implementation/collaboration with partners from the consortium, CPs primarily named challenges.
Communication
Temporary personnel changes, failure to clarify uncertainties, especially in technical aspects, delayed or insufficient feedback, and other communicative difficulties led to uncertainties in project progress. A person from the consortium formulated this as follows: "And then coordinate everything with project partners. So there was a lot of communication work that was not considered beforehand and could not be considered" (CP 5, 41).
The two thematic areas in the challenges section implementation/process flow and implementation/collaboration in the project were exclusively mentioned by CPs (13 mentions each).
Personnel effort
First, it is worth mentioning the personnel effort and role definition of staff in social care within the project (seven mentions). Due to significantly higher time and care requirements than initially planned for participants, some CPs, especially employees of the social service provider, felt like "buffers for everything" (CP 4, 42). A person from the consortium confirmed this with the following words: I would say that at first we had much more educational work. But that was also because normally in planning it was that we have a personal conversation, a personal consultation before the study starts so that we advise them at home and explain the technology and social care a bit more in depth (CP 5, 56).
Motivation/recruitment
Second, difficulties in recruiting and motivating participants were listed here. Local public recruitment actions to promote the project were limited by the emerging pandemic situation at that time and distributing flyers did not initially lead to numerous participants. Only a large-scale telephone campaign conducted by one participating health insurance company showed success: "We started calling after receiving poor feedback on letters. And that was quite successful with patients, insured persons" (CP 8, 120).
In the area of implementation/framework conditions, two themes emerged in this evaluation.
COVID-19 related uncertainties
First, the uncertainty caused by the COVID-19 pandemic greatly hindered the project implementation (nine mentions). The actual procedure for installing the sensors had to be constantly adjusted due to changing hygiene measures and required a certain level of "caution" (CP 2, 21). Since seniors, as the target group for the intervention, were also categorized as a “high-risk group,” questions of accountability and the design of personal contacts arose during the project. A person from the project team formulated this as follows: "So the first thing is of course uncertainty with the (…) new situation. That was also a big topic for us" (CP 5, 125).
Data protection
Second, five participants as well as CPs expressed their concerns regarding data protection (five mentions). One senior was concerned that their conversations might be listened to. This concern was also shared by other seniors. A CP expressed it as follows: "Data protection is, I believe, really a big issue, which also concerns roles, as you rightly said. So we were not allowed to go on-site, so to the patients really. That was already a problem" (CP 4, 105). It should be noted that the topics “uncertainty due to the pandemic,” “adjustments to project processes,” and “personnel changes” are not completely distinct but overlap.
Potentials
In the section for positive aspects, most mentions were in the area of project conception. A total of 22 statements referred to potentials in this area.
Fit accuracy
The most frequently mentioned aspect, exclusively by CPs, was the alignment of the project with the target group (eight mentions). They described the project concept as audience-appropriate, barrier-free, and thus an easily usable and actual relief in the daily lives of seniors. A person from the consortium expressed it as follows: So the (project) title immediately appeals to me again [laughs], that was the credo, to say 'Self-determined and safe - Living in your own four walls’ and that was also the experience, it should address those needs. And that's something that has proven true, that many wanted to use this offer to see if it could be an additional help for them (…) (CP 2, 16).
Care in one's own home
An important category was the possibility of ensuring care at home through the project (seven mentions). This would decrease the attraction of moving to a nursing home, causing this move to be delayed. A CP summarized it as follows: "I believe that technology can really help (…) people live independently in their own homes for longer" (CP 7, 56).
Increase in the sense of security
Related to this, participants and two individuals from the consortium mentioned the increase in a sense of security through fall sensors and emergency call systems (seven mentions). An example of this is: "So especially when you are alone in the apartment at night and (…) you have this feeling of security that you can quickly get medical help if needed. Yes, the need for security" (PP 1, 36).
Good participant use and evaluation
In the implementation/technology category, there were five mentions stating that technology is user-friendly for participants. One person summarized it as follows: "So I would say they handle technology relatively well, especially with emergency call systems. As I said, it's very simple technology. They don't have to do much" (CP 5, 60). In four mentions, the theoretical potential of risk assessment was rated as very high.
In terms of implementation/collaboration with the CPs, there were a total of 16 mentions.
Practical implementation
Twelve praised the joint practical implementation of the project and four (additional) highlighted good communication between project partners. A person from the consortium commented on this: What do we do now? How do we deal with this? And despite these many new challenges that led to individual project partners being overloaded at times, it was always a collaborative effort. And that was very pleasant and I think mainly because everyone was convinced by idea and implementation to really create added value for participants (CP 2, 100).
Volunteer telephone mentoring program
Thirteen mentions from both participants and the consortium addressed social care offered to participants during the project. A person from the consortium explained it as follows: Well, what we do is low-threshold and very personal social care. That means we have everything from A to Z. We have participants who almost never contact us, where we naturally offer services, whether it's tablet training or soon we will also offer a coffee afternoon, but they simply don't want to take advantage of the offer because maybe they don't need it. So we have participants who hardly need social care, I would say. Then we have a good middle ground who are occasionally in contact with us. And then we also have participants who are strongly in contact with us. And these are usually those whose health is rather poor, who may not be well-connected socially, who need more care from us and want to take advantage of it (CP 5, 66).
Participants stated that they experienced a basis of trust in social care situations and through existing connections could access further necessary social assistance or support when needed. Eight mentions highlighted the positive impact of volunteer telephone partnerships which provide an important social component enabling interpersonal exchange. A person from the consortium explained the process as follows: Telephone partnerships are being greatly expanded. So we have a large pool of volunteers there, and we look at who fits with whom. Participants are then called at regular intervals by one person each time. And a personal level is built up there too. And then there is this neutral third person who may not be part of their family circle whom they can tell something different and who then provides a neutral perspective and gives some courage or just listens where they can tell a joke (CP 5, 68).
Four mentions from the consortium additionally pointed out the positive effect of telephone contacts for participants.
Future wishes/requirements
More documentation and transparency
Within the project conception, there were a total of six mentions of the desire for more transparency and documentation throughout the project—from both participants and the consortium. This includes creating an information base for all stakeholders through appropriate informational materials, meeting notes, or interim reports. A CP expressed it as follows: Well, what we have learned now is that there was always a desire from participants to know during the study what the results are. We were constantly asked about the status of things, if we could say something already, when there would be a brochure, when there would be an event. It is important to consider this from the beginning so that those who want to participate in such a project also naturally have an interest and want to be informed (CP 5, 144).
Use of the latest digital technology
The CPs expressed the desire to use the latest technology in the future. This includes not only tablets but also smartwatches, better software, and AI-based systems. This involves more comprehensive systems and improved range, as evidenced by the following quote from a CP: And technology is constantly being improved. There are always further connections. I believe, especially towards smartwatches, which are becoming increasingly popular. Smartwatches that are also connected to the emergency call system, which then not only work well in the apartment but probably also in the garden (CP 5, 84).
Clarification of data protection issues
In the area of implementation/framework conditions, three CPs expressed their wish for a more concrete and practical data protection concept for information pathways and workflows in five statements, as exemplified by the following quote: "This data protection issue is always relevant. I believe that when approaching future projects, one should always consider data protection in all processes" (CP 5,72).
Results of the questionnaire
A total of N=165 participants were surveyed. One of the participants was surveyed after 9 months of intervention shortly after they dropped out. The remaining 164 participants were surveyed after completing the full 12 months of the intervention. They were between 76 and 94 years old (M = 83.14 years; SD = 4.08 years) and 112 participants (68%) were female. The participants rated the intervention positively for the most part. Almost 80% of the participants agreed with the statement that there were always contact persons available through the care services. Just under three-quarters of the participants (72%) reported flawless functioning of the emergency call system and sensors. Slightly lower (71%) was the proportion of those who agreed with the statement that the care services helped them to continue living in their home. Nearly three-quarters (72%) stated that they felt secure through the care services. Overall, between 10% and 13% of participants were unable to provide an assessment of the intervention (Figure 1).

Evaluation of the intervention by participants.
The individual components of the care services were rated as follows: Nearly 80% of the participants rated having a contact person at the social service provider as very good or good. About half of the participants (52%) found the motion sensors to be good or very good. Similarly, around half of the participants viewed the automatic fall detection positively. Approximately 40% of the participants could not assess the motion sensors and automatic fall detection and chose the response option "cannot say." In contrast, three-quarters of the participants rated the emergency call system as good or very good. Nearly 12% could not evaluate the availability of the social service provider, and almost 14% could not rate the emergency call system in general. The percentage of "cannot assess" responses was higher for motion sensors (almost 43%) and automatic fall detection (almost 39%; Figure 2).

Evaluation of the individual components of the intervention by participants.
Discussion
Our study aimed to evaluate the implementation of a new care model for older adults living at home, combining monitoring technology with social service support. The intervention involved installing emergency call systems with automatic fall detection and motion sensors in the homes of 181 seniors. Its goals were to prevent emergencies, monitor mobility patterns, detect risks early, and reduce falls. The evaluation included two focus groups, three individual interviews, and 165 questionnaires. Focus groups identified challenges in technical implementation and team communication, while highlighting the potential of the project's design and social support. Interviewees expressed a desire for improved technology in the future, and the questionnaires showed overall positive results.
Combining social and technical support
The VAL project distinguishes itself by combining social services with technology. Participants receive ongoing support not only through monitoring technology but also from volunteers and social service staff, helping with daily tasks. The social aspects were particularly valued, as seen in the high ratings for the question about contact persons. This focus on social support illustrates the unique and innovative nature of the VAL project.
Enhancing security and support for participants
As this study shows, social support and assistive technologies enhance the feeling of security for older adults. Interviews and focus groups highlight the project's strength in integrating technology and social support to help participants live independently at home. Both study participants and the project consortium identified this approach as a key success factor, supported by international findings. A study by Lukas et al. 32 demonstrated that emergency call systems can have a positive impact on the sense of security held by older people and their relatives. In the context of this study, an emergency call system was installed for 39 older participants and evaluated by the participants themselves after 6 months of use. Just like in the present survey, around 70% reported feeling secure with it. However, the flawless functioning of the system was rated differently (15% in the study by Lukas et al. 32 vs. 72.1% in the study presented here).
A study by Vimarlund et al. 33 on assessing the needs for technical services among older people also found that the latter prefer support tools that prevent social isolation, preserve self-determination and autonomy, and stimulate and maintain physical and mental activities. Adequate social support is a crucial factor for physical and psychological quality of life.34,35 In the present project, remaining in one's home, a sense of security, and social support emerged as central concerns and the second greatest implication of this analysis.
Challenges in technical implementation
The technology, particularly the sensors, occasionally faced functionality issues, reflected in slightly lower satisfaction with the technical aspects compared to the social components. Despite this, most participants rated the technology as "very good" or "good." Notably, about 40% could not evaluate the motion sensor, likely due to limited interest, understanding, or a lack of relevant incidents during the study period. Participants understood the study aimed to inform integration into standard healthcare.
Interviews and focus groups identified technical challenges, such as frequent false alarms and dashboard failures, as key obstacles. Stakeholders expressed a need for updated technology, greater transparency, and better progress documentation to enhance project implementation. False alarms were also a common issue in the study by Lukas et al. 32 They were documented by 27 out of 39 participants, with one person experiencing them 150 times. Targeted measures were implemented to reduce false alarms, which improved the situation. Tsertsidis et al. 36 noted that many of the support services in recent years aimed at enabling independent aging at home integrated little technology. In the future, more focus should be placed on support services with assistive technology. This includes addressing the wishes of participants who, for example, preferred a wristband instead of a necklace for emergency/fall alarms. Additionally, more suitable pre-tests or usability studies of assistive technology should be conducted in the future to generate user- and needs-oriented assistive technology instruments.
The “reassurance paradox”
Participants reported feeling more secure, despite the technology occasionally triggering false alarms. This reaction likely stems from their perception of these alerts, not as malfunctions, but as evidence that the system is actively monitoring their safety. The lack of clear differentiation between various alarm types may have contributed to this misunderstanding. However, high satisfaction with prolonged home stays and the resulting sense of security appeared to outweigh any concerns about false alarms. Additionally, the participants’ willingness to adopt such systems at home supports this trend.
As highlighted in a 2014 literature review by Chaudhari et al., 37 many studies on fall detection technologies have shown that older adults tend to view these systems favorably. They value the sense of security these devices provide, even when false alarm rates are high. The interpretation suggests that the emotional reassurance of "being watched" often has a more significant impact than the system's functional accuracy. Trust in the technology, alongside technology-related anxiety, emerges as a critical factor influencing the perceived sense of safety. 38
This “reassurance paradox” underscores the complex interplay between technological functionality and emotional well-being in older adults. In this study, the paradox is further amplified by the participants’ positive reception of the social support element, which enhanced their overall experience with the system.
Future research and implementation
Central themes for future projects highlighted in this analysis primarily revolve around data protection and privacy.39,40 This includes secure communication channels between project partners and structured responsibilities. Furthermore, both qualitative and quantitative results from the project underscore the significant importance of involving and having access to social service providers. Quality of life, especially among older people, cannot be solely improved through technology but is primarily influenced by their overall social environment. 41 Technological advancements in healthcare are often rushed to market without sufficient testing of their benefits. To better serve the growing number of older adults seeking independence, evaluating such care systems is crucial. Both objective measures (e.g. falls, hospital visits, emergency calls) and users’ subjective experiences are essential for assessing and improving these technologies.
Limitations
The findings generated here allow statements about the perceived functionality and subjective assessment of the intervention. A limiting factor is the relatively small number of participants (Focus Group I N = 5 and Focus Group II N = 6) as well as the holding of only one focus group with participants and the project consortium each (it is recommended to have a group size of six to twelve individuals and to conduct three to five focus groups. 28 Furthermore, no data material from the interviews or focus groups was returned to the participants for verification. The concept of data saturation is not directly applicable to the interviews conducted as part of the study. As only three employees of the social service provider worked on the project, it was not possible to conduct more interviews to gain their perspectives. Nevertheless, the different participants in the two focus groups (study participants and consortium staff) provided a broad perspective. Another limitation involves the selection of participants for Focus Group I, who were in relatively good health. A certain level of mobility to attend the focus group location and the ability to communicate openly and clearly were beneficial for participation. This meant that individuals with more severe health or cognitive limitations did not participate. However, these groups were also considered and discussed in the focus groups. An interesting phenomenon in the participant focus group was that participants did not perceive false alarms as problematic but rather viewed them as indications of a functioning system. Therefore, the study participants in Focus Group I were well-suited to subjectively assess the system they tested, but they likely lacked the technical expertise to evaluate false alarms as such. This is also evident from the high number of missing values in the results presented earlier. It would be beneficial to use qualitative methods to address this issue. In this analysis, we only included people who had already chosen to use technology so the results cannot be applied to less tech-savvy people. The positive overall evaluation certainly has to do with the fact that mainly people with a positive attitude toward technology took part in the study. This means that the results apply primarily to these people and further studies would be needed to generalize the results. We did not survey dropouts, resulting in a lack of their perspective. Furthermore, we did not use any validated survey instruments and had not pre-tested the follow-up questionnaire and the interview and focus group guidelines prior to using them.
Conclusion
The intervention aimed to prevent emergencies, analyze mobility patterns, identify risk situations early, and reduce the likelihood of falls. The assessment of the intervention examined from the perspective of participants and project stakeholders in the consortium varies depending on the individual components of the new form of care. Interviews and focus groups suggest that the project's greatest potential lies in its conception. The approach of helping participants live independently and safely at home through interconnected technology and social support was recognized as a key success factor by both the consortium and participants. Participants generally rated the combination of being connected to a monitoring system in their home and having access to a contact person for social and health-related issues positively. The malfunctioning technical component highlighted a need for revision for all parties involved. The insights gained here show that surveying individuals who implement and use an intervention can provide valuable information on potential reasons for its effectiveness or ineffectiveness, which can then be captured by other parameters. The experiences of those involved in testing a new form of care should therefore always be captured both quantitatively (surveys) and qualitatively (focus groups, interviews). The study on the process evaluation of supported technology for older adults within the framework of this project is of great importance, as it provides valuable insights into the acceptance and effectiveness of digital aids for seniors. By applying a mixed-method approach, both quantitative and qualitative data were collected, creating a comprehensive picture of user experiences and challenges. These findings are crucial for improving the quality of life for older individuals and promoting the integration of technology into their daily lives. Additionally, the study helps to shape and adapt future developments in assistive technology to meet the needs of this population group. Future research on assistive technologies for older adults should focus on integrating technological and social support while also examining secure methods for data protection.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076241308445 - Supplemental material for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project
Supplemental material, sj-docx-1-dhj-10.1177_20552076241308445 for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project by Eva Jansen, Juliana Supplieth, Sonia Lech, Jan Zöllick and Johanna Schuster in DIGITAL HEALTH
Supplemental Material
sj-docx-2-dhj-10.1177_20552076241308445 - Supplemental material for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project
Supplemental material, sj-docx-2-dhj-10.1177_20552076241308445 for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project by Eva Jansen, Juliana Supplieth, Sonia Lech, Jan Zöllick and Johanna Schuster in DIGITAL HEALTH
Supplemental Material
sj-docx-3-dhj-10.1177_20552076241308445 - Supplemental material for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project
Supplemental material, sj-docx-3-dhj-10.1177_20552076241308445 for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project by Eva Jansen, Juliana Supplieth, Sonia Lech, Jan Zöllick and Johanna Schuster in DIGITAL HEALTH
Supplemental Material
sj-docx-4-dhj-10.1177_20552076241308445 - Supplemental material for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project
Supplemental material, sj-docx-4-dhj-10.1177_20552076241308445 for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project by Eva Jansen, Juliana Supplieth, Sonia Lech, Jan Zöllick and Johanna Schuster in DIGITAL HEALTH
Supplemental Material
sj-pdf-5-dhj-10.1177_20552076241308445 - Supplemental material for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project
Supplemental material, sj-pdf-5-dhj-10.1177_20552076241308445 for Process evaluation of technologically assisted senior care using mixed methods: Results of the virtual assisted living (VAL, German: VBW Virtuell Betreutes Wohnen) project by Eva Jansen, Juliana Supplieth, Sonia Lech, Jan Zöllick and Johanna Schuster in DIGITAL HEALTH
Footnotes
Authors’ information
Dr Eva Jansen (female) is an anthropologist and qualitative health researcher. In this project, she was responsible for the qualitative analysis and wrote the main body of the article.
Juliana Supplieth (female) is a nurse and health scientist (B.Sc). She has 7 years of experience in quantitative and qualitative data collection. In this project, she was the study nurse and responsible for collecting standardized pre- and post-survey data, leading the focus groups with the participants, conducting the qualitative analyses, and interpreting the findings. She also contributed to writing the body of the article.
Dr Sonia Lech (female) is a psychologist and senior researcher. In this project, she was conducting the interviews, led the focus group with the consortium, and assisted in the interpretation of the findings. She also contributed to writing the body of the article.
Dr Jan Zöllick (male) is a psychologist and senior researcher with a focus on mixed-method studies in the context of digital healthcare interventions. In this project, he was responsible for data collection, and quantitative analyses; he led the focus group with the consortium, and he contributed to writing the body of the article.
Dr Johanna Schuster (female) is a psychologist and senior researcher in the field of health research in older age. In this project, she was the principal investigator for the evaluation, leading the study group. She also collected data within the focus group of the participants, and she contributed to writing the body of the article. Before the study, none of the authors knew the people in the consortium or the participants of the project. During the project, the evaluation team and project consortium got to know each other through regular meetings on the implementation of the study. Accordingly, they were professionally acquainted with each other at the time of the individual interviews, focus group, and the end of the project. JS knew the participants in the focus group and the PP through the standardized telephone surveys that were conducted at the beginning and end of the project. JSc was unknown to them at this point. The participants were informed about the aim and purpose of the focus group and the evaluation of the project.
Contributorship
Conceptualization: JZ,SL, JSc; Data collection: JZ,SL, JS, JSc; Data curation: EJ, JS, JZ; Formal analysis: EJ, JS, JZ; Funding acquisition: JSc; Investigation: EJ, JS, JZ; Methodology: JZ, EJ; Project administration: JSc; Resources: JSc; Software: EJ, JS, JZ; Supervision: JSc; Validation: JS; Visualization: JS; Writing—original draft: EJ; Writing —review & editing: JSc, JZ,SL, JS; All authors approved the final version for publication.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
Gemeinsame Bundesausschuss (grant number 01NVF18035).
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
