Abstract
Background:
Personalized music listening can be highly effective in supporting people experiencing changes in mood and behavior due to Alzheimer’s disease and other conditions causing dementia. However, a lack of staff education can be a barrier to effective use of music in residential aged care.
Objective:
This study investigates the usability, appeal and relevance of an online training course designed to help care staff develop personalized music programs for people living with dementia in their care.
Methods:
A mixed methods approach was taken in which 13 participants took part in a think-aloud session while using the training. Qualitative data from the think-aloud session and interviews were triangulated with results of a survey evaluation of the training.
Results:
Themes relating to Engagement, Usability and Appeal, Pedagogical Design and Content, and Transfer, Impact and Barriers to Uptake were discussed. Results indicated that participants regarded the virtual environment and pedagogical design as of high quality, although some minor issues in navigability were identified. Participants also found the training to be highly relevant to their roles as caregivers and reported ways they would incorporate key concepts into care practices. However, ongoing barriers to implementation of such training were identified including systemic issues that contribute to a culture of focus on physical care.
Conclusions:
Consistency of navigation and clear instructions for interacting with content are important in training of time-poor care staff. Linking training to industry standards can provide important motivation for implementation of training, although systemic barriers can impede real-world change.
INTRODUCTION
Personalized music listening can be highly effective in supporting people living with dementia [1]. Music listening can reduce agitation [2] and depression [3] and help reduce behavioral disturbances [4]. As such, music has the potential to be a useful alternative to pharmacology as a frontline approach to supporting people living with dementia.
However, research shows that a lack of staff education is a barrier to the effective use of music in aged care [5]. Staff often lack knowledge about the potential benefits of music, as well as the know-how to create and implement successful personalized music programs targeted to individual needs [6, 7]. Training is crucial to enable staff to provide quality care in a variety of health contexts [8]. To address the lack of education regarding the effective and targeted use of music to support people living with dementia, we developed an online training course, “Music playlists for people living with dementia” 1 .
The training was co-designed with aged care staff and family members of people living with dementia [9], and takes carers through a 5-step process for developing a personalized music program for people under their care [10]. The process can be used to address a number of diverse circumstances, such as to prevent the agitation that typically occurs at a certain time of day or in a particular care situation, or to decrease symptoms of withdrawal or depression that an individual is experiencing. Although further clinical testing is required, the program draws on evidence-based strategies from music therapy and music psychology and has been found to have positive outcomes in both residential and home-based care. In a pre-post experimental study in which participants were randomly allocated either to the 5-step music program or to a waitlist control group, improvements to quality of life over a 6-week period were found in the treatment group as well as improved mood and increased engagement with the environment during individual listening sessions [11]. Caregivers using the program have also reported positive effects on mood and reduced agitation in the people they care for [12].
The 5 steps covered in the training include: 1) collecting background information about symptoms and music preferences, 2) creating playlists, 3) trialing the playlists and observing responses, 4) refining and revising playlists based on responses, and 5) incorporating music into individual care plans. As well as outlining the benefits of using music for people living with dementia, the training also seeks to provide a realistic viewpoint of music’s value within overall approaches to non-pharmacological behavior support. For example, the importance of first ensuring that physical needs are addressed as a possible reason for agitation is outlined. It is also emphasized that the five steps occur in a cyclical process, with frequent revisiting of an individual music program. Indeed, the user is helped to understand how people at different stages of cognitive decline may respond to music differently and how individual programs can be revised as needs change. Tools and templates for evaluating the impact of the program on individuals and incorporating it into care plans are provided within the training. Information is also included in the training about equipment choices and considerations in selecting appropriate equipment.
This study examines the usability, appeal, and relevance of the online training course to aged care staff, people living with dementia and their family members. In particular, this study aims to: 1) Evaluate the usability and appeal of a training course for carers; 2) Evaluate the degree of user understanding of course concepts and relevance of information; 3) Understand possible barriers to uptake of the training or application in practice.
METHODS
A mixed methods approach was taken using a concurrent nested design in which qualitative and quantitative data were collected within a single session [13]. Qualitative data were comprised of verbalizations and observer notes during a ‘think-aloud’ session and interview feedback, a common method for evaluating eHealth materials [14]. During the think-aloud session participants viewed the module or modules of their choice in real-time during a zoom session with a member of the research team. Qualitative data were triangulated with results from a survey questionnaire.
Participants
Purposive sampling was used to recruit 13 participants (9 female; 4 male) including one person living with dementia, 5 family carers, and 7 participants working in residential aged care (Table 1). Participants were recruited by emails to stakeholder mailing lists and consumer advisory groups. Care staff needed to have 2 or more years’ experience in aged care. All participants needed a sufficient level of English to be able to participate in discussions and access to a device with a reliable internet connection. No participants who expressed an interest were ineligible and there were no withdrawals from the study. Participants were offered a $30AUD voucher for participation.
Participant characteristics
Participant characteristics
Music playlists for people living with dementia consists of a number of different modules that are available depending on the role of the person undertaking the training, including care staff and lifestyle workers, family members or friends, executive staff and managers, and allied health professionals. Each module covers the 5 steps, with the Carer & Lifestyle module (designed for care staff and staff employed to coordinate leisure activities in aged care) being the most detailed, since these are likely to be the staff most involved in implementing music programs. The course in its entirety takes about 20-40 min to complete. While some small amounts of reading are involved, the training is primarily presented using video content and scenario-based learning to help users envisage real-life applications. Quizzes and interactive activities, such as assessing the likely effect of particular pieces of music on mood, consolidate learning and help users develop hands-on experience in selecting music for various care scenarios.
Procedures
After obtaining ethics approval from the Human Research Ethics Committee of Western Sydney University (Approval number: H14462), emails were sent to potential participants. When interest was expressed, eligibility was determined via email correspondence, and eligible participants provided with an information sheet before signing a consent form.
Sessions of approximately one hour were held individually over Zoom with participants joining the session from their home or work in an isolated room. Participants were provided with a link to the online training. The session facilitator explained the think-aloud procedure, requesting that participants share their screen for the video recording. Participants were informed that all forms of feedback, both negative and positive, would be helpful to the research team in improving the training in the future. Participants explored whichever modules of the course they were interested in (Table 1), independently navigating through the materials whilst articulating their thoughts out loud [15]. Participants were prompted to continue speaking if they were silent for 20 consecutive seconds. Other than that, the facilitator remained silent and off-camera in order to avoid disrupting the train of thought or influencing responses [16]. The facilitator also took notes throughout the session.
Once participants had explored as many modules of the training as they wished, a follow up discussion was conducted in which the session facilitator asked questions about events or responses that arose during the think-aloud protocol. Finally, participants completed a short online survey.
Materials
Most participants viewed the training course on a computer, although one viewed it on a smartphone and another on an iPad. Zoom record function captured the shared screen to allow researchers to see which part of the course the participant was viewing as they spoke.
A broad interview guide covered 6 topics including: 1) general impressions, 2) relative appeal of features, 3) needed changes or improvements, 4) relevance, clarity and comprehensiveness of content, 5) ease of use, and 6) visual appeal. The Qualtrics survey completed by participants consisted of 29 items rated on a scale of 1 (Highly disagree) to 5 (Highly agree). Questions were mostly drawn from the Questionnaire to Evaluate Online Training in the Workplace (CEFOAL) [17, 18] and the Mobile App Rating Scale (MARS) [19]. The CEFOAL was designed to assess online training across a range of workplace contexts on five dimensions: pedagogical design, tutor performance, virtual environment design, timing, and transfer of learning. Since items related to tutor performance were not relevant, 14 items relating to the other four dimensions were retained. Although the MARS was developed for evaluating smartphone applications, 10 items relating to visual design and course content which were not covered by the CEFOAL were used with minor adaptations (such as changing “app” to “course”). An additional 5 items were included to consider potential impact of the course on approaches to care such as increasing “awareness of how to use music”, and “motivation of people caring for someone with dementia to use music as part of the care routine”.
Data analysis
Quantitative data analysis. Due to the small sample size and relative homogeneity of responses, factor analysis could not be performed on the survey items. However, data were logically grouped into 3 primary dimensions of 9 items corresponding to dimensions of the CEFOAL: 1) pedagogical design, 2) virtual environmental design, and 3) transfer and impact. Reliability analyses returned Cronbach’s alphas of 0.63, 0.85, and 0.85 for these dimensions respectively. One question each addressed timing/duration (Item 7), and overall quality (Item 24).
Qualitative data analysis. Analysis followed principles of Framework Analysis [20], which is commonly used in health research and allows data to be analyzed both deductively with reference to an overall framework and inductively to derive novel themes. Think-aloud sessions and interviews were transcribed verbatim and screen shots of shared screens added into the transcriptions for reference. Frequency counts of usability issues were tallied to identify problematic sections by an independent researcher (non-author). Overall, responses to 139 screens in the training were analyzed. In a second wave of analysis, coding and thematic analysis was conducted by the first author and independently by a non-author as a check to reduce potential bias using the table function in Microsoft Word. Data were categorized according to three dimensions: 1) virtual environmental design, 2) pedagogical design, 3) transfer, impact and barriers to uptake. In a final stage of analysis, qualitative and quantitative results were considered together to derive an overall picture of participant response and identify key areas for improvement.
Reflexivity
Sessions were facilitated by a Research Assistant (non-author, female), since both authors were directly involved in developing the training, a fact which may have been known to participants. The use of a think-aloud method was utilized as a means of reducing bias as well, since participants have the opportunity to provide an initial response to the training without prompting from the session facilitator. None of the research participants were previously known to the session facilitator. No information about the session facilitator other than job title was provided to participants.
Both the first and second authors have previously conducted research about personalized music programs for people with dementia and were involved in developing the training, creating some risk of bias in data analysis. The use of a non-author to conduct the think-aloud sessions and as an independent coder was used in an attempt to reduce this risk.
RESULTS
Engagement, usability, and appeal
Quality of the training and duration each received an average rating of 4.92 (SD = 0.28, possible scores 1–5). With possible scores ranging from 9 to 45, the Virtual Environment Design dimension returned a mean of 42.23 (SD = 3.17), indicating a high level of satisfaction with the design.
Overall, 28 usability issues were identified in the think-aloud study, falling into two main categories: readability and navigation issues. Qualitative analysis of the data about the virtual environmental design identified 2 main themes and sub-themes including 1) visual elements, and 2) technical difficulties (Table 2).
Themes relating to virtual environmental design
Visual elements. In general, participants reported appreciating the simplicity of the overall visual design, which was “nicely laid out” (P2), “visually interesting” (P6), and “appealing” (P5). However, on some screens people reported that color choices or insufficient contrast made it difficult to read. Participants made suggestions about text or navigation keys that could be visually bolder so that they stand out, “I’d really like my arrows to be a bit bolder to tell me where I have to go” (P13).
Participants appreciated the personal touch that images of people provided but reported that they would have preferred animations rather than “static” images (P5), or would have preferred closer face shots to make the person more central. Others noted that the exclusive use of images of older people could make carers of people with younger onset dementia feel uncomfortable, “You’re defining dementia as an old person’s disease. My husband was 58 when he started on his dementia journey, so looking at old lady Shirley probably would be a turn-off” (P10). Others appreciated the use of graphs and other visualizations that communicated concepts without dense text.
Technical difficulties. Most participants said that the course had been easy to navigate, “My knowledge of anything technical is very limited, so I appreciated the fact that it was easy to use” (P7). However, others found it non-intuitive to navigate in places. One participant stated, “There were a couple of points where it wasn’t obvious which button I should press next” (P2). Participants noted that consistency was important in this regard, with one participant observing that it was confusing when buttons changed format:
“You changed the visual connection when you made those other things buttons, instead of a ‘Next’. You changed the visual connection so I had to discover my error in clicking on that to realize that ‘oh, it’s not a button now.”’ (P10)
Ease of navigation was considered to be particularly important given the diversity of technical know-how among potential users, with many being “people who are not tech savvy” (P13). Facilitator notes also noted a need for clarity about how to engage with the interactive elements of the course, recording several instances of participant confusion.
Pedagogical design and content
With possible scores ranging from 9 to 45, ratings of the Pedagogical Design returned a mean of 42.54 (SD = 2.37), indicating a high level of satisfaction. Participants commented that the messages were easy to understand and clearly communicated. One participant said, “I think it’s really insightful. I think it gives a lot of good information” (P5). Another stated:
“For people like me who, you know, I don’t find it easy to sit at a computer and take it all in, it was good. I was more nervous about coming off as stupid, not understanding, but it’s very good for people who have all sorts of different understandings.” (P7)
Language. While some people thought the language used was clear, home-based carers tended to find the language too formal or academic especially when viewing the modules designed for aged care staff. Two participants in fact noted that they had not understood that the “Carer and Lifestyle” module was designed for aged care staff (P10, P12) and that this labelling itself was somewhat confusing. There were also some particular terms that users were not familiar with, for example expressions such as “partnering in care” (P8). However, some terminology was noted as potentially confusing for aged care staff as well, with one participant pointing out that, “There’s quite a lot of staff we have here that would not have a clue what you’re talking about when you say ‘playlists’. They still live in the 1990s” (P3).
Pedagogical techniques. Participants appreciated a number of techniques used in the training to communicate information, such as the use of scenario-based learning to help users see real-life applications of concepts. Interactive activities such as song selection, and quizzes to reinforce learning were also appreciated.
Content. The elements of content most frequently commented upon by participants were the focus on person-centered care and the need for personalization. One participant said, “I really like the focus on individual preferences and asking the person or those who know them really well” (P9).
A participant with dementia stated:
“I really loved how person-centred this was. You know, you’ve really got the person at the centre of their care, so I really want to congratulate you on that. That really warms my heart.” (P13)
Others agreed that giving information about equipment and technology was useful and appreciated the inclusion of information about the potential for negative responses. One participant noted that it could be quite discouraging to carers if there was no acknowledgement of times that music may not work. She said:
“If you say ‘here are some of the benefits’, ‘oh, what have I done wrong? He’s not getting any of these benefits’, you know? ‘Am I doing it the wrong way?’. So I don’t think you can be black and white, ever, with someone who is living with dementia.” (P10)
Themes relating to pedagogical elements
Other content that was noted as useful included the importance of considering carer wellbeing, the need to observe responses rather than to put music on and then leave, suggestions for how family members can communicate with staff, and clear explanation of musical concepts for people without music training.
Transfer, impact, and barriers to uptake
The Transfer & Impact dimension returned a mean score of 39.70 (SD = 4.64, possible range 9 to 45), suggesting a high level of agreement that the training would be useful and relevant to participant roles as carers, and likely to achieve the intended purpose. Qualitative data in this data category revealed four broad themes: 1) incorporation in care plans, 2) changes to practice encouraged, 3) increased awareness, and 4) barriers to implementation (Table 4).
Themes relating to transfer & impact
Incorporation in care plans. A number of participants liked the fact that the training talked about incorporating music use into the overall care plans of the individual, “It’s great that you mention the care plan, because it can be such a good tool” (P9). Others liked the fact that the training linked to national government standards in Australia and a recently introduced requirement that care plans outline non-pharmacological strategies to supporting people experiencing changes in behavior.
Changes to practice encouraged. Participants spoke about aspects of the training that they would incorporate into their own approach to care or that would be useful for encouraging change in other staff. One key area of change highlighted in the training was the need for personalization of a music program, rather than using generic forms of music in communal living areas. One participant described ideas that the training had sparked for her:
“It just gave me an idea that you can play selected music for that one resident who might be having certain behaviors, so that’s something that we should probably look in to. Because we have the sound system, we have a TV, etc., in the activities room where a lot of things take place during the day for our residents. I’m thinking of one in particular who does have certain behaviors. Music in that situation would be really good, actually, so thank you, might give me an idea.” (P7)
Another described her increased awareness about personalization saying:
“You need to use more playlists than only one, and I think that’s missing actually at the moment in how we use it. We often use the same - because we know they like Elvis, then we play always Elvis. Or we know that they like to sing ‘Daisy Daisy’ and that’s why we sing all the time ‘Daisy Daisy’. I think it’s good to have a little bit more of a variety and to make it more suitable and more for the specific resident.” (P4)
Increased awareness. Participants also believed the training would support increased awareness of a number of important issues including consent, how the needs of people with dementia change over time, technology choices, and risks. In relation to consent it was noted that the training drew attention to the fact that it is important to make sure the person living with dementia consents to have headphones put on them. Several other participants noted that the training would help staff to be more aware that music programs need to be revised as needs change over time, “You need to change the playlists as the person goes on with their dementia progression. It may change the whole thing, the whole game. They may not like it anymore.” (P1)
Another issue that the training raised awareness of is the need for carers to consider other contextual factors before using music to address behaviors or moods. As one participant said, “You can’t just shove on a song and think that’s going to fix everything - there’s obviously other things that we’ve got to keep in mind to help them with. Like pain, depression, that sort of thing.” (P7)
Barriers to implementation. Two important barriers to implementation of the training were identified by participants: 1) an ingrained culture of focusing on physical needs, and 2) time-poor staff. Some staff argued that limited budgets were responsible for a focus on caring for physical needs, while others suggested this was a mindset that could slowly be shifted. Lack of time was also suggested as a contributing factor to the focus on necessities of care, with one staff member commenting:
“If I was a care staff at one of our cottages, I think I’d be going ‘oh, I don’t have time for that’. It’s pretty time consuming, isn’t it, to sit down with someone and go through and develop a playlist and then to go out and then get the device.” (P6)
DISCUSSION
This study aimed to evaluate the usability, appeal, and relevance of a training course for carers of people living with dementia, as well as to understand possible barriers to implementation of the training in practice. Overall, the training received high ratings from participants for quality and design as well as content. The interactive elements, person-centered approach, and information about technology choices, managing negative responses and personalization were particularly appreciated by participants.
Despite the positive response from participants, some issues were identified with consistency of navigation and a lack of instructions in how to use interactive elements. Clearer labelling of modules and different use of language between modules designed for people in different roles was also noted as important. Previous research has similarly found a strong need for consistency of placement of navigational buttons [21, 22]. Other studies have demonstrated that clarity of instructions is closely linked to student acceptance of online training [23]. Given the heavy reliance on online learning that has occurred during the COVID-19 pandemic and further reliance on hybrid learning that is likely into the future [24], ensuring student engagement with any kind of online learning is imperative. Future iterations of the training materials will need to ensure increased consistency and clarity of navigation.
Care staff could see clear relevance of the training to industry standards of care that they were required to implement and noted ways that viewing the training had caused them to think about new strategies for caring for individuals living with dementia. Linking of the training to such standards can serve not only as a motivation for implementation of the training, but also as an important ingredient in the implementation of the standards themselves.
However, despite its relevance, participants noted that time pressures would still be a barrier to implementation of the training in real-life settings. According to the Capability, Opportunity and Motivation (COM-B) model [25], for behavior changes to occur there must be both the capability and opportunity to engage in a particular behavior as well as a motivation that outweighs the motivation for any competing behavior. Training such as that being evaluated in this study may help enhance capability as well as motivation to develop personalized music programs as part of care plans by aged care staff. However, systemic barriers still exist which may reduce the opportunity for changing approaches to care. While a gradual shift from task-oriented care may be occurring in aged care in recent years, additional obstacles such as time pressure and a high workload can lead care staff to see anything beyond physical care as an additional burden [26]. Some studies have suggested that the increased requirements to provide person-centered care has in some cases, resulted in a reduced focus on the personhood of care staff with priority being given to their value as facilitators of increased resident personhood [27]. In Australia, workforce shortages and high staff turnover rates affect capacity for systemic change towards person-centered approaches to care [28]. The current study highlights that education and training alone cannot facilitate change.
A further challenge that exists is the need for standardized tools for measuring the effectiveness and cost-benefits of psychosocial interventions in dementia care [6]. Many studies that examine the effectiveness of psychosocial programs assess immediate, short-term outcomes such as enjoyment or engagement. However, studies that examine long-term clinical outcomes are more scarce. Measures used are often developed on a symptom-focused or a loss/deficit model [29]. Other studies use broad quality of life measures that assess dimensions that not all psychosocial interventions are likely to influence, such as financial security, and thus fail to detect long-term change. Thus, measures that are sensitive to the long-term benefits of psychosocial interventions such as music programs, are important so that aged care facilities are able to determine the cost-effectiveness of such programs.
This study was limited by the fact that participants were largely carer and lifestyle staff or family members. The involvement of additional care home managers would have been useful, and, despite the fact that one of the modules was for allied health professionals, no allied health professionals were recruited. In addition, the think-aloud process itself can influence responses to the training. For example, users can become distracted by what they are talking about and fail to read instructions properly or may stop talking out loud when they are concentrating on the training. Participants are also conscious of the session facilitator observing their response and therefore some response bias may occur. For this reason, larger sample sizes to facilitate the use of objective measures such as facial analysis would be useful in future research. This study has focused on the usability, appeal, comprehensiveness and relevance of the training to people who care for someone living with dementia. Future studies should consider the effectiveness of the training in increasing awareness and effecting changing approaches to care by the assessment of performance on specific learning outcomes as well as on measures which indicate changes in attitudes and practice in real-life care situations. Nevertheless, the study indicates that the training developed is likely to be of assistance to carers aiming to integrate personalized music programs into care plans for people living with dementia, contributing to shifting cultures of care in residential aged care settings.
Footnotes
ACKNOWLEDGMENTS
The authors have no acknowledgments to report.
FUNDING
This research was funded by a grant to the first author from the Dementia Australia Research Foundation.
CONFLICT OF INTEREST
The authors have no conflict of interest to report.
DATA AVAILABILITY
Data is available upon request to the first author for research with similar aims as the original research.
