Abstract
New or adapted methods and tools are needed to ensure the voices of older people with cognitive impairment and dementia are included in evaluations of care services which aim to support their quality of life (QoL). In this study, cognitive interviewing practices were used with a group of 26 older service users with cognitive impairment from two service providers in New South Wales, Australia, to test and modify the Adult Social Care Outcomes Toolkit Easy Read (ER) survey to improve its suitability for this cohort. We used Antonovsky’s “sense of coherence” framework to describe our research approach and how it was adapted to provide a manageable, meaningful, and comprehensible experience for our participants. While the modified ER format made the survey more comprehensible and meaningful, it was the techniques of cognitive interviewing that made the research approach manageable. We argue that while ER does support the research process for older service users with cognitive impairment, combining ER pictorials with the qualitative interactions with the researcher, in particular cognitive interviewing strategies, is needed to support a cohesive approach to assess care-related QoL in this vulnerable group.
Keywords
Background
Globally, the number of people living with dementia is expected to increase from 47 million in 2015 to 122 million by 2050 (Prince, Wimo, et al., 2015). Dementia is a major cause of disability, and people with dementia and their caregivers are high users of social and health-care services (Prince, Guerchet, & Prina, 2015). Dementia is a collection of symptoms that are caused by disorders affecting the brain, resulting in cognitive impairment that impacts on thinking, behavior, and ability to manage everyday life (World Health Organization, 2017). Dementia can be difficult to diagnose, and many people also delay help-seeking for their symptoms. So there are also many who live with progressive cognitive impairment as they age, who have not been diagnosed with dementia (Bradford, Kunik, Schulz, Williams, & Singh, 2009). While often high users of health and social care services, historically, the perspectives of people with dementia and other forms of cognitive impairment have been excluded from health and social research (Taylor, DeMers, Vig, & Borson, 2012). There is a critical need for new methods and tools which can help to illuminate their care experiences and needs (Heggestad, Noetvedt, & Slettebo, 2012; McKeown, Clarke, Ingleton, & Repper, 2010; Phillipson & Hammond, 2018) and promote their social health (Dröes et al., 2017).
With older people (defined in this study as 65 years or over) accessing support services to help them stay at home longer, there is a specific need to develop research tools to assess and understand domains of care-related quality of life (QoL) for older service users living in the community. The Adult Social Outcomes Toolkit (ASCOT) is one valid and reliable measure used extensively in the United Kingdom (UK; Bauer et al., 2017; Callaghan, Brookes, & Palmer, 2017; Department of Health, 2014; Netten et al., 2012) and more recently in the Australian older service using population (Cardona, 2018; Cardona et al., 2017; Kaambwa et al., 2015; Milte et al., 2014). It measures social care–related QoL (SCRQoL) (Netten et al., 2012) across eight domains. The lower order domains include personal cleanliness and comfort, accommodation cleanliness and comfort, food and drink, and feeling safe. Higher order domains include social participation, occupation, and control over daily life. The eighth domain, dignity, asks participants to consider how the way they are treated by care staff makes them feel (Netten et al., 2012). However, despite being widely adopted, barriers exist to the ASCOT engaging sections of the aged and disabled populations who use care and support services to assist them at home (Rand, Caiels, Collins, & Forder, 2017; Turnpenny et al., 2018).
To address these limitations, the Adult Social Care Outcomes Toolkit (ASCOT) applied Easy Read (ER) principles in the development of the ASCOT-ER questionnaire (Turnpenny et al., 2015). It was originally co-developed and beta tested with younger people with intellectual disability and autism to address their reported difficulties with access to the standardized research tools being used in the UK (Turnpenny et al., 2015). Initial testing of the tool showed great potential for use in this cohort of younger people with disability (Turnpenny et al., 2015; Turnpenny et al., 2018). However, the suitability of the ASCOT-ER has not been assessed for an older community-dwelling service using population with cognitive impairment. Distinct issues must be considered when designing accessible research approaches for older people. For example, questionnaire design considerations include the impact of declining information processing and working memory on both question order effect and response option effect (Schwarz, 2006).
Additional considerations also exist when designing research tools for use with people living with cognitive impairment or dementia. These include the capacity of individuals for orientation and attention, as well as language and communication skills (Mozley et al., 1999; Ready & Ott, 2003). For people with dementia, standard interview formats continue to have limitations due to their reliance on abstraction, recall, and verbal reporting (Beuscher & Grando, 2009). The utility of health-related QoL tools also remains limited for those with moderate or advanced dementia (Bowling et al., 2015). In social and health research with people with dementia, tailoring of the research approach has supported improved consent procedures, maximized responses, and promoted more positive participant experiences (Cridland, Phillipson, Brennan-Horley, & Swaffer, 2016; Murphy, Jordan, Hunter, Cooney, & Casey, 2015). Innovations have also led to the development of new more accessible quantitative and mixed methods tools to assess health-related QoL (Hoe, Katona, Roch, & Livingston, 2005; Smith et al., 2006; Trigg, Jones, & Skevington, 2007).
Visual methods, like ER, have been useful to explore and contextualize interview questions for people with dementia (Banks & Zeitlyn, 2015). However, ER also has limitations and requires a careful and contextual approach to promote accessiblity, as emphasized by Turnpenny et al. (2018) who originally developed the ASCOT-ER. Despite ER’s growing use, there is limited research on its effectiveness, with some questioning the usefulness of images in creating clarity for people with cognitive impairment (Chinn & Homeyard, 2017; Codling & Macdonald, 2008; Hurtado, Jones, & Burniston, 2014). Indeed, in some cases, adding pictorials has been shown to decrease comprehension rather than aid it (Brennan, Worrall, & McKenna, 2005; Poncelas & Murphy, 2007; Strydom & Hall, 2001). A recent literature review of ER and accessible information for people with intellectual disabilities (Chinn & Homeyard, 2017) found that personalized and individualized approaches were the best way of supporting access and that co-production of resources with intended audiences was essential.
So while visual methods have been useful to explore and contextualize interview questions for people with dementia (Banks & Zeitlyn, 2015), the above challenges around accessibility and ER encouraged us to not only test the ASCOT-ER with older service users with suspected cognitive impairment or dementia but also to develop a theoretical framework that would help us frame our approach to this testing.
In this article, we have utilized Antonovsky’s (1979) “sense of cohesion” framework for promoting health which highlights the importance of people experiencing their world as manageable, comprehensible, and meaningful (Golembiewski, 2017). This framework was developed and applied within studies of mental or social health (Huber et al., 2011). However, we found it a useful framework for describing a coherent research approach and it has supported us to reflect on whether our research methods, including ER, created a sense of coherence for our participants.
Research Aims and Objectives
The research aims for this study were to: explore the extent to which qualitative cognitive interviewing and the ASCOT-ER supported a manageable research data collection process with community dwelling older service users with cognitive impairment; assess whether visual images and accompanying text used in the ASCOT-ER communicated the concepts intended (i.e., were comprehensible and meaningful); and respond to any confusion around the ASCOT-ER and modify it to make it more comprehensible and meaningful for older people living in the community with cognitive impairment.
Method
Ethics Approval and Consent Process
Approval for the conduct of the study was provided by the University Human Research Ethics committee (HREC Approval 16/236). ER formats were used to support the comprehensibility of both the research information and consent forms. Participants were provided with this written information by their service provider prior to the interview but were also supported to read through the information, ask any questions, and discuss their understanding with the researcher. In this context, 23 participants demonstrated a clear understanding of the research and were supported to provide written informed consent; 3/16 participants expressed willingness to take part but showed some confusion related to the issues concerning data use and confidentiality. In these cases, written proxy consent of a care partner was also obtained. In 1/3 cases, the care partner was also present during the conduct of the interview at the request of the participant. In these cases, the care partner also signed a consent form. All participants were also observed to monitor process assent through their willingness and interest to discuss and answer the questions (Dewing, 2007).
Study Design and Procedure
The study had three phases: assessing the comprehensibility and meaningfulness of the original ASCOT-ER using a cognitive interview which incorporated staggered reveal and “talk-aloud” methods; modifying the ASCOT-ER by responding to the insights gained from Phase 1; and retesting the modified ASCOT-ER using the cognitive interview protocol in Phase 2.
The ASCOT-ER was used in the study with permission from and in collaboration with the ASCOT team at the Personal Social Services Research Unit at the University of Kent, who developed the measure. The University of Kent is also the copyright holder (the full questionnaire is available on www.pssru.ac.uk/ascot). The ASCOT-ER questionnaire uses three modes of communication to support the participant to engage images that orient the participant to a topic, an explanation, and question in plain English and a Visual Response Scale with small facial expressions to support the response (https://www.pssru.ac.uk/ascot/).
Firstly, to support the manageability of the testing phases (1 and 3), we adapted the Turnpenny et al. (2015) cognitive interviewing protocol. Cognitive interviewing has been used extensively to improve questionnaire design by getting participants to make the usually hidden processes of answering questionnaires overt (Willis, 2005). This involves a range of practices such as think-aloud interviews and verbal probing (Willis, 2005). In our study, as in Turnpenny et al. (2015), this involved first showing the participant the image associated with each question in the ASCOT-ER (with the text covered). The researcher then asked the participant to talk aloud about what they see in the image and what they think the topic of the question is about. The explanatory text and question stem was then revealed, and participants were asked to discuss how they feel in response to the question. This helped to pace or limit the amount of information participants are required to consider at any one time. It also provided opportunities to explore whether the person had found both the picture and the text “comprehensible” (i.e., relevant and oriented to the topic) and “meaningful” (i.e., applicable to them). Finally, following these discussions, the interviewer revealed the visual scale and asked participants to choose a response which corresponded with how they felt about this domain in their life. The cognitive interviewing practices not only allowed for testing of the ASCOT-ER with older service users but also supported a cohesive research approach. After the interviews, researchers wrote field notes reflecting on the comprehensibility, manageability, and meaningfulness of the ASCOT-ER and interview process for the participants.
Phase 2 involved analyzing the data from interviews and research notes (discussed below) to identify and develop modifications to the ASCOT-ER to increase comprehensibility and meaningfulness. Suggestions were marked up on the original survey and discussed by all members of the research team. The aim was to keep changes to the minimum while adapting it where necessary to make the ASCOT-ER more suitable for the cohort.
Recruitment and Characteristics of Participants
A purposeful sample of people with confirmed or suspected cognitive impairment was recruited via two service providers in the Illawarra-Shoalhaven and Southern Highlands regions of NSW, Australia. Inclusion criteria for the sample included that participants were community dwelling and recipients of aged care supports and services through the Home Care Packages Program (HCPs; Department of Social Services, 2015). HCPs support older people (65 years and over) who need coordinated care and services to help them live at home. HCPs are delivered under a model of consumer directed care and provide a choice of provider and services through access to an annual budget care budget—from Level 1 to support basic needs (US$8,250) to Level 4 to support higher needs (US$50,250; My Aged Care, 2019). Types of supports and services can include assistance with personal care (e.g., bathing, showering, dressing); nutrition, hydration, and meal preparation; continence management; support with equipment and aids to promote mobility and dexterity; and support for access to nursing, allied health, or other clinical services (high-level packages only).
All participants in this study had been identified by their service providers as having confirmed or suspected cognitive impairment or dementia. Recruitment via service providers proved necessary to support the complex needs of this cohort to ensure appropriate assistance with recruitment and follow-up with participants if they needed any further assistance or support as a result of taking part in the research. Service providers provided potential participants and their care partners with a written information sheet. Participants who provided verbal consent were contacted by the research team to arrange an interview in their homes in which their consent was also confirmed (see description above).
An important aspect of the study was to test the ASCOT-ER for older people with cognitive decline and to understand the impact of this decline on the usefulness of the approach. People were asked whether they had ever been told by a doctor that they had memory problems or dementia but were not asked to specify the type. The cognition of participants was also screened by the researchers—both of whom have clinical experiences with people with dementia—using the Mini-Cog© tool. In this community dwelling population, a score of 3 or less was used as suggestive of cognitive impairment to include identification of people with milder forms of disease (Lorentz, Scanlan, & Borson, 2002). Conducting the cognitive screening was not about determining inclusion or exclusion to the study but allowed reflection on the suitability of the ASCOT-ER for people with varying levels of cognitive ability. General functional ability was also assessed using the Home and Community Care (HACC) functional screener (maximum total score of 16), with a lower score indicating more difficulty managing with daily activities of living (Eagar, Owen, Marosszeky, & Poulos, 2006; Green, Eagar, Owen, Gordon, & Quinsey, 2006).
Twenty-six older adults took part in the cognitive interviews in their own homes, 16 in Phase 1 (before modifications to the measure) and 10 in Phase 3 (after minor modifications). Participants who took part in the study ranged in age from 63 to 99 years (mean 82.51 years). Only 2/26 (8%) spoke a language at home in addition to English at home, just over half were female (15, 58%), 19 (73%) reported having a family member provide them with care, though only 6 (23%) had a coresident care partner and most of this group received a lower level of funded support (Level 2; 16, 67.7%). When asked about their family finances, most reported they had some finances left over at the end of the month (65.4%) as opposed to having “just enough” or “not enough.”
As discussed above, cognitive screening was used not as an inclusion criterion but instead to allow conclusions to be drawn about the usefulness of the ASCOT-ER for older people with different degrees of cognitive impairment. Cognitive screening confirmed the suspected impairment of 24/26 participants. The two participants whose Mini-Cog score did not confirm impairment were still included in the study as they had “suspected” memory impairment and had been referred by their service provider as they felt they would benefit from a modified approach to assessing their outcomes. Just over a third (9, 35%) reported they had a medical diagnosis of dementia. As shown in Table 1, there were no significant differences between participant characteristics of the two groups.
Participant Characteristics for Groups Testing Different Versions of the Adult Social Care Outcomes Toolkit Easy Read (ER).
Note. Comparison of the groups, p values are not included as all were nonsignificant.
Interviews varied from 45 min to 2 hr. One participant who presented as having more advanced cognitive impairment Mini-Cog score of 0 showed limited ability to converse and was unable to use the visual scale to rate QoL within the various domains. As such, data from this participant were not included in the analysis. Another participant (also with a Mini-Cog score of zero) was able to converse about the domains but was not able to respond to all of the questions. Data from this interview were included as it was still useful to inform the comprehensibility and meaningfulness of the tool. The researchers involved in the conduct of the interviews have research and clinical experience (Phillipson) or clinical experience (Jenkins) working with people with dementia.
Data Analysis
Interviews were audio-recorded and transcribed verbatim for analysis. Transcripts were analyzed in NVivo 11 by two researchers. To provide insight into the comprehensibility of the ASCOT-ER, transcripts and field notes on research reflections were deductively coded under each ASCOT domain and then under six overarching themes relating to the “clarity” or “confusion” of the pictures, the explanatory text, and the question stem. For example, themes included “clarity pictures” and “confusion pictures.” This initial step in Phase 1 informed modifications that were made to the questionnaire. It also supported later comparison between responses to the original (Phase 1) and modified version (Phase 3) of the ASCOT-ER. While this preliminary analysis informed the modifications, secondary analysis was required to understand themes around access.
The second stage of analysis involved coding data to understand how the clarity and confusion related to Antonovsky’s (1979) three dimensions of cohesion. As such, data were recoded deductively under the themes drawn from Antonovsky’s notion of a cohesion, that is, comprehensible, meaningful, and manageable. Table 2 includes the definitions of these three dimensions of cohesion for a research approach. Subthemes under these overarching themes were simultaneously developed to determine what aspects of cohesion might be relevant to the research. Table 3 shows an example of this process unfolded from the data to the subtheme.
Definitions of Three Dimensions of Cohesion.
Stages of Analysis.
Results
Central to the suitability of the ASCOT-ER was that all aspects of it (the images, the text, and the rating scale) were comprehensible to older adults with cognitive impairment. By comprehensible we mean that people could recognize and understand what they were being asked to consider by the ASCOT-ER questionnaire (in the way is intended) and could respond to it in a way that represented their evaluation of their own situation.
Lower Order Domains: Clarity and Comprehension
In our testing, we found the pictorials representing the lower order domains were mostly fit for purpose and frequently prompted a useful, clear, and direct context for the topic of the question. For example, in the Accommodation domain, most participants correctly described the pictures in the ER question as concerning their home environment and noted that they anticipated the question would invite them to reflect on the state of different rooms within their house.
P4: The living room and the kitchen. Maybe about housework?
P3: Yeah, I guess if you’re able to clean [your home] properly, or it’s all cluttered up with stuff or, yeah.
However, the comprehensibility was compromised for some participants by the fine detail within the pictures. For example, in the Accommodation domain, not all participants could see that the kitchen was dirty.
I: So what about the kitchen, what do you think it happening there?
P9: Oh I don’t know, it just looks like a kitchen to me.
I: So can you see the flies there and the sauce bottle, the spills?
P9: No not really. It’s all a bit small…
When choosing their response options, participants emphasized cleanliness as central to their considerations, particularly bathrooms, toilets, and kitchens, so it was considered important that the dirty room was clear in the image. As such, the question was modified to enlarge the image. In Phase 3, this resulted in more participants identifying the lounge room as clean and the kitchen as dirty.
P23: Oh it’s a lounge room, a relaxing place, where you watch tele or read a magazine or something…and a dirty kitchen.
P25:…that’s certainly clean and neat and tidy, but this [the kitchen] is a complete mess.
Again, the fine details within the pictorial image representing the Food and Drink domain also lowered the comprehensibility of the question for some participants. With regard to the pictorial, most were able to recognize a variety of food portrayed in the image.
P11: Well this is a meal and I would have to say because there’s orange juice on the table and I would say, it’s bread, I would say it’s breakfast, yeah, I’m going to say that’s breakfast.
P6: Oh, food. Cereal. I think that’s cereal. That looks like fish and chips and maybe even peas…Yoghurts and juices, mixed nuts, fruit. Like a healthy diet actually.
But some did experience difficultly identifying some of the finer details, for example, of food on a plate.
P3: [The picture is about] Food
I: So can you identify all the different things in that picture easily enough?
P3: Not all. They’re quite small. What is that? Peas and fish?
In the revised version, the picture was enlarged. However, when exposed only to the image, some still wondered if the question would concern only breakfast food. However, once the wording of the question was also revealed, they understood that it was asking them to consider all of their food and drink. This was important as it demonstrates that both the pictorial and the explanatory text together were needed to make the whole context of the question comprehensible to participants.
P23: Most of it, I thought originally it was a breakfast menu but it’s not. It’s food. I had thought I had to be more specific than that.
Lower Order Domains: Emotion, Meaning, and Complexity
Most lower order domains were well comprehended, albeit requiring some practical/physical modifications to improve clarity. However, the Safety in the Home domain was more complex. This domain seeks to establish the extent to which people feel safe from threats, bullying, theft, or falls in their own home. Responses to the pictorials in this domain were often emotionally charged—highlighting the meaningfulness of the topic for the participants. Comments highlighted concerns about threats of intrusion or intimidation and being robbed.
P6: She’s getting bullied. My son talks like that to me sometimes…it’s not nice
P7: Oh, she’s going to lock [the door]…She won’t let him in. Like me. I never, I never let anybody in past that gauze door…
Despite their emotional connection with the topic, in Phase 1, participants only discussed their fear of falls after reading the explanatory text. For retesting in Phase 3, one of the pictures representing interpersonal threat was replaced with a picture which showed someone mobilizing with the assistance of an aide and a person (see Figure 1). Retesting of the adapted version confirmed that this was a helpful change to highlight the issue of safe mobility in both pictorial and text. Unfortunately, however, removal of the image of a woman being shouted out inadvertently resulted in no participants discussing verbal abuse when testing the modified version in Phase 3. As such, the use of three pictures may be more suitable to support consideration of all relevant dimensions of safety in future. This again highlights the importance of all concepts being represented both pictorially and in text to ensure comprehensibility and to prompt participants to reflect on all the dimensions and potential meanings of each domain.
Original (left) and modified (right) Easy Read pictures and stem for safety at home.
Similar to “Safety in the Home,” participants found the images in the “Safety in the Local Area” domain somewhat emotive. The “Safety in the Local Area” domain explores the extent to which participants feel safe when out and about in their local area and is inclusive of both fear of robbery or abuse and fear of falling. In this cohort, few participants went outside alone, and going out at night posed particular fears for their safety. In line with this, responses to the pictures stimulated an emotional reaction and also prompted appropriate responses which reflected the dimensions of interpersonal safety, but not falls.
P3: Fighting. Someone’s getting bullied. Oh, he’s threatening her. Yeah, and she’s scared.
P6: She’s getting mugged. There’s people drinking and they’ve graffiti’ed on the wall. Not safe outside the home.
In the modified version, the image of the young people drinking in front of a wall with graffiti was replaced to show a couple walking to represent the mobility aspect within the domain. This was recognized by some as representing someone walking with another person to aide their safety, but not all participants were clear about its intention.
P21: Bag snatching. And this one, I’m not sure of…she must be looking after him. She’s walking alongside him. Probably protection.
P22: Theft, or robbery. The second one is, I don’t know what that one means.
In future revisions, the need to retain the graffiti image to reveal general perceptions of safety should also be considered.
Higher Order Domains—Comprehensibility and Meaning
Overall, as with the “Safety” domains above, there was some confusion around higher order domains, largely due to the more conceptual and life course–specific nature of their content. For example, the “Occupational” domain considers what people do with their time and the extent to which people feel they engaged with enough meaningful activities. Testing in Phase 1 highlighted that the pictorials were not always interpreted as intended. For example, rather than illustrating volunteering, many identified the central picture as shopping.
P3: Looking in the pantry, yeah, and shopping, yeah. Is that shopping, yeah?
P4: The cooking and the shopping, and the swimming…But where is housework?
Some participants did not relate to the examples in the explanatory test and suggested alternatives as more appropriate for an older audience.
P16: M’mm…all the things you do during the day…I…don’t go to work, college or volunteering…
P3: I don’t think somebody with dementia would be going to college. But volunteering’s good. I volunteer…and gardening, yeah. Good. [And I] go to groups…like, you’ve got exercise or walking [groups] or those sorts of things.
To improve the relevance, both the pictures and the examples in the explanatory text were modified to be more meaningful for the older cohort (see Figure 2). This included a removal of the option for work, retaining volunteering and suggesting activities such as gardening or attending groups. Retesting in Phase 3 highlighted the improved understanding of the domain and suggested that the amendment encouraged reflection on activities that were meaningful for their life stage and context.
Original (left) and modified (right) Easy Read pictures and explanatory text for the occupation domain.
P26: Yes. I could see that as cooking or whatever, gardening, reading the newspaper or reading. I used to do a lot of reading but I don’t do that much now either…
Responses to the “Dignity” domain are used to assess whether people feel about how their home support workers treat them. Participants are prompted to consider both kindness and respect. Responses to the pictorials suggested that participants were responding to images they believed represented a conversation between themselves and their service provider and were prompting them to reflect on whether the conversation was thoughtful and inclusive.
P11: Well they’re having a chat, those two. That looks like [my service provider] sitting talking to me, [but] that one…she must be left out, yeah, feeling left out, yes.
One participant felt that the picture showing two staff talking with one another in the presence of the third person was not relevant to the community setting.
P4: I think this never occurs actually, with these people, this picture. This actually never occurs…[two staff in my home]. That’s wrong. This picture is wrong.
Some participants expressed uncertainty in relation to the word “dignity” and the term “paid support.” However, once the text was available, the meanings for both appeared to be adequately clarified.
P5: What’s dignity? Oh I see…being treated nicely and kindly. Oh right. Yes, I know what that means then.
In Phase 3 testing, the term “paid support” was amended to “care staff” to better reflect the language in the aged care sector in Australia. This improved comprehensibility for some participants, but for others, the meaning of the question was only fully realized when the interviewer specifically named the staff from a particular agency, or used the vernacular of the participant, for example, “It is asking about ‘the girls’ who come to your home to help you from Agency X.”
Interestingly, much like the lower order domains, responses to the “Control Over Daily Life” domain reflected high degrees of comprehension. Responses to the “Control Over Daily Life” domain assess whether overall people feel they have adequate choices in their daily life. The question prompts consideration of choices including what they eat, the clothes they wear, and what they spend their money on. In response to the pictorials, most participants identified the domain as concerning food, clothes, and money, and it is perhaps the tangible nature of these choices which allowed higher levels of comprehension in this domain. Some also identified that the money picture contained a symbol for pounds rather than dollars, but this did not appear to affect the comprehensibility of the question.
P4: So the lady asks, “what would you like for dinner?” Yeah, the man is ask[ed] what he likes for dinner…He’s probably ask[ed] what he likes to wear…[and] He’s ask[ed] how much money he has.
When choosing their response options, participants identified that feeling in charge was important.
P2: No I wouldn’t like somebody bossing me all the time, love, not while I’m capable of cleaning up the mess I make…
Overall, the pictorials were helpful to support reflections about choices across these different dimensions of life and to support a rating. However, participants mostly reflected only on the examples given. The other choice that was frequently mentioned was the choice to stay living in their own home.
P9: Oh yes I have the choice, I have the choice of what I do in my daily life, I don’t have any problems with that one. The other one is money…I make my own choices with money…What was the other one…well it was, oh clothes, well, yes I don’t make any choices, I don’t buy any but and choices in your food, I have that. So I don’t have any problem in any of those.
However, for others, the “Control” domain question was difficult to answer, particularly for those who felt limited in their functioning and choices:
P2: What is in my daily life? I get up, have my breakfast, if any, if I feel like it, if not I don’t. I go to bed when I want to, sometimes I go to bed in the afternoon ‘cause I’m bored and I’m fed up. I kick the dog, go out and see what’s outside…What choices? What choice am I supposed to have? Can you tell me?
While possible changes were discussed by the research team, there was no clear consensus reached on how the question or pictorials could be amended to make them more meaningful and comprehensible. Indeed, the domain remained meaningful to the above participant [P2] because of the marked limitations he perceived around choice. As such, it was retested in the second round of cognitive interviews in the hope that more insights could be gained. Again some participants were able to understand it and responded appropriately; however, others continued to find it difficult to answer, except as it related to the choice to “stay at home.”
Comprehensibility, Meaning, and the Visual Response Scale
There were mixed responses to the visual scale used to represent the four text response options. Some found it useful to informing their response.
I: And did you look at the smiley faces this time, to help you answer the question?
P24: Yeah, very happy, happy, not so happy, very unhappy.
P18: No, I’d say, I think I’m satisfied. So I’ll put, I’ll put a smile in there.
A few participants appeared to choose the answer next to the “smiley face” as a way to reflect their own positive disposition.
P7: I don’t know. No good getting around grumpy, is it. [You need] a big smile on your face. What have we got here? We’ve got a smile on our face so that’s all right.
On the whole though, most participants were not conscious of using them and only three described them negatively.
P11: No I didn’t look at any of those smiley things at all.
P19: No, they’re not what I feel. That’s just dumb.
Overall, qualitative responses indicated that changes to the wording and pictures of the questionnaire improved the cohesiveness of the questionnaire for the cohort. Researcher reflections also suggest that the cognitive interviewing protocol used in conjunction with the ER format was useful to focus and refocus the participants on each domain and to assist transitioning the interview discussion from one domain to the next.
Discussion
Research approaches that are comprehensible, manageable, and meaningful for people with cognitive impairment are needed to ensure health and care research is informed by user perspectives, a growing number of whom live with cognitive impairment or dementia. With an aging population and changing policy to manage this, more older people are being supported to “age in place” rather than in residential facilities. Such changes make it particularly important to evaluate SCRQoL in community dwelling older populations with cognitive impairment. This study used qualitative methods to test and adapt the ASCOT-ER to support assessment of SCRQoL in older community dwelling service users with suspected or confirmed cognitive impairment. In particular, the study highlights the value of cognitive interviewing techniques, such as using staggered reveal method, to support engagement with an ER tool.
The study is the first to apply Antonovksy’s framework of cohesiveness to the practices of dementia research. While originally the study set out to use a cognitive interview protocol to test the appropriateness of the ASCOT-ER for older service users with cognitive impairment, it emerged that combining the practices of cognitive interviewing with a modified ASCOT-ER created a cohesive research approach which enabled a manageable, comprehensible, and meaningful opportunity to reflect on their SCRQoL. Through seeking to improve the comprehensibility, manageability, and meaningfulness of the ASCOT-ER, these participant-led modifications also highlighted the benefits of a cohesive approach to research in drawing on a range of other qualitative practices.
Creating a Cohesive Research Approach
Comprehensibility
“Comprehensibility” refers to providing research artefacts (e.g., information sheets, consent forms, and questionnaires) that enable participants to make sense of the content in the way intended and relate them to their narrative, context, and current circumstances. Conventional research approaches, such as questionnaires, surveys, and interviews, rely on advanced language and communication skills, recall, abstraction, and verbal reporting all of which are particularly difficult for older people with dementia and cognitive impairment to engage in, comprehend, and manage (Beuscher & Grando, 2009; Cridland et al., 2016; Ready & Ott, 2003). While visual methods have been found to mediate some of the challenges around comprehensibility in health information and research (Banks & Zeitlyn, 2015; Department of Health, 2010), this study revealed the specific conditions around which images in ER increased comprehension.
The images in the ASCOT-ER were more likely to assist comprehensibility if they were visually clear, communicated specific concepts, and represented all dimensions of the intended topic. Firstly, while design accessibility such as the use of high contrast colors, large font sizes, and plain English questions has been previously highlighted as important when designing questionnaires for older people (Fryrear, 2016), the testing of the ER format also prompted the need to consider picture size, contrast, graphic resolution, and clarity for images. As the use of visual methods and ER formats are increasingly used to support the provision of health information and health research (Department of Health, 2010), age considerations for picture clarity and relevance should also be included as part of standard design recommendations.
Secondly, while the ER pictorials were particularly useful in supporting participants to comprehend topics in more tangible domains, such as food and drink and accommodation, in more complex and abstract domains, the images were sometimes confusing or limited the scope of what participants’ considered when responding to the domain. One example was in the adaptation of the “safety” domain where a picture of an older woman being yelled at was replaced with a picture of someone mobilizing with assistance. While the text description of “bullying and intimidation” remained, without the prompt of the picture, there were no respondents in the retest group who reflected on verbal abuse. This highlights that while pictures can increase comprehension, they can also constrain it, meaning it is important to visually representing all aspects of the concept when using an ER format to ensure that respondents are prompted comprehend both the text and the images.
Manageability
“Manageability” refers to providing a research approach that assists people to manage the essential steps involved in completing the questionnaire. This includes attending to each question, looking at the pictures, reading the explanatory text, reading the question stem, choosing a response option, and marking the page to indicate that response option. Difficulties with promoting the focus and engagement of people with dementia within a research interview have been acknowledged in previous research, particularly around strategies to manage the research process itself, such as orientation and attention (Mozley et al., 1999; Ready & Ott, 2003). While some strategies, such as supporting consent and use of visual methods, have been suggested to support research engagement (Banks & Zeitlyn, 2015; Cridland et al., 2016), it is still considered innovative for research approaches, particularly in health and care research and outcomes assessment, to consider the manageability of the research process for a person with cognitive impairment as central to their research design, development, and research engagement (Phillipson & Hammond, 2018).
This study has shown how the manageability of the research approach was supported by using a range of qualitative research practices that scaffolded engagement with research processes visually, verbally, textually, and through relational engagement with the researcher. This involved combining cognitive interviewing, “staggered reveal,” and “talk-aloud” methods to test an “ER” questionnaire. In addition, researchers experienced in working with people with dementia supported the management of the research process by using the different methods to focus and refocus participants on the domain being investigated, support the transition between domains, and help transfer their discussion into a response for each domain. In line with research into increasing access in research for people with intellectual disabilities, this approach to supporting the management of research process suggests that personalized and individualized approaches are the most effective (Chinn & Homeyard, 2017). This means that it is less feasible to conduct the assessment as a self-complete survey and instead requires more personalized support. While this takes more time, it does allow otherwise unheard voices to be heard and could be conducted as part of care plan reviews or annual assessments.
Meaningfulness
Meaningfulness is found in the intensity of personal connections, with questions and the extent to which participants engage with concepts and ideas which are the intended focus of the exploration. In this study, meaningfulness was affected by the ways in which the ASCOT domains (from food and drink to dignity) adequately reflected the concerns of their life phase. This was particularly evident in the “occupation” domain. Occupational identity is a composite definition of “self,” incorporating roles, relationships and responsibilities, values, self-concept, and goals and aspirations and is known to change across life stage (Kielhofner & Wilkins, 2008). The planned move from paid employment to retirement particularly requires considerable adjustment, as one’s occupational identity needs redefinition (Hewitt, Howie, & Feldman, 2010). The ASCOT-ER, being codesigned with a younger cohort, did not reflect this readjustment in occupation, causing confusion and strong emotional reactions from participants who did not feel the images meaningfully reflected their life course. The cognitive interviewing methods allowed meaningful engagement with participants around what occupations were important to them, facilitating modifications to this domain in the ASCOT-ER to ensure that it was meaningful.
Importantly, the images in the ASCOT-ER often prompted meaningful emotional responses in the participants, encouraging them to engage with difficult topics, as we saw above with the discussion of “safety.” The meaningfulness of images has been recognized and utilized in other sensitive areas, for example, the development of “Talking Mat” conversation sets to engage older people in conversations around end of life issues (Talking Mats, 2018). Given the increasing recognition of elder abuse in this vulnerable population (Kaspiew, Carson, & Rhoades, 2018), this study suggests that both visual and text prompts are important to promote meaningful discussion of their experiences. This requires that we consider a broad range of multimodal methods when we are conducting research with people with dementia (Phillipson & Hammond, 2018). It also highlights the importance of being prepared for additional support that participants may need to process emotional distress and, where relevant, to take actions as required to improve participant safety. An advantage of researchers working in collaboration with care providers and care partners, as was the case in this study, is that it allows for researchers, with the permission of the study participants, to discuss concerns and options for additional supports or equipment to meet their needs.
Age and life stage may also have had an influence on the use of visual “smiley faces” response scale which was not consistently meaningful for the older cohort. This was in contrast to the younger cohort in the original ER study who discussed their own familiarity with “smiley faces” through use of social media and emoji (Turnpenny et al., 2018). However, while some had difficultly relating to the faces, the visual scale was useful to some participants who took part in the testing. This indicates that in terms of creating a cohesive research approach, a range of approaches are necessary to ensure that strategies, tools, and resources are meaningful for all older people with cognitive impairment.
Limitations
This study was conducted with a small purposeful sample of older service users with suspected or confirmed cognitive impairment recruited through two local services providers. Only two of the participants screened had results and presentations which suggested they had more moderate or advanced cognitive impairment, meaning we could only develop our cohesive research approach and test the ASCOT-ER with a cohort with milder symptoms. This limitation suggests that use of other tools in the ASCOT (e.g., the ASCOT Multi-Method CH3) may also be of benefit to support a cohesive approach for the cohort in need of SCRQoL assessment. The small sample also means that the generalizability of the adapted version of the ASCOT-ER may be limited and would benefit from further testing and validation in a larger cohort which was purposively sampled to represent diversity in other terms such as geographic and cultural factors. Finally, as most of the adaptations were made to improve the suitability of the ASCOT-ER for life stage and age, it is unlikely the adaptations will make the questionnaire more suitable for service users with younger onset dementia.
Other limitations of the study come from the benefit of hindsight. While consent processes were conducted using ER formats, cognitive interviewing practices such as talk aloud and staggered reveal may have increased the comprehensibility of the process for those who required assent from a care partner. In addition, while the use of the Mini-Cog was important to confirm the likely presence of cognitive impairment for the purposes of the study, participants themselves were often unaware of their cognitive impairment. In future research, we would consider an action research component in regard to how to use the information collected in cognitive assessments to benefit the participant.
Conclusions
Promoting a cohesive research approach for older services users with cognitive impairment and dementia to share their experiences of care-related QoL required thinking broadly about the implications of research processes, practices, and design. While the ASCOT-ER tool provided a useful starting point for our investigations of care-related QoL, adaptations and personalized approaches were needed to make it comprehensible, meaningful, and manageable. This study demonstrated the value of combining interviewing, staggered reveal of information, and talk-aloud approaches with an ER tool format. The ER format when embedded in a personalized interviewing process was useful to focus and refocus participants on the domain of interest, support the transition in questioning between domains, and support the selection of a response for each domain. The ASCOT-ER provided a useful starting point for this, but questions remain as to whether the modified version would be manageable for community dwelling older service users with cognitive impairment as a self-complete questionnaire.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by National Health and Medical Research Council (NHMRC-ARC Dementia Development Fellowship [APP1107401]).
