Abstract
Evaluation of efforts to support family caregivers of people with dementia in their daily medical management responsibilities requires a measure of caregiver self-efficacy (confidence). This article describes the development and psychometric properties of the Caregiver Confidence in Sign/Symptom Management (CCSM) scale, the only available instrument in this area. Measurement development included literature and expert panel review, cognitive testing, and field testing. The CCSM is a 25-item measure (α = .92) composed of confidence in relation to 4 subscales: knowledge of signs/symptoms (α = .83), management of cognitive signs/symptoms (α = .85), management of medical signs/symptoms (α = .87), and general medication management/responsiveness (α = .85), all of which relate to caregiver role strain. The CCSM is a reliable and valid instrument to assess caregiver confidence in sign/symptom management and is useful to determine caregiver needs and outcomes of related interventions. Additionally, it furthers understanding of the role of self-efficacy in caregiver quality of life.
Introduction
Self-efficacy is an individual’s belief in his or her ability to accomplish a specific task in various situations. 1 There is substantial evidence that higher self-efficacy relates to numerous beneficial outcomes, including better physical health and less depression. 2,3 According to the theory, an individual’s self-efficacy is significantly influenced by external factors such as circumstances and the amount of support received. 1 For caregivers of people living with dementia, these external circumstances include the caregiving demands with which they are faced, central among them the medical needs of their care recipient.
Medical needs are common among older adults. More than 80% of Medicare beneficiaries have at least 1 chronic condition, and nearly 25% have 5 or more chronic conditions. 4 Acute medical conditions are common as well; for example, more than 30% of older community-dwelling adults fall each year, 5 which is a leading cause of death. 6 Notably, older adults with Alzheimer’s disease and related dementias (ADRD) have more comorbid conditions, higher illness burden, and a greater risk of injury than those without ADRD. 7 For these individuals, the responsibility of providing daily care and managing chronic and acute health conditions often falls on a family member; in fact, 75% of home care provided to individuals with ADRD is provided by family members. 8 More than 20% of ADRD family caregivers report difficulty with medical or nursing tasks such as administering medications, attending to wound care, monitoring blood pressure, or operating oxygen tanks. 9
In addition, the economic impact of health care and supportive care for people with ADRD is immense. In 2016, Medicare and Medicaid spent approximately US$160 billion in health-care and long-term care payments for individuals with ADRD. 10 The supports provided by caregivers are additionally costly, with 2015 estimates that the care provided by the nearly 16 million family and other unpaid caregivers of ADRD in the United States was over US$220 billion. 10
To complicate the challenges caregivers face to providing care to their relative, the signs and symptoms associated with medical conditions in people with ADRD can be difficult to recognize and manage due to the individual’s cognitive and functional impairment. Many caregivers find it difficult to communicate with health-care professionals concerning their relative’s medical problems. Additionally, health-care providers themselves note that their own biggest challenge responding to telephone calls from caregivers is the caregivers’ inability to assess vital signs and communicate relevant medical information. 11 Having to address medical concerns can also be frustrating for caregivers who feel overwhelmed in their daily responsibilities or ill-prepared to address medical issues. Even worse, the stress and burden experienced by family caregivers while caring for their relative with dementia often leads to their own adverse psychological and physical health outcomes. 8,10 Perhaps not surprisingly, family caregivers of people with dementia have lower self-efficacy than caregivers of people with nondementia-related diseases. 12 What is surprising, though, is that resources to help family caregivers assess and manage new or worsening medical conditions are scarce, especially because family caregivers themselves have expressed a need for more resources to fill this gap. 13
Developing strategies to improve family caregiver’s self-efficacy regarding care for the medical needs of their relative with ADRD is a key need; further, to fully determine whether the strategies are effective, it is necessary to be able to measure caregiver’s self-efficacy in relation to medical sign and symptom management. No such tools are currently available. A few validated self-efficacy and confidence tools focus exclusively on disease self-management (rather than caregiver management) 14,15 or on caregiver mental and/or physical health, especially related to safety. 16 Two validated self-efficacy tools that seek, in part, to address some of the medical problems seen in dementia are the Revised Scale of Caregiving Self-Efficacy (RSSE) 17 and the Self-Efficacy Questionnaire for Chinese Family Caregivers (SEQCFC). 18 The RSSE addresses disruptive behaviors in dementia that may pertain to medical conditions but does not address self-efficacy more broadly related to acute and chronic medical conditions those with ADRD may have. The SEQCFC (which has only been validated with Chinese caregivers) similarly addresses disruptive behaviors and includes a subscale that asks several self-efficacy questions related to medical symptoms, treatment, and health care. 18 What it does not do, however, is address in the larger context of confidence deciding what to do when a medical event arises, or to manage the many medical conditions that arise outside of disruptive behaviors. Having a measure that addresses this gap will better complement the tools available to assess and support caregivers of people with ADRD.
The Caregiver Confidence in Sign/Symptom Management (CCSM) scale was developed for exactly that purpose. Specifically, the aim of this project was to develop an instrument to assess caregivers’ confidence related to (1) knowledge about new or worsening medical events, (2) ability to decide what to do about medical events, and (3) ability to take care of medical events. This article reports on the development and psychometric properties of the CCSM.
Methods and Participants
Conceptually, the CCSM was largely informed by Albert Bandura’s work on self-efficacy instrument development, specifically his recommendations on content validity, minimizing response bias, conceptual analysis, and item analysis. 19 Initial item construction was informed by previous self-efficacy tools in both the dementia and the chronic disease management literature. 14 -18 Experts in geriatrics (a geriatrician, gerontological nurse, and gerontological social workers) determined the content for the specific items (the wording related to “confidence” and the medically related stem for each item), and they were discussed with an expert advisory board (composed of 2 gerontological nurses and an expert in family caregiving, all of who have expertise in measurement development).
Participants for psychometric testing of the CCSM were those participating in a research study that evaluated a web-based educational resource called the Alzheimer’s Medical Advisor (AlzMed for short). AlzMed includes more than 50 evidence-based and professionally developed guides for the assessment and management of common medical signs or symptoms pertinent for caregivers of people with ADRD. The goal of AlzMed is to help caregivers feel more confident in communicating with health-care professionals effectively, making initial management decisions, and (when appropriate) providing care at home.
Potential participants were identified though flyers, websites, and outreach by various organizations (eg, the Alzheimer’s Association), and an effort was made to solicit white/nonwhite, college/noncollege–educated, and spouse/nonspouse caregivers. To be eligible, caregivers had to be at least 21 years of age, the person most involved in care for their relative, and live with or visit their relative at least weekly. The care recipient had to have a diagnosis of ADRD, reside in the community, not be on hospice or in a comatose state, and not be expected to transition to nursing home, assisted living, or hospice care within 6 months (because the AlzMed project included 6 months of follow-up data collection). Telephone interviews were conducted by trained data collectors. Caregivers provided baseline data (used for the analyses in this article) and information about themselves and their relative; they then learned how to use AlzMed and participated in 2 follow-up interviews. They received US$100 for their involvement, and all procedures were approved by the institutional review board of the University of North Carolina at Chapel Hill. Further information on the details and results of this intervention are currently under review. 20
Description and Scoring of the Initial Instrument
The initial instrument included 2 sections. The first section asked if any of 30 specific medical events had been experienced in the recent months; the conditions included, for example, falls, pain with urination, and vomiting, as well as new or increasing agitation, depression, or confusion (because these signs can indicate an underlying medical problem). As noted above, the medical items themselves were selected based on the expert knowledge of the geriatric/dementia experts participating in this effort, informed by the literature on medical problems commonly experienced by older adults, including persons with dementia. With input from the advisory board, 11 confidence questions were included for each such event, tapping the domains of knowledge, ability to decide, and ability to act in relation to that event.
The second section aggregated medical events into 5 categories: dementia-related behaviors (eg, agitation), mood changes (eg, depression), mental status changes (eg, confusion), new medical problems (eg, falls, infections, fever), and ongoing chronic disease (eg, diabetes, hypertension) and asked 1 knowledge, decision, and action item for each of the 3 areas (ie, 15 items in total). The rationale to develop aggregate items was the anticipated infeasibility of asking numerous items for any and all individual conditions that occur, and the potential benefit of having summative items to provide a more global indicator of self-efficacy. The second section also included 11 items to assess general confidence understanding what to do and enacting generally responsive behaviors (not specific to a given medical condition) such as talking to a medical provider or taking vital signs.
In total, there were 37 items in the initial version of the measure (11 items of which were either not asked if no new or worsening medical conditions were encountered or were asked repeatedly for each new or worsening condition, in the first section of the measure). Response options were constructed on a 5-point Likert scale, ranging from 1 = not at all true/confident to 5 = extremely true/confident. As noted above, an expert panel provided input on the measure prior to testing the measure on family caregivers.
Psychometric Testing
Data collection included cognitive testing (n = 5, conducted February 2014), test–retest reliability (n = 17, conducted April 2014), and field testing including evaluation of validity (n = 194, conducted September 2014 to November 2016).
Participants for the cognitive testing and test–retest reliability were not included in the field testing and were recruited from the Duke Family Support Program. E-mails and notices in the newsletter were sent out to this Alzheimer’s disease support program, located in North Carolina, in order to recruit 17 caregivers who met the same inclusion criteria described above for participants involved in the field testing. Five of these individuals participated in the cognitive testing while all of the 17 participated in the test–retest reliability.
Cognitive testing
To assure construct validity, cognitive interviews were conducted by telephone with 5 caregivers. The cognitive interview was designed using the Question Appraisal System-99 technique. 21 This technique is a systematic approach containing several steps that include, but are not limited to, clarifying the measure’s instructions, assessing the measure’s assumptions and biases, and assessing adequacy or range response.
First, participants responded to the 37 interview items without assistance (ie, no clarification by the interviewer), and their unsolicited comments were recorded. Then, during the cognitive interview process, participants were asked to restate the question using their own words, explain how they arrived at their answer, and expound upon any difficulty they may have had answering the question. Also, respondents were asked to share questions they had or felt the instrument did not address and to elaborate on the comments they had provided spontaneously. The measure was modified based on the results of the cognitive test.
Test–retest reliability
Test–retest reliability was assessed by 17 caregivers who responded to the items by telephone interview. The entirety of the instrument was asked approximately 2 days apart, and reliability was determined using Pearson r and intraclass correlation coefficients (ICCs). Results of these analyses informed the version of the instrument that was field tested.
Field testing (internal reliability and validity)
The resulting items were administered to 194 caregivers upon their entry into the AlzMed project (ie, administration of the instrument constituted baseline data). An exploratory factor analysis was conducted requesting an oblique (promax) solution, on the assumption that factors would be correlated. In addition, convergent validity examined the association (Pearson r) between the caregiver’s self-report of medical training and the CCSM scale and subscales, with the expectation that having medical training would be related to more confidence in sign/symptom management. Concurrent validity of the CCSM scale and subscales was assessed by associations with 3 widely used valid and reliable caregiver measures: (1) the Zarit Caregiver Burden (short version), which includes subscales of personal strain (eg, “because of the time you spend with your [relative] you don’t have enough time for yourself”) and role strain (eg, “do you feel uncertain about what to do about your [relative]?” 22 ; (2) the Generalized Anxiety Disorder 7-item scale (a sample item being “worrying too much about different things”) 23 ; and (3) depression, measured using the 9-item Patient Health Questionnaire (a sample item being “feeling bad about yourself—or that you are a failure or have let yourself or your family down”). 24 It was expected that higher scores on the CCSM scale would be associated with lower scores on the burden, anxiety, and depression measures. All analyses were conducting using SPSS for Windows version 24.
Results
Cognitive Testing
The 5 individuals participating in the cognitive testing were predominately white (n = 4) and female (n = 5) and included spouses (n = 3) and daughters (n = 2). All lived with the care recipient, and the majority were retired (n = 3) and college graduates (n = 4); 1 had medical training. During cognitive testing, all 5 participants reported having no trouble in answering the questions, and they felt the measure did not have items that were difficult to answer or not relevant, nor response options that prevented accurate answers. They had no recommendations on additional questions to ask concerning their relative’s medical problems or about their own self-efficacy.
However, results from the first 4 interviews indicated that when some participants were asked to restate a question in their own words, the concepts of “deciding what to do” and “actually take care of” a medical problem were not discretely different to all participants—in essence, some felt confident taking care of matters in accordance with their confidence in being able to decide what to do. Consequently, we explicitly asked the fifth participant if she felt these were 2 different concepts, and we learned she did. At that point, we decided to retain them as distinct items and asked the same question of the 17 participants who participated in the interrater reliability testing (described in the next section). They were asked initially and 2 days later whether they considered confidence in deciding what to do and actually doing it to be distinct concepts. Twenty-seven unique answers were obtained, and on 20 occasions (74%) they felt these were different concepts. Additionally, correlations within participant’s answers for the overall questions related to knowledge about, deciding what to do, and actually taking care of medical problems and events, varied significantly (P < .05). Consequently, it was decided to continue to use these as separate concepts in the measure. Notably, the expert panel had strongly advocated for the distinction between confidence in deciding and enacting, which was supported by these results.
Test–Retest Reliability of the Condition-Specific Items
The 17 individuals participating in test–retest reliability testing were predominately white (n = 15) and female (n = 15) and included spouses (n = 14) and daughters (n = 3). Sixteen lived with the care recipient, and the majority were retired (n = 11) and college graduates (n = 15); 3 had medical training. Assessment of the reliability of all 37 items indicated concern about the reliability of the condition specific items (11 items in total); specifically, only 3 had a Pearson r > .40 with the total scale score, and their intraclass correlation values for test–retest reliability averaged .45, with 5 values being <.30. 25 Consequently, it was decided to omit these items from the field testing version for 3 key reasons: poor performance in reliability analyses, the overall length of the questionnaire (for each new or worsening event, the 11 items were repeated), and the general utility of the data. That is, in reality, condition-specific medical events often are not isolated (eg, falls and skin injuries often occur in conjunction, as can pain and urinary tract infections, as but a few examples) and self-confidence related to 1 event may vary depending on the other events that accompany it. The test–retest reliability of the items remaining in the CCSM is described in the next section.
Field Testing: Internal Reliability and Validity, and Test–Retest Reliability of the CCSM
Demographic characteristics of the 194 caregivers and their care recipients are shown in Table 1. The majority of caregivers were white (79%), female (83%), daughters (43%), with a college or graduate degree (53%), self-rated health as very good or excellent (59%), and an average age of 62. Of the care recipients, the majority were females (58%) with an average age of 79; they were in less than very good/excellent health (69%), and almost one-half were diagnosed with ADRD 5 or more years ago (48%).
Demographic Characteristics of Caregivers and Care Recipients.a
aN = 194.
The instrument that was administered to 194 caregivers included 26 items. Based on results of the factor analysis and guidance for measurement development, 18 1 item was omitted from the final measure because of its low item–total correlation (ie, ability to look out for your own health and safety). Consequently, the final CCSM scale included 25 items (α = .92); the items are listed in Table 2 and described below.
Caregiver Confidence in the Sign/Symptom Management (CCSM) Scale, Subscales, and Items.a
aN = 194.
bFor individual items, the statistic shown is the corrected item–total correlation.
Examination of the scree plot from the factor analysis indicated a 4-factor solution. In a few cases where an item had a strong secondary loading (>.35) on a factor that was a closer conceptual fit than the items in the factor with the primary loading, the item was assigned to the secondary factor. Using these criteria, the CCSM includes 4 distinct factors related to caregiver confidence: confidence related to knowing enough about signs/symptoms (5 items, related to behaviors, cognition, mood, chronic disease, and new medical problems; α = .83); confidence related to management of cognitive signs/symptoms (6 items, reflecting confidence in deciding what to do and ability to take care of signs/symptoms related to behaviors, cognition, and mood; α = .85); confidence related to management of medical signs/symptoms (4 items, reflecting confidence in deciding what to do and ability to take care of chronic disease and new medical problems, α = .87); and confidence related to general medical management/responsiveness (10 items, including, for example, confidence to make a decision about whether to contact a medical provider and to take vital signs; α = .85).
As shown in Table 3, the test–retest reliability of the overall CCSM was strong (r = .92, ICC = .91); in addition, all item–total correlations were ≥ .40, with an average of .64 (data not shown). Further, test–retest reliability of 3 of the 4 subscales was strong (management of cognitive signs/symptoms, management of medical signs/symptoms, and general medical management/responsiveness; r ≥ .78 for both r and ICC); the reliability of the knowledge subscale was more moderate (r = .57, ICC = .56).
Test–Retest Reliability of Caregiver Confidence in Sign/Symptom Management (CCSM) Overall Scale and Subscales.a
aN = 17.
Finally, the CSSM exhibited both convergent and concurrent validity (Table 4). Caregivers with more medical training had significantly more confidence in all areas than did those with less medical training (correlations ranged from .27 for general medical management/responsiveness to .15 for knowledge about signs/symptoms). In addition, caregivers with less role strain reported more confidence in all areas (correlations ranged from −.37 for general medical management/responsiveness to −.15 for knowledge about signs/symptoms).
Convergent and Concurrent Validity: Association of Caregiver Confidence in Sign/Symptoms Management (CCSM) With Related Concepts.a
aN = 194.
Discussion
The CCSM is a valid and reliable scale to assess the medical-related caregiving confidence of family members who oversee the health of a relative with dementia. It is a 25-item measure; each item is scored on a 5-point Likert scale, with a resulting final score ranging from 25 to 125 (Appendix A). If a shorter scale is desired, each of the 4 subscales (which include between 4 and 10 items) can be separately administered and scored. Considering that family caregivers provide care for longer than in the past and that the majority must attend to medical care needs, 9 a measure of caregiver confidence in sign/symptom management is highly beneficial to assess the need for caregiver support and evaluate the results of related interventions. Such interventions include, but are not limited to, self-efficacy training, psychoeducational approaches, and medical/nursing training for family caregivers. The CCSM can be used by clinicians to help assess the needs of caregivers and also to evaluate the results of related interventions.
It is informative to find that caregivers consider confidence related to management of cognitive signs/symptoms to be distinct from management of medical signs/symptoms (ie, they constitute 2 separate factors). Existing caregiver supports largely address dementia-related behaviors, mental status change, and mood change, and these are the domains typically associated with ADRD. To date, the medical needs of people with ADRD have had less attention, and so conceptualizing them differently is not surprising. In fact, this differentiation highlights the complexity of measurement in this area. This finding also supports self-efficacy theory, which states it is not a global measure but instead task/domain specific. 1,16,19
Most notably, several challenges exist in measuring the self-efficacy (confidence) among caregivers of people with dementia. One challenge is that overall measures of self-efficacy often do not discriminate the complexity of an individual’s true self-efficacy regarding a specific task. 19 For example, a single self-efficacy question asking about confidence in dealing with dementia-related medical problems is not specific enough to elucidate the probable wide range of self-efficacy an individual may feel regarding different events. Additionally, a person’s confidence is not always stable even in a short period of time, because self-efficacy can vary widely depending on recent circumstances and emotional state; for that reason, global items such as those included in the CCSM are desirable. 19 Further, self-efficacy is a dynamic construct and a person must consider multiple informational cues and experiences to form a judgment, 26 a point that supports inclusion of the numerous items in the CCSM to assess self-efficacy. In addition, valid and reliable self-efficacy constructs must be able to isolate perceived self-efficacy from outcome expectations, self-esteem, and locus of control, 19 consistent with the separation within the CCSM of knowledge, decision, and taking action regarding events.
Another significant challenge in measurement is that individuals may have difficulty assessing their self-efficacy related to medical problems or situations that do not happen often, as is the case, for example, with isolated acute medical events. As familiarity with a task or situation decreases, so does the ability to predict self-efficacy. 27,28 Of note, this point supports the removal of the condition-specific items in our initial testing. Another challenge is that individuals may think about their own self-efficacy in a different manner than the measure intended; for example, a measure may try to differentiate 2 concepts that are thought of as a single concept by a caregiver or may categorize items in a different way. These issues with conceptualization can be problematic when developing an appropriate measure 26 and were addressed through our use of cognitive interviewing during measurement development.
To put our findings in an even broader context, Crellin and colleagues recently developed a hypothesized model for caregiver self-efficacy in health-related quality of life based on their systematic review of the self-efficacy literature. 16 When determining positive or negative effects on caregiver physical and mental health, the main branching point in the model was the caregiver’s appraisal of stressors. The model hypothesized that caregivers’ appraisal of stressors is based on both caregiving stressors and their self-efficacy for controlling upsetting thoughts, responding to disruptive behaviors, and obtaining respite. 16 We suggest that self-efficacy regarding knowing, deciding, and responding to medical signs and symptoms plays an important role in caregiver self-efficacy in health-related quality of life not explicitly detailed in Crellin and colleague’s model. Future research using measures such as the CCSM are needed to test this hypothesis.
Strengths of the CCSM are the theoretical basis underlying its conceptualization, psychometric separation of 4 distinct areas of self-efficacy, test–retest reliability (although we acknowledge the limitation that it was based on only 17 respondents), and validity. Further, this instrument fills a gap in the literature, because other self-efficacy tools that are domain/task-specific for medical signs and symptoms in this population often focus predominately on disruptive behaviors related to ADRD. 16
Although the predictive validity of the CCSM has yet to be established, its generalizability is evident in that the field test sample included a large sample size of caregivers across the country whose demographics largely represent the range of caregivers of people with dementia. Some populations were underrepresented, and future research should especially include more persons without a college degree, who represent 60% of the caregiving population. 10
In conclusion, the CCSM is a robust measure that fills an important gap in understanding the confidence of caregivers to address medical needs of their relative with dementia. The 25 items are short and easy to understand and score, and the results provide guidance suggesting interventions to increase confidence and perhaps also health-related outcomes of people with dementia.
Footnotes
Appendix A
Acknowledgments
The authors thank Diane C. Calleson, PhD, for her guidance during this work, and Kimberly Ward for her expert data collection. The authors also acknowledge the input of our expert panel, which consisted of Drs Barbara Resnick, Ellie McConnell, and Carol Whitlatch. Finally, we would like to thank all caregivers that participated in this study.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by grant R01 NR014199 from the National Institute of Nursing Research.