Abstract
The Questionnaire on Palliative Care for Advanced Dementia (qPAD) is a 2-part instrument that measures long-term care staff knowledge, and beliefs, perceptions, and attitudes about palliative and end-of-life care for persons with advanced dementia. Factor analyses of the Knowledge Test (coefficient α = .81) produced 3 factors: Anticipating Needs, Preventing Negative Outcomes, and Insight and Intuition (coefficient α = .75, .73, and .58, respectively), explaining 67% of the total variance. Factor analyses of the Attitude Scale (coefficient α = .83) produced 3 factors: Job Satisfaction, Perceptions and Beliefs, and Work Setting Support of Families (coefficient α = .90, .64, and .67, respectively), explaining 68% of the total variance. These initial findings hold promise for an instrument that measures both knowledge and attitudes of long-term care staff in the care of persons with advanced dementia.
Keywords
In 2012, the Alzheimer’s Association estimates that approximately 5.4 million Americans have Alzheimer’s disease, a number projected to grow markedly in the near future. 1 Many of these individuals in the early stages of Alzheimer’s disease and other dementias live at home where more than 80% of their care is supplied by family or other unpaid caregivers. 1 However, persons in advanced stages typically live in institutional or long-term care settings, such as nursing homes, where paid staff provide most of their care. 2-3
The deficits and needs of persons with advanced dementia are vast. Individuals with advanced dementia often exhibit conditions such as apraxia, insomnia, restlessness, incontinence, eating difficulties, or confinement to bed, among others. 4 Furthermore, dementia is considered a terminal condition for which there is no cure at present. Because the medical model offers few benefits, persons with advanced dementia require comfort or palliative care. Comfort care is doing for persons what they would do for themselves if they could. A focus on comfort includes attention to knowing the person and executing care practices that support physical, psychological, social, and spiritual needs. These comfort principles incorporate strategies to ensure that pain is assessed and addressed, meaningful connections are maintained for all activities, and dementia-related behaviors are minimized. Comfort care for persons with advanced dementia also includes impeccable attention to care needs at the end-of-life. Thus, caregivers should possess the knowledge, beliefs, attitudes, and skill sets that support palliative and end-of-life care principles.
In long-term care settings, care is primarily delivered by staff with limited training in dementia-specific care. 2,5,6 In addition, staff may be exposed to highly stressful conditions due to the nature of the caregiving experience, long work hours, low pay, and challenging work. With limited training and stressful conditions, staff may develop poor attitudes and high staff turnover may result, which is problematic not only for long-term care staff but also for people with dementia. 1,7 To promote the overall well-being of staff as well as people with dementia, it is essential to study knowledge, attitudes, perceptions, and beliefs of staff in long-term care settings and ensure that persons with advanced dementia receive comfort care from staff who are competent. This research may reveal information about what caregiver attributes and skill sets are linked to positive workplace conditions, satisfied staff, and improved outcomes for persons with dementia.
In an effort to institute comfort-care practices, the Palliative Care for Advanced Dementia: A Model Teaching Unit (PCAD) program was launched in 2005 at Beatitudes Campus, Phoenix, Arizona. This grant-funded training program was designed to help long-term care staff learn about and adopt best practices in the care of persons with advanced dementia. 4 The 6-month training program consists of didactic and experiential education on-site in a 30-bed neighborhood called the Vermilion Cliffs located in the Health Care Center on Campus. Here the caregiving staff embrace comfort care practices and extend training to other long-term care organizations by providing peer training and instilling a sense of interdisciplinary teamwork within a person-directed framework. Direct care staff and management peers from guest long-term care organizations learn about fundamental caregiving principles related to advanced dementia care. Accordingly, determining the impact of the training program on residents, staff, and the facility has been a primary objective with program evaluation that addresses measurable outcomes over time. The PCAD program staff wanted to determine how best to ascertain change in knowledge, attitudes, perceptions, and beliefs acquired after a staff member from a guest institution attended the PCAD training program. Thus, the purpose of this article is to describe the processes undertaken to review existing instruments, the development of the Questionnaire on Palliative Care for Advanced Dementia (qPAD), and report findings that support a reliable and valid tool useful in long-term care settings, such as nursing homes.
Literature Review
Training should help all long-term care staff not only to understand the needs of people with dementia and deliver care effectively but also to improve the work experience. An integral part of such training is measurement of long-term care staff’s knowledge of advanced dementia care and assessment of their attitudes, beliefs and perceptions of their roles as care providers. Moreover, these measures must be appropriate for repeated use over the period of training and application as determined by their psychometric properties.
Existing knowledge tests of dementia care typically do not serve these purposes, largely because they measure general knowledge of dementia care. For example, the revised Alzheimer’s Disease Knowledge Scale reflects contemporary knowledge of dementia, and though useful in evaluations of general knowledge of dementia, it does not measure knowledge of advanced dementia care. 8 Both the knowledge test for general practitioners 9 and the test for family caregivers 10 measure general knowledge of dementia specialized for those audiences.
The Knowledge of Alzheimer’s Test (KAT), developed at the University of Iowa College of Nursing Research Center, assesses the knowledge about the etiologic pathology of Alzheimer’s disease, its signs and symptoms as well as the treatment and care for persons with the disease. 11 The KAT has been used in different studies in which the knowledge of certified nursing assistants was assessed. Reported measures of reliability and validity findings vary. In 3 studies the reported reliability of the original KAT was .80. 11 -13 In 1 study, reported alphas ranged from .90 to .96. 12 Content validity was established by a literature review and a panel of gerontological nurses expert in working with Alzheimer’s disease patients. 11,13
Other dementia care knowledge tests include a 15-item multiple choice test developed and used by Hobday and others, which demonstrated reliability at pretest (Cronbach’s α = .94). 5 The Advanced Nursing Assistant Knowledge Test, a 12-item knowledge test for advanced nursing assistant training, showed appropriate validity, as measured by INFIT-OUTFIT statistics in the acceptable range of .6 to 1.4 and a high reliability coefficient (KR-20 = .88). This test measured areas such as recognizing acute medical problems in older adults with dementia and managing dementia-related behaviors. 2 Hughes and colleagues reported no psychometric information for a 12-item knowledge test of dementia care practice. 3
In addition to the knowledge of dementia care, assessment of staff beliefs and their perceptions of staff roles can provide insight into job performance. As examples, do long-term care staff believe they can have an impact in end-of-life care discussions? How do staff perceive their collaboration with other team members and families or decision makers? How satisfied are staff with their job performance? 4 Answers to these questions can signal how long-term care staff view their work experience. 14
Three reports were located that involved instruments for end-of-life decision making. Stoeckle and associates developed and tested the End-of-Life Care Decision Questionnaire (EOLCDQ). Content validity was assessed by a panel of experts who were in total agreement about the clarity and appropriateness of the items. No information was available about reliability of the EOLCDQ. 14 The same instrument was modified as EOLCDQ II and found to have a Cronbach alpha of .82. Used in a second study with a different sample, EOLCDQ II had a coefficient of .81. 15,16 Lacey’s end-of-life decision-making questionnaire for nursing home social services staff was evaluated as having face and content validity but no statistics are reported. 17 Questions included items on advance directive responsibilities, perceptions of end-of-life issues, and palliative care principles.
Several studies reported using measures of job satisfaction among long-term care staff in nursing homes or other long-term health care organizations. An English translation of the Swedish Satisfaction with Nursing Care and Work Assessment Scale provided reliable data (Cronbach α = .86) for Brodaty and his colleagues about nursing staff work strain and satisfaction, quality of care, attitudes, and knowledge of people with dementia. 7
Coogle and others, who used the Minnesota Satisfaction Questionnaire, reported appropriate reliability (Cronbach’s α for overall scale = .90) and validity (no indicators provided) for measuring participants’ perceptions of pay, work environment, and management style in dementia care settings. 18 In a study with certified nurse assistants, 12 both the General Job Satisfaction (GJS) Scale and the Grau Satisfaction Scale were used to identify job satisfaction in a pre- and post-dementia training program. Whereas the GJS Scale measures job satisfaction generally, the Grau Satisfaction Scale measures intrinsic job satisfaction. The GJS Scale was construct validated (construct unspecified) and showed low Cronbach’s alpha values of .45 to .58 over 3 time points. The Grau Satisfaction Scale has validity scores ranging between 0.84 and 0.85; its coefficient alphas range from .81 to .84.
As this review shows, several instruments can be used for measuring the knowledge and affective behaviors of staff caring for people with dementia. As yet, however, none merges these domains within a single instrument with acceptable validity and reliability measurements. Therefore, the purpose of this study was two-fold: (1) to create an instrument for measurement of knowledge and attitudes of long-term care staff for people with advanced dementia and (2) to determine its psychometric properties based on a sample of staff members with whom it might be used.
Methods
Instrument Development
Background
Because evaluation of the impact of training is an important part of PCAD, program developers determined that staff knowledge of the care of persons with dementia as well as staff attitudes, beliefs, and perceptions of caregiving acquired as a result of a training program was important. A thorough literature search failed to reveal instruments that could serve these purposes directly. Hence, the need arose to develop an instrument that would capture these elements considered essential in the care of persons with advanced dementia.
A literature search located two instruments that lent themselves to modification for use in PCAD. One was a knowledge test of Alzheimer’s disease and the other, an attitude scale for caregivers.
Initial development
The qPAD was developed by clinical nurse specialists, a psychiatric nurse practitioner, 2 nurses with doctorate degrees and active in dementia research, and a behavioral health practitioner, all of whom were very familiar with advanced dementia care and best practices. This group started with the 20-item KAT developed at the University of Iowa College of Nursing Research Center. 11 This instrument had been tested with nursing home staff and results showed a reliability coefficient of .80. With permission from the KAT developers, the current research team expanded the test to include 50 items that reflected care for persons with advanced dementia. These items assessed staff knowledge and beliefs along with their satisfaction and participation in care. In the end, modifications consisted chiefly of incorporating key concepts related to palliative care and program-specific principles of advanced dementia care. However, in a pilot test of this revised instrument known as the Modified Knowledge of Alzheimer’s Test or mKAT, 12 caregiving staff took 1 hour to answer the items.
Because this time requirement was unreasonable, the researchers reduced the number of items to 25 carefully selected forced-choice, true–false knowledge questions. These items focused on general caregiving principles, activities, nutrition, mobility, weight loss, toileting, bathing, depression, pain, and behavioral approaches to care. These items became the Knowledge Test portion of the qPAD.
In addition, the researchers realized a need to inquire about long-term care staff’s attitudes, perceptions, and beliefs about dementia, palliative care, and end-of-life issues. They developed an attitude scale by modifying the EOLCDQ II originally created to evaluate health care professionals’ perceptions of the end-of-life process and their beliefs and practices. 14 A Cronbach’s alpha of .82 suggests that the EOLCDQ II demonstrated adequate reliability. Although the scale itself contains no questions specific to palliative care or dementia, several items approximated the content needed in qPAD. With permission from its first author, researchers selected 6 items from the EOLCDQ II and created 6 additional items for the qPAD Attitude Scale. These 12 Likert-type questions focus on end-of-life care, beliefs about staff practice/approaches to care, and teamwork. Responses are registered on a scale of 1 (Strongly agree) to 5 (Strongly disagree).
Initial testing of qPAD
The 2-part qPAD, originally developed in 2006, was administered to a multidisciplinary team of long-term care staff at 3 nursing homes with dementia care special units and 1 assisted living organization. These 86 staff included licensed nursing personnel, social workers, dietary professionals, activity professionals, administration, caregivers, and certified nursing assistants. Each participant was provided with a description of the study and a consent form prior to completing a demographic questionnaire and the qPAD. Anonymity was guaranteed as a unique identifier known only to the research team was used to match pre- and post-training results, deemed necessary for future studies, and only aggregate data were reported. All procedures had been approved by an institutional review board in advance of the study to assure the protection of human subjects.
Although qPAD had suitable indices of construct validity and reliability, the researchers determined that further development of the Knowledge Test was necessary. The Knowledge Test was revised and expanded. 4 Additional items encompassed physical restraint use, the onset of delirium, cause of death, changes in sleep patterns, and resisting care. A third response option of “don’t know” was added to the true/false options. This revised Knowledge Test along with the original qPAD Attitude Scale became qPAD Version 2. Finally, qPAD Version 2 requires approximately 20 minutes for completion. Anecdotal information from caregivers indicated they experienced relative ease in completing the instrument in terms of time.
Evaluation of Instrument Psychometric Properties
Participants
A convenience sample of 85 caregiving staff from 4 nursing homes participated in the evaluation of qPAD Version 2. Staff members had been employed an average of 61.8 months and had an average of 107 months or 9 years’ experience in long-term care. Most staff (67.5%) worked the 7-
Staff Characteristics (n = 85)
Abbreviations: SD, standard deviation
aFour respondents who indicated “Other” as their job description listed the following: Nursing Director, Assistant Nursing Director, and Unit Clerk.
All participants were in the process of enrolling in the PCAD education program and research study. These individuals, and their respective long-term care organizations, demonstrated significant interest in ascertaining knowledge, beliefs, attitudes, and perceptions of caregiving staff involved in dementia care services. 4
Administration of instruments
Participants were informed of the study with written materials and at an oral presentation by the project principal investigator. Staff gave their consent to participate and then voluntarily completed the instrument. Information and testing sessions occurred in the long-term care organization’s education or conference room setting and were held at different times during the day to accommodate all shifts.
Procedures for determining instrument psychometric properties
Data generated by the administration of the qPAD were subjected to statistical analyses to determine the reliability and validity of the instrument. Item difficulty and item discrimination procedures were calculated for the Knowledge Test. Also, internal consistency reliability for the Knowledge Test and the Attitude Scale was determined by examining interitem correlations, item-total correlations, and by computing a coefficient alpha. A principal components analysis provided information about the underlying dimensions of both the Knowledge Test and Attitude Scale.
Results
Table 2 provides item discrimination values, item difficulty, item means, and factor loadings for 23 knowledge test items. Discrimination was determined by using the point-biserial correlation index 19 which provides an estimation of the contribution of an individual item to the overall test score. 20 A high point-biserial correlation coefficient suggests that persons who answer items correctly obtain high scores and those who answer incorrectly receive low scores. Each item retained in the test had a point-biserial correlation of .24 or higher. Two items with poor discrimination values (less than .10) were dropped from the test.
Analysis of qPAD Knowledge Test (N = 85)
Abbreviation: qPAD, Questionnaire on Palliative Care for Advanced Dementia.
Knowledge Test items were scored as 1 (correct) or 0 (incorrect). Therefore, item difficulty is represented by mean values. Low mean values represent difficult items and high mean values represent easy items. 19 In this test, means ranged from .28 to .70, indicating that no item was either exceptionally difficult or exceptionally easy. Knowledge Test items are shown in Table 2 nested within the factor with which they are associated.
Means and factor loadings from the Attitude Scale are presented in Table 3. Possible responses for the Attitude Scale ranged from a value of 1 (Strongly disagree) to 5 (Strongly agree). A high mean value represents a more positive response than a low mean value. In this situation, item means ranged from 3.929 (item 12) to 3.060 (item 3). The Attitude Scale items are shown in Table 3 nested within the factors with which they are related.
Analysis of qPAD Attitude Scale (N = 85, except where noted)
Abbreviation: qPAD, Questionnaire on Palliative Care for Advanced Dementia.
a N= 83.
Reliability and validity
Establishing reliability and validity for the qPAD required gathering evidence from several sources. One source is evidence of score consistency commonly called reliability, which is the degree to which scores obtained with these instruments are free from unexplainable random error. Coefficient alpha, which measures the internal consistency of test items, produces scores ranging from 0 to 1.0. Values for both the Knowledge Test (coefficient α = .81) and the Attitude Scale (coefficient α = .83) are considered moderately strong and quite acceptable for measurements of this type. 20
A second source of evidence, needed to establish construct validity, employed a principal components factor analysis for each of the 2 parts of the qPAD. The principal components model was selected because the researchers believed that the knowledge and attitude portions of the qPAD would reflect clearly definable elements or components.
The analysis for the Knowledge Test employed an oblique rotation model due to the high level of multiple correlations among items. Review of this analysis provided evidence of the 3 principal factors shown in Table 2. These factors were defined and subsequently named (1) Anticipating Needs (coefficient α = .75), (2) Preventing Negative Outcomes (coefficient α = .73), and (3) Insight and Intuition (coefficient α = .58). The 3 factors (correlational loadings after rotation) account for slightly more than 67% of the total variance and are consistent with the underlying theoretical components of the instrument.
Attitude Scale items were also analyzed using a principal components factor analysis but with an orthogonal rotation because there was no indication of high interitem correlations. Review of these results provided evidence of the 3 principal factors shown in Table 3 defined as (1) Job Satisfaction (coefficient α = .90), (2) Perceptions and Beliefs (coefficient α = .64), and (3) Work Setting Support of Families (coefficient α = .67). Together these 3 factors (correlational loadings after rotation) accounted for 68% of the variance and provide supportive evidence for construct validity of the Attitude Scale. Evidence gathered in these processes provides preliminary support for the construct validity of the qPAD. It is acknowledged that this is the initial stage of instrument validation and that further evidence should be collected.
Discussion
This study provides preliminary evidence for the reliability and validity of the qPAD, a 2-component instrument for professional staff working in long-term care organizations, such as nursing homes, who care for persons with advanced dementia. With the elimination of 2 items, a 23-item Knowledge Test assesses caregivers’ knowledge about advanced dementia care and a 12-item Attitude Scale measures caregivers’ attitudes toward dementia, palliative care, and end-of-life issues.
It is very important to note that the 2 concepts of reliability and validity are, by definition, highly related to one another. According to Urbina, reliability may often be seen as an initial source of evidence that a valid measure has been attained. Reliability should never be seen as a sufficiency for validity, only a necessity. 20 The reliability coefficient alphas of .81 for the knowledge portion and .83 for the attitude portion provide initial support of validity evidence.
The factor analyses that were performed on the 2 measures within the qPAD were utilized in order to verify appropriate constructs upon which inferences could be drawn. 20 The knowledge items as well as the attitude items were developed by experts within the field. It was with this understanding that the 3 factors identified for each of the 2 measures were felt to provide supportive evidence of construct validity.
Through the use of intuitive logic, the factors were labeled by determining distinguishing and common features of the items or attitudes based upon the factor loadings. 20,21 It was the goal of these factor analyses to reduce a large number of observations or items into a smaller number of factors that would aid in defining the domain being measured. 21
Data from convenience samples totaling 85 long-term care staff suggest that both components provide reliable and valid measures. However, the researchers realize that the instrument should be used with greater numbers of health caregivers before definitive statements can be made.
Use of this instrument in its current form is intended for long-term care staff in the United States. Its use by other audiences, such as family caregivers or health caregivers in non-English speaking countries, would require changes.
The implications of using this tool are promising. The Knowledge Test results identifies not only the acquisition of knowledge as an important construct, but also ways that staff can minimize the challenges that persons with dementia face as they lose independence over time. As long-term care staff acquire knowledge to promote positive care and minimize negative outcomes, they develop insight and a depth of understanding about the dementia and care management strategies that support comfort.
The Attitude Scale findings identify the importance of job satisfaction, staff beliefs and perceptions, and the inclusion of families within the dementia work setting. Job satisfaction includes elements of teamwork, individual self-worth, and respect. Positive beliefs and care practices are necessary in the care of persons with advanced dementia. Augmenting family support and interdisciplinary team collaboration are important considerations in a palliative care model.
The prospects of measuring a change in knowledge and attitudes at pre- and post-training are important in promoting and validating best practices in the care of persons with dementia. Pre- and post-test measurement will also help in determining how best to support long-term care staff in acquiring dementia care skills and traits. Long-term prospects for qPAD include the opportunity to examine staff turnover, job satisfaction, and the sustainability of knowledge and changed or improved attitudes, perceptions, and beliefs over time.
This project was not designed to answer questions about the extent to which assessment of caregivers’ beliefs and perceptions of caregiving roles actually provide insights into their job performance or workplace conditions. As intriguing as answers might be, they must be determined in future research projects.
Footnotes
Acknowledgments
The authors express gratitude to Jan Dougherty, MS, RN; Bradley Fulton, PhD; Maribeth Gallagher, DNP, Psych NP-BC; and Geri Hall, PhD, ARNP, GCNS, FAAN, for their contributions to the initial development of qPAD.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported, in part, by a grant from the BHHS Legacy Foundation, Phoenix, Arizona.