Abstract
Introduction
Measuring health activation in general population using valid instruments is needed to facilitate the evaluation of health education and behavioral programs in community. The 13-item Patient Activation Measure was well validated in patients with different chronic diseases but rarely validated in general population. The objective of this study was to assess the psychometric properties of the Patient Activation Measure among community-dwelling adults in Singapore.
Methods
Data of participants having valid responses to the English-version measure (N = 824) were analyzed. The psychometric properties were assessed by demonstrating evidence for uni-dimensionality using Rasch Principal Component Analysis of Residuals, known-group validity, convergent and divergent validity, and internal consistency reliability using Cronbach’s alpha.
Results
The uni-dimensionality of the Patient Activation Measure was supported by the Rasch Principal Component Analysis of Residuals results. Participants having multimorbidity or polypharmacy and being inactive in physical activity had significantly lower activation scores. The activation score was positively and moderately correlated with health confidence measured by the Health Confidence Measure (r = .38, P < .001), and negatively and weakly correlated with depressive symptoms measured by the Patient Health Questionnaire (r = − .13, P < .001). The internal reliability was good with a Cronbach’s alpha of .82.
Conclusion
The 13-item Patient Activation Measure has acceptable construct validity and good internal consistency among community-dwelling adults. It is a potential instrument to measure health activation in this population. Further research is required to investigate the expansion of response options, validate the cut-off scores for the activation levels and examine the test-retest reliability and responsiveness.
Keywords
Highlights
• In recent years, healthcare consumers are increasingly recognized as critical partners in health services planning, delivery, and evaluation; and are increasingly required to play a more active role in their own care. • The 13-item Patient Activation Measure is the most widely used measure for activation which has been validated in many countries among various disease population.
• The findings of our research provide additional evidence for the validity and reliability of Patient Activation Measure from Asian perspective. • Different response patterns were observed in community-dwelling adult population compared to patients with specific or mixed chronic conditions as reported in other Patient Activation Measure validation studies. • Patient Activation Measure had good structural and construct validity and acceptable internal consistency reliability in community-dwelling adult population.
• Patient Activation Measure can be used among community-dwelling adult population. However, further research is required to refine the instruction for administration, investigate the expansion of response options, validate the cut-off scores for the activation levels, and examine the test-retest reliability and responsiveness in community-dwelling adult population.
Introduction
With the ageing population and changes in societal behaviors, chronic diseases are on a steady rise and are the leading contributors to death and disability globally. 1 The prevalence of chronic diseases greatly exceeds the capacity of health system to care for patients. To combat this growing health crisis in a sustainable way, healthcare systems around the world have recognized the importance of activating and empowering individuals to take active roles in their own health and health care, and multi-level strategies have been taken to prevent and improve outcomes of chronic diseases.2,3
Consumer or patient activation refers to how well an individual understands one’s own role in their health management and how competent they are to fulfill that role.4,5 Numerous studies examining the relationship between an individual’s level of activation and engagement in health management have consistently reported that highly activated individuals were more likely to engage in a variety of healthy behaviors such as regular physical exercise, healthy diet, maintaining recommended weight, and managing stress6-8; monitor health conditions; and adhere to treatment.
Several instruments have been developed to measure patient activation or engagement. The 13-item Patient Activation Measure (PAM-13), which was developed to assess an individual’s self-reported knowledge, skills and confidence to manage one’s own health and healthcare needs,9,10 has become the most widely used generic measure for patient activation irrespective of the underlying health condition. The PAM-13 had shown good psychometric properties in the English-speaking general 10 or specific disease-specific population.11,12 It has been translated into many languages and validated in disease-specific outpatient and inpatient care settings11,13-18 with mixed but generally good psychometric properties. However, its applicability to community-dwelling adults with diverse health status is not well established.
In Singapore, multiple health education and behavioral programs are taking place in community to improve community-dwelling adults’ health behaviors and capability of health management. Measuring activation of this population not only allows for designing tailored programs for individuals at different activation levels, but also for monitoring their changes in activation as one component of short-term outcomes of the programs. As such, validating the PAM-13 in this population becomes necessary. As a multiracial and multicultural country with prevalent multi-language literacy (74.3% in 2020), English is the most predominant language in Singapore. 19 The objective of this study was to assess the psychometric properties of the English version PAM-13 among community-dwelling adults in Singapore.
Methods
Study Design and Participants
Data of this cross-sectional study was collected face-to-face by trained interviewers during a follow-up survey of the Population Health Index (PHI) study. Details of the PHI study has been previously published.20,21 Briefly, PHI participants were aged 21 years old and above and residing in the randomly selected households at Central region of Singapore for at least 6 months in the past year when recruited. A total of 824 participants who 1) declared without any pre-existing intellectual disability, cognitive impairment, or psychiatric diseases; 2) completed the English version of the PAM-13 independently during the second follow-up survey; and 3) had valid responses to the PAM-13 (“Not Applicable” response to less than three items, and at least one non “Agree” response to any of the 13 items) were included in the analysis.
To ensure the sufficiency of sample size for the study, we calculated the required sample size based on the main analyses being conducted. Firstly, when using Rasch analysis to examine item characteristics of rating scale tools, a minimum of 250 subjects were recommended for polytomous items to ensure the stability and robustness of Rasch analysis. 22 Secondly, a power analysis for simple correlation was conducted. To detect a correlation of −.16 between PAM-13 and the Patient Health Questionnaire (PHQ-9) 11 using a two-sided test at 5% significance level (α = .05) with power of 80% (β = .2), the required sample size would be 304. As such, the sample size of 824 participants was sufficient.
The PHI study was approved by the ethics review committee of the National Healthcare Group Domain Specific Review Board (Reference Number: 2015/00269). Written informed consent was obtained prior to the study initiation from all individual participants after they were fully informed of the study objectives and procedures.
Measures
Participant characteristics
Self-reported demographics used for this analysis included age, gender (male or female), ethnicity (Chinese, Malay, Indian, or others), highest education level, employment status, and self-perceived money sufficiency for essential daily living. A participant was considered to have multimorbidity if he/she had two or more self-reported chronic diseases. Participants were also asked to report the number of prescribed medications taken regularly for chronic diseases. The use of three or more medications daily was considered as polypharmacy.
Patient Activation Measure −13
The PAM-13 is a unidimensional, Guttmann-like measure that has 13 items measuring self-assessed knowledge, skills and confidence for self-management of health. 10 The items are ordered by level of difficulty of activation. Each item has four response options: 1=strongly disagree, 2=disagree, 3=agree, 4=strongly agree, and additional “Not Applicable (NA)” option. Raw total PAM scores (derived by [raw score]/[number of items answered excepting non-applicable items] * 13) can be transformed to a scale (PAM score) with a theoretical range of 0-100 based on calibration tables, with higher scores indicating higher activation. The scaled score can be converted into four activation levels using the pre-defined cut-offs. 16 Individuals at Level 1 tended to be more passive recipients of their own healthcare; Level 2 had the confidence and knowledge to take actions; Level 3 were taking actions to maintain or improve health; and Level 4 were maintaining guideline health behaviors under stress. In this study, each item score was entered into the PAM-13 scoring spreadsheet 2017 provided by the developer to derive the PAM score and activation level for each participant.
Patient Health Questionnaire
The PHQ-9 is a short, well-validated screening tool for symptoms of depression using 9 items. It captures self-reported response to each item based on past two weeks condition using a 4-point scale (0 = not at all, 1 = several days, 2 = more than half the days, 3 = nearly every day). The total score is calculated by summing up each item score and ranges from 0 to 27 with higher scores indicating more severe depressive symptoms. PHQ-9 had been validated among primary care patients in Singapore with good internal consistency reliability (Cronbach’s alpha .87). 23
Physical activity participation and self-perceived health status
A single item phased “How often do you take part in regular fitness program?” was used to measure the physical activity participation with five response options: “very often”, “often”, “once in a while”, “almost never”, and “never”. Participants were categorized into two groups based on their response to the question: physically active group if they reported “very often” or “often”, and physically inactive group if they reported “once in a while”, “almost never” or “never”. Self-perceived health status was determined based on the response to a single question phased “In comparison with other people of the same age, how do you consider your health status?” with four response options: “not as good”, “do not know”, “as good”, and “better”. The first two options were grouped together as “not as good/do not know” due to the small number of participants.
Health Confidence Measure
The Health Confidence Measure is considered as a simple and effective proxy for health engagement. 24 It measures a person’s health engagement using a single question: “How confident are you that you can control and manage most of your health problems?”. Individuals were asked to rate their health confidence on a scale from 0 (not very confident) to 10 (very confident). A cut-off score of 7 was used to categorized individuals into two groups: high health confidence (scored 7-10), and low health confidence (scored 0-6).
Statistical Analysis
Descriptive statistics were employed to summarize participants’ characteristics with continuous variables being described by the mean and standard deviation (SD), and categorical variables being described using the frequency and percentage.
Item-level descriptive statistics including responses distribution (including “NA” responses) and floor and ceiling effects of individual PAM-13 items were reported. Floor and ceiling effects were considered present if more than 15% of the participants achieved the lowest (disagree strongly) or highest (agree strongly) possible score. 25
The construct validity of the PAM-13 was examined from three aspects: dimensionality and Rasch item fit, known-group validity, and convergent and divergent validity. The internal consistency reliability was examined in this study.
Uni-Dimensionality and Rasch Item Fit
Rating Scale Model (RSM) 26 was used to estimate the primary measurement dimension for polytomous data from the Likert response format since the PAM-13 is anticipated to be unidimensional. Rating scale model requires each item to have the same number of response options and have one set of threshold values for all the items in the scale. As “disagree strongly” option was not selected for 6 out of 13 items and rarely selected for the rest of the items, it was combined with “disagree” option for the estimation of the RSM. As such, the initial five responses for each PAM-13 item were recoded as follows: 0=disagree strongly or disagree, 1=agree, 2=agree strongly, and NA=not applicable.
Analysis of uni-dimensionality of the PAM-13 was a two-step process. Firstly, the measurement dimension of the scale was estimated using the RSM. Secondly, a principal component analysis of residuals (PCAR) was used to determine whether substantial subdimensions existed within the items after fitting the RSM. 27 This analysis involves computing the standardized residuals, that is, (observed responses- expected values)/(model standard error). If the items measured a single latent dimension as estimated by the Rasch model, the remaining residual variance should reflect random variation. In operation, uni-dimensionality was evaluated based on four criteria: 1) at least 50% of the total variance should be explained by the measurement dimension 28 ; 2) the variance explained by the first principal component of the residuals be no more than 15% 29 ; 3) A minimum ratio of 3:1 for the variance in the measurement dimension compared to the variance of the first principal component of residuals 30 ; and 4) the Eigenvalue of the first contrast 31 should be less than 2.0.
Rasch item fit statistics were used to examine how accurately the data fit the Rasch measurement model, i.e. how well item difficulty or an individual’s ability contributes to the underlying construct of the test. 32 Infit statistics including Infit mean-square (MNSQ) and Infit value of t-test (ZSTD) and Outfit statistics including Outfit MNSQ and Outfit ZSTD were reported. The reasonable Infit and Outfit MNSQ range was .6 - 1.4 for a rating scale 33 and the reasonable Infit and Outfit ZSTD range 34 was between −2 and 2. Items not fitting the Rasch model implied a lack of uni-dimensionality which could be either due to misunderstanding of the item or that it was measuring another construct.
Known-Group Validity
Known-group validity was conducted to evaluate the ability of the PAM-13 to discriminate among known distinct groups. The PAM scores and the activation levels were compared between subgroups of multimorbidity, polypharmacy, and physical activity using independent samples t-tests and Chi-squared tests, respectively. The difference in PAM scores and activation levels across self-perceived health status groups was evaluated using the one-way ANOVA test and Chi-squared test, respectively. We hypothesized that individuals without multimorbidity or polypharmacy, being more active in physical activity, and having better self-perceived health would have higher PAM scores and higher activation levels compared to their respective counterparts.
Convergent and Divergent Validity
The convergent validity of the PAM-13 was measured by assessing the association between PAM score and health confidence score measured by Health Confidence Measure using Pearson’s correlation coefficient (r) and association between activation level and health confidence level using Chi-squared test. We conjectured that there was at least moderate (r > .30) and positive relationship between PAM score and health confidence score and participants with higher activation levels were more likely to have higher health confidence. The divergent validity of the PAM-13 was measured by assessing the association between PAM score and PHQ-9 depressive symptom score using Pearson’s correlation coefficient. We hypothesized that there was little or weak correlation (|r| < .30) between these two. We expected that the correlation coefficient of PAM score and health confidence score was higher than that of PAM score and depressive symptom score.
Reliability
Internal consistency reliability was measured using Cronbach’s alpha, average inter-item correlation and item-to-rest correlation. An alpha of .70 or higher was considered as the acceptable value. 35 Average inter-item correlation was derived by taking the average of all correlation coefficients for each pair of items on a test that measured the same construct. The ideal range of average inter-item correlation 36 was .15 to .50. Item-rest correlation was to evaluate the correlation between an item and the scale formed by all other items. A correlation of .3 or higher indicated a moderate correlation while a correlation of .5 or higher indicated a strong correlation. 37
The RSM and PCAR were conducted using the statistical program R with the extended Rasch model package (eRM) and Rasch Model Parameters by Pairwise Algorithm package (pairwise). All the other analyses were performed using Stata 16.0 for Windows. The result was considered significant if a P value was <.05 (one-tailed tests for independent t test and one-way ANOVA).
Results
Participant Characteristics
Participant characteristics and PAM scores (N = 824).
Item-Level Descriptive Statistics
Distribution of responses for the items in the PAM-13, n (%).
Floor and ceiling effects, and item level correlations.
Uni-Dimensionality and Rasch Item Fit
Assessing uni-dimensionality of the PAM-13 using the PCAR analysis showed that the variance explained by the measure was 63.3% (above 50% of the total variance), and the variance explained by the first principal component of residuals was 14.2% (slightly lower than 15%), forming a ratio of 4.5:1, which met the 3:1 criterion for uni-dimensionality. The eigenvalue of the first contrast was 1.85 (lower than 2.0). Overall, the uni-dimensionality of the PAM-13 was supported by the data.
Item difficulty, standard error and infit and outfit statistics.
Note. MNSQ: mean square (values between .60 and 1.40 are within acceptable limits for the Rasch model). ZSTD: value of t-test (values between 2 and 2 are within acceptable limits for the Rasch model).
Based on the difficulty parameters (Table 4), the ranking of items derived from this cohort was different as compared to the ranking of items in the original PAM-13. Participants in this study found it easier to agree to PAM_7, PAM_10 and PAM_13. On the other hand, items including PAM_3, PAM_6, PAM_8 and PAM_9 were harder to be endorsed by the participants. The item location parameter ranged from .65 for PAM_1, which was the easiest item, to 4.09 for PAM_12, the most difficult one.
Known-Group Validity
Associations between PAM-13 and other variables or measures.
aChi-squared test.
bIndependent sample t-test.
cOne-way ANOVA.
dPearson’s correlation test.
Convergent and Divergent Validity
PAM score was positively associated with health confidence (r = .38) as shown in Table 5. Individuals with high health confidence level had significantly higher PAM scores compared to those with low health confidence level (mean score: 67.1 and 58.7, respectively). However, the association between PAM score and PHQ-9 depressive symptom score was low (r = − .13) although higher activation levels corresponded to lower depressive symptom scores. The correlation between PAM score and health confidence score (r = .38) was higher than that between PAM score and depressive symptom score (r = −.13), providing the evidence of good convergent and divergent validity.
Internal Consistency Reliability
Cronbach’s alpha for the PAM-13 was .82 and item-rest correlations ranged from moderate (items PAM_1 - PAM_10, and PAM_13) to strong (items PAM_11 and PAM_12). The average inter-item correlation was .26 with individual inter-item correlations ranging from .24 to .27 (Table 2), well within the ideal range of .15 to .50, indicating strong internal consistency reliability.
Discussion
This study examined the psychometric properties of the English-version PAM-13 in terms of construct validity (including uni-dimensionality, known-group validity, and convergent and divergent validity) and internal consistency reliability among community-dwelling adult population in Singapore.
Findings from the item-level descriptive analysis showed that “disagree strongly” and “disagree” were the options least selected by this population. The ceiling effect existed in all 13 items and was the highest for the first two items and the lowest for item PAM_12. This pattern was consistent with the study conducted among cardiac patients in outpatient clinics 11 although the ceiling effect for each item was more prevalent in our cohort. While high ceiling effect was expected for PAM_1 and PAM_2 as they are the easiest items in the measure, the high ceiling effect for the rest of the items suggests that the scaling of the PAM-13 may not be adequate to capture the appropriate responses for this population. This might also suggest that the PAM-13 with existing rating scale may have low discriminating power or be less responsive to changes when being used among a relatively healthy population. Increasing the response categories towards the high end may allow participants to provide a more accurate response to what they truly feel and reduce ceiling effects. 38
In this study, the “NA” option was mostly selected for items PAM_4 (use of prescribed medications), PAM_9 (available treatment options), and PAM_8 (knowledge on cause of health problems), which was different from the response pattern reported in other studies in patients with either specific (eg, cardiac conditions or mental problems) or mixed chronic diseases.11,16,39 This could be partially explained by the difference in health profile of the participants as these three items might be less applicable to very healthy individuals, especially those without any health problems. The relatively higher responses of “NA” option among community-dwelling adults brings up the necessity of rephasing the instructions of the measure while using it in a population with diverse health states to make it clearer that the health problems stated in the items describe any health problems that are not specific to any chronic conditions.
The uni-dimensionality of the PAM-13 was generally supported by the data from our study, however, the high Infit and Outfit ZSTD for seven out of 12 items suggested for further refinement of these items. Furthermore, as what have been found in a cognitive interviewing study conducted among cardiac patients in Singapore, 38 some of the terms/phases such as “when all is said and done” in item PAM_1 or the sentence structures such as “tell doctor concerns I have” in item PAM_6 are not commonly used by local population, and the interpretations of some wording such as “active role” in item PAM_2 and “medical treatments” in item PAM_7 varied due to ambiguity. This is probably due to the differences in health beliefs and needs or practices of self-management of health linked to the local cultural background. 38 Another issue highlighted by the trained interviewers was that participants with multiple conditions in our study were more inclined to ask questions like “Which condition should the answer be based on?”, suggesting that the settings where the participants are recruited or interviewed determine the “default” condition the responses are based on.
Similar to the validation studies of either the English10,12 or other language versions of the PAM-13 conducted in other countries,13,17,40 ranking of items according to levels of difficulty in this study was different from the original ranking of items derived from an American population with chronic diseases. Interestingly, within the same country, the ranking of items and their response patterns derived from outpatient cardiac patients and the community-dwelling adult population also differed. 11 This highlights the necessity in validating the measure when using in different populations or settings.
The differences in PAM scores between subgroups based on multimorbidity, polypharmacy, and activity participation suggest that these three conditions have good distinguishing capability for activation. Similar to other validation studies,15,18,40,41 the significant difference in PAM scores across three self-perceived health status groups in our study population provided further evidence for known-group validity.
Confidence is a postulated aspect of activation. The convergent validity of the PAM-13 was supported by its positive and moderate correlation with the confidence in terms of controlling and managing most of health problems. The PHQ-9, a screening tool for depressive symptoms, was not constructed to measure any aspects of activation. As such, its negative and weak association with the PAM-13 provided the evidence of divergent validity.
Similar to the validation studies conducted in different disease-specific populations,11-13,15,40 this study adds additional evidence for good internal consistency reliability.
Examining the distribution of activation levels between/across known-groups and comparison of PHQ-9 and health confidence scores across activation levels produced consistent findings as compared to examining using PAM scores, suggesting that the current cut-off scores for activation levels have the discriminating power based on the variables examined in this study. However, as the existing four levels of activation were derived based on the original item difficulty, further research is required to determine the validity of the cut-off scores of the four levels of activation in different populations, especially if the ranking of the item difficulty differs from its original one.
There are several limitations for the study. Firstly, the original PAM-13 was directly used for data collection without any adaption to the health beliefs of local population, this might result in unintended variation in responses. Secondly, the interviewers were instructed not to interpret the statements to individuals as suggested by the administration guideline of PAM-13, variation in interpretation or comprehension from participants could not be avoided, which might increase the variation in responses. Thirdly, due to the cross-sectional study design, we were unable to examine the test-retest reliability and the responsiveness of the PAM-13, which is of interest for future research.
Conclusions
This study demonstrates that the PAM-13 has acceptable construct validity and good internal consistency among community-dwelling adults in Singapore. Further research is required to refine the instruction of PAM administration, investigate the expansion of response options, validate the cut-off scores for the activation levels and examine the test-retest reliability and responsiveness.
Footnotes
Acknowledgments
The authors thank Reuben Ong for administrative assistance with project management, all the trained surveyors for assistance with data collection and the subjects whose participation made this study possible.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Availability of data and materials
According to the Data Protection Act Commission Singapore-Advisory Guidelines for the Healthcare Sector, all the individual data collected for the Population Health Index study are protected under the Act. As such, the datasets analyzed during the current study are not publicly available. However, minimal dataset underlying the findings in the manuscript is available from the corresponding author on reasonable request.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Healthcare Group Pte Ltd in the form of salaries for all authors.
Ethics approval and consent to participate
The study was approved by the ethics review committee of the National Healthcare Group Domain Specific Review Board (Reference Number: 2015/00269). The study was conducted in accordance with the Declaration of Helsinki. Written informed consent was obtained from all individual participants after they were being informed about the study objectives and the safeguards put in place so that confidentiality of the collected data is maintained.
