Application of the PAPERS Grading Criteria Within a Rapid Evidence Review to Determine the Psychometric and Pragmatic Properties of Patient Empowerment Tools

Abstract

Self-management of long-term conditions requires health professionals to understand and develop capabilities that empower the population they serve. A rapid evidence review was undertaken to assess the current evidence based on the psychometric properties of patient empowerment tools. MEDLINE was searched, and data were extracted for each publication and scored using a modified Psychometric and Pragmatic Evidence Rating Scale (PAPERS) evidence rating scale. The results were grouped into the following domains: (a) health literacy; (b) patient activation; (c) long-term conditions; (d) self-management needs and behaviors. A full-text review of 65 publications led to the inclusion of 29 primary studies. The highest scoring tools were selected with respect to performance for each domain: (a) Newest Vital Sign and the Brief Health Literacy Screen; (b) Consumer Health Activation Index and PAM-13; (c) LTCQ and LTCQ8; and (d) SEMCD and Patient Enablement Instrument. PAPERS was a useful tool in determining the generalizability, validity, and reliability of these patient empowerment tools. However, further research is required to establish whether an individual's health literacy status influences patient empowerment tool outcomes.

Keywords

patient participation patient activation patient outcome assessment health literacy

Key Points

Determining the generalizability, validity, and reliability of patient empowerment tools was assisted using a modified Psychometric and Pragmatic Evidence Rating Scale.

A lack of consistency in reporting the psychometric outcome of patient empowerment tools was observed.

Practical consideration outside of the research environment should be considered to guide implementation.

Based on the evidence in this review, there is a knowledge gap around the impact of health literacy on the acceptability and outcomes of the tools.

Introduction

Patient empowerment is increasingly being recognized as an important driver of behavior change interventions within the healthcare environment. By empowering individuals, they are more likely to make rational, informed decisions about their care, which reduces dependence on health service use and improves the utilization of more cost-effective services.¹ Therefore, it is important for health professionals to be able to understand the level of empowerment their patient has and develop capabilities to empower individual to adopt self-management strategies.

The value-based healthcare model seeks to address health inequalities across the population it serves by promoting self-management strategies and supporting patients to make healthier lifestyle choices.^2,3 It can facilitate tailoring the needs of the patient, using the best evidence to hand, and avoiding any unnecessary waste, harm, and variation in care.⁴ However, previous critical appraisals of the literature have reported that a substantial proportion of validated patient empowerment tools has significant limitations related to their acceptability and generalizability to the general population despite demonstrating relatively robust psychometric measurement properties.^5–8

The primary aim of this rapid evidence review was to examine the range of patient empowerment tools available within the published literature and to measure their respective psychometric and pragmatic properties against the Psychometric and Pragmatic Evidence Rating Scale. The secondary aim was to establish which tools performed best against these psychometric and pragmatic criteria.

Methods

Scoping Exercise–Semi-Structured Survey

In July 2023, a scoping exercise was performed by CEDAR researchers at Cardiff and Vale University Health Board, Wales, United Kingdom. An online survey (MS Forms) link was sent using a QR code to a representative cohort of the allied health professional (AHP) staff working within the Therapies Directorate at Cardiff and Vale University Health Board. The survey was designed to capture “real-world” evidence on what patient empowerment tools therapists were regularly using in routine clinical practice. The survey used both open and closed questions to collect information on the tools used, opinions of tools, perceived experiencing rating, perceived patient experience, and factors that affect usability and acceptability (Appendix 1). A descriptive and thematic analysis of the open questions from 69 completed surveys was then conducted by one researcher (NJJ).

Modification to PAPERS

Implementation of patient empowerment tools with the Welsh NHS health system is a complex intervention that requires the consideration of pragmatic measures (eg, time, training requirements, relevance to practice, cost, digital literacy, cultural acceptability), which goes beyond what is covered in more traditional evidence-based assessment and methodological quality of studies (eg, Hunsley and Mash's criteria and Consensus-based Standards for the Selection of Health Status Measurement Instruments (COSMIN)).⁹ The Psychometric and Pragmatic Evidence Rating Scale (PAPERS) takes into consideration the view of stakeholders to generate a stakeholder-driven rating criteria that are relevant to the context of implementation.⁵ PAPERS has previously been shown to assess the quality of existing tools and inform development of implementation measures.⁵

As PAPERS is stakeholder-driven, the items in the pragmatic criteria can be modified to reflect the current situation within a complex system. Relevant data from the thematic analysis from the scoping exercise was used to determine these new items (Table 1). Each new item has the original PAPERS evidence rating scale applied (“poor” (minus 1), “none” (0), “minimal/emerging” (1), adequate (2), “good” (3) and “excellent” (4)).

Table 1.

PAPERS Evidence Rating Criteria.

Pragmatic Criteria*	Psychometric Criteria**
Self-administered	Analysis
Relevant to practice^†	Reliability (Test Retest)
Easy to understand^†	Reliability (Internal Consistency)
Time to complete^†	Validity (Construct Validity — Convergent)
Tool complexity^†	Validity (Construct Validity — Discriminant)
Digital literacy^†	Validity (Construct Validity — Known Groups)
Generalisability^†	Validity (Criterion Validity — Predictive)
Inclusive^†	Validity (Criterion Validity — Concurrent)
Cost	Validity (Structural Validity — Dimensionality)
Availability	Responsiveness
Sustainability	Norms
Cultural adaptability

*Modified pragmatic rating scale from PAPERS.

**Standardized psychometric rating scale from PAPERS.

†

Items added to the pragmatic criteria based on the results of the scoping exercise.

Literature Review

Database and Search Strategy

MEDLINE was searched in September 2023 by one member of the research team (AR). The search strategy was made of up of a combination of free text and index terms relating to “Activation tools” and “psychometric properties” (Appendix 2). Named tools identified during the scoping exercise were also incorporated into the search strategy.

Inclusion and Exclusion Criteria

The inclusion criteria for primary studies were limited to peer-reviewed, English language articles published between 2013 and August 2023, which had conducted Rasch analysis or item response theory analysis, and reported statistically significant correlations between the tool in question and other high-quality generic tools (eg, EQ-5D).

The exclusion criteria included (a) research conducted on a rare disease population, (b) tool developed for a specific condition (eg, not a generic tool), (c) not a patient-specific PROM (eg, FROM or CROM), (d) lack of reliability (α) and validity (r) reporting, and (e) publications that applied the classical test theory (CTT).

Study Selection

One reviewer (AR) screened the search results by title and abstract. Another reviewer (NJJ) then undertook the full text review to identify relevant publications, the results of which were then discussed with a second review (RL). At each stage, any discrepancies were discussed and resolved with a third reviewer (KLW). Roles and responsibilities of each of the authors were designated based on their previous experience as information specialist (AR), clinicians with a research background to provide clinical context (NJJ, RL) and research oversight (KLW).

Data Extraction, Analysis, and Synthesis

Information for the PAPERS grading criteria was extracted from the primary studies by one review (NJJ) into Microsoft Excel. The PAPERS grading for each study was discussed a second reviewer (KLW). The results were discussed narratively under the following domains: (a) Health Literacy; (b) Patient Activation; (c) Long-term Conditions; and (d) Self-management Needs and Behaviors.

Results

The database searches yield 413 unique results (Figure 1). After screening at title and abstract and full-text review, 29 publications were included (Appendix 3). The included studies were pooled under the following domains: Health Literacy (n = 11);^10–20 Patient Activation (n = 8);^21–28 Long-term Conditions (n = 6);^29–34 and Self-management Needs and Behaviors (n = 4).^35–38

Figure 1.

PRISMA flow diagram.

Pragmatic Properties

The grading for the pragmatic qualities from each of the patient empowerment tools is illustrated in Table 2. Sixteen^{11,12,14,15,19–22,24,26,29,31,33,36–38} (55%) of the 29 primary studies reported that the tools were self-administered. A further 21/29^{11,12,14,15,18,19,23–25,27–38} (72%) of tools were regarded as relevant to practice, and 62% (18/29)^{10,11,13,14,16–18,28–38} were deemed easy to understand.

Table 2.

PAPERS Modified Pragmatic Evidence Rating According to the Scoping Exercise on Patient Empowerment Tools.

Pragmatic Criteria	Health Literacy Tools (n = 11)	Patient Activation Tools (n = 8)	Long-term Conditions Tools (n = 6)	Self-management Needs and Behaviors Tools (n = 4)
Self-administered	1	0	0	3
Relevant to practice	1	2	6	4
Easy to understand	4	−6	6	4
Time to complete (+3 minutes)	6	3	0	9
Tool complexity	15	−6	6	4
Digital literacy	7	7	4	2
Generalisability	9	8	−6	4
Inclusive	5	8	6	4
Cost (Free of charge)	7	−6	4	2
Availability	5	−8	4	2
Sustainability	28	23	12	10
Cultural adaptability	5	2	6	3
Total pragmatic score	93	27	48	51

Only 31% (9/29)^{10,13,16–18,28,35,37,38} of the tools could be completed in three minutes or less. The tools within the Long-term Conditions domain were the worst performing, with 5/6 (83%)^30–34 taking longer than 10 minutes to complete. With respect to tool complexity, Long-term Conditions (100%),^29–34 Self-management Needs and Behavior (100%),^35–38 and Health Literacy (64%)^10–17 were assessed as being the best performing tools with basic and/or simple text. While the complexity of the Patient Activation tools was regarded as above normal comprehension levels for typical healthcare service-users, only one study²⁸ (13%) has basic text.

Over 72% (21/29) of primary studies reported that the tools were digitally inclusive^{10,12–17,20–27,29,31,33,34,36,38} and could be conducted either digitally, over the phone, or face-to-face in a clinical environment. Tools within the Health Literacy, Patient Activation, and Self-management Needs and Behavior domains all performed very well with respect to generalizability, whereas tools in the Long-term Conditions domain were the worst. Tools within all domains performed well in view of their inclusivity. However, cost was more difficult to establish, as it was often not reported within the publications. Only 48% of studies included in this rapid evidence review documented whether the tools were free of charge to use without permission from the developer or a specific license. The Patient Activation domain was the worst performing with respect to cost and availability given the fact that a license must be purchased prior to access. In contrast, over 60% (13/21) of the tools in the other domains were reported to be free of charge and easily accessible online.^{10,11,13,14,16–18,29,31,33–35,37} As regards the sustainability and cultural adaptability, these scores were very much co-dependent on the other outcomes generated and were difficult to extrapolate independently from the findings reported in the primary studies included in this rapid evidence review.

Psychometric Properties

The grading for the psychometric properties for each tool based on the standardized PAPERS psychometric evidence rating criteria is outlined in Table 3. Twenty-two (76%)^{12,15,17–26,28–34,36–38} of the 29 primary studies documented the methods of data analysis undertaken in each respective study. The most common method was Rasch analysis, which accounted for 32% (7/22)^{15,21,24,26,29,30,32} of all analyses performed. In contrast, only four (14%) of studies^17,20,28,35 conducted reliability (Test Retest) measures to assess the consistency or stability of the patient empowerment tools over time. Internal consistency was reported in 100% of the primary studies included in this rapid evidence review. With respect to internal consistency, Health Literacy was the worst performing domain overall, with alpha Cronbach values ranging between α = 0.70 and α = 0.90 (mean α = 0.80) when compared to the other domains; Patient Activation (range α = 0.81-0.99, mean α = 0.87), Long-term Conditions (range α = 0.82-0.95, mean = α = 0.88), and the Self-management Needs and Behaviors (range α = 0.87-0.9, mean = α = 0.89). This was despite attaining the highest psychometric score (Health Literacy = 90, Patient Activation = 67, Long-term Conditions = 56, Self-management Needs and Behaviors = 41) when graded against the standardized PAPERS evidence rating scale (Table 3).

Table 3.

Overall PAPERS Standardized Psychometric Evidence Rating for Each Patient Empowerment Domain.

Psychometric Criteria	Health Literacy Tools (n = 11)	Patient Activation Tools (n = 8)	Long-term Conditions Tools (n = 6)	Self-management Needs and Behaviors Tools (n = 4)
Analysis	7	7	7	3
Reliability (Test Retest)	14	5	4	2
Reliability (Internal Consistency)	34	28	21	15
Validity (Construct Validity — Convergent)	4	7	12	8
Validity (Construct Validity — Discriminant)	0	0	0	2
Validity (Construct Validity — Known Groups)	14	8	8	0
Validity (Criterion Validity — Predictive)	0	2	0	0
Validity (Criterion Validity — Concurrent)	18	2	3	7
Validity (Structural Validity — Dimensionality)	−1	4	1	1
Responsiveness	−1	4	0	3
Norms	1	0	0	0
Total psychometric score	90	67	56	41

All reported studies that looked at construction and criterion validity reported acceptable Cohen d and Pearson r values when graded against the standardized PAPERS psychometric evidence rating criteria; however, overall validity reporting across all domains was inconsistent. Only one³⁸ (3%) of the 29 primary studies tested discriminant construct validity (Cohen's d = 0.37) when comparing the structural and convergent validities of three generic patient empowerment tools in a cohort (n = 742) of healthcare service-users. Predictive criterion validity analyses were also limited with only a single study (n = 1/29) exploring the development and validation of a novel patient activation tool reporting a Pearson's r value (r = 0.32).²⁸ Furthermore, 41% (12/29)^{10,11,13,16,18–20,23,24,31,36,37} of the primary studies included in this rapid evidence review reported concurrent criterion validity, and only a third (31% — 9/29)^{12,21,22,24,26,30,32,37,38} conducted a dimensional structural validity analysis. As regards responsiveness and norms, these scores were very much co-dependent on the other psychometric outcomes generated and were difficult to extrapolate independently given the poor quality of evidence reported in the published literature.

Combined PAPERS Evidence Scores

Health literacy tool scores had the highest overall domain score of 90, comparative to the Self-management Needs and Behaviors Tools domains, which had the lowest score of 41 (Table 3). With respect to both psychometric and pragmatic performances, the highest scoring tools within each domain were 1) Health Literacy = Newest Vital Sign (NVS) and the Brief Health Literacy Screen (BHLS), 2) Patient Activation = Consumer Health Activation Index (CHAI) and PAM-13, 3) Long-term Conditions = LTCQ and LTCQ8, and 4) Self-management Needs and Behaviors = SEMCD and Patient Enablement Instrument (PEI-2).

Discussion

Patient empowerment is increasingly being recognized as an important driver behind the behavioral shift toward self-management within the healthcare environment and contributes to positive health outcomes (ie, reduced recovery time, altering disease progression). The value-based healthcare model seeks to address health inequalities across the population it serves by promoting self-management strategies and supporting patients to make healthier lifestyle choices. However, while there are a number of patient empowerment tools that can enable healthcare clinicians to tailor specific interventions to the need of patients, there is a lack of empirical evidence to support what constitutes the optimal patient empowerment tool. The majority of validated instruments employed to evaluate the psychometric performance of patient empowerment tool across different contexts and populations often fail to consider the practical properties required to guide implementation outside the research environment. In this rapid evidence review, an opportunity was presented to challenge current practice and adopt a novel method of evaluating both psychometric and practical properties on a range of evidence-based patient empowerment tool using PAPERS. It demonstrated the effective use of stakeholder-driven components within the pragmatic criteria and a lack of consistency in reporting psychometric outcome measures. There are implications for the wider use of PAPERS within implementation science, where complex interventions require the consideration of situational and individual level factors to ensure successful implementation.

The generalizability of any healthcare intervention is important; therefore, having a measure that can confidently provide psychometric evidence in a variety of contexts with diverse populations is key. Previous critical appraisals of the literature have reported that a substantial proportion of validated patient empowerment tool has significant deficiencies limiting their acceptability and generalizability to the general population despite demonstrating relatively robust psychometric measurement properties.^5–8 The ability to detect diverse population with the PAPERS criteria has arguably been achieved within this study as those tools included under the Long-term Conditions domain score the worse for generalizability given the specificity of the long-term conditions tools to target a particular population of typical healthcare service-users.

To be adopted for practical use, these measures also need to have pragmatic properties to address challenges faced by stakeholders both at a healthcare system level and wider social-determinants of health. Identifying and providing solutions appropriate to these barriers are regarded as critical for successful implementation and also drive quality improvement.^9,39 The PAPERS criteria allowed for modifications based on key implementation barriers within the NHS, which had been identified by staff within the survey. This is where the PAPERS criteria set differ from other tools, such as the COSMIN criteria, which main focus is methodological quality. COSMIN has previously been criticized for being subjective, requiring a high level of expertise to implement, and can produce contradictory conclusions.⁴⁰ Furthermore, PAPERS is more suited to the rapid review methodology, as using the COSMIN checklist is a very time-consuming process, which may be challenging for a rapid evidence review.⁴¹ Having said that, there is merit in exploring the difference in between PAPERS and COSMIN as further research, taking into consideration the pragmatic and methodological properties of the work being undertaken. It should be noted that pragmatic properties extend beyond psychometric properties to include features like sustainability and cost, which are essential drivers of value-based healthcare.

The value-based healthcare model seeks to address health inequalities across populations. Both patient expectations and health literacy are factors that are linked to patient activation.⁴² Understanding the role of health literacy is important as it impacts the ability to complete tools, but also impacts how decision making and assessments of health outcomes are made by individuals, which ultimately impacts the ability to adopt positive lifestyle changes. The NVS tool, the highest rated health literacy tool against the PAPERS criteria, identifies healthcare service-users at risk for low health literacy using a nutritional label. It is regarded as one of the most widely used validated health literacy screening instruments in clinical practice due to its 6-item response and short administration time (2-3 minutes).⁴³ In contrast, the Brief Health Literacy Screen (BHLS), which is ranked second against the PAPERS criteria, uses a five-point Likert scale to quantify healthcare service-users responses to question relating to confidence and issues related to the understanding of medical information.¹⁸ This tool can be administered in less than two minutes, and when compared to the NVS, it is likely to have more practical relevance to the typical healthcare service-user than deciphering a nutritional label. Despite the relevance to health care service-users, a systematic review identified during the sift of search results for this study and set aside for comparison concluded that the NVS was a practical health literacy tool due to containing both communicative and numeracy elements.⁴⁴ It is encouraging that a similar tool preference has been found within the literature, but there is still merit in considering local relevance.

The Patient Empowerment tools domains that demonstrated robust psychometric properties in this rapid evidence review often failed to perform optimally when graded against the modified pragmatic evidence rating criteria. The Patient Activation domain is a good example where the reliability and validity of these tools were graded between “good” and “excellent” when scored against the PAPERS psychometric criteria. However, when scored against the PAPERS modified pragmatic criteria, they were graded as “poor” across four of the twelve items. This was predominantly attributed to tool complexity and lack of availability given the fact that a license must be purchased prior to access. In comparison, the majority of tools in the other three domains were regarded as relatively easy to understand, and over 60% of these patient empowerment tools were reported to be free of charge and easily accessible online. However, these results were based on the reported details in the included publications, with a focus on validity and reliability. The individual tools were not obtained within this study to assess content or the ability of the tools to produce the desired outcomes. These findings highlight the need to measure both psychometric and pragmatic performance when exploring the evidence to derive, which properties constitutes the optimal patient empowerment tool in clinical practice. PAPERS evidence rating scale may move the field toward measures that inform robust research evaluations and practical implementation efforts.

Limitations and Further Research

This scoping review managed to collate and grade multiple patient empowerment tools rapidly using the PAPERS grading criteria. As this was a rapid review, only one database was used. It was limited to publications after 2013, and only English language publications were included to reduce the need for translation. Although publications before 2013 were excluded, this was deemed appropriate to reflect current healthcare practices and population needs.

The PAPERS criteria did enable the consideration of the practical properties of implementation; however, the analysis only looked at the PAPERS scores and did not assess on study setting, population, and publication quality. Thus, a full appraisal of the included publication and the comparability between the different studies were not fully explored, including the role of the social determinant of health. One such factor was the impact of an individual's health literacy status; therefore, further qualitative and quantitative research is recommended to determine the role of health literacy on tool usability and acceptability and the associated patient empowerment outcomes. Consideration should also be given to the association between patient empowerment and patient-reported outcome measures (PROMs) both as a promoter of healthy-behaviors and as outcome measurement tools for assessing the impact of the use of patient empowerment tools. Any further research could consider using large diverse samples of patients that are representative at a population level and have appropriate control groups and longitudinal follow-up to understand the long-term impact of the implementation of patient empowerment tools.

Conclusion

PAPERS was a useful tool to determine the validity and reliability of different patient empowerment tools while also considering the practical aspects of implementation in clinical practice. However, further research requires the relevance of these tools in guiding behavior change across a diverse and complex population of healthcare service-users in Wales. Future work should include qualitative and quantitative insights into relationships between an individual's health literacy status and capacity to understand the purpose and relevance of these tools in guiding behavior change and longitudinal implication of patient empowerment tool use, with the appropriate control group with changes attributed to the intervention.

Footnotes

Appendix

Acknowledgments

The authors would like to thank the therapies staff at Cardiff and Vale University Health Board who engaged in and completed the scoping exercise. The results of this activity were invaluable in informing the pragmatic criteria of PAPERS.

Author Contributions

Conceptualization was done by NJJ, KLW, RP; methodology was performed by NJJ, AR, KLW; validation was done by KEW, KLW, RL; formal analysis and data curation were done by NJJ; writing the original draft was done by NJJ and KEW; writing the review and editing were done by AR, KLW, RL; funding acquisition was done by NJJ and KLW.

Consent Statement

Implied consent was obtained.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical Statement

Ethical approval was not applicable for this study.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Welsh Value in Health Centre.

ORCID iD

Katherine E Woolley

References

McAllister

Dunn

Payne

Davies

Todd

. Patient empowerment: the need to consider it as a measurable patient-reported outcome for chronic conditions. BMC Health Serv Res. 2012;12(1):157.

Gibbs

Guarnieri

Chu

Murdoch

Asif

. Value-based chronic care model approach for vulnerable older patients with multiple chronic conditions. J Gastrointest Oncol. 2021;12(Suppl 2):S324-s338.

Lewis

. Value-based healthcare: is it the way forward? Future Healthcare J. 2022;9(3):211-5.

Kim

Daly

, et al. An evidence review of low-value care recommendations: inconsistency and lack of economic evidence considered. J Gen Intern Med. 2021;36(11):3448-55.

Stanick

Halko

Nolen

, et al. Pragmatic measures for implementation research: development of the Psychometric and Pragmatic Evidence Rating Acale (PAPERS). Transl Behav Med. 2021;11(1):11-20.

Cooper

Clatworthy

Harding

Whetham

. Measuring empowerment among people living with HIV: a systematic review of available measures and their properties. AIDS Care. 2019;31(7):798-802.

Jordan

Osborne

Buchbinder

. Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses. J Clin Epidemiol. 2011;64(4):366-79.

Eyles

Hunter

Meneses

SRF

, et al. Instruments assessing attitudes toward or capability regarding self-management of osteoarthritis: a systematic review of measurement properties. Osteoarth Cartil. 2017;25(8):1210-22.

Lewis

Mettert

Stanick

, et al. The Psychometric and Pragmatic Evidence Rating Scale (PAPERS) for measure development and evaluation. Implement Res Pract. 2021;2:26334895211037391.

10.

Caldwell

Killingsworth

. The online use of the newest vital sign in adolescents. Clin Nurs Res. 2022;31(1):55-9.

11.

Chakkalakal

Venkatraman

White

Kripalani

Rothman

Wallston

. Validating health literacy and numeracy measures in minority groups. Health Literacy Res Pract. 2017;1(2):e23-30.

12.

Finbråten

Pettersen

Wilde-Larsson

Nordström

Trollvik

Guttersrud

. Validating the European Health Literacy Survey Questionnaire in people with type 2 diabetes: latent trait analyses applying multidimensional Rasch modelling and confirmatory factor analysis. J Adv Nurs. 2017;73(11):2730-44.

13.

Kordovski

Woods

Avci

Verduzco

Morgan

. Is the newest vital sign a useful measure of health literacy in HIV disease? J Int Assoc Provid AIDS Care. 2017;16(6):595-602.

14.

Mock

Sethares

. Concurrent validity and acceptability of health literacy measures of adults hospitalized with heart failure. Appl Nurs Res: ANR. 2019;46:50-6.

15.

Morris

Soh

Hill

, et al. Measurement properties of the health literacy questionnaire (HLQ) among older adults who present to the emergency department after a fall: a Rasch analysis. BMC Health Serv Res. 2017;17(1):605.

16.

Rowlands

Khazaezadeh

Oteng-Ntim

Seed

Barr

Weiss

. Development and validation of a measure of health literacy in the UK: the newest vital sign. BMC Public Health. 2013;13(1):116.

17.

Russell

Patel

Curtis

, et al. Test-retest reliability of the newest vital sign health literacy instrument: in-person and remote administration. Patient Educ Couns. 2019;102(4):749-52.

18.

Wallston

Cawthon

McNaughton

Rothman

Osborn

Kripalani

. Psychometric properties of the brief health literacy screen in clinical practice. J Gen Intern Med. 2014;29(1):119-26.

19.

Weinert

Shreffler-Grant

Nichols

. Psychometric evaluation of the MSU CAM health literacy scale. Complement Ther Med. 2019;42:156-7.

20.

Zegers

Gonzales

Smith

Pullen

De Alba

Fiandt

The psychometric testing of the functional, communicative, and critical health literacy tool. Patient Educ Couns. 2020;103(11):2347-52.

21.

Eyles

Ferreira

Mills

, et al. Is the patient activation measure a valid measure of osteoarthritis self-management attitudes and capabilities? Results of a Rasch analysis. Health Qual Life Outcomes. 2020;18(1):121.

22.

Hendrikx

RJP

Spreeuwenberg

Drewes

Ruwaard

Baan

. How to measure population health: an exploration toward an integration of valid and reliable instruments. Popul Health Manag. 2018;21(4):323-30.

23.

Hung

Carter

Hayden

, et al. Psychometric assessment of the patient activation measure short form (PAM-13) in rural settings. Qualf Life Res. 2013;22(3):521-9.

24.

Lightfoot

Wilkinson

Memory

Palmer

Smith

. Reliability and validity of the patient activation measure in kidney disease: results of Rasch analysis. Clin J Am Soc Nephrol: CJASN. 2021;16(6):880-8.

25.

Melby

Nygård

Brobakken

, et al. Test–retest reliability of the patient activation measure-13 in adults with substance use disorders and schizophrenia spectrum disorders. Int J Environ Res Public Health. 2021;18(3):1185

26.

Packer

Kephart

Ghahari

Audulv

Versnel

Warner

The patient activation measure: a validation study in a neurological population. Qual Life Res. 2015;24(7):1587-96.

27.

Prey

Qian

Restaino

, et al. Reliability and validity of the patient activation measure in hospitalized patients. Patient Educ Couns. 2016;99(12):2026-33.

28.

Wolf

Smith

Pandit

, et al. Development and validation of the consumer health activation index. Med Decis Making. 2018;38(3):334-43.

29.

Batchelder

Fox

Potter

, et al. Rasch analysis of the long-term conditions questionnaire (LTCQ) and development of a short-form (LTCQ-8). Health Qual Life Outcomes. 2020;18(1):375.

30.

Boecker

Iglesias

Peytremann-Bridevaux

. Psychometric evaluation of the patient assessment of chronic illness care instrument using item analysis according to the Rasch model. J Eval Clin Pract. 2022;28(4):670-9.

31.

Kelly

Fitzpatrick

Kurinczuk

Rivero-Arias

Alderdice

. Assessing the validity of the long-term conditions questionnaire (LTCQ) in women during pregnancy and the first year following birth. Patient Relat Outcome Meas. 2022;13:221-8.

32.

Lambert

McCusker

Belzile

, et al. Using confirmatory factor analysis and rasch analysis to examine the dimensionality of the patient assessment of care for chronic illness care (PACIC). Qual Life Res. 2021;30(5):1503-12.

33.

Potter

Batchelder

A'Court

, et al. Long-term conditions questionnaire (LTCQ): initial validation survey among primary care patients and social care recipients in England. BMJ Open. 2017;7(11):e019235.

34.

Potter

Peters

Cundell

McShane

Fitzpatrick

. Use of the long-term conditions questionnaire (LTCQ) for monitoring health-related quality of life in people affected by cognitive impairment including dementia: pilot study in UK memory clinic services. Qual Life Res. 2021;30(6):1641-52.

35.

Bedford

Yeung

MHY

, et al. The validity, reliability, sensitivity and responsiveness of a modified patient enablement instrument (PEI-2) as a tool for serial measurements of health enablement. Fam Pract. 2021;38(3):339-45.

36.

Eikelenboom

Smeele

Faber

, et al. Validation of self-management screening (SeMaS), a tool to facilitate personalised counselling and support of patients with chronic diseases. BMC Fam Pract. 2015;16(1):165.

37.

Enthoven

Peolsson

Ludvigsson

Wibault

Peterson

Öberg

. Validity, internal consistency and self-rated change of the patient enablement instrument in patients with chronic musculoskeletal pain. J Rehabil Med. 2019;51(8):587-97.

38.

Kephart

Packer

Audulv

Warner

. The structural and convergent validity of three commonly used measures of self-management in persons with neurological conditions. Qual Life Res 2019;28(2):545-56.

39.

Cook

Beckman

. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119(2):166.e7-16.

40.

McKenna

Heaney

. Setting and maintaining standards for patient-reported outcome measures: can we rely on the COSMIN checklists? J Med Econ. 2021;24(1):502-11.

41.

Kwok

EYL

Rosenbaum

Thomas-Stonell

Cunningham

. Strengths and challenges of the COSMIN tools in outcome measures appraisal: a case example for speech–language therapy. Int J Lang Commun Disord. 2021;56(2):313-29.

42.

Smith

Curtis

Wardle

Wagner

Wolf

. Skill set or mind set? Associations between health literacy, patient activation and health. PLOS ONE. 2013;8(9):e74373.

43.

Weiss

Mays

Martz

, et al. Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med. 2005;3(6):514-22.

44.

Duell

Wright

Renzaho

Bhattacharya

. Optimal health literacy measurement for the clinical setting: a systematic review. Patient Educ Couns. 2015;98(11):1295-307.