Though the interest in community engagement in research (CEnR) protocols has increased, studies reporting on the findings of tested CEnR engagement measurement scales for health studies are sparse. A systematic review was conducted from January 1 to March 1, 2023, to identify validated, quantitative CEnR engagement measurement tools for health studies. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology was employed. The rigor of scale development, testing, and implementation was explored, and a `best practices evaluation conducted. Themes on the readiness of scales for implementation in health research studies were narratively compiled. Nineteen studies met the search inclusion criteria—reporting on the development, testing, and implementation of seven CEnR engagement measurement scales for health studies. Scale implementation studies precipitated only two of the studies. None of the scales followed the rigorous process dictated in best practices; however, at this time, three scales have gone through the most robust testing processes. Advancement of the science of engagement measurement requires consensus on terminology, application of best practices for scale development and testing protocols, and consistency of reporting findings.
AbelsonJ.LiK.WilsonG.ShieldsK.SchneiderC.BoesveldS. (2016). Supporting quality public and patient engagement in health system organizations: Development and usability testing of the Public and Patient Engagement Evaluation Tool. Health Expectations, 19(4), 817–827. https://doi.org/10.1111/hex.12378
2.
BoatengG. O.NeilandsT. B.FrongilloE. A.Melgar-QuiñonezH. R.YoungS. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6(1), 344. https://doi.org/10.3389/fpubh.2018.00149
3.
BoivinA.L'EspéranceA.GauvinF.-P.DumezV.MacaulayA. C.LehouxP.AbelsonJ. (2018). Patient and public engagement in research and health system decision making: A systematic review of evaluation tools. Health Expectations, 21(6), 1075–1084. https://doi.org/10.1111/hex.12804
4.
BombakA. E.HansonH. M. (2017). A critical discussion of patient engagement in research. Journal of Patient-Centered Research and Reviews, 4(1), 39–41. https://doi.org/10.17294/2330-0698.1273
5.
BowenD. J.AckermannN.ThompsonV. S.NederveldA.GoodmanM. (2022). A study examining the usefulness of a new measure of research engagement. Journal of General Internal Medicine, 37(2), 50–56. https://doi.org/10.1007/s11606-021-06993-1
6.
BowenD. J.HyamsT.GoodmanM.WestK. M.Harris-WaiJ.YuJ.-H. (2017). Systematic review of quantitative measures of stakeholder engagement. Clinical and Translational Science, 10(5), 314–336. https://doi.org/10.1111/cts.12474
7.
BrettJ.StaniszewskaS.MockfordC.Herron-MarxS.HughesJ.TysallC.SulemanR. (2014). Mapping the impact of patient and public involvement on health and social care research: A systematic review. Health Expectations, 17(5), 637–650. https://doi.org/10.1111/j.1369–7625.2012.00795.x
8.
CarmanK. L.DardessP.MaurerM.SofaerS.AdamsK.BechtelC.SweeneyJ. (2013). Patient and family engagement: A framework for understanding the elements and developing interventions and policies. Health Affairs, 32(2), 223–231. https://doi.org/10.1377/hlthaff.2012.1133
9.
ChungJ.SarathyA.HsiehY. G.EsteyG.TorresA.PatsiogiannisV.DonelanK.SkotkoB. G. (2021). Assessment of stakeholder engagement in a Down syndrome research study. Journal of Patient-Centered Research and Reviews, 8(1), 64–67. https://doi.org/10.17294/2330-0698.1777
DomecqJ. P.PrutskyG.ElraiyahT.WangZ.NabhanM.ShippeeN.BritoJ. P.BoehmerK.HasanR.FirwanaB.ErwinP.EtonD.SloanJ.MontoriV.AsiN.DabrhA. M. A.MuradM. H. (2014). Patient engagement in research: A systematic review. BMC Health Services Research, 14(1), 89. https://doi.org/10.1186/1472-6963-14-89
13.
EderM. M.EvansE.FunesM.HongH.ReuterK.AhmedS.CalhounK.Corbie-SmithG.DaveG.DeFinoM.HarwoodE.KissackA.KleinmanL. C.WallersteinN. (2018). Defining and measuring community engagement and community-engaged research: Clinical and translational science institutional practices. Progress in Community Health Partnerships: Research, Education, and Action, 12(2), 145–156. https://doi.org/10.1353/cpr.2018.0034
14.
EpsteinS. (1996). Impure science: AIDS, activism, and the politics of knowledge. University of California Press.
15.
ForsytheL. P.CarmanK. L.SzydlowskiV.FayishL.DavidsonL.HickamD. H.HallC.BhatG.NeuD.StewartL.JalowskyM.AronsonN.AnyanwuC. U. (2019). Patient engagement in research: Early findings from the patient-Centered outcomes research Institute. Health Affairs, 38(3), 359–367. https://doi.org/10.1377/hlthaff.2018.05067
16.
GarrattA.SagenJ.BørøsundE.VarsiC.KjekenI.DagfinrudH.MoeR. H. (2022). The public and patient engagement evaluation tool: Forward-backwards translation and cultural adaption to Norwegian. BMC Musculoskeletal Disorders, 23(1), 89. https://doi.org/10.1186/s12891-022-05514-3
GoodmanM. S.AckermannN.BowenD. J.ThompsonV. (2019). Content validation of a quantitative stakeholder engagement measure. Journal of Community Psychology, 47(8), 1937–1951. https://doi.org/10.1002/jcop.22239
19.
GoodmanM. S.AckermannN.Haskell-CraigZ.JacksonS.BowenD. J.ThompsonV. L. (2022). Construct validation of the research engagement survey scale (REST). Research involvement and engagement, 8, 2022. https://doi.org/10.1186/s40900-022-00360-y
20.
GoodmanM. S.AckermannN.PierceK. A.BowenD. J.ThompsonV. S. (2021). Development and validation of a brief version of the research engagement survey tool. International Journal of Environmental Research and Public Health, 18(19), 33. https://doi.org/10.3390/ijerph181910020
21.
GoodmanM. S.ThompsonV. L.JohnsonC. A.GennarelliR.DrakeB. F.BajwaP.WitherspoonM.BowenD. (2017). Evaluating community engagement in research: Quantitative measure development. Journal of Community Psychology, 45(1), 17–32. https://doi.org/10.1002/jcop.21828
22.
HahnD. L.HoffmannA. E.FelzienM.LeMasterJ. W.XuJ.FagnanL. J. (2017). Tokenism in patient engagement. Family Practice, 34(3), 290–295. https://doi.org/10.1093/fampra/cmw097
23.
HamiltonC. B.HoensA. M.BackmanC. L.McKinnonA. M.McQuittyS.EnglishK.LiL. C. (2018a). An empirically based conceptual framework for fostering meaningful patient engagement in research. Health Expectations, 21(1), 396–406. https://doi.org/10.1111/hex.12635
24.
HamiltonC. B.HoensA. M.McKinnonA. M.McQuittyS.EnglishK.HawkeL. D.LiL. C. (2021). Shortening and validation of the Patient Engagement in Research Scale (PEIRS) for measuring meaningful patient and family caregiver engagement. Health Expectations, 24(3), 863–879. https://doi.org/10.1111/hex.13227
25.
HamiltonC. B.HoensA. M.McQuittyS.McKinnonA. M.EnglishK.BackmanC. L.AzimiT.KhodarahmiN.LiL. C. (2018b). Development and pre-testing of the Patient Engagement in Research Scale (PEIRS) to assess the quality of engagement from a patient perspective. PLoS One, 13(11), 208. https://doi.org/10.1371/journal.pone.0206588
26.
HanH.-R.XuA.MendezK. J. W.OkoyeS.CudjoeJ.BahouthM.ReeseM.BoneL.Dennison-HimmelfarbC. (2021). Exploring community engaged research experiences and preferences: A multi-level qualitative investigation. Research Involvement and Engagement, 7, 2021. https://doi.org/10.1186/s40900-021-00261-6
27.
HarringtonR. L.HannaM. L.OehrleinE. M.CampR.WheelerR.CooblallC.TesoroT.ScottA. M.von GizyckiR.NguyenF.HareendranA.PatrickD. L.PerfettoE. M. (2020). Defining patient engagement in research: Results of a systematic review and analysis. Value in Health, 23(6), 677–688. https://doi.org/10.1016/j.jval.2020.01.019
28.
HumphriesD.GuptaR.DukpaT.WangmoD. (2021). Assessing community health research capacity across stakeholders: Adapting a tool. Health Promotion International, 36(4), 1198–1208. https://doi.org/10.1093/heapro/daaa105
29.
HumphriesD.MaM.CollinsN.RayN.WatE.BazelonJ.PettinelliJ.FiellinD. A. (2019). Assessing research activity and capacity of community-based organizations: Refinement of the CREAT instrument using the Delphi method. Journal of Urban Health, 96(3), 912–922. https://doi.org/10.1007/s11524-019-00374-0
30.
HumphriesD. L.Carroll-ScottA.MitchellL.TianT.ChoudhuryS.FiellinD. A. (2014). Assessing research activity and capacity of community-based organizations: Development and pilot testing of an instrument. Progress in Community Health Partnerships: Research, Education, and Action, 8(4), 421–432. https://doi.org/10.1353/cpr.2014.0067
31.
KhodyakovD.StockdaleS.JonesA.MangoJ.JonesF.LizaolaE. (2013). On measuring community participation in research. Health Education & Behavior, 40(3), 346–354. https://doi.org/10.1177/1090198112459050
32.
KhodyakovD.StockdaleS.JonesF.OhitoE.JonesA.LizaolaE.MangoJ. (2011). An exploration of the effect of community engagement in research on perceived outcomes of partnered mental health services projects. Society and Mental Health, 1(3), 185–199. https://doi.org/10.1177/2156869311431613
33.
KwokE. Y. L.RosenbaumP.Thomas-StonellN.CunninghamB. J. (2021). Strengths and challenges of the COSMIN tools in outcome measures appraisal: A case example for speech–language therapy. International Journal of Language & Communication Disorders, 56(2), 313–329. https://doi.org/10.1111/1460-6984.12603
34.
KyriazosT. A. (2018). Applied psychometrics: Sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology, 9(8), 2207–2230. https://doi.org/10.4236/psych.2018.98126
35.
LammK. W.LammA. J.EdgarD. (2020). Scale development and validation: Methodology and recommendations. Journal of International Agricultural and Extension Education, 27(2), 24–35. https://doi.org/10.4148/2831-5960.1115
36.
MaccarthyJ.GuerinS.WilsonA. G.DorrisE. R. (2019). Facilitating public and patient involvement in basic and preclinical health research. PLoS One, 14(5), 445. https://doi.org/10.1371/journal.pone.0216600
37.
MarlettN.ShklarovS.MarshallD.SantanaM. J.WasylakT. (2015). Building new roles and relationships in research: A model of patient engagement research. Quality of Life Research, 24(5), 1057–1067. https://doi.org/10.1007/s11136-014-0845-y
38.
MellingerC. D.HansonT. A. (2021). Methodological considerations for survey research: Validity, reliability, and quantitative analysis. Linguistica Antverpiensia, New Series – Themes in Translation Studies, 19(1), 2021. https://doi.org/10.52034/lanstts.v19i0.549
39.
MokkinkL. B.de VetH. C. W.PrinsenC. A. C.PatrickD. L.AlonsoJ.BouterL. M.TerweeC. B. (2018). COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27(5), 1171–1179. https://doi.org/10.1007/s11136-017-1765-4
40.
MrklasK. J.MeraliS.KhanM.ShergillS.BoydJ. M.NowellL.PfadenhauerL. M.PaulK.GoertzenA.SwainL.SibleyK. M.Vis-DunbarM.HillM. D.Raffin-BouchalS.TonelliM.GrahamI. D. (2022). How are health research partnerships assessed? A systematic review of outcomes, impacts, terminology and the use of theories, models and frameworks. Health Research Policy and Systems, 20(1), 22. https://doi.org/10.1186/s12961-022-00938-8
41.
PageM. J.McKenzieJ. E.BossuytP. M.BoutronI.HoffmannT. C.MulrowC. D.ShamseerL.TetzlaffJ. M.AklE. A.BrennanS. E.ChouR.GlanvilleJ.GrimshawJ. M.HróbjartssonA.LaluM. M.LiT.LoderE. W.Mayo-WilsonE.McDonaldS.MoherD. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. British Medical Journal, 372(71), 332. https://doi.org/10.1136/bmj.n71
PhoenixM.NguyenT.GentlesS. J.VanderKaayS.CrossA.NguyenL. (2018). Using qualitative research perspectives to inform patient engagement in research. Research Involvement and Engagement, 4(1), 2018. https://doi.org/10.1186/s40900-018-0107-1
44.
ShufutinskyA. (2020). Employing use of self for transparency, rigor, trustworthiness, and credibility in qualitative organizational research methods. Organization Development Review, 52(1), 50–58.
45.
SimmonsL. A.WoleverR. Q.BechardE. M.SnydermanR. (2014). Patient engagement as a risk factor in personalized health care: A systematic review of the literature on chronic disease. Genome Medicine, 6(1), 201. https://doi.org/10.1186/gm533
46.
SlatteryP.SaeriA. K.BraggeP. (2020). Research co-design in health: A rapid overview of reviews. Health Research Policy and Systems, 18(1), 18. https://doi.org/10.1186/s12961-020-0528-9
47.
StaniszewskaS.BrettJ.SimeraI.SeersK.MockfordC.GoodladS.AltmanD. G.MoherD.BarberR.DenegriS.EntwistleA.LittlejohnsP.MorrisC.SulemanR.ThomasV.TysallC. (2017). GRIPP2 reporting checklists: Tools to improve reporting of patient and public involvement in research. British Medical Journal, 358(3), 337. https://doi.org/10.1136/bmj.j3453
48.
SteeleB.NeelakantanL.JochimJ.DaviesL. M.BoyesM.Franchino-OlsenH.DunneM.MeinckF. (2023). Measuring violence against children: A COSMIN systematic review of the psychometric and administrative properties of adult retrospective self-report instruments on child abuse and neglect. Trauma, Violence, & Abuse, 24(3), 1832–1847. https://doi.org/10.1177/15248380221145912
49.
SwitzerS.FlickerS.McClellandA.CarusoneS. C.FergusonT. B.HerelleN.YeeD.GutaA.StrikeC. (2020). Journeying together: A visual exploration of “engagement” as a journey in HIV programming and service delivery. Health & Place, 61(3), 455. https://doi.org/10.1016/j.healthplace.2019.102247
50.
TerweeC. B.BotS. D. M.de BoerM. R.van der WindtD. A. W. M.KnolD. L.DekkerJ.BouterL. M.de VetH. C. W. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34–42. https://doi.org/10.1016/j.jclinepi.2006.03.012
51.
ThompsonV. L. S.LeahyN.AckermannN.BowenD. J.GoodmanM. S. (2020). Community partners’ responses to items assessing stakeholder engagement: Cognitive response testing in measure development. PLoS One, 15(11), 22. https://doi.org/10.1371/journal.pone.0241839
VanArsdallE. (2009). [Review of Handbook of usability testing: How to Plan, design, and conduct effective tests, 2nd ed., RubinJ.ChisnellD.]. Technical Communication, 56(1), 70–71. http://www.jstor.org/stable/43095435
54.
WolfC.JoyeD.SmithT. W.FuY. (2016). The Sage handbook of survey methodology. Sage Publications Ltd.
55.
YasuiM.PottickK. J.ChenY. (2017). Conceptualizing culturally infused engagement and its measurement for ethic minority and immigrant children and families. Clinical Child and Family Psychology Review, 20(3), 250–332. https://doi.org/10.1007/s10567-017-0229-2
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
