Changing health-care climates mean evaluators need to provide findings within shorter time frames, but challenges remain in the creation of rapid research designs capable of delivering quality data to inform decision-making processes. We conducted a review of articles to grapple with these challenges and explore the ways in which rapid evaluations have been used in health care. We found different labels being used to define rapid evaluations and identified a trend in the design of evaluations, where evaluators are moving away from short studies to longer evaluations with multiple feedback loops or cyclical stages. Evaluators are using strategies to speed up evaluations: conducting data collection and analysis in parallel, eliminating the use of transcripts, and utilizing larger evaluation teams to share the workload. Questions persist in relation to the suitability of rapid evaluation designs, the trustworthiness of the data, and the degree to which evaluation findings are used to make changes in practice.
AnkerM.GuidottiR. J.OrzeszynaS.SapirieS. A.ThuriauxM. C. (1993). Rapid evaluation methods (REM) of health services performance: Methodological observations. Bulletin of the World Health Organization, 71, 15–21.
2.
AsprayT. J.NesbitK.CassidyT. P.HawthorneG. (2006). Rapid assessment methods used for health-equity audit: Diabetes mellitus among frail British care-home residents. Public Health, 120, 1042–1051.
Bjornson-BensonW. M.StiboltT. B.ManskeK. A.ZavelaK. J.YoutseyD. J.BuistA. S. (1993). Monitoring recruitment effectiveness and cost in a clinical trial. Controlled Clinical Trials, 14, 52S–67S.
5.
ChowdhuryS. N. M.MoniD. (2004). A situation analysis of the menstrual regulation programme in Bangladesh. Reproductive Health Matters, 12, 95–104.
6.
CroweM.SheppardL. (2011). A review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed. Journal Clinical Epidemiology, 64, 79–89.
7.
FelisbertoE.FreeseE.NatalS.AlvesC. K. (2008). A contribution to institutionalized health evaluation: A proposal for self-evaluation. Cadernos de saude publica, 24, 2091–2102.
8.
GaleN.HealthG.CameronE.RashidS.RedwoodS. (2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13, 117.
9.
GaleR.WuJ.ErhardtT.BounthavongM.ReardonC.DamschroderL.MidboeA. (2019). Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans health administration. Implementation Science, 14, 11.
10.
GlasgowR.KesslerR. S.OryM. G.RobyD.GorinS. S.KristA. (2014). Conducting rapid, relevant research: Lessons learned from the My Own Health Report project. American Journal Preventive Medicine, 47, 212–219.
11.
GoughD.ThomasJ.OliverS. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1, 28–37.
12.
GrantL.BrownJ.LengM.BettegaN.MurrayS. A. (2011). Palliative care making a difference in rural Uganda, Kenya and Malawi: Three rapid evaluation field studies. BMC Palliative Care, 10, 8.
13.
GrassoP. (2003). What makes an evaluation useful? Reflections from experience in large organizations. American Journal of Evaluation, 24, 507–514.
14.
HargreavesM. B. (2014). Rapid evaluation approaches for complex initiatives. Mathematica Policy Research. White Paper, March 31, 2014, submitted to U.S. Department of Health and Human Services, Assistant Secretary for Planning and Evaluation.
15.
JohnsonG.A.Vindrola-PadrosC. (2017). Rapid qualitative research methods during complex health emergencies: A systematic review of the literature. Social Science & Medicine, 189, 63–75.
16.
KeithR.CrossonJ. C.O’MalleyA. S.CrompD.TaylorE. F. (2017). Using the consolidated framework for implementation research (CFIR) to produce actionable findings: A rapid-cycle evaluation approach to improving implementation. Implementation Science, 12, 15.
17.
LandisJ. R.KochG. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159.
18.
McNallM.Foster-FishmanP. (2007). Methods of rapid evaluation, assessment, and appraisal. American Journal of Evaluation, 28, 151–168.
19.
McNallM.WelchV.RuhK.MildnerC.SotoT. (2004). The use of rapid-feedback evaluation methods to improve the retention rates of an HIV/AIDS healthcare intervention. Evaluation and Program Planning, 27, 287–294.
20.
MoherD.LiberatiA.TetzlaffJ.AltmanD. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. The British Medical Journal, 339, 332–336.
21.
MundayD.HaraldsdottirE.ManakM.ThyleA.RatcliffC. M. (2018). Rural palliative care in North India: Rapid evaluation of a program using a realist mixed method approach. Indian Journal Palliative Care, 24, 3–8.
22.
NunnsH. (2009). Responding to the demand for quicker evaluation findings. Social Policy Journal of New Zealand, 34, 89–99.
23.
O’CathainA. (2010). Assessing the quality of mixed methods research: Towards a comprehensive framework. In TashakkoriA.TeddieC. (Eds.), Handbook of mixed methods in social and behavioural research (pp. 531–555). Thousand Oaks, CA: Sage.
24.
O’CathainA.MurphyE.NichollJ. (2008). The quality of mixed methods studies in health services research. Journal Health Service Research Policy, 13, 92–98.
PluyeP.BartlettG.MacaulayA. C.SalsbergJ.JagoshJ.SellerR. (2012). Testing the reliability and efficiency of the pilot mixed methods appraisal tool (MMAT) for systematic mixed studies review. International Journal Nursing Studies, 49, 47–53.
27.
PluyeP.HongQ. N. (2014). Combining the power of stories and the power of numbers: Mixed methods research and mixed studies reviews. Annals Review Public Health, 35, 29–45.
28.
PluyeP.RobertE.CargoM.BartlettG.O’CathainA.GriffithsF.RousseauM. C. (2011). Proposal: A mixed methods appraisal tool for systematic mixed studies reviews. Retrieved August 31, 2019, fromhttp://mixedmethodsappraisaltoolpublic.pbworks.com
29.
RileyW.GlasgowR. E.EtheredgeL.AbernethyA. P. (2013). Rapid, responsive, relevant (R3) research: A call for a rapid learning health research enterprise. Clinical and Translational Science, 2, 10.
30.
SandisonP. (2003). Desk-based review of real-time evaluation experience. New York, NY: United Nations Children’s Fund.
31.
SchneeweissS.ShrankW. H.RuhlM.MaclureM. (2015). Decision-making aligned with rapid-cycle evaluation in health care. International Journal of Technology Assessment in Health Care, 31, 214–222.
32.
ShrankW. (2013). The Center For Medicare And Medicaid Innovation’s blueprint for rapid-cycle evaluation of new care and payment models. Health Affairs, 32, 807–812.
33.
SkillmanM.Cross-BarnetC.Friedman SingerR.RotondoC.RuizS.MoiduddinA. (2019). A framework for rigorous qualitative research as a component of mixed method rapid-cycle evaluation. Qualitative Health Research, 29, 279–289.
34.
SonnichsenR. (2000). High impact internal evaluation: A practitioner’s guide to evaluation and consulting inside organizations. Thousand Oaks, CA: SAGE.
35.
TaylorB.HenshallC.KenyonS.LitchfieldI.GreenfieldS. (2018). Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open, 8, e019993.
36.
Vindrola-PadrosC.Vindrola-PadrosB. (2018). Quick and dirty? A systematic review of the use of rapid ethnographies in healthcare organisation and delivery. BMJ Quality & Safety, 27, 321–330.
37.
WholeyJ. S. (1983). Evaluation and effective public management. Boston, MA: Little Brown.
38.
ZakocsR.HillJ. A.BrownP.WheatonJ.FreireK. E. (2015). The data-to-action framework: A rapid program improvement process. Health Education & Behavior, 42, 471–479.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
