Mixed methods research (MMR) is versatile, pragmatic, and adaptable to constraints and opportunities during a research process. Although MMR has gain popularity in health services management research, little is known about how the research approach has been used and the quality of research. We conducted a systematic review of 198 MMR articles published in selected U.S.-based and international health services management journals from 2000 through 2018 to examine the extent of MMR application and scientific rigor. Results showed limited, yet increasing, use of MMR and a high degree of correspondence between MMR designs and study purposes. However, most articles did not clearly justify using MMR designs and the reporting of method details and research integration were inadequate in a significant portion of publications. We propose a checklist to assist the preparation and review of MMR manuscripts. Additional implications and recommendations to improve transparency, rigor, and quality in MMR are discussed.
AguinisH.RamaniR. S.AlabduljaderN. (2018). What you see is what you get? Enhancing methodological transparency in management research. Academy of Management Annals, 12(1), 83-110. https://doi.org/10.5465/annals.2016.0011
2.
AllenH.WrightB. J.HardingK.BroffmanL. (2014). The role of stigma in access to health care for the poor. Milbank Quarterly, 92(2), 289-318. https://doi.org/10.1111/1468-0009.12059
3.
BazosD. A.LaFaveL. R. A.SureshG.ShannonK. C.NuwahaF.SplaineM. E. (2015). The gas cylinder, the motorcycle and the village health team member: A proof-of-concept study for the use of the microsystems quality improvement approach to strengthen the routine immunization system in Uganda. Implementation Science, 10(1), Article 30. https://doi.org/10.1186/s13012-015-0215-3
CampbellL.LovingT. J.LebelE. P. (2014). Enhancing transparency of the research process to increase accuracy of findings: A guide for relationship researchers. Personal Relationships, 21(4), 531-545. https://doi.org/10.1111/pere.12053
6.
CastroF. G.KellisonJ. G.BoydS. J.KopakA. (2010). A methodology for conducting integrative mixed methods research and data analyses. Journal of Mixed Methods Research, 4(4), 342-360. https://doi.org/10.1177/1558689810382916
7.
CollinsK. M. T.OnwuegbuzieA. J.JiaoQ. G. (2007). A mixed methods investigation of mixed methods sampling designs in social and health science research. Journal of Mixed Methods Research, 1(3), 267-294. https://doi.org/10.1177/1558689807299526
8.
CreswellJ. W.(2015). A concise introduction to mixed methods research. Sage.
9.
CreswellJ. W.KlassenA. C.Plano ClarkV. L.SmithK. C. (2011). Best practices for mixed methods research in the health sciences. National Institute of Health.
10.
CreswellJ. W.Plano ClarkV. L. (2017). Designing and conducting mixed methods research (3rd ed.). Sage.
11.
CurryL. A.NumbhardI. M.BradleyE. H. (2009). Qualitative and mixed methods provide unique contributions to outcomes research. Circulation, 119(10), 1442-1452. https://doi.org/10.1161/CIRCULATIONAHA.107.742775
12.
EvansB. A.SnooksH.HowsonH.DaviesM. (2013). How hard can it be to include research evidence and evaluation in local health policy implementation? Results from a mixed methods study. Implementation Science, 8(1), 1-9. https://doi.org/10.1186/1748-5908-8-17
13.
FettersM. D.CurryL. A.CreswellJ. W. (2013). Achieving integration in mixed methods designs: Principles and practices. Health Services Research, 48(6pt2), 2134-2156. https://doi.org/10.1111/1475-6773.12117
14.
FettersM. D.FreshwaterD. (2015). Publishing a methodological mixed methods research article. Journal of Mixed Methods Research, 9(3), 203-213. https://doi.org/10.1177/1558689815594687
15.
FinleyE. P.HuynhA. K.FarmerM. M.Bean-MayberryB.MoinT.OishiS. M.MoreauJ. L.DyerK. E.LanhamH. J.LeyhumL.HamiltonA. B. (2018). Periodic reflections: A method of guided discussion for documenting implementation phenomena. BMC Medical Research Methodology, 18, 153. https://doi.org/10.1186/s12874-018-0610-y
16.
GilmerT. P.KatzM. L.StefancicA.PalinkasL. A. (2013). Variation in the implementation of California’s full service partnerships for persons with serious mental illness. Health Services Research, 48(6pt2), 2245-2267. https://doi.org/10.1111/1475-6773.12119
17.
GilsonL.HansonK.SheikhK.AgyepongI. A.SsengoobaF.BennettS. (2011). Building the field of health policy and systems research: Social science matters. PLOS MEDICINE, 8(8), e1001079. https://doi.org/10.1371/journal.pmed.1001079
18.
GreeneJ. C.CaracelliV. J.GrahamW. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274. https://doi.org/10.3102/01623737011003255
19.
HadiM. A.AlldredD. P.ClossS. J.BriggsM. (2013). Mixed-methods research in pharmacy practice: Basics and beyond (Part 1). International Journal of Pharmacy Practice, 21(5), 341-345. https://doi.org/10.1111/ijpp.12010
HusseinA. (2009). The use of triangulation in social sciences research: Can qualitative and quantitative methods be combined?Journal of Comparative Social Work, 4(1), 1-12. https://doi.org/10.31265/jcsw.v4i1.48
23.
JohnsonR. B.OnwuegbuzieA. J.TurnerL. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133. https://doi.org/10.1177/1558689806298224
24.
LeechN. L.OnwuegbuzieA. J. (2009). A typology of mixed methods reearch designs. Quality & Quantity: International Journal of Methodology, 43(2), 265-275. https://doi.org/10.1007/s11135-007-9105-3
25.
LeechN. L.OnwuegbuzieA. J. (2010). Guidelines for conducting and reporting mixed research in the field of counseling and beyond. Journal of Counseling & Development, 89(1), 61-70. https://doi.org/10.1002/j.1556-6678.2010.tb00151.x
26.
LeechN. L.OnwuegbuzieA. J.CombsJ. P. (2011). Writing publishable mixed research articles: Guidelines for emerging scholars in the health sciences and beyond. Mixed Methods Research in the Health Sciences, 5(1), 7-24. https://doi.org/10.5172/mra.2011.5.1.7
27.
McCulloughJ.Eisen-CohenE.LottB. (2018). Barriers and facilitators to intraorganizational collaboration in public health: Relational coordination across public health services targeting individuals and populations. Health Care Management Review, 45(1), 60-72. https://doi.org/10.1097/HMR.0000000000000203
28.
MeadH.RamosC.GranthamS. C. (2016). Drivers of racial and ethnic disparities in cardiac rehabilitation use: Patient and provider perspectives. Medical Care Research and Review, 73(3), 251-182. https://doi.org/10.1177/1077558715606261
29.
O’CathainA.MurphyE.NichollJ. (2008). The quality of mixed methods studies in health services research. Journal of Health Services Research & Policy, 13(2), 92-98. https://doi.org/10.1258/jhsrp.2007.007074
30.
O’CathainA.MurphyE.NichollJ. (2010). Three techniques for integrating data in mixed methods studies. British Medical Journal, 341, c4587. https://doi.org/10.1136/bmj.c4587
31.
RiddeV.Olivier de SardanJ. (2015). A mixed methods contribution to the study of health public policies: Complementarities and difficulties. BMC Health Services Research, 15(Suppl. 3), Article S7. https://doi.org/10.1186/1472-6963-15-S3-S7
32.
ShoreS.HoP. M.Lambert-KerznerA.GloriosoT. J.CareyE. P.CunninghamF.LongoL.JackeviciusC.RoseA.TurakhiaM. P. (2015). Site-level variation in and practices associated with dabigatran adherence. JAMA, 313(14), 1443-1450. https://doi.org/10.1001/jama.2015.2761
33.
StangeK. C.CrabtreeB. F.MillerW. L. (2006). Publishing multimethod research. Annals of Family Medicine, 4(4), 292-294. https://doi.org/10.1370/afm.615
34.
Tapia-ConyerR.Saucedo-MartinezR.Mujica-RosalesR.Gallardo-RinconH.Campos-RiveraP. A.LeeE.WaughC.GuajardoL.Torres-BeltranB.Quijano-GonzalezU.Soni-GallardoL. (2016). Enablers and inhibitors of the implementation of the Casalud model, a Mexican innovative healthcare model for non-communicable disease prevention and control. Health Research Policy and Systems, 14(1), 1-12. https://doi.org/10.1186/s12961-016-0125-0
35.
TashakkoriA.TeddlieC. (2003). Handbook of mixed methods in social & behavioral research. Sage.
36.
ThononF.BoulkedidR.TeixeiraM.GottotS.SaghatchianM.AlbertiC. (2015). Identifying potential indicators to measure the outcome of translational cancer research: A mixed methods approach. Health Research Policy and Systems, 13(1), 72. https://doi.org/10.1186/s12961-015-0060-5
WeinerB. J.AmickH. R.LundJ. L.LeeS.-Y. D.HoffT. J. (2011). Use of qualitative methods in published health services and management research: A ten-year review. Medical Care Research and Review, 68(1), 3-33. https://doi.org/10.1177/1077558710372810
39.
WisdomJ. P.CavaleriM. A.OnwuegbuzieA. J.GreenC. A. (2012). Methodological reporting in qualitative, quantitative, and mixed methods health services research articles. Health Services Research, 47(2), 721-745. https://doi.org/10.1111/j.1475-6773.2011.01344.x
