Process Evaluation of a Retreat for Scholars in the First Cohort: The NIH Mixed Methods Research Training Program for the Health Sciences

Abstract

The Mixed Methods Research Training Program for the Health Sciences aims to enlarge the national pool of trained investigators in mixed methods and improve the quality of grant applications to the National Institutes of Health. Selected scholars are assigned a consulting team, participate in webinars, and attend an annual “retreat” focused on learning mixed methods through application to their research. Our article summarizes the process evaluation of the retreat. Scholars identified strengths in small interactive groups to discuss individual projects and the opportunity to apply learning. Scholars wanted further opportunity to discuss individual projects, understanding interventions and mixed methods, and finding collaborators. Our findings will be useful to leaders developing workshops or similar programs at the faculty level.

Keywords

health sciences methodological training mixed methods training student perspective

Get full access to this article

View all access options for this article.

References

Board on Life Sciences Bridges to Independence. (2005). Fostering the independence of new investigators in biomedical research. Washington, DC: National Academies Press.

Bruce

M. L.

Bartels

S. J.

Lyness

J. M.

Sirey

J. A.

Sheline

Y. I.

Smith

(2011). Promoting the transition to independent scientist: A national career development program. Academic Medicine, 86, 1179-1184. doi:10.1097/ACM.0b013e3182254399

Bruning

R. H.

Schraw

G. J.

Norby

M. M.

(2011). Cognitive psychology and instruction (5th ed.). Boston, MA: Pearson.

Christ

T. W.

(2009). Designing, teaching, and evaluating two complementary mixed methods research courses. Journal of Mixed Methods Research, 3(4), 292-325. doi:10.1177/1558689809341796

Coyle

C. E.

Schulman-Green

Feder

Toraman

Prust

M. L.

Plano Clark

V. L.

Curry

(2016). Federal Funding for Mixed Methods Research in the Health Sciences in the United States: Recent Trends. Journal of Mixed Methods Research. doi: 10.1177/1558689816662578

Creswell

J. W.

(2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Thousand Oaks, CA: Sage.

Creswell

J. W.

(2015). 30 Essential skills for the qualitative researcher. Thousand Oaks, CA: Sage.

Creswell

J. W.

Klassen

A. C.

Plano Clark

V. L.

Smith

K. C.

(2011). Best practices for mixed methods research in the health sciences. Washington, DC: National Institutes of Health.

Creswell

J. W.

Tashakkori

Jensen

K. D.

Shapley

(2003). Teaching mixed methods research: Practices, dilemmas, and challenges. In Tashakkori

Teddlie

(Eds.), Handbook of mixed methods in social & behavioral research (pp. 619-638). Thousand Oaks, CA: Sage.

10.

Curry

Nunez-Smith

(2014). Mixed methods in health sciences research: A practical primer. Thousand Oaks, CA: Sage.

11.

Dahlberg

Wittink

M. N.

Gallo

J. J.

(2010). Funding and publishing integrated studies: Writing effective mixed methods manuscripts and grant proposals. In Tashakkori

Teddlie

(Eds.), Handbook of mixed methods in social & behavioral research (pp. 775-802). Thousand Oaks, CA: Sage.

12.

Damschroder

L. J.

Aron

D. C.

Keith

R. E.

Kirsh

S. R.

Alexander

J. A.

Lowery

J. C.

(2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. doi:10.1186/1748-5908-4-50

13.

Datta

(1997). A pragmatic basis for mixed-methods designs. New Directions for Evaluation, 74, 33-46.

14.

Denzin

N. K.

(1978). The research act: A theoretical introduction to sociological methods. New York, NY: McGraw-Hill.

15.

Earley

M. A.

(2007). Developing a syllabus for a mixed-methods research course. International Journal of Social Research Methodology, 10, 145-162. doi:10.1080/13645570701334118

16.

Epstein

R. M.

Hundert

E. M.

(2002). Defining and assessing professional competence. Journal of the American Medical Association, 287, 226-235.

17.

Fink

L. D.

(2013). Creating significant learning experiences: An integrated approach to designing college courses (2nd ed.). Somerset, NJ: Jossey-Bass.

18.

Frels

R. K.

Newman

(2015). Mentoring the next generation in mixed methods research. In Hesse-Biber

Johnson

R. B.

(Eds.), The Oxford handbook of multimethod and mixed methods research inquiry (pp. 333-353). New York, NY: Oxford.

19.

Frels

R. K.

Onwuegbuzie

A. J.

Leech

N. L.

Collins

K. M. T.

(2012). Challenges to teaching mixed research courses. Journal of Effective Teaching, 12(2), 23-44.

20.

Frels

R. K.

Onwuegbuzie

A. J.

Leech

N. L.

Collins

K. M. T.

(2014). Pedagogical strategies used by selected leading mixed methodologists in mixed research courses. Journal of Effective Teaching, 14(2), 5-34.

21.

Greene

J. C.

(2010). Foreword: Beginning the conversation. International Journal of Multiple Research Approaches, 4, 2-5. doi:10.5172/mra.2010.4.1.002

22.

Guetterman

Creswell

J. W.

Wittink

M. N.

Barg

F. K.

Castro

Dahlberg

… Gallo

J. J.

(2016). Development of a self-rated mixed methods skills assessment: The NIH Mixed Methods Research Training Program for the Health Sciences. Manuscript in preparation.

23.

Hansson

(2010). Program implementation in health services. International Journal of Multiple Research Approaches, 4, 40-48.

24.

Jones

D. P.

Tucker-Allen

(2000). Mentor/mentee relationship with the focus on meeting promotion/tenure guidelines. ABNF Journal, 11, 113-116.

25.

Kuckartz

(2014). Qualitative text analysis: A guide to methods, practice and using software. London: Sage.

26.

Lincoln

Y. S.

Guba

E. G.

(1985). Naturalistic inquiry. Beverly Hills, CA: Sage.

27.

McKim

C. A.

(2015). The value of mixed methods research: A mixed methods study. Journal of Mixed Methods Research. Advance online publication. doi:10.1177/1558689815607096

28.

National Academy of Science. (2000). Addressing the nation’s changing needs for biomedical and behavioral scientists. Washington, DC: National Academies Press.

29.

National Advisory Mental Health Council Workgroup on Research Training. (2008). Investing in the future. Retrieved from http://www.nimh.nih.gov/about/advisory-boards-and-groups/namhc/reports/investing-in-the-future_42525.pdf

30.

National Institutes of Health. (2001). Qualitative methods in health research: Opportunities and considerations in application and review. Washington, DC: Author.

31.

O’Cathain

Murphy

Nicholl

(2007). Integration and publications as indicators of “yield” from mixed methods studies. Journal of Mixed Methods Research, 1(2), 147-163. doi:10.1177/1558689806299094

32.

Onwuegbuzie

A. J.

Leech

N. L.

(2007). Validity and qualitative research: An oxymoron? Quality & Quantity, 41, 233-249. doi:10.1007/s11135-006-9000-3

33.

Plano Clark

V. L.

(2010). The adoption and practice of mixed methods: U.S. trends in federally funded health-related research. Qualitative Inquiry, 16, 428-440. doi:10.1177/1077800410364609

34.

Poth

(2014). What constitutes effective learning experiences in a mixed methods research course? An examination from the student perspective. International Journal of Multiple Research Approaches, 8, 74-86.

35.

Ramanan

R. A.

Taylor

W. C.

Davis

R. B.

Phillips

R. S.

(2006). Mentoring matters: Mentoring and career preparation in internal medicine residency training. Journal of General Internal Medicine, 21, 340-345. doi:10.1111/j.1525-1497.2006.00346.x

36.

Reynolds

H. Y.

(2007). Mentoring: Nurturing clinician and physician scientists in an academic career. Pharos of Alpha Omega Alpha Honor Medical Society, 70(3), 26-28.

37.

Reynolds

H. Y.

(2008). In choosing a research health career, mentoring is essential. Lung, 186, 1-6. doi:10.1007/s00408-007-9050-x

38.

Rossi

P. H.

Lipsey

M. W.

Freeman

H. E.

(2004). Evaluation: A systematic approach. Thousand Oaks, CA: Sage.

39.

Saint Arnault

Fetters

M. D.

(2011). RO1 funding for mixed methods research: Lessons learned from the “Mixed-Method Analysis of Japanese Depression” project. Journal of Mixed Methods Research, 5(4), 309-329. doi:10.1177/1558689811416481

40.

Sung

N. S.

Crowley

W. F.

Jr. Genel

Salber

Sandy

Sherwood

L. M.

… Rimoin

(2003). Central challenges facing the national clinical research enterprise. Journal of the American Medical Association, 289, 1278-1287.

41.

Varki

Rosenberg

L. E.

(2002). Emerging opportunities and career paths for the young physician-scientist. Nature Medicine, 8, 437-439. doi:10.1038/nm0502-437

42.

Weinert

C. R.

Billings

Ryan

Ingbar

D. H.

(2006). Academic and career development of pulmonary and critical care physician-scientists. American Journal of Respiratory and Critical Care Medicine, 173, 23-31. doi:10.1164/rccm.200503-325OC

43.

Wisdom

J. P.

Fetters

M. D.

(2015). Funding for mixed methods research: Sources and strategies. In Hesse-Biber

Johnson

R. B.

(Eds.), The Oxford handbook of multimethod and mixed methods research inquiry (pp. 314-332). New York, NY: Oxford.

44.

Zerhouni

E. A.

(2005). Translational and clinical science—Time for a new vision. New England Journal of Medicine, 353, 1621-1623.