Accelerating Mixed Methods Research With Natural Language Processing of Big Text Data

Abstract

Situations of catastrophic social change, such as COVID-19, raise complex, interdisciplinary research questions that intersect health, education, economics, psychology, and social behavior and require mixed methods research. The pandemic has been a quickly evolving phenomenon, which pressures the time necessary to perform mixed methods research. Natural language processing (NLP) is a promising solution that leverages computational approaches to analyze textual data in “natural language.” The aim of this article is to introduce NLP as an innovative technology to assist with the rapid mixed methods analysis of textual big data in times of catastrophic change. The contribution of this article is illustrating how NLP is a type of mixed methods analysis and making recommendations for its use in mixed methods research.

Keywords

mixed methods research natural language processing qualitative analysis content analysis big data

Get full access to this article

View all access options for this article.

References

Baumer

E. P.

Mimno

Guha

Quan

Gay

G. K.

(2017). Comparing grounded theory and topic modeling: Extreme divergence or unlikely convergence? Journal of the Association for Information Science and Technology, 68(6), 1397-1410. https://doi.org/10.1002/asi.23786

Bazeley

(2018). Integrating analyses in mixed methods research. Sage.

Boyatzis

R. E.

(1998). Transforming qualitative information: Thematic analysis and code development. Sage.

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Colditz

J. B.

Welling

Smith

N. A.

James

A. E.

Primack

B. A.

(2017). World Vaping Day: Contextualizing vaping culture in online social media using a mixed methods approach. Journal of Mixed Methods Research, 13(2), 196-215. https://doi.org/10.1177/1558689817702753

Creswell

J. W.

Plano Clark

V. L.

(2018). Designing and conducting mixed methods research (3rd ed.). Sage.

Crowston

Allen

E. E.

Heckman

(2012). Using natural language processing technology for qualitative data analysis. International Journal of Social Research Methodology, 15(6), 523-543. https://doi.org/10.1080/13645579.2011.625764

DeJonckheere

Nichols

L. P.

Moniz

M. H.

Sonneville

K. R.

Vydiswaran

V. G. V.

Zhao

Guetterman

T. C.

Chang

(2017). MyVoice national text message survey of youth aged 14 to 24 years: study protocol. JMIR Research Protocol, 6(12), Article e247. https://doi.org/10.2196/resprot.8502

Devlin

Chang

M.-W.

Lee

Toutanova

(2018). BERT: Pre-training of deep bidirectional transformers for language understanding. Paper presented at the Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, MN. https://www.aclweb.org/anthology/N19-1423.pdf

10.

Fetters

M. D.

(2020). The mixed methods research workbook: Activities for designing, implementing, and publishing projects. Sage.

11.

Guetterman

Creswell

J. W.

Kuckartz

(2015). Using joint displays and MAXQDA software to represent the results of mixed methods research. In McCrudden

Schraw

Buckendahl

(Eds.), Use of visual displays in research and testing: Coding, interpreting, and reporting data (pp. 145-175). Information Age Publishing.

12.

Guetterman

T. C.

Chang

DeJonckheere

Basu

Scruggs

Vydiswaran

V. G. V.

(2018). Augmenting qualitative text analysis with natural language processing: methodological study. Journal Medical Internet Research, 20(6), Article e231. https://doi.org/10.2196/jmir.9702

13.

Hannigan

T. R.

Haans

R. F.

Vakili

Tchalian

Glaser

V. L.

Wang

M. S.

Kaplan

Jennings

P. D.

(2019). Topic modeling in management research: Rendering new theory from textual data. Academy of Management Annals, 13(2), 586-632. https://doi.org/10.5465/annals.2017.0099

14.

Hsieh

H.-F.

Shannon

S. E.

(2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288. https://doi.org/10.1177/1049732305276687

15.

Jordan

M. I.

Mitchell

T. M.

(2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255-260. https://doi.org/10.1126/science.aaa8415

16.

Johnson

S. C.

(1967). Hierarchical clustering schemes. Psychometrika, 32(3), 241-254. https://doi.org/10.1007/BF02289588

17.

Kuckartz

(2014). Qualitative text analysis: A guide to methods, practice and using software. Sage.

18.

Duan

Wang

Zhang

Sun

(2017). Enhancing topic modeling for short texts with auxiliary word embeddings. ACM Transactions on Information Systems (TOIS), 36(2), 1-30. https://doi.org/10.1145/3091108

19.

Wang

Zhang

Sun

(2016). Topic modeling for short texts with auxiliary word embeddings. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval (pp. 165-174). https://doi.org/10.1145/2911451.2911499

20.

Loper

Bird

(2002). NLTK: The natural language toolkit. https://arxiv.org/pdf/cs/0205028.pdf

21.

Mikolov

Chen

Corrado

Dean

(2013). Efficient estimation of word representations in vector space. https://arxiv.org/abs/1301.3781

22.

Miller

G. A.

(1995). WordNet: A lexical database for English. Communications of the ACM, 38(11), 39-41. https://doi.org/10.1145/219717.219748

23.

Németh

Sik

Máté

(2020). Machine learning of concepts hard even for humans: The case of online depression forums. International Journal of Qualitative Methods, 19, Article 9338. https://doi.org/10.1177/1609406920949338

24.

O’Halloran

K. L.

Tan

Pham

D.-S.

Bateman

Vande Moere

(2016). A digital mixed methods research design: Integrating multimodal analysis with data mining and information visualization for Big Data Analytics. Journal of Mixed Methods Research, 12(1), 11-30. https://doi.org/10.1177/1558689816651015

25.

Onwuegbuzie

A. J.

Collins

K. M.

(2007). A typology of mixed methods sampling designs in social science research. Qualitative Report, 12(2), 281-316.

26.

Pennington

Socher

Manning

C. D.

(2014). GloVe: Global vectors for word representation. Paper presented at the Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Doha, Qatar.

27.

Renz

S. M.

Carrington

J. M.

Badger

T. A.

(2018). Two strategies for qualitative content analysis: An intramethod approach to triangulation. Qualitative Health Research, 28(5), 824-831. https://doi.org/10.1177/1049732317753586

28.

Silver

Lewins

(2014). Using software in qualitative research: A step-by-step guide (2nd ed.). Sage.

29.

Stenetorp

Pyysalo

Topić

Ohta

Ananiadou

Tsujii

J. i.

(2012). BRAT: A web-based tool for NLP-assisted text annotation. Paper presented at the Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics, Minneapolis, MN. https://www.aclweb.org/anthology/E12-2021

30.

Stephens

(2007). Collecting data from elites and ultra elites: Telephone and face-to-face interviews with macroeconomists. Qualitative Research, 7(2), 203-216. https://doi.org/10.1177/1468794107076020

31.

Tashakkori

Johnson

R. B.

Teddlie

(2020). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage.

32.

Taylor

Henshall

Kenyon

Litchfield

Greenfield

(2018). Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open, 8(10), Artcle e019993. https://doi.org/10.1136/bmjopen-2017-019993

33.

The Biosemantics Group. (2007). Jane/Journal Author Name Estimator. https://jane.biosemantics.org

34.

U.S. Department of Health and Human Services. (2020). Telehealth: Delivering care safely during COVID-19. https://www.hhs.gov/coronavirus/telehealth/index.html

35.

van der Maaten

Hinton

. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9(November), 2579-2605. https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf

36.

Veinot

T. C.

Mitchell

Ancker

J. S.

(2018). Good intentions are not enough: How informatics interventions can worsen inequality. Journal of the American Medical Informatics Association, 25(8), 1080-1088. https://doi.org/10.1093/jamia/ocy052

37.

Verdugo-Castro

Sánchez-Gómez

M. C.

García-Holgado

Costa

A. P.

(2019). Mixed methods and visual representation of data with CAQDAS: Empirical study. In Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality (pp. 511-517). Association for Computing Machinery.

38.

Watkins

D. C.

(2017). Rapid and rigorous qualitative data analysis: The “RADaR” technique for applied research. International Journal of Qualitative Methods, 16(1), 1-9. https://doi.org/10.1177/1609406917712131