Generative AI: Is Authentic Qualitative Research Data Collection Possible?

Abstract

With the advent of readily accessible generative artificial intelligence (GAI), a concern exists within the academic community that research data collected in the context of conducting doctoral dissertation research is authentic. The purpose of the present study was to explore the role of GAI in the production of new research paying particular attention to the use of GAI in collecting new data for a doctoral dissertation. This study employed qualitative methodology examining how GAI, specifically ChatGPT, responded to interview questions from a previously published article by the researchers to determine how closely chatbots mimic responses from the actual study participants. The researchers found that data integrity in qualitative research may be at risk if higher education institutions do not set clear policies and specific parameters for how doctoral research data is obtained and validated in light of GAI.

Keywords

artificial intelligence chatbots communication conversation agents data validity doctoral research generative AI

Get full access to this article

View all access options for this article.

References

Alexander

(2024, January 10). AI in higher education: Implications for learning, curricula, and institutions [Webinar]. AAC& U. https://www.aacu.org/webinars/ai-and-higher-education

Arif

T. B.

Munaf

Ul-Haque

(2023). The future of medical education and research: Is ChatGPT a blessing or blight in disguise? Medical Education Online, 28(1). https://doi.org/10.1080/10872981.2023.2181052

Barabino

G. A.

(2024, January 10). AI in higher education: Implications for learning, curricula, and institutions [Webinar]. AAC& U. https://www.aacu.org/webinars/ai-and-higher-education

Benner

Hughes

R. G.

Sutphen

(2008). Clinical reasoning, decision making, and action: Thinking critically and clinically. In Hughes

R. G.

(Ed.), Patient safety and quality: An evidence-based handbook for nurses (pp. 87–109). Agency for Healthcare Research and Quality.

Blackman

(2021, April 1). If your company uses AI, it needs an institutional review board . Harvard Business Review. https://hbr.org/2021/04/if-your-company-uses-ai-it-needs-an-institutional-review-board

Browne

(2023, March 6). Elon Musk, who co-founded firm behind ChatGPT, warns A.I. is ‘one of the biggest risks’ to civilization . CNBC. https://www.cnbc.com/2023/02/15/elon-musk-co-founder-of-chatgpt-creator-openai-warns-of-ai-society-risk.html

Burleigh

Wilson

A. M.

(2022). Mobile technology acceptable use policies and teaching in high school classrooms: Do boundaries exist? International Journal of Multidisciplinary and Current Educational Research, 4(1), 71–86. https://www.ijmcer.com/wp-content/uploads/2022/01/IJMCER_I041071086.pdf

Chan

C. K. Y.

(2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education, 20(38). https://doi.org/10.1186/s41239-023-00408-3

ChatGPT and generative AI in the classroom. (n.d.). Office of the Vice-Provost, Innovations in Undergraduate Education. University of Toronto. https://www.viceprovostundergrad.utoronto.ca/strategic-priorities/digital-learning/special-initiative-artificial-intelligence/

10.

Choi

E. P. H.

Lee

J. J.

M. H.

Kwok

J. Y. Y.

Lok

K. Y. W.

(2023). Chatting or cheating? The impacts of ChatGPT and other artificial intelligence language models on nurse education. Nurse Education Today, 125, 105796–105796. https://doi.org/10.1016/j.nedt.2023.105796

11.

Cotton

D. R. E.

Cotton

Shipway

J. R.

(2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International. https://doi.org/10.1080/14703297.2023.2190148

12.

Eaton

(2023). Class policies for AI tools. Northern Illinois University, Center for Innovative Teaching and Learning. https://www.niu.edu/citl/resources/guides/class-policies-for-ai-tools.shtml

13.

Foltynek

Bjelobaba

Glendinning

Khan

Z. R.

Santos

Pavletic

Kravjar

(2023). ENAI Recommendations on the ethical use of artificial intelligence in education. International Journal of Educational Integrity, 19(2023), 12. https://doi.org/10.1007/s40979-023-00133-4

14.

Fraenza

(2023, December 12). AI done rights: Ethical use of AI in research & writing. [PowerPoint Slides]. Walden University. https://event.on24.com/wcc/r/4407337/705A4A5516C468410D107D7A795DC09D

15.

Generative AI usage guidance. (n.d.). University of North Carolina at Chapel Hill, Office of the Provost. https://provost.unc.edu/generative-ai-usage-guidance-for-the-research-community/

16.

Generative artificial intelligence. (n.d.). Laurier. https://www.wlu.ca/about/governance/senior-leadership/vice-president-academic/generative-artificial-intelligence.html

17.

Grájeda

Burgos

Córdova

Sanjinés

(2024). Assessing student-perceived impact of using artificial intelligence tools: Construction of a synthetic index of application in higher education. Cogent Education, 11(1). https://doi.org/10.1080/2331186X.2023.2287917

18.

Guidance for effective and responsible use of AI in research. (2023, October 30). Georgia Tech. https://grad.gatech.edu/sites/default/files/documents/Guidance%20for%20Effective%20and%20Responsible%20Use%20of%20AI%20in%20Research.pdf

19.

Guidance on the appropriate use of generative artificial intelligence in graduate thesis. (2023, October 23). School of Graduate Studies, University of Toronto. https://www.sgs.utoronto.ca/about/guidance-on-the-use-of-generative-artificial-intelligence/

20.

Hart

(2023, March 30). Elon Musk and tech leaders call for AI ‘pause’ over risks to humanity . Forbes. https://www.forbes.com/sites/roberthart/2023/03/29/elon-musk-and-tech-leaders-call-for-ai-pause-over-risks-to-humanity/?sh=515afc7d6dfc

21.

Hill

(2023, October 31). Harnessing AI responsibly and effectively in research. Graduate Education Georgia Tech. https://grad.gatech.edu/news/harnessing-ai-responsibly-and-effectively-research

22.

Hilton

(2020). Open educational resources, student efficacy, and user perceptions: A synthesis of research published between 2015 and 2018. Educational Technology Research and Development, 68, 853–876. https://doi.org/10.1007/s11423-019-09700-4

23.

Hoefer

(2021). The surprising usefulness of the policy stages framework. Journal of Policy Practice & Research, 2, 141–145. https://doi.org/10.1007/s42972-021-00041-2

24.

Holden

O. L.

Norris

M. E.

Kuhlmeier

V. A.

(2021). Academic integrity in online assessment: A research review. Frontiers in Education, 6, 639814. https://doi.org//10.3389/feduc.2021.639814

25.

Hosseini

Rasmussen

L. M.

Resnik

D. B.

(2023). Using AI to write scholarly publications. Accountability in Research, 1–9. https://doi.org/10.1080/08989621.2023.2168535

26.

Hsieh

H. F.

Shannon

S. E.

(2005). Three approaches to qualitative content analysis . Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687

27.

Introducing GPTs. (2023, November 6). OpenAI. https://openai.com/index/introducing-gpts/

28.

Ivanov

(2023). The dark side of artificial intelligence in higher education. The Service Industries Journal, 43(15-16), 1055–1082. https://doi.org/10.1080/02642069.2023.2258799

29.

Ivanov

Soliman

(2023). Game of algorithms: ChatGPT implications for the future of tourism education and research. Journal of Tourism Futures, 9(2), 214–221. https://doi.org/10.1108/JTF-02-2023-0038

30.

Kelton

(2023, May 15). Committee to explore applications of generative AI. The Record. https://record.umich.edu/articles/committee-to-explore-applications-of-generative-ai-at-u-m/

31.

Khedkar

(2023, August 11). Guidelines for generative AI use from universities worldwide. Editage Insights. https://www.editage.com/insights/guidelines-for-generative-ai-use-from-universities-worldwide

32.

C. K.

(2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13, 410. https://doi.org/10.3390/educsci13040410

33.

MacKenzie

(2014). Global mental health from a policy perspective: A content analysis. Overseas Development Institute.

34.

McAdoo

(2024, February 23). How to cite ChatGPT. APA Style. https://apastyle.apa.org/blog/how-to-cite-chatgpt

35.

McKlveen

(2023, June 29). Think again before using generative AI during peer review or as you prspare an application. National Center for Complementary and Integratie Health. https://www.nccih.nih.gov/research/blog/think-again-before-using-generative-ai-during-peer-review-or-as-you-prepare-an-application

36.

McNutt

Hoefer

(2021). Social welfare policy: Responding to a changing world (2nd ed.). Oxford University Press.theory.

37.

Moorhouse

B. L.

Yeo

M. A.

Wan

(2023). Generative AI tools and assessment: Guidelines of the world's top-ranking universities. Computers and Education Open, 5(2023), 100151. https://doi.org/10.1016/j.caeo.2023.100151

38.

Nickerson

(2023, October 10). Albert Bandura’s social cognitive theory: Definition & examples. Simple Psychology. https://www.simplypsychology.org/social-cognitive-theory.html

39.

Roehl

J. M.

Harland

D. J.

(2022). Imposter participants: Overcoming methodological challenges related to balancing participant privacy with data quality when using online recruitment and data collection. The Qualitative Report, 27(11), 2469–2485. https://doi.org/10.46743/2160-3715/2022.5475

40.

Roth

M. S.

(2024, January 10). AI in higher education: Implications for learning, curricula, and institutions [Webinar]. AAC& U. https://www.aacu.org/webinars/ai-and-higher-education

41.

Savoy

(2023, August 15). Institutional statement on the use of generative AI. Institutional Review Board, University of Wisconsin-Madison. https://irb.wisc.edu/2023/08/15/institutional-statement-on-the-use-of-generative-ai/

42.

Scarlatos

(2024, March 7). Editorial overview: Special issue on artificial intelligence in education. Journal of Educational Technology Systems. https://doi.org/10.1177/00472395241236997

43.

Shen

Heacock

Elias

Hentel

K. D.

Reig

Shih

Moy

(2023). ChatGPT and other large language models are double-edged swords. Radiology, 307(2), e230163. https://doi.org/10.1148/radiol.230163

44.

Shoja

M. M.

Van de Ridder

J. M. M.

Rajput

(2023). The emerging role of generative artificial intelligence in medical education, research, and practice. Cureus, 15(6), e40883. https://doi.org/10.7759/cureus.40883

45.

Tools such as ChatGPT threaten transparent science; here are our ground rules for their use. (2023, January 24). Nature, 613(7945), 612. https://doi.org/10.1038/d41586-023-00191-1

46.

Watson

C. E.

(2024, January 10). AI in higher education: Implications for learning, curricula, and institutions [Webinar]. AAC& U. https://www.aacu.org/webinars/ai-and-higher-education

47.

Weatherton

Von der Mehden

B. M.

Schussler

E. E.

(2023). I don't know what I would do without it” how life science graduate students describe resource value. CBE Life Sciences Education, 22(4), ar34. https://doi.org/10.1187/cbe.22-11-0241

48.

Wilfrid Laurier University’s generative artificial intelligence committee (2023, September 23). Laurier. https://www.wlu.ca/news/spotlights/2023/sept/lauriers-generative-artificial-intelligence-committee.html

49.

Young

Mendizabal

(2009). Helping researchers become policy entrepreneurs. http://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/1730.pdf

50.

Zawacki-Richter

Marín

V. I.

Bond

Gouverneur

(2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16, 39. https://doi.org/10.1186/s41239-019-0171-0