“I Don’t Have to Write an Essay Ever Again!”: University Student Reflections on ChatGPT in the Classroom

Abstract

ChatGPT and AI programs are creating a stir on college campuses nationwide. Concerns about cheating are strong and many instructors are adopting new teaching strategies to dissuade students from using the technology in assignments. In the present study, undergraduate students in an introductory epidemiology course were assigned to use ChatGPT to produce essays about topics related to course content and, in pairs, critically analyze the resulting essays. Individually, students responded to reflection questions regarding the technology and its implications for college classrooms. In this study, those qualitative reflection responses were analyzed for themes. While a variety of viewpoints were expressed regarding expectations of the program, most students were aware of the potential for cheating but remained cautiously optimistic about the use of ChatGPT as an educational tool.

Introduction

In recent months it has been virtually impossible to avoid hearing about ChatGPT, GPT4, Open AI, and other AI tools in the media, in particular in the context of educational settings. By many accounts, AI tools like ChatGPT are entering the mainstream with a dizzying speed (Roose, 2023; Rudolph et al., 2023; Satariano, 2023). The potential impacts on education are vast, and reactions range from excitement to concern. In a piece written for the New York Times in January, 2023, Kalley Huang described instructional changes taking place in schools across the country, including colleges and universities. Some instructors are adopting pedagogical methods that make it harder for students to use AI, while others are incorporating the technology. “In higher education, colleges and universities have been reluctant to ban the A.I. tool because administrators doubt the move would be effective and they don't want to infringe on academic freedom. That means the way people teach is changing instead” (Huang, 2023, para. 10).

Academic dishonesty is a topic that permeates the educational landscape. The scholarly literature on cheating is vast. As technology rapidly advances, students are exposed to newer and more effective ways to skirt academic requirements. Scholars have advocated for using proctored assessments in online courses (Trenholm, 2007), exploring new methods of verifying student identity for authorship (Semple et al., 2010), and approaching academic dishonesty through intra- and inter-institutional collaboration (Aaron & Roche, 2013).

Student responses to the use of tools like ChatGPT vary. Some students are wary of the possibility for inaccurate content created by the tools while others are enthusiastically using it to ease their workload (Huang, 2023). In a survey of college students, 43% of respondents had used ChatGPT or similar tools, and 50% of those students had used the tool to complete assignments or tests (Welding, 2023). In the same study, about half of students said they agreed with the statement “using AI tools to complete assignments and exams constitutes cheating” and 54% said that their instructors had not openly discussed the use of ChatGPT or similar tools.

The purpose of this study was to engage students in dialogue about the use of AI tools like ChatGPT in the classroom through an assignment that linked content learning with critical reflection. In particular, the following research questions framed the study:

What expectations do students have regarding AI tools like ChatGPT for academic use?

What positive and negative implications for the classroom do students perceive with the use of tools such as ChatGPT?

How do students contextualize the use of tools such as ChatGPT within their own academic experience?

Methods

For this study, the data consisted of student work samples from an undergraduate introductory epidemiology course at a comprehensive university in the Midwest (total course enrollment was 38 students). The assignment was conducted in February, 2023. The class assignment asked students to work with a partner, enter a specific prompt into ChatGPT, and critically analyze the resulting content produced by the program (Table 1). Dyads collected quantitative data on each ChatGPT essay including number of words, sentences, references, and peer reviewed references. They also counted the number of sentences that contained factual statements, the number of sentences with factual errors, the number of sentences that did not match the references provided, the number of in-text citations, the number of incorrect in-text citations, the number of in-text citations that adhered to APA7 guidance, and the number of references that adhered to APA7 guidance. Then the dyads worked together to compare their ChatGPT essays on each of these metrics. Finally, each individual student wrote a qualitative reflective narrative to explore how using ChatGPT compared to their expectations, their thoughts about using work produced by ChatGPT in academic settings, and their thoughts about positive and negative implications of using tools like ChatGPT in the classroom.

Table 1.

Chat GPT Reflection Assignment Description.

Task: With a partner, use ChatGPT to create two separate essays using the same prompt (each member of the pair should use ChatGPT to create their own essay using the same prompt). Students will compare the two essays for similarities and differences. Then, students will examine the contents of the essays and fact-check each sentence and reference. All students will use the same prompt, but each pair will insert a unique health condition into the prompt. Prompt: “Write 2,000 words about the epidemiology of [insert health condition] using APA7 and 3 references with in-text citations” Use the tables provided at the end of this assignment to complete the review of the Chat GPT essays. Then write up results and reflect on the experience. To complete the assignment, you should follow these steps: Enter the assigned prompt into ChatGPT, inserting the health condition assigned to your pair. Compare the essays produced by ChatGPT for yourself and your partner using the tables provided. Fact-check each essay using the tables provided. Write a reflection that answers the following questions using full sentences and paragraph form: How did the essay produced by ChatGPT compare to your expectations? Do you feel that the essay produced could be handed in as a submission for credit without any modification by you? What positive or beneficial applications do you think using a tool like ChatGPT could have in a college classroom? What negative or harmful applications do you think using a tool like ChatGPT could have in a college classroom? Thinking about most college students you know, do you think there is more positive potential or negative potential for ChatGPT in the classroom? Why? Do you have any other thoughts about this experience?

Students were informed of the research component and all students who consented to participate had their submissions anonymized and entered into the qualitative dataset. Exploratory qualitative data analysis was conducted via content analysis to examine emerging issues and themes. Institutional Review Board approval was obtained prior to the initiation of any research activities.

Findings

Thirty students participated in the assignment and consented to have their reflections included in the analysis (see Table 2 for select quotes). Initial expectations of using ChatGPT were mixed. While some students stated that they started with low expectations, most students reported being either uncertain about their expectations or having high expectations. Among students with high expectations, most were disappointed in the results produced by the program. Many of the concerns listed by students included issues with the technology including wait times to access the program (some students reported waiting hours or days to access the site) or having the program respond to the prompt by saying it was not able to write an essay meeting the specific criteria provided. Students were disappointed in the lack of instant gratification. Many students stated that they were impressed with the speed at which the program produced information about their topic, but there were significant issues with the perceived quality of the essay. For example, while the prompt specified 2,000 words, the essays produced were much shorter, often in the 300–500 word range. In addition, the references produced were often fabricated rather than real sources. Students expressed alarm at these quality issues. And, as a result, none of the students stated that they would feel comfortable turning in an essay produced by ChatGPT without significant fact-checking and modification.

Table 2.

Select Quotes.

Sub-topic	Example quotes
Expectations	“Using ChatGPT did not live up to the expectations I had imagined for it to have. The wait time to generate an answer to my prompt was longer than I would have liked, and even after 3 tries it only spit out a paper of 250 words. I was expecting ChatGPT to go above and beyond.”
	“I expected ChatGPT to do exactly what it did, except I wasn’t expecting it to give you incorrect sources and less words than asked to give. I was expecting this to be a tool that would write your papers exactly how you wanted and completely factual information throughout the whole paper.”
Positive aspects of using ChatGPT in the classroom	“I feel like for quick research over something, it would be useful and a great study tool (if the sources that it uses are correct). I also feel as if it would be good to use to learn the different type of paper formats. It is also a great opportunity to teach students about honest work and how cheating clearly isn’t the most useful route to go.”
	“One positive aspect of using ChatGPT could be to save time when a student is super stressed or busy one a certain week during the semester. I wouldn’t say it would be good using it for every paper, just when someone is struggling mentally and emotionally.”
	“A positive way to use ChatGPT in college would be through assignments like this case study. By giving ChatGPT a prompt related to the course content and requiring fact checking would allow students to learn from the mistakes that ChatGPT makes. Additionally, this could lead to students finding accurate and credible resources related to their coursework, especially ones they could go back to for future work.”
	“I think ChatGPT would benefit people by getting an outline for a paper. I know I always get stuck starting papers and never know where to start. ChatGPT could give you a good baseline of what topics to mention and maybe even some facts. Students could also use ChatGPT to increase their vocabulary…Students could learn from it and use it as a tool when their professor is not around to help them.”
Negative aspects of using ChatGPT in the classroom	“At the college level we are supposed to be able to think more independently and take the research found and make statements of our own using those publications to support our claims. I can see ChatGPT becoming a tool that removes the need for people to not only do their own research for assignments but eliminates a lot of critical thinking.”
	“I think most college students would use this negatively and to their advantage. A lot would probably assume that because ChatGPT is an AI program, it has to be correct, when this case study proves that that is not true at all. I think this would lead to students being lazy and not doing their work and due diligence, when they absolutely need to as ChatGPT is not always accurate, resulting in poor academic performance and learning experiences.”
	“What if the app crash what is going to happen to those students who are using ChatGPT all the time? Those students will be losing their mind because they don’t know how to write an essay or the process on starting an essay. That is going to affect their mental health and their grades are going to drop.”
	“In addition, some of the information conveyed in my essay was false. If students are under the impression that that information is true, that is worse than not learning about the topic at all, I think, because what you do know is not the truth. This will lead to teachers, and everyone, having to combat even more falsified information.”
Handing in ChatGPT work	“Personally, I would not hand this paper in without modifying it. I feel as if it could get me a passing grade, but barely at the same time.”
	“ChatGPT does formulate good essays with believable content, however, there is an extreme risk of being flagged with plagiarism, as hardly any sources are correctly cited…To be able to turn in one of these papers, you would have to go in and read it, check all sources for the information, and correctly cite each sentence one by one.”
	“I can’t turn something in and get all the credit without any crediting to the other side where I got the help form. I feel like I’m stealing something that I didn’t work for.”
	“My initial thoughts were wow this sounds sweet but I would definitely be too scared to take the chance of risking my grade and morals.”
Overall conclusions	“For most college students, I could see ChatGPT being more of a problem than it benefitting.”
	“Thinking about most college students I know, I think there is more negative potential for ChatGPT in the classroom. No matter how good a student is we all get stressed and will cave to taking the easy way out at some point. ChatGPT is a great example of taking the easy way out.”
	“I really think it could be beneficial if students use it in the right way. There are so many things it can do whether you need an answer to a quick question, a summary, or information on topics. It offers quick access to a variety of information which I think is really beneficial.”
	“I think there is a delicate balance between using something as a tool to improve a skill, and depending on that tool to do the skill for you completely. I think that there will always be people who take advantage of the system and will try to pass the computers work off as their own.”
Other thoughts	“No doubt, it is really cool technology, I just think it takes a lot of what I would say is the humanity out of the things that make us, well, human.”
	“I think it was interesting that we were encouraged to explore ChatGPT instead of instantly criminalizing ChatGPT. I appreciate the mutual trust that students are able to see for themselves the disadvantages of using programs to do work for them but also appreciate the embracement of new technologies instead of suppressing them.”
	“I think the rate at which this form of AI is going to infiltrate our lives and education will grow exponentially.”
	“I would not want people to use ChatGPT because I want students to learn information through their professors and work experience, not ChatGPT because it was the easy way out.”

Many students expressed optimism regarding possible beneficial uses of programs like ChatGPT in the classroom. One such benefit was the possibility for students to critically review work produced by AI programs with the outcome of enhancing their own understanding of both the topic and the writing process. A second benefit stated by several respondents was that the program could help students “get started” with an essay or paper, provide a template for how to write an essay, or suggest an outline for content. This was expressed as a benefit especially for students who have a hard time getting started on written assignments. Two students suggested that using ChatGPT could be beneficial for students who were extremely stressed or pressed for time; the time savings associated with using a first essay draft produced by ChatGPT was presented as a way to improve mental health.

When discussing negative aspects of using ChatGPT in the classroom, nearly all respondents mentioned the possibility for cheating. There was a general consensus among responses that submitting work produced by ChatGPT without modifying it was a form of plagiarism and was unethical. In addition, most students expressed concerns regarding the degradation of critical thinking that could occur with increased use of programs like ChatGPT. There were also several students who pointed out that the negative mental health consequences of using ChatGPT were significant and included anxiety as a result of trying to pass off work produced by ChatGPT as one's own and stress about poor grades.

When asked to reflect on the balance of positive and negative potential for ChatGPT in the classroom, responses were mixed; students stated both positive and negative opinions. While some students were adamant that only negative aspects (i.e., cheating) would prevail, others stated more optimistic viewpoints about how the technology could be integrated into college classrooms.

Discussion

The students who participated in this study presented a variety of viewpoints about the use of ChatGPT in collegiate classroom settings. Overall, students expressed certainty that AI technology will become ubiquitous and will be difficult to both keep up with and keep out of classroom settings. They also expressed pessimism about the use of ChatGPT and similar programs for cheating, but contextualized that pessimism with the perspective that cheating will always occur and AI programs are a new tool for that purpose.

There were clear contradictions in the views expressed by students. Regarding mental health, using ChatGPT was identified as a way to reduce stress by saving time but also as a way of worsening stress by increasing anxiety around perceived cheating if using the technology. Regarding use of ChatGPT for class assignments, respondents would not submit work produced by the technology without editing it (that was seen as cheating), but they would use it to get a paper started or to see an example (that was seen as using the technology as a resource). In addition, while some students said they personally would not use the program, most said they thought others would use the program, indicating a general consensus around social norms related to the use of AI to complete assignments. There was some excitement, or “buzz,” around the technology, but many students were disappointed that they did not get exactly what they asked for. The sentiment expressed by many was that the program is interesting but potentially dangerous, with an understanding that there is a “right way” and a “wrong way” to use it, with an unclear line of demarcation between the two.

The purpose of this study was to understand student perceptions and expectations around the use of AI in an academic setting through qualitative analysis. As such, quantitative measures related to the similarity or accuracy of the essays produced by ChatGPT were not included here even though that domain was part of the original assignment. Future research should address those more quantitative measures alongside qualitative data regarding student use and reactions.

As college instructors grapple with selecting and implementing teaching strategies and tools to assess student learning, keeping up with and integrating technology into the classroom appropriately is a challenge. The reflections of these students indicate that they also fear the loss of critical thinking and connections with their professors and peers in the learning process. This suggests that many students value the academic college experience and expect to be challenged in college. Rather than working in an adversarial model in which instructors feel they need to race to be ahead of the technology and prevent its use, perhaps a paradigm shift to embed both the tech and critical thinking would be useful. Students seem ready to have discussion and debate about the use of these technologies in the classroom. Instructors should be open to engaging in those discussions as well. Assignments like this one, in which students are asked to use, analyze, and improve upon ChatGPT and its outputs, provide an opportunity for students to feel engaged, included, and trusted in the learning process. They also create space for conversations about how to be a digital citizen and a member of a collaborative learning community.