Practical Applications of ChatGPT in Undergraduate Medical Education

Abstract

ChatGPT is a chatbot developed by OpenAI that has garnered significant attention for achieving at or near a passing standard on the United States Medical Licensing Exam (USMLE). Currently, researchers and users are exploring ChatGPT's broad range of potential applications in academia, business, programming, and beyond. We attempt outline how ChatGPT may be applied to support undergraduate medical education during the preclinical and clinical years, and highlight possible concerns regarding its use which necessitates the creation of formal policies and training by medical schools.

Introduction

Developed by OpenAI, ChatGPT (generated pretrained transformer) is a robust artificial intelligence chatbot that debuted to the general public in November 2022. ¹ Following its release, ChatGPT gained attention for its ability to “perform at or near the passing threshold” of the three-step, United States Medical Licensing Exam (USMLE) per Kung et al. ²

ChatGPT relies on GPT-3.5, a large language model that synthesizes and explores relationships between the wealth of existing internet data to generate conversational-style text in response to user-generated prompts, or questions. In March 2023, GPT-4 was released, a subscription-based, powerful update to GPT-3.5 that can process both text and images, among other features. Early adopters are exploring the utility of ChatGPT in a variety of fields, including: mathematics, ³ coding, ⁴ and academic writing. ⁵

ChatGPT in Medical Education

Research has consistently demonstrated that medical students make extensive use of point-of-care tools and mobile medical applications to support their learning, citing knowledge augmentation, reliability, and efficiency. ⁶ While a dedicated mobile application is not currently available, ChatGPT can be accessed through web browsers on mobile devices, allowing users to utilize the tool on-demand. Currently, little exists that focuses on practical applications of ChatGPT in undergraduate medical education (UGME). We attempt to enumerate and describe potential uses of ChatGPT in UGME and discuss concerns and limitations surrounding its use by medical students.

Facilitating Evidence-Based Decision Making

Literature suggests that while medical students attempt to practice evidence-based medicine, they are often limited by time constraints and lack of technical expertise.^7,8 Reviewing and synthesizing the sheer quantity of research while balancing the stressors of clinical duties can be overwhelming. ⁹ ChatGPT can provide succinct summaries of clinical trials with key practice points derived from them. For example, if one enters the prompt “What are the clinical implications of the EMPEROR-PRESERVED trial?” ChatGPT will return a succinct, point-by-point response highlighting its practice implications, including the crux that it “provides strong evidence that empagliflozin can improve outcomes in patients with HFpEF…that SGLT2 inhibitors may be a useful treatment option [in HFpEF] which…has few effective therapies.”

Generating a Differential Diagnosis

One of the key competencies expected of a medical graduate is the ability to generate a reasonable differential diagnosis list for a given clinical presentation. ¹⁰ For junior medical students, generating a broad differential, including “can’t miss” or “red flag” diagnoses is frequently cited as a major challenge due to knowledge limitations at their stage of training. ChatGPT can be utilized as a tool to broaden their differentials and trigger self-directed reading on diagnoses unfamiliar to the student. In the clinical environment, students can enter pertinent elements of the patient's history to generate a provisional differential diagnosis. This has the potential to be immensely helpful to students who lack experience in certain specialties. Moreover, having a differential diagnosis framework prior to meeting the patient can inform the acquisition of a targeted history and focused physical exam that can refine the post-encounter differential and influence downstream diagnostic studies and treatment.

A Knowledge Resource

ChatGPT is an effective general resource when knowledge-based questions are clearly elaborated in the prompt. For example, inputting “What are the three categories of AKI etiologies?” will produce a response outlining the pre-renal, intrinsic, and post-renal causes of AKI complete with common cause of each. While students can obtain this information through a “Google search” ¹¹ or using a point-of-care resource or mobile app, ⁶ ChatGPT excels at presenting the information in a concise, practical manner for ease of understanding. Similarly, it provides succinct summaries of clinical practice guidelines.

OSCE Preparation

Studies have often demonstrated that OSCE exams are associated with considerable stress for medical students, perhaps even more so than conventional exams. ¹² Mock OSCEs, or practice OSCEs, have been identified as a method to build exam competence and alleviate stress. ¹³ ChatGPT has the ability to generate level-appropriate OSCE scenarios with pertinent patient details, including demographics, history of presenting illness, and medical history. If prompted, ChatGPT can also provide possible answers to the scenario, complete with focused history questions, physical exam maneuvers, and potential investigations. Further, students can tailor the OSCE scenario to a topic of their choosing, eg “Can you provide a third-year General Surgery OSCE scenario?” This offers students another resource to hone their OSCE skills and a dynamic means to host mock OSCEs with their colleagues.

Exam Preparation

Multiple-choice examinations represent the most common examination format in medical school. ¹⁴ Medical students make frequent use of third-party question banks such as UWorld and AMBOSS to prepare for licensing exams. These resources often provide answer explanations that offer teaching points around specific topics behind each question. Likewise, medical schools frequently release practice exams to assist students in exam preparation and to guide their studying. ChatGPT can be utilized in this context as a “virtual teaching assistant” as described by Lee, ¹⁵ to offer insight into each question and provide feedback on questions and clarifying concepts the student may find particularly challenging. By extension, it can also be leveraged to generate knowledge-check questions to solidify conceptual understanding in an interactive manner.

Literature Survey and Academic Writing

Many medical students conduct research during their medical school careers. ¹⁶ ChatGPT can be leveraged to conduct a cursory survey of the literature, with limitations, prior to formal literature review for the purpose of familiarization, owing to its ability to analyze high volumes of data several magnitudes quicker than a human can achieve. Further, medical students can achieve additional time savings by training the tool to identify and summarize specific papers, again with limitations, pertaining to their research question. While ChatGPT can generate seemingly coherent essays, it is often based on a combination of verifiable and fabricated data. ¹⁷

Future Applications

Recently, GPT-4 was released to developers and comes with the capacity to accept and process image inputs, including photographs, screenshots, and hand-drawn images. ¹⁸ While this function is not based on a computer vision model and remains in its infancy, the ability to interpret relevant information from visual inputs offers dynamic possibilities as a “virtual tutor.” These include: providing differential diagnoses based on submitted images (eg skin rashes and radiographs), identifying pertinent histological features on slides, and generating practice questions based on image inputs.

Concerns with ChatGPT

Despite the excitement surrounding ChatGPT, several critical concerns exist regarding its use. First, prompts entered, including personal information, are collected and processed by ChatGPT, raising ethical and privacy concerns. ¹⁹ This data could be compromised in the event of data breaches and could even be utilized for malicious intent. Given the growing popularity of AI chatbots, medical schools should consider tailoring their lectures on privacy to specifically address the use of AI chatbots. Privacy policies similar to those widely adopted by medical schools to minimize privacy breaches and guide online professional conduct when social media use among students became commonplace should be implemented. ²⁰

Educators have astutely raised concerns over academic integrity with the use of ChatGPT. Medical students can conceivably submit high-quality written passages generated by ChatGPT as their own work which may constitute academic misconduct. ²¹ A major publisher has since developed policies which prohibit the listing of ChatGPT as a contributing author in an attempt to safeguard integrity in scholarly work. ²² Similarly, universities have begun implementing specific policies around generative AI tools; some have banned their use outright although many appear to be in the process of reviewing and updating their policies. ²³ Given the growing popularity of these tools, medical students will likely make increasing use of ChatGPT and its alternatives in the coming years, hence establishment of policy and guidance is a definite priority. To safeguard academic integrity, medical schools should utilize AI detection tools (eg Open Text Classifier, GPTZero, etc.) to review submitted assignments although such tools remain unsophisticated at this point in time. ²³ With respect to its usage in the clinical environment, explicit wording must be defined to ensure ChatGPT is utilized as a resource only, in conjunction with the trainee's clinical experience and knowledge. This is clearly outlined by OpenAI's usage policies, ie “You should never use our models to provide diagnostic or treatment services for serious medical conditions.” ²⁴ Clinical practice demands both accuracy and precision, and the provision of inaccurate or misleading information by ChatGPT is a concerning possibility. ¹⁵

An additional limitation to ChatGPT is that up until March 2023, it did not have access to real-time information, ie GPT-3.5 is trained on data up until September 2021. ¹ Therefore, guidelines, recommendations, or publications published after that date will not be included in responses. However, OpenAI released robust plugins for ChatGPT to subscribers and developers in March 2023, providing it with real-time web access. ²⁵ Theoretically, this would allow it to provide factual, up-to-date information to support students. However, OpenAI previously trialed WebGPT, a prototype ChatGPT variant with web access, and found that it tended to “cherry pick” sources that did not necessarily provide an accurate representation of the literature. ²⁶ Moreover, ChatGPT is not immune to AI hallucinations, whereby it generates plausible-sounding but ultimately non-sensical or factually incorrect responses that are not supported by evidence. ¹⁷ In fact, ChatGPT has be found to generate non-existent references upon which it attempts to substantiate its output.^27,28 It has also been reported that ChatGPT has a tendency to produce inconsistent responses to identical prompts and has incomplete access to major databases, including PubMed, and therefore would not be included in its training data.^28,29 The value and utility of ChatGPT to support literature review and academic writing in the context of medical student research is therefore questionable if not limited given these concerns. Ultimately, even OpenAI acknowledges the limitations of ChatGPT, stating that, “Great care should be taken when using language model outputs, particularly in high-stakes contexts.” ¹⁸

Conclusions

ChatGPT is a robust, evolving AI chatbot that has considerable potential to augment UGME in both the preclinical and clinical years. While its future potential and applications remain unenumerated, the current version can aid exam preparation and knowledge development, although concerns over AI hallucinations and unreliable sources limits its present utility as an evidence-based clinical support resource and research tool. It remains to be seen how future updates and improvements to ChatGPT, including real-time, web access, can be utilized to augment the UGME experience. Nonetheless, explicit policies surrounding its use should be developed by medical schools in light of their widespread popularity and uptake among students.