ChatGPT: A Double-Edged Sword?

Dear Editor,

Chat Generative Pre-trained Transformer, commonly known by its acronym ChatGPT is an Artificial Intelligence (AI) model which leverages Natural Language Processing (NLM) and sophisticated machine learning algorithms, including Reinforcement Learning from Human Feedback (RLHF) to comprehend and generate human-like dialogues. ¹ According to its own assertion, ChatGPT-3 had been trained on an extensive dataset comprising roughly 45 terabytes of textual data. This dataset encompasses a diverse array of text sources, including web pages, books, articles and other materials, and spans multiple languages and domains. It is, of course, beyond the capacity of any individual to achieve within a lifetime. The potential applications of this software are limitless, spanning from literature and medicine to space exploration. It is widely regarded as the most significant breakthrough in the current era of information technology.

Despite initial criticism, ChatGPT’s content is now being accepted by numerous prestigious universities, institutions and publications worldwide for inclusion in academic and medical writing. Notable examples include the International Baccalaureate, ² Elsevier and Cambridge University Press ³ all of which mandate that due credit be given if used.

However, in the context of medical literature, significant importance should be placed on verifying the data obtained from ChatGPT, given the potential impact it may have on human life. There have been instances where ChatGPT has generated inaccurate or contradictory information, a phenomenon commonly known as ‘hallucination’. According to BBC Science Focus, ‘ChatGPT could become confused, offering up a nonsensical answer to your question, or even inputting stereotypes or false information’. ⁴

I encountered this phenomenon in December 2022 while using ChatGPT to search for research related to the topic of ‘Adherence to Diabetes medication and Body Mass Index’. The programme promptly responded with the following statement: ‘1) A study published in the Journal of Diabetes and Its Complications in 2016 found that higher BMI was associated with lower adherence to diabetes medications in a sample of adults with type 2 diabetes. 2) A review published in the Journal of Diabetes and Metabolic Disorders in 2015 concluded that obesity (as measured by BMI) is a significant predictor of poor adherence to diabetes self-management behaviours, including medication taking, diet, and physical activity’. However, upon conducting a thorough literature search, the aforementioned studies were nowhere to be found.

After the most recent update of ChatGPT-3, the search was conducted again at the end of March. The results are presented in Figure 1. It is important to acknowledge that the referenced studies and the links do not lead to a genuine literature pertaining to the mentioned topic. There are numerous underlying issues that have yet to be addressed, and this is merely a superficial representation of the problem.

Figure 1.

ChatGPT response to the prompt ‘find researches related to Adherence to Diabetes medication and BMI with links’.

The purpose of this letter is not to dissuade the utilisation of ChatGPT. Although it is a powerful creation, it remains a mere tool. It is important to bear in mind that the power ultimately lies with user. A researcher must use it wisely and rationally, ensuring the authenticity of the data at every stage. The ultimate objective of medical research is to enhance the well-being and health of individuals globally and to discover more effective approaches to manage and prevent diseases. ⁵ What could be more honourable than utilising all available resources to achieve this objective?