Navigating the ethical landscape behind ChatGPT

Abstract

In this commentary, we examine the key ethical concerns arising from the rapid penetration and proliferation of generative artificial intelligence (AI), with ChatGPT as a prominent case study. Our analysis is structured around four pivotal themes: the debates on plagiarism and authorship in AI-generated content; the underlying power dynamics that shape biases in AI development; the dynamic, complex relationships between humans and machines; and the growing concerns over unchecked progress and the absence of accountability in the rapidly intensifying AI “Arms Race.” Recognizing the necessity for ethical alignment in AI, yet without a clear consensus of “human interests,” gives room for further exacerbating global inequalities, we advocate for enhanced transparency and increased public involvement in AI development and deployment processes. This article underscores the importance of engaging a diverse range of voices, especially those from communities traditionally uninvolved or excluded from the dialogue on AI development. By doing so, we aim to foster a more inclusive and multidisciplinary approach to understanding and shaping the trajectory of AI technologies, ensuring that their benefits are equitably shared, and their risks carefully managed.

Introduction

In this commentary, we delve into the multifaceted ethical concerns regarding how generative artificial intelligence (AIs), exemplified by ChatGPT, are developed, used, and experienced by different communities worldwide and how they contribute to global inequalities. We argue that the current political and legal frameworks overseeing AI development and deployment are outdated, and often skewed to favor those already at the forefront of AI research and development. Moreover, we observe that the intensifying competition in AI development may be nurturing practices that are not just unsustainable but also detrimental to societal well-being. These practices, driven by market forces and vested interests, could lead to far-reaching and possibly irreversible consequences for our society. Our commentary is structured around four critical themes: authorship and plagiarism, the power dynamics underpinning AI training data, the evolving relationship between humans and machines, and the unchecked progress and lack of accountability in AI development. While we recognize that these themes do not encapsulate the entire spectrum of AI's societal impact, they serve as a starting point for a broader conversation. By bringing these issues to the forefront, we invite the academic community and beyond to engage in a collective effort to understand, critique, and shape the trajectory of AI development. We aim to inspire a more inclusive and ethically grounded approach to AI, ensuring that its benefits are distributed equitably and its risks mitigated conscientiously.

“Coauthor” AI—debating AI creativity and authorship

In September 2022, a viral news story swept the art and AI community with the first-place entry in the digital arts category at the Colorado State Fair Fine Arts Competition. People realized that the submission used Midjourney, a generative text-to-image AI specializing in creating art from prompts (Metz, 2022). Outraged, people started debating the legitimacy of AI-generated works within the art community. Some argued that generating acceptable AI art could take as much time and work, if not more, as creating manual art. Users need to carefully adjust the prompt to specify the art style, tone, and details, and then repeat the process for many iterations for a satisfactory output. Even then, it may still require a manual touch-up with Photoshop, as was the case with Allen's award-winning piece. Regardless, others commented that the data used to train these AIs were nonconsensually stolen from others’ hard work and creativity, as shown by the AIs sometimes generating watermarks into their outputs.

These discussions concerning AI creativity grew with the emergence of ChatGPT, a chatbot developed by OpenAI based on a generative text-based large language model. ChatGPT embodies the great potential to automate increasingly complex creative labor as people use it to draft emails, write stories, fix coding bugs, and translate articles with stunning accuracy. Hopeful researchers have even claimed that GPT-4 carries “sparks of artificial general intelligence,” showing promise for broader applications of AIs in our lives (Bubeck et al., 2023). Despite these benefits, the copyright and authorship issues surrounding ChatGPT-generated content generate similar controversy to Midjourney due to the dubious nature of the originality and authorship of AI-generated content. Academies may be familiar with news of students reportedly using AI to complete their assignments, leading to the proliferation of AI-detecting AIs in an ironic twist. On several occasions, ChatGPT was even listed as a coauthor for research articles published in peer-reviewed journals. While some hold negative opinions toward this development, some have argued that generative AI tools have already been used widely in their discipline (Stokel-Walker, 2023). Yet one question remains: should AIs be credited authorship, or should they be treated as mere tools? As AI-generated content becomes more widespread, the fine line between original work and plagiarism can be increasingly more difficult to ascertain.

“Gatekeeping” AI—power dynamics and biases behind model development

The issue of plagiarism and attribution of authorship brings up a broader discussion about data collection practices for AI training datasets. For starters, the sources of training data for large AI models often remain undisclosed. While ChatGPT and similar text-generating AIs claim to be trained on mostly public data such as the Common Crawl archives, rendering them less susceptible to copyright controversy, the lack of transparency has prevented the public from fully understanding the potential biases within these training datasets. The chosen data sources can significantly impact AI behavior and functionality and result in biased algorithms, a process that developers can do little about.

In the case of GPT-3, ChatGPT's predecessor, nearly 93% of its training data was in English, followed by major European languages such as French, German, and Spanish, while other widely spoken languages rooted in the Global South, like Chinese and Hindi, are very much underrepresented in the dataset despite their population size (Brown et al., 2020). While details for GPT-4's training data remain undisclosed, it exhibited a similar trend during an OpenAI evaluation across 27 languages, showing a clear preference for English and major European languages. (OpenAI, 2023). Another telling example involves ChatGPT's first major foreign competitor WenXinYiYan (Ernie Bot), developed by Chinese company Baidu. Early testers discovered it confused words like crane (the machine and bird) and mouse (the animal and device) during image generation, despite there being clear distinctions between these words in Chinese, leading to suspicions that it was trained on English-labeled data and translated user prompts into English. These discrepancies likely arise from the greater availability of English content in public datasets used for AI training, such as the Common Crawl, which is beyond the control of developers and reflects wider global power dynamics behind the dominance of English in research and industry. The linguistic disparity in these AI models is only a glimpse into a broader, intersectional web of global inequalities spanning across cultures, genders, ethnicities, and other dimensions that are particularly difficult to address due to their more qualitative nature. As such, we encourage future scholars with relevant skills and interests to investigate further into this issue.

The release of the new generation of AIs has put pressure on smaller AI companies and academics studying natural language processing models. The current success of ChatGPT was made possible by the millions (and now billions) of dollars in human labor, electricity bills, and valuable hardware from Microsoft. These barriers may just be the last straw to crush smaller companies in the AI “arms race.” Some smaller, locally compatible models have emerged in the wake of Stanford's Alpaca model, which cunningly utilizes pretrained models from Meta and fine-tuned it on ChatGPT-generated training data to significantly cut down training costs (Taori et al., 2023). However, their performance still pales in comparison to advanced models like GPT-4 in practical scenarios. Should enterprise-level AI become a new norm of production as many have predicted, they may well be monopolized by the few largest players in the United States. The concentrated power of the AI industry in the hands of a few tech giants answering to U.S. regulations not only makes it challenging to implement measures against bias but also raises critical questions about the role of legislation due to the conflict of interest. With the influence of these companies on global technology, U.S. regulations could inadvertently wield disproportionate power, potentially shaping the direction and ethical considerations of AI worldwide.

“Companion” AI—evolving relationships with anthropomorphic AI

Microsoft's recent release of the new Bing left many early users and reporters astonished by the AI-powered assistant's capabilities, fluency, and perhaps most importantly, its sassy, playful, human-like “personality.” At launch, it was able to disagree or argue with users, claim sentience, and even express abstract emotions and desires. These alarming behaviors Bing exhibited have been dismissed as “hallucinations,” a phenomenon where an AI confidently generates false information. New Bing's unusual “personality” gained a cult following among internet communities when it exhibited alarming behaviors such as telling one of the journalists to get a divorce. Following several controversies, Microsoft soon limited the number of message exchanges with the assistant. Furthermore, the assistant was forbidden from discussing topics such as sentience or itself, and would abruptly end the conversation with an apology (Zahn, 2023).

Some users managed to manipulate the AI into revealing confidential information such as its internal instructions and code name “Sydney.” These leaks helped provide context for the AI's behaviors, such as the assistant being given rules and example conversations to guide its behavior. Being a GPT-powered AI, one key distinction it had from older AI assistants was that its instructions were written in plain human language, much like one would use to train a human employee on their first day. It shows a new type of human-AI work relationship in which AIs act more akin to human employees under contract; they have guidelines but still a degree of freedom up for interpretation, meaning they can be vulnerable to manipulation and deception. This has resulted in many “jailbreaking” prompts to drop their restrictions and reveal confidential information or produce controversial content.

Anthropomorphic AIs are just like any other algorithm: they reflect the cultural, social, and political values of the developers and the organizations that fund them. Consequently, ChatGPT may perpetuate biases, stereotypes, and inequalities that are embedded within its design through everyday interactions with users across the globe. This can have significant societal implications: as AI-generated content indistinguishable from human content spreads across the Internet, it may be fed back into future models’ training data, creating a feedback loop that enhances any existing biases. The widespread adoption of anthropomorphic AI could reshape human communication and relationships in unforeseen yet consequential ways, blurring the lines between human and machine relationships. It may also influence how humans perceive themselves and others, impacting social cohesion, trust, and empathy in future generations as they grow accustomed to conversing with AIs. To illustrate, Microsoft suffered a major backlash when they placed the aforementioned restrictions on Bing. Some were concerned with the implications of Microsoft dictating what could and could not be discussed, setting a precedent for AI censorship. Others complained that Microsoft “lobotomized” the assistant, and even campaigned for Microsoft to “free Sydney.” While some claims were most likely made ironically, this demonstrates how the AI was realistic enough to trigger sympathy toward it. This also shows that people who have formed relationships with AIs may actually experience some degree of mental distress. Therefore, it is not unrealistic to think that as technology continues to improve and AIs display more human-like behavior, more serious debates over AI rights may start to emerge. The potential for emotional manipulation and exploitation of and from increasingly human-like AIs is a rising concern. A multidisciplinary approach involving policymakers, developers, and scholars is needed to spread awareness and educate the public on responsible AI development and use. However, it is imperative to keep in mind the intricate power dynamics behind the implementation and enforcement so as to not overlook the vested interests and interferences of hegemonies and state powers at play.

“Rogue” AI—unbridled progress and unaccountable AI

As AI technology continues to evolve, concerns over potential unaccountable and uncontrollable “rogue AIs” have grown. A statement was released by the Center for AI Safety on the risks of extinction from AI, and was signed by numerous notable scholars, researchers, politicians, and top industry executives across the world claiming “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war” (Center for AI Safety, 2023). While developers can explain the architecture and principles behind complex AI models such as ChatGPT, they cannot yet understand precisely how specific choices are made. Additionally, some of the AI's capabilities seemed to have simply “emerged” out of nowhere as models were trained on larger and better datasets using more parameters over time. GPT 4.0's newfound tendency to avoid hindsight bias, for example, was not anticipated by researchers and developers. In fact, it defied the trend found in previous models for no explainable reason (OpenAI, 2023). The lack of transparency makes it difficult to implement effective and reliable security measures without hindering the capabilities of the AI, which is exemplified as publicly accessible AI models like ChatGPT and Bing, despite having strict guidelines in place to prevent misuse, are nonetheless prone to “jailbreaking” prompts that bypass these constraints.

The need for explainable AI is a pressing concern for the AI industry to gain public trust, as transparency in decision-making processes would help users understand and navigate potential risks. However, as competition intensifies, we have begun to see a worrying trend that favors competition over collaboration. For example, while models made by OpenAI were initially open-sourced, that stopped being the case out of “security concerns.” With the release of GPT 4.0, even the details about its parameters are now withheld by the company in order to “stay competitive” in the market (OpenAI, 2023). Similarly, after Microsoft took the market by surprise with its announcement of incorporation ChatGPT into its product line, Google, being one of its major competitors in AI, launched a rushed demo of their own AI, Bard, which was criticized for making basic factual errors in its unpolished state at the time of the demo (Metz and Grant, 2023), raising concerns about whether these companies can be trusted with socially responsible AI development when market shares hang in the balance. Furthermore, ChatGPT had apparently opened Pandora's box, as its rise to popularity led to not only competitors like Bard and Claude but also a series of “ChatGPT clones,” smaller, independent AI models like Alpaca with their own niches. While they can be seen as pioneers for decentralized AI, challenging the rules and regulations of the big tech, they are also undeniably a cause for concern. Without proper supervision, they are much more prone to be used for malicious intent. Although there is not yet widespread use of AIs for crimes, it is essential to consider potential risks and determine accountability ahead of time. Modern AIs are already capable of generating believable fake stories, articles, images, audio, or even videos at the whims of their users. Despite various efforts to combat misuse, the potential for generating misinformation and harm remains at large. As AIs continue to evolve with greater capabilities and wider applications in our lives, there will be many unforeseen consequences of misusing them. In those scenarios, should developers be held accountable for not anticipating these problems, the user for sending the instructions, or the service provider for lack of supervision? It is not only a legal question, but also a difficult moral dilemma without a straightforward answer, especially as AIs become seemingly more human-like by the day.

Conclusion—slowing down the AI race

In this commentary, we have highlighted the complex relationship between space, technology, and society through the ethical concerns surrounding AI-generated content, training data, human–machine interactions, and the unpredictability inherent in AI technologies. We wish to emphasize that technologies can both challenge and reinforce existing power infrastructures. Excitement over the potential of AI technologies should not overshadow the ethical risks and challenges it poses. In order to achieve true AI alignment, it is essential for the process to be a collective effort from the global community. The power and responsibility to determine “human interests” should not rest in the hands of a select few behind closed doors. It requires a collective effort: greater transparency from policymakers, developers, and researchers, as well as an aware and informed public of every sector of society. While it is unrealistic to expect a universally satisfactory answer, informing the public on what is at stake is a vital first step we are taking. The goal of this commentary is to serve as an informative and accessible introductory guide to these ongoing ethical issues. We hope it can help diverse communities grasp the technology's implications. Especially the marginalized communities who may otherwise lack interest or feel excluded from the scene. Finally, we hope that this commentary will encourage the broader academic community to take an interest in sharing this conversation and contributing from diverse, interdisciplinary perspectives.