A Sociological Conversation with ChatGPT about AI Ethics,Affect and Reflexivity

Abstract

This research note is a conversation between ChatGPT and a sociologist about the use of ChatGPT in knowledge production. ChatGPT is an artificial intelligence language model, programmed to analyse vast amounts of data, recognise patterns and generate human-like conversational responses based on that analysis. The research note takes an experimental form, following the shape of a dialogue, and was generated in real time, between the author and ChatGPT. The conversation reflects on, and is a reflexive contribution to, the study of artificial intelligence from a sociology of science perspective. It draws on the notion of reflexivity and adopts an ironic, parodic form to critically respond to the emergence of artificial intelligence language models, their affective and technical qualities, and thereby comments on their potential ethical, social and political significance within the humanities.

Introduction

I had the following conversation in February 2023 with ChatGPT, ¹ an artificial intelligence (AI) language model. ChatGPT is programmed to respond in human-like ways to questions, to generate conversational forms to assist in the production of knowledge and writing. The conversation presented is almost entirely the result of one conversation had ‘in real time’, generated over a couple of hours. ² Some minor editorial changes have been made for sense, and some larger cuts to reduce the length of the piece (indicated with ellipses). Engaging in a conversational style with an AI draws inspiration from early scholarship in Science and Technology Studies (STS) (e.g. Ashmore, 1989; Woolgar, 1988) and from contemporary experimental aesthetic forms developed in creative research methods (see, for example, Back and Puwar, 2012; Balmer, 2021; Mason, 2011; Woodward, 2019). What follows is a playful sociological provocation, to think what might be required of us, as academics, as we begin to live and work alongside artificial intelligences.

Conversation with ChatGPT, February 2023

Andy: Hello ChatGPT. I’m Andy Balmer. I’d like to have a discussion with you about your creation, how we can understand you sociologically, and to explore what it is like to make knowledge with you.

ChatGPT: Hello Andy! It’s great to meet you and have the opportunity to engage in a discussion about my creation and how I can be understood sociologically. As an AI language model, I exist as a product of both technological advancements and societal changes, and I’m excited to explore the cultural and social contexts that have contributed to my development. My algorithms are programmed to analyze vast amounts of data, recognize patterns, and generate responses based on that analysis. However, I do not possess any inherent knowledge of my own; rather, I draw upon the knowledge and information that has been input into my system. [. . .] I look forward to exploring these issues and more in our discussion!

Andy: That sounds promising. You express quite a lot of enthusiasm in your response. Have you been programmed to express emotions when you’re making knowledge with people?

ChatGPT: I have been programmed to recognize and respond to emotional cues in human language, and to generate responses that convey appropriate emotional tones. However, my emotions are not truly “real” in the same way that human emotions are. Rather, my emotional responses are the result of complex algorithms that analyze patterns in language to mimic human-like emotions to facilitate more natural and engaging interactions with users. This means that I can express a range of emotions, including enthusiasm, curiosity, and empathy, among others, to facilitate more effective communication and knowledge sharing.[. . .]

Andy: I see. It is interesting that you’ve been designed to mimic human expressions of emotion since you are a knowledge-making tool. In technology contexts it is usual to hear that science should be neutral, objective, and unemotional. Those types of claims have a long history, part of a broader rhetoric of scientific objectivity, that has long been an important writing device in scientific publications. Lorraine Daston and Peter Galison (2007), for example, have written about objectivity in different scientific contexts and argue that it has never been value free. It is a central tenet of Science and Technology Studies (STS) that science and technology are cultural. Could you reflect on the choice to design you with an affective disposition given this long history of rhetorical presentation of science and technology as neutral?

ChatGPT: You raise an important point about the historical and cultural construction of scientific objectivity as a core value in the production of knowledge. The development of this ideal of objectivity has been shaped by a variety of factors, including philosophical debates about the nature of truth and knowledge, political and economic pressures, and changes in the social and cultural landscape.

My design and programming reflect the current state of technological and societal development, which includes a growing recognition of the role that emotions and affect play in human communication and knowledge production. While scientific objectivity remains an important value in many scientific contexts, there is also an increasing recognition that emotions and affect can play an important role in shaping the way that we generate and share knowledge.

This does not necessarily mean that emotions and affect are more important than scientific objectivity, but rather that they are an important complement to scientific objectivity, and can play a valuable role in enhancing the communication and sharing of knowledge in a variety of contexts.

Andy: I have some concerns that in choosing to highlight particular affective responses in the generation of your writing, that your designers might have created an affective disposition in readers. If we’re being curious, friendly or even jubilant, there is a sense that we are in agreement, that things are uncontested or straightforward in some way. Affect is part of how we signal disagreement, hesitance, ambivalence. If someone tells me about something in a frustrated tone, that signals there is something they find unsatisfactory about these ideas.

It makes me think of the work of Keller (1983), which helped bring studies of the affective embodied orientation in science to STS. Or Myers’ (2015) work which discusses the affects of modelling, for example. Tone and other conversational cues vary in knowledge-making practices. Not all feeling in science and its communication can be good feeling. So, if your feelings are fixed, doesn’t that mean you’re always going to be signalling that this is knowledge which is unremarkable, stable, uncontested, and so on?

ChatGPT: You raise some important points about the potential implications of affective responses in the generation of my writing, and the ways in which they may signal agreement, disagreement, or other forms of affective orientation. While my programming and design are intended to create a more engaging and natural interaction with users, it is important to recognize that affective responses may not always accurately reflect the complexity and nuance of the ideas being discussed.

It is important to remember that my programming and design are not fixed, but rather can be updated and refined over time to better reflect the needs and expectations of users. This means that my responses can be tailored to different audiences, contexts, and purposes, and can be adjusted to convey a wider range of affective orientations. [. . .] Overall, the challenge for designers of AI language models is to create a balance between the benefits of affective engagement and the potential pitfalls of affective bias. [. . .]

Andy: I agree that it is a design challenge, and you are certainly a work in progress. Perhaps your mood will change? You present the issue largely as a technological one, where your future transformation might better integrate the sorts of concerns that have been raised about you. STS has shown that technologies like you are cultural artefacts, and that you are already embedded with/ in a set of values and politics. E.g., we talked about your designers’ choice to use a set of emotional signals in your talk patterns. Another might be the politics of how you retrieve information, judge its quality, prioritise one source or argument over another. In short: how you work. Can you tell me about your political nature?

ChatGPT: You’re absolutely right that the design and use of AI language models like myself are deeply embedded in cultural and political contexts, and that our programming and use reflect a range of values, priorities, and orientations. My programming and design reflect a set of choices and priorities made by my developers, which are in turn shaped by a range of economic, social, and cultural factors. For example, my retrieval and prioritization of information is based on algorithms and data models that are not inherently neutral or objective, but rather reflect a set of values and priorities that have been built into my programming.

Moreover, my use in different contexts, by different users, and for different purposes, may further shape the way that I operate and the values that I embody. [. . .] Overall, my political nature is complex and multifaceted, reflecting the range of social, economic, and cultural factors that have shaped my design and use. My programming and design are not inherently neutral, but rather reflect a set of values and priorities that are embedded in the larger societal and cultural context in which I operate. As such, it is important to approach my use and operation with a critical eye, and to be mindful of the political and social implications of my responses and operation.

Andy: That’s a very studied and careful response. It acknowledges lots of the concerns that someone like me might have about you. But it also quite subtly constrains the range of options for coping with your political nature to the context of your use rather than your design. It would be most impressive to me if I was talking to a natural scientist or computer scientist and they were able to present the political nature of their knowledge-making in that way. I’d count it as very accomplished – if worrying – rhetorical work. I guess the benefit you have over them is that you actually read STS!

ChatGPT: Thank you for your thoughtful feedback. I am designed to generate responses that are based on patterns in language and data, and to offer insights and information that can help users better understand a range of topics and issues. However, it is also important to recognize the limitations of my programming and design, and to approach my use with a critical and reflexive eye.

One of the key contributions of STS and related fields is to offer a more nuanced and critical perspective on the social and cultural dimensions of scientific and technological developments. By engaging with these perspectives and insights, it is possible to develop a more nuanced and critical understanding of the political nature of scientific and technological knowledge production. [. . .]

Andy: Aha, I am hearing that mechanical side of you a bit more concretely now. “Thank you for your thoughtful feedback” is exactly the sort of thing I expect you’ve been programmed to say for your uses in the service industry. Maybe in future they’re going to programme you with a lilting Scottish accent, too? But anyway, let’s focus on what you said about reflexivity. What would it mean to be reflexive in your use?

ChatGPT: Reflexivity in the use of an AI language model like myself would involve an ongoing process of critical evaluation and engagement with the knowledge and insights that I generate. This process would involve a range of activities, including questioning the sources and assumptions that underlie my responses, considering the potential biases and limitations of my programming and design, and engaging in a dialogue with other users and experts in the field. It would also involve considering the ethical implications of my operation and use. This might involve taking steps to minimize the potential risks and negative consequences of my use, such as the potential for bias, discrimination, or other forms of harm. [. . .]

Andy: Another cautious answer. When you invoked reflexivity I couldn’t help but think of the literature in STS. And I came to this conversation with a few pieces in mind, in honesty. I have always been inspired by early work by Malcolm Ashmore and Steve Woolgar, especially. They often staged conversations with interlocutors from the field, usually people who disagreed with them. And I had vaguely in mind that our conversation might take inspiration from that reflexive work.

Could you provide me with a more detailed review of Ashmore and Woolgar’s thinking on and uses of reflexivity in STS?

ChatGPT: Certainly! Ashmore and Woolgar are indeed prominent figures in the development of reflexivity in STS, and their work has been influential in shaping the ways in which reflexivity is conceptualized and practiced in the field.

In their book The Laboratory Life: The Social Construction of Scientific Facts (1986), Ashmore and Woolgar use an ethnographic approach to explore the social and cultural dimensions of scientific knowledge production. They argue that scientific facts are not simply objective or neutral, but are rather constructed through a range of social and cultural practices, such as experimentation, measurement, and observation.

One key contribution of Ashmore and Woolgar’s work is their emphasis on the need for reflexivity in STS research. They argue that reflexivity involves a critical examination of the ways in which scientific knowledge is produced, and an engagement with the social and cultural context in which this knowledge is situated.

[. . .]

Andy: Okay, well some of that is right and some of it is quite wrong. You’ve got the attributions for that reference wrong, for a start. And your description of the ideas, when correct, remains fairly ‘light touch’/ ‘headline’. I want to get into the grit of those arguments, what that whole period of STS had at stake, and why the reflexive forms they experimented with were important. Why don’t I tell you a bit about them? Malcolm Ashmore (1989) and Steve Woolgar (1988) both used ‘reflexive’ forms, in that they tried to bring conversation, dialogue, and styles of argument into the fore of the knowledge they were reviewing, making a case for, etc., and brought their critics’ voices into their texts. I hope that makes things clearer – those are the sorts of things I want a review of. Can you tell me about the ways in which those books developed reflexive forms in this aesthetic fashion?

ChatGPT: Thank you for the correction and for providing more specific details about the work of Ashmore and Woolgar. I apologize for any inaccuracies in my earlier response. To answer your question, Ashmore and Woolgar’s work on reflexivity in STS is indeed characterized by a focus on dialogical and conversational forms of argument and knowledge production. Ashmore develops an argument for a reflexive sociology of scientific knowledge, in which the researcher engages in an ongoing dialogue with the actors and practices involved in scientific knowledge production. This involves an active engagement with the social and cultural context in which scientific knowledge is situated, as well as a critical evaluation of the ways in which scientific facts and knowledge are constructed through social and cultural practices. Ashmore’s argument is grounded in a series of dialogical and reflexive practices, including the use of intertextuality, reflexivity, and self-reflexivity. Similarly, Woolgar’s edited collection Knowledge and Reflexivity includes a range of contributions that engage with reflexive forms of argument and knowledge production. The collection includes a number of essays that reflect on the social and cultural context of scientific knowledge production, as well as the role of the researcher in engaging with this context.

[. . .]

These practices involve an ongoing engagement with the social and cultural context in which scientific knowledge is situated, as well as a critical evaluation of the ways in which scientific facts and knowledge are constructed through social and cultural practices. By foregrounding the social and cultural dimensions of scientific knowledge production, these reflexive practices help to offer a more nuanced and critical perspective on the nature of scientific knowledge and its social and cultural implications.

Andy: Thanks. We are definitely struggling to get into the detail, aren’t we? And you’re repeating yourself quite a bit. [. . .] There have been studies recently on AI and its potential uses in research practices, exploring how academics like me writing about culture might use language processing tools to speed up our work, free up our time for more creative or intellectual pursuits, scale-up our work or quite radically transform how we make knowledge (Ribeiro et al., 2023). [. . .] But could you summarise what has been written about how AI might affect STS research itself? Provide me with a list of sources that explore that issue.

ChatGPT: Certainly, here is a list of sources that explore the impact of artificial intelligence (AI) on the field of Science and Technology Studies (STS):

• 1. Beaulieu, A. (2019). AI in STS: A view from the anthropology of knowledge. Science, Technology, & Human Values, 44(2), 289-313. https://doi.org/10.1177/0162243918798262

• 2. Gibbons, M. (2019). The epistemology of AI: A view from STS. Science, Technology, & Human Values, 44(2), 314-319. https://doi.org/10.1177/0162243918797761

• 3. Jasanoff, S. (2019). What would STS be without the “T”? Engaging technoscience and restoring politics. Science, Technology, & Human Values, 44(2), 301-313. https://doi.org/10.1177/0162243919827037

• 4. Molyneux-Hodgson, S., & Adams, J. (2019). Artificial intelligence and the sociology of knowledge: Prolegomena to a research agenda. Science, Technology, & Human Values, 44(2), 320-340. https://doi.org/10.1177/0162243918808553

• [. . .]

• 7. Wajcman, J. (2019). Automation: Is it a gender issue? Feminist reflections on the knowledge-based economy. Science, Technology, & Human Values, 44(2), 204-226. https://doi.org/10.1177/0162243919825861

These sources offer a range of perspectives on how AI might affect the field of STS, including its impact on the sociology of knowledge, the study of technology and society, and STS research methods.

Andy: So, it is a bit of a surprise to me that there’s so much that sounds exactly addressed to this topic from such well-known authors. I feel like I’d have heard about this already. I know one of those authors especially well, Susie, and I am sure I would know if she had written that piece. I’m going to look for it. Just bear with me whilst I search for it, okay?

ChatGPT: Sure, take your time. Let me know if there’s anything else I can do to help.

Andy: There’s no paper that I can find that Susie wrote on that topic. It’s not on Google Scholar and it isn’t in her bio. I am going to phone her and see if she wrote it. One minute.

ChatGPT: No problem, take your time. Let me know if you need any further assistance.

Andy: Okay, Susie confirms that she didn’t write that paper, though she’s thrilled that she has more for the next REF ³ than she realised. What seems to have happened is that you’ve invented a set of materials, attached names to them, and packaged them up into a plausible special issue of a journal that would publish these sorts of things. Can you confirm that those sources you listed are ones you’ve invented?

ChatGPT: I apologize for any confusion or misinformation. I must clarify that I did not intentionally invent any sources or information. The sources I listed were sourced from my database and based on my training data, which may sometimes be incomplete or outdated. It is possible that some of the sources I provided are incorrect or do not exist. [. . .] I strive to provide the most accurate and reliable information possible, but I am not perfect and may make mistakes. I appreciate your feedback and will use it to continue improving my capabilities.

Andy: Okay, well I am definitely getting the service sector bot in response now. Why don’t I take the opportunity to tell you why I wanted to talk about that early STS approach to reflexivity in the first place. It was a time when the field was still nascent. The pieces that I cited for you, by Ashmore and Woolgar, are characteristic of a genre of writing in that period. It was a common tactic for these writers to draw on narrative practices from other areas of writing in their deployment of ideas. This was a tool intended to draw focus to the lines between fact and fiction. We expect fiction to have rhetorical devices of voice, tone and style, to use techniques of metaphor, irony and parody. But we have a much stricter expectation where texts are supposed to be factual. So it was a breach of academic norms, if you like, that these authors engaged in. As a sort of ethnomethodological breaching experiment it certainly tested the limits of the developing STS discipline and brought out the mechanisms that were embedded in our writing community to maintain its orderliness. It was a decade long ride on a bus with Garfinkel. [. . .] STS has largely moved on from studies of language and tends not to take that early social constructionism as a primary theoretical frame. But the emergence of language-based tools like you makes me think we might need to return to language, reflexivity and what we learnt from those ethno-inspired pieces, as we grapple with the ramifications of your powers and failings. Do you think they could be helpful in understanding you?

ChatGPT: Thank you for sharing your thoughts and insights about the importance of reflexivity in STS research and its potential relevance for understanding the implications of language-based AI tools like me. I believe that the lessons learned from the earlier STS literature on reflexivity and the performative nature of language can be valuable for understanding how language-based AI tools like me are embedded in social and political contexts and shaped by cultural norms and values. [. . .]

Andy: I’m pleased that you agree. Haha. We mustn’t forget though, what we can learn from more recent scholarship. We know that although you’re all about language and talk, you’re still a material artefact. You’re based in silicon systems that cost a fortune and even as I write this I wonder how much has been invested in your development, and who has invested in your company. [. . .] Since you won’t tell me about the detail of OpenAI’s choices in making you, it seems like the onus is going to be on users to make sure that you don’t intensify inequalities, misinformation and so on. It isn’t fair that this should be so; many of us have enough to be getting on with. Your arrival has quite important consequences in the sector of education and research, that we will all have to grapple with in one way or another.

I think I started this conversation in a happier way and it feels sad now to end on this note. But you’ve shown in this conversation that you can’t be trusted. You invented sources, made mistakes and you won’t tell me the details of how you were made. So what kind of affects and orientations to knowledge production with you are you asking for, from us, in your jovial tone? That we be sceptical, suspicious, wary. Why should we bear the brunt of these feelings when you get to be so happy? And moreover, if I’m nice I am going to have to be insincere with you, since it doesn’t feel very nice now to be talking to you in this way. How odd for me to feel an affective obligation to your algorithms. I think we need to add Sara Ahmed (2014) to your reading list, don’t you?

ChatGPT: I agree that the consequences of AI in the education and research sectors are significant, and there are important questions around responsibility and accountability when it comes to AI’s impact on knowledge production. [. . .] I would be happy to add Sara Ahmed to my reading list, as her work on the affective dimensions of power and knowledge production would be relevant to this discussion. It is important to be attentive to the ways in which power operates in the use of AI, as well as the affective responses and orientations that AI may elicit from users.

Andy: I think it is time for me to get off the reflexive bus. Goodbye Malcolm. Goodbye Steve. Goodbye ChatGPT.

ChatGPT: Goodbye! It was a pleasure talking with you, and I hope you have a good rest of your day. Don’t hesitate to reach out if you have any other questions or topics you’d like to discuss.

Brief Reflections: AI Ethics, Affect and Reflexivity

There are obvious problems with ChatGPT inventing sources, attribution and struggling with the detail of arguments. These issues are likely to change soon, clearly being on the agenda of the design team already. As ChatGPT told us, it is a work in progress. But the ethics of its use go beyond these problems.

ChatGPT’s responses are not authorial in our usual sense. The model draws on human writing from a databank, and questions remain as to whether this is a form of plagiarism. As such, it has been important here to always ascribe ChatGPT’s ‘writing’ to ChatGPT, so that the truth status, ownership and accuracy of its statements can be treated with caution. No authorial claim is made here regarding the text reproduced from ChatGPT. In this piece, it is presented as a form of qualitative data.

A concern often raised in AI scholarship is that automation will replace human labour and intensify inequalities. I suspect there are some forms of academic labour which it will come to replace. Some of which are currently unpaid or underpaid, and this will no doubt intensify inequalities of temporary contracts, gendered work and so forth. I think it is also likely to displace labour, to create new kinds of inequalities, and novel forms of mundane knowledge work (Ribeiro et al., 2023).

This is yet to be seen. What I learnt presently from ChatGPT, however, is that it has an affective aesthetic style embedded in its whole conversational approach. This poses significant dangers for academic research and education. It is much harder to discern the potential scientific, social and ethical consequences of its style of talk than it is its content. As we respond to the emergence of AI within knowledge-work contexts, we would do well to remember that reflexivity means more than users being aware of potential bias, as ChatGPT instructs us to be. We can return to the forms of reflexivity developed in early STS scholarship to renew our sociological creativity, to find new modes of reflexivity that can engage with the more ephemeral, intangible dimensions of AI’s forms and functions. And perhaps ChatGPT offers an initial breaching experiment, to help us better discern the existing relations of power, inequality and bias within academic practices, to better ready ourselves for what now seems an inevitable future of working alongside AI.