One AI’s Hallucination is Another Human’s Writing Prompt: Notes From an Autoethnography Journey

Abstract

To catch an AI in a hallucination, the human must know the answer or know where to find the answer. This raises the question of what happens when both the AI and the Human don’t know the answer. What if the AI's output, the hallucination, was its superpower? Does the ‘prompt and response’ exchange between AI and Human become a discussion between two entities searching at the edges of understanding for an answer?

Engaging in a dialogue with technologies challenges our preconceptions of ‘knowing’ in that technology disrupts traditional thinking practices while offering alternative pathways to discovery. Disrupting traditional methods of inquiry gives license to tackle the difficult and subversive questions in the liminal spaces between the layers of knowing, stepping past conventional disciplinary methods of inquiry and opening possibilities for radical change. We are then free to question the boundaries between theory, practice, and representation, creating space to embrace alternative, experimental, and arts-based methods to generate and interpret data.

My reminder to myself, “The future must be a form of resistance, struggling against the limitations of our present modes of thinking,” floats across my screen saver, challenging me to ask questions. It serves as a likely beginning to examine how changing human-technology relationships struggle against our images of creativity, for without a catalyst to challenge our own self-interests, we are living the future we deserve, not the one we need.

Reframing AI hallucinations as an alternative pathway to understanding provides a means of challenging the everyday metaphors about creativity. To alter how human creativity is valued and perceived, we must let go of the human-centred ideologies that shackle us to the past. We aren't diminished by the technologies we employ; they are already part of us and have always been.

The struggle is not solely about the AI but is part of the larger conversation that seeks to reveal the authenticity of writing, a craft and art form that would not exist without technology. The dominant narratives on writing and research point to authentic writing being a human-centred struggle that, somehow, we have lessened when we accept technological aids. While the dialogue may spark polarizing debate surrounding the authenticity of writing and creativity, it is an important one, as emergent technologies challenge conventional notions of what it is to know and how that knowing comes into being.

Writing technology that acts as both a mediator and medium continually reshapes our perception of the authenticity of the creative int the process. Our words are never just ours. They are informed by friends, enemies, dictionaries, public libraries, Google searches, emotions, social environment, word processors, grammar checkers, and our readings of other authors’ work before they coalesce into the stories we create. What changes if we add AI to the ever-growing list of influences?

It is time to let go of the limitations of the belief that, somehow, humans are the islands of creativity surrounded by technologies that challenge our creative agency. Our conception of what it means to be human needs updating. Not just to understand creativity but the larger questions surrounding the welfare of humanity and our planet. Change needs room to happen, driving the idea of resistance to create room to understand there is more to being human than humanism permits.

Technology mediates and is the medium of my storytelling, art, writing and creative practices. That said, I appreciate the skepticism, especially as digital aids have the potential to lead our research processes. Despite the tensions, it is crucial to accept that technology plays a significant role in shaping human evolution, understanding and self-expression. We never question the use of a laptop; we expect the author to be the final arbitrator of their content and attribute the original creator if necessary. At this moment in the AI’s development, they are not original sources.

My journey with technology began as I explored the complexities of dyslexia in a world driven by the format of text, not the value of ideas. Dyslexia has gifted me with a unique way of seeing the world, entangled and intra-active, where the boundaries between humans and technology blur. This brought into sharp focus that the pen was a poor technology to mediate the streams of ideas from my dyslexic mind. It wasn't until the advent of word processors, with their forgiving backspaces and endless edits, that my ideas would finally become visible to a text-driven society.

This revelation challenged me to accept a broader truth: rather than diluting authenticity and creativity, technology provided a means to amplify my unique vision. At a much deeper level, the awareness that the human never stands as an isolated entity but is entangled in a complex relationship with technology, ecology, and other non-human agencies. The 97.6 million words (a ridiculous number based on how the words are counted) that Grammarly tells me we have argued over and disputed since 2018 are not just strings of machine-generated text but a realization that my words were now accessible to a text-based society.

My words become a means of writing myself into ‘knowing’ with the reader. Writing is a liminal construct, inhabiting a space where answers are always an ongoing negotiation between what I know in this present moment and what I could know in the next. Neither I nor the technologies are infallible, nor are the answers final. It was the “in-betweenness” that provoked me to rethink subjectivity, ethics, and values in a more inclusive and distributed manner, thereby challenging the assumptions that have traditionally dominated our understanding of creativity and authenticity narratives.

The authenticity and originality of my work are maintained through my acceptance and willingness to be held accountable for the ideas and assertions produced; technologies are an attribute of who I am and, therefore, a natural part of my creative process. However, the narrative surrounding AI in writing often echoes past fears reminiscent of the disruption of the concentration of power that once accompanied the printing press. Yet history teaches us that all technology is political; adaptation to or rejection of technology is not the solution. Instead, through public policy, social understanding, and individual self-awareness, we can understand the essence of technology is to reveal the human and point to who we might become as a species. We must let go of the human-centred ideologies that shackle us to the past to open pathways to alternative human creativity. We aren't diminished by the technologies we employ; they are already part of us.

That is not to say there are no problems with emergent technologies. The freedom allowed to those who create them is unfettered and popularized as progress, playing into our conditioning of technology as inevitable and a means of solving world problems. Stretching the concepts of sowing the seeds of legends, myths, and metaphors from the Dune novel, we can imagine the “Missionaria Protectiva of Technologies” creating dominant narratives through public media, science fiction books, popular litanies and social expectations that advocate for the rapid advancement and widespread adoption of AI as beneficial progress. They do this by sowing litanies of progress that create social structures that lag behind technology, allowing those with power over the technologies to ignore ethical responsibility, social considerations or regulatory oversight.

AI’s increasingly pervasive role in the creative process is more an ethical question for the user than a technological one. The pen and the keyboard are both technologies humans use to co-create masterpieces. We research online libraries, snipping and repurposing ideas from fellow authors. AI, like the disruptive technologies that preceded it, confronts us with the question of what creativity means and how we value that creativity. It is not technology alone that infringes on copywriting, creates divisive narratives online, replaces human workers or refuses to pay royalties to original creators. It requires human intervention.

When the barriers between humans and technology are removed, technology augments human creativity rather than replaces it. AI, in essence, is not the danger. It is merely a technological aggregation of human textual expressions, biases, and worldviews, a hallucination of the worldviews embedded in the data poured into the LLMs. All technology is inherently political, either directly or indirectly. The real challenge lies in making the policies and social contracts that harness the intentions and actions of the humans who have exercised power through the technology that inhabits society.

As an AI said, mimicking Yoda,” “But a mirror AI is reflecting our intentions, it does. Not in the technology, the danger lies, but in the hand that wields it”.