For people with mental illnesses that impair reality testing, such as psychosis, severe depression and bipolar disorder, Artificial Intelligence (AI) Large-Language Models (LLMs) may represent threats to mental health. LLMs are unable to detect delusional beliefs, may encourage and validate delusions and cognitive distortions, miss opportunities to reinforce reality-based thinking, and exacerbate risks of self-harm and harm to others. Psychiatrists need to understand these risks of LLMs for people with severe mental illnesses, and educate patients and carers on avoiding these potential harms. Risk assessments need to be informed by an awareness of the inputs that patients receive from LLMs.
BenderEMHannaA. The AI con: how to fight big tech's hype and create the future we want. London, UK: Bodley Head, 2025.
3.
BenderEMGebruTMcMillan-MajorA, et al.On the dangers of stochastic parrots. In: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, Virtual Event Canada, 3–10 March, 2021, 2021, pp. 610–623.
4.
FuchsT. Understanding sophia? On human interaction with artificial agents. Phenomenol Cogn Sci2024; 23: 21–42.
ZuboffS. The age of surveillance capitalism. London, UK: Profile Books, 2019.
7.
FisherM. The chaos machine. London, UK: Quercus, 2023.
8.
MorrinHNichollsLLevinM, et al.Delusions by design? How everyday AIs might be fuelling psychosis (and what can be done about it). https://osf.io/cmy7n_v5/download/?format=pdf (2025, accessed 20 August 2025).
9.
ØstergaardSD. Will generative artificial intelligence chatbots generate delusions in individuals prone to psychosis?Schizophr Bull2023; 49: 1418–1419.
KoeneckeAChoiASGMeiKX, et al.Careless whisper: speech-To-text hallucination harms. In: The 2024 ACM conference on fairness, accountability, and transparency, Rio de Janeiro Brazil, 3–6 June, 2024, 2024, pp. 1672–1681.
