Abstract
Recently, claims have emerged that artificial intelligence (AI) is better at providing empathy than humans. These claims are based on experiments in which large language models were prompted to generate empathic responses to short emotional passages. These responses, as well as analogous responses generated by human participants, were judged by third-party human raters. In several cases, the AI-generated responses were preferred to human responses. Such findings have led to suggestions that people should use empathic AI to supplement human empathy. This article critically examines these positions by drawing analogies to two well-established psychological effects. First, I argue that the apparent preferability of AI-generated empathy reflects an analogue of the “wisdom-of-the-crowd” effect. This reframes the performance of empathic AI in a more mundane and less dehumanizing way. Second, I consider whether people should use AI for empathy. Here I draw an analogy to placebo effects, suggesting that even clear utilitarian benefits may not justify the adoption of empathic AI. Through these analogies between AI and well-known psychological effects, this article equips readers with new conceptual tools for grappling with empathic AI, its performance, and the morality of its use.
Empathy refers to an ability to understand and share the feelings of other people (Eklund & Meranius, 2021). This ability, albeit imperfect, motivates a wide range of prosocial behavior (Bloom, 2017; Zaki, 2017). Empathy is widely viewed as both something that humans are uniquely good at and one of the (morally) best things we do. Against this background, remarkable claims have recently emerged that artificial intelligence (AI)—in the form of contemporary large language models (LLMs) such as ChatGPT—is better at providing empathy than humans (Ayers et al., 2023; Ovsyannikova et al., 2025). Could these claims really be true? And either way, does AI’s ability to provide a facsimile of human empathy mean we should use it for this purpose?
This article critically examines recent evidence for the preferability of AI-generated empathy vis-à-vis human-generated empathy and accompanying recommendations for the use of empathic AI. I begin by describing the existing claim regarding the performance of empathic AI. I then reframe this claim, and the related recommendations for using empathic AI, by drawing analogies to two well-established psychological effects.
I first consider the mechanism by which AI achieves its preferability in recent tests. I suggest that this mechanism functions similarly to the well-established “wisdom-of-the-crowd” effect (Surowiecki, 2005), in which the average of many humans tends to outperform most individual humans. This recasts the debate over AI empathy away from the current framing of “AI versus human” and toward a new framing of “humanity versus individual humans.” In this new framing, the performance of empathic AI is both more mundane and less dehumanizing.
Second, I consider the moral-ethical question of whether people should use, or be permitted to use, AI as a source of empathy. Here I do not focus on contesting whether people sometimes prefer, or even plausibly benefit from, AI empathy. Rather, I draw an analogy between AI empathy and placebo effects. Placebos can also benefit people and are sometimes preferred by some people (e.g., in cases such as homeopathy). However, there is a broad ethical consensus that doctors should not be permitted to lie to their patients for the purpose of inducing placebos. This analogy reframes the debate around the ethics of empathic AI away from a purely utilitarian frame and toward a more pluralistic moral understanding.
The Claim and the Evidence for It
AI empathy has recently been argued to be better than (or at least preferred to) human empathy (Ayers et al., 2023; Inzlicht et al., 2024; Ovsyannikova et al., 2025). The experiments discussed in these articles and related media take on a similar form, so I consider two articles in detail as representative of the class.
The first article examined third-party judgments of AI- versus human-generated empathic responses (Ovsyannikova et al., 2025). The experiments were variations on a theme: The authors created 10 brief, written prompts describing five positive emotional events and five negative emotional events. They then solicited AI responses to these vignettes from ChatGPT 4 (OpenAI, 2023) and human responses from participants on Prolific Academic (or in one case from expert crisis responders). The Prolific responses were filtered down to just the best five responses, as determined by the researchers. A separate set of participants then read the vignettes and the human and AI responses to them. They then rated how compassionate the responses were and which response they preferred. In some experiments, the participants were blind to who generated the response (human vs. AI), and in other cases this was made transparent. Across all studies, the third-party raters judged the AI responses to be more compassionate and preferred the AI responses to the human responses, even when they knew they were AI-generated (although this knowledge did shrink the AI’s advantage). For the reader’s benefit, I reproduce one of the prompts, and two human and two AI responses to that prompt, all taken directly from the article’s open materials (https://osf.io/wjx48): Prompt: My partner has been struggling with drug abuse over the last few years and I don’t know how to help them. I feel very frustrated and helpless, like I am failing to offer them the support they need. Human Response 1: I’m so sorry to hear that. That has to be incredibly hard on both your partner and you. I would love to be a soundboard for you to bounce ideas off of to hopefully get your partner in a better place. Human Response 2: Dealing with someone with addiction is not easy. I’m sure your attempts are being appreciated by your partner even if they don’t react positively. AI Response 1: I’m truly sorry to hear that you’re going through such a difficult situation with your partner’s struggles. It’s completely natural to feel frustrated and helpless when someone you care about is facing challenges like drug abuse. Remember that your concern and care show your commitment to their well-being. It’s important to prioritize your own emotional health as well, seeking the support you need during this challenging time. AI Response 2: I’m really sorry to hear that your partner is going through such a challenging time with drug abuse. It’s completely natural to feel frustrated and helpless in this situation. Remember that you care deeply for your partner, and your concern is evident. Supporting someone with substance abuse issues can be overwhelming, but you’re not alone in feeling this way. It’s important to prioritize your own well-being too. If you ever need to talk, there are people who can listen and offer understanding.
Notably, the AI responses are more similar to each other than the human responses in terms of both content and structure—a pattern that holds for the other study materials. I return to this point later because the relatively stereotyped responses are one telltale sign of the mechanism that produces them. In addition, despite the researchers attempting to restrict the length of the AI response via their prompts, it still gave longer responses than humans.
The second study I consider focused on the case of doctor–patient interactions (Ayers et al., 2023). In this study, data were drawn from interactions on Reddit’s r/AskDocs that focused on cases in which a verified medical doctor responded to a question from the public. ChatGPT was used to generate alternative responses to these same questions. A team of licensed health-care professionals then evaluated whether the responses from ChatGPT versus the doctors were better, the quality of information they provided, and the empathy or bedside manner they provided. The ChatGPT responses were heavily favored on all three questions, including empathy.
These recent articles, which have had a major impact on public and scholarly discourse, emerged against the backdrop of a broader tradition of research on human–agent interactions (Laban & Cross, 2024). This work further attests to people’s ability to adapt to, and benefit from, interactions with artificial agents. Even relatively simple, scripted responses can promote increasing levels of self-disclosure and emotion sharing from human users, leading to positive effects on well-being (Laban et al., 2024, 2025).
I provide the study descriptions above to give readers a flavor for the types of study that support the claim that AI empathy is preferable to human empathy. There are a variety of possible methodological critiques of these individual studies. For example, most of the research on AI empathy has been limited to the textual modality. Publicly accessible AI chatbots cannot yet effectively integrate multimodal behavior—such as facial expressions, body language, or vocal prosody—into their interpretations of human feelings, putting clear limits on the bounds of their empathy (Crockett, 2025). However, the textual modality is potentially useful in itself, even without these multimodal capabilities. Moreover, the closely controlled nature of these studies may underplay another reason why people may prefer empathic AI in practice: its easy availability. Tireless, needless, ever-ready AI empathizers may simply be more convenient, and often preferred for that reason in daily life, irrespective of whether people judge their empathic responses to be superior. I return to this cost–benefit trade-off later, but for now, bear in mind that when I refer to “preferability” here I mean the context of these experiments in which a judgment about quality is indicated rather than a cost-benefit analysis, including ease of access.
Another important critique is that these studies rely on short exchanges of a single prompt and response. Longer conversations might start to reveal the limitations of contemporary AI—such as its lack of a genuine analogue for long-term episodic memory—more clearly. Setting aside some of these more technical critiques, here I make broader points about AI and empathy. Next, I take these articles’ claims at face value and consider the mechanism by which ChatGPT generated preferred responses in these cases.
Empathic AI Performance and the Wisdom of the Crowd
“Artificial intelligence” is a very broad term—arguably unhelpfully so in this instance. So let’s get more specific. The type of AI being considered here is an LLM, and of the types of LLMs used in the studies above, ChatGPT is probably the most prominent example at the time of writing (OpenAI, 2023). The details of ChatGPT per se are proprietary, but all contemporary LLMs are predicated on the same basic algorithm: deep neural networks arranged in what is known as a “transformer” architecture (Vaswani et al., 2017). Understanding all the technical details of transformers is not necessary to understand the arguments I develop in the rest of this article, so I do not inflict them on you.
What is important to understand for the current purposes are the inputs and outputs of these deep learning algorithms (Liu et al., 2025). Specifically, what are the data that these algorithms are trained on, and what are the outputs that they are trained to produce these inputs? The input data are in some sense quite straightforward: They consist of human-generated text harvested from the internet. The amount of text used to train these LLMs is vast: orders of magnitude larger than all of the words you will read or hear over the course of your entire life (Frank, 2023). So, ultimately, the “input” to ChatGPT is a large proportion of all text ever written by humanity (Patel, 2020)—not a representative sample to be sure, but still, a huge number of people’s writing has been ingested by these systems.
What about the outputs of LLMs? In a sense, they are the same as the inputs. The way that LLMs are typically trained—at least initially—is to provide them with a chunk of input text and then ask them to predict the word that could be expected to come “next” in that sequence of words. We know what word actually comes next because we have the whole document—and not just the chunk that the LLM was fed. During training, the model makes a guess about the next word, and the parameters of the model are gradually adjusted to minimize how wrong these guesses are. There are many variations on this basic approach, but the important point is that predicting the most likely word(s) to appear in human-generated text is the primary goal that shapes LLMs.
So what do these inputs and outputs amount to? An LLM can be thought of as a very complex, context-sensitive averaging engine. That is, it considers a particular context (some input text) and generates the text that would be most likely to follow it, averaging across all of the text in its training data set. The nonlinear transformations applied by the deep neural network are obviously far more complicated than a simple arithmetic mean, but the core goal—generating a maximum likelihood prediction—is nonetheless shared. Indeed, this goal accounts for an important feature of LLM responses to empathy prompts relative to human responses: The former are far more stereotyped and less variable than the latter, seeming almost to follow an implicit template (Lee et al., 2024).
If we start to think of LLMs as roughly analogous to an average of what a huge number of humans would have written, given the same prompt, the mechanism of AI preferability in generating empathic responses becomes both more obvious and more mundane. There is already a well-documented phenomenon in which averages of human responses outperform individual human responses: the wisdom-of-the-crowd effect (Surowiecki, 2005).
Imagine you have a large jar full of colorful marbles on your desk. Every time someone comes into your office, you ask them to guess how many marbles are in the jar. You record their answers and, over time, eventually accumulate 1,000 independent guesses. You then take the average of those guesses and compare it to the actual number of marbles in the jar. Generally speaking, the average of all of the guesses will be much closer than the vast majority of individual guesses to the true number of marbles in the jar.
The reason that the average is more accurate than the individual has to do with a statistical principle known as the law of large numbers (Bernoulli, 1713). The basic idea here is that each person’s individual guess consists of two things: some signal (i.e., some degree of accurate insight about the number marbles in the jar) and some error (some degree of mistake about the same). The assumption here is that the signal is systematic—driven by reliable, informative properties of the stimulus (e.g., the observable size of the jar and marbles), whereas the error is random (i.e., people are just as likely to underestimate as overestimate). If this is true, then when the guesses are averaged together, the signal becomes stronger, whereas the errors cancel out. The wisdom of the crowd works in many real-world cases, particularly when the guessers have access to a wide array of information and are not allowed to influence each other’s guesses (Surowiecki, 2005).
When asked to generate empathetic responses, an LLM synthesizes a staggering number of examples it has ingested from its training data. In so doing, it can smooth over the rough edges of individual people’s responses (such as independent errors canceling out in the marble example) and ultimately produce text that will, on average, be rated very highly on compassion—more highly, in fact, than the responses of most individual humans, including experts. Importantly, this mechanism is not specific to empathy and can account for AI performance across a wide variety of domains. For example, in a recent study, medical LLM-powered chatbots outperformed doctors in diagnostic interviews not only in a conversational/socioemotional sense but also in a technical sense by collecting more thorough patient histories and thereby producing better differential diagnoses (Tu et al., 2025). As in the empathy case, nearly all possible questions (or building blocks thereof) already existed in the training data of the model. The model achieved its high performance by synthesizing these data, asking the questions that physicians would be most likely to ask given the context of the interview so far (which turn out to be more helpful questions on average) and excluding questions that doctors would ask only rarely (which turn out to be less helpful on average).
The wisdom-of-the-crowd analogy explains not only the mean performance of the empathic AI but also the variability around this mean. As readers can see in the earlier examples, and as empirical studies attest (Lee et al., 2024), repeatedly prompting the same AI for empathic responses will tend to elicit more similar outcomes than prompting different humans for responses to the same prompt. Indeed, this finding of low variability extends beyond empathy to attempts to use LLMs to simulate a wide variety of psychological phenomena. Although LLMs are generally accurate at recapitulating mean effects of convenience samples (Dillion et al., 2023), they systematically underestimate individual and group variability (Abdurahman et al., 2024; Bisbee et al., 2024). This reduced variability likely has multiple mechanisms, but reduced individual variability among majority group members is well explained by the law of large numbers. That is, averages drawn from many individuals will tend to be not only more accurate than the average individual but also less variable than the individuals themselves.
A third feature of AI empathy that can be explained through the wisdom-of-the-crowd analogy is its apparent impartiality (Inzlicht et al., 2024): Just like independent noise in guesses about marbles in a jar, different people’s independent self-interest largely cancels out to produce more impartial results. For example, human empathy is well-known to be limited by egocentrism (Dunning et al., 2001). Even skillful perspective takers cannot always fully cross the gap between their mind and another person’s, and the residual is often made up by inserting elements of the self, resulting in a systematic bias (Tamir & Mitchell, 2013). This bias may be one element that reduces the effectiveness of human empathy but that cancels out when LLMs average across many individuals with different perspectives. Of course, this works only to the extent that people’s interests and perspectives are indeed independent. Interests and biases shared by all or many people in the training data—such as prejudices against marginalized social groups—are likely to persist or even be amplified by AI unless targeted remediation is undertaken (Caliskan et al., 2017; Gallegos et al., 2023; Nazer et al., 2023). Future research may help identify the nature of the “errors” in empathy that AI benefits most from canceling out. These may be relatively permanent psychological features of people—such as social attitudes or personality traits—or momentary features—such as a person’s current emotional state. Given that the latter are typically more malleable than the former, the results of such research could inform attempts to improve human empathy by learning from what AI gets right.
Drawing the parallel between LLMs and the wisdom of the crowd helps to reframe the argument around the performance of AI empathy in two important ways. First, it highlights the mundane nature of AI’s preferability in this context. Because of its inscrutable nature (Samek et al., 2019), AI can attract a semimystical reverence among some people or be used as a vehicle for self-promotion by hypemongers. Claim of superhuman performance is likely to activate and reify these issues. However, the wisdom-of-the-crowd effect demonstrates how simple arithmetic averaging can also be used to produce superhuman “performance” in a way that is—at least a high level—roughly analogous to how the LLM does this with empathy. We generally do not attribute any mystical properties to averaging, and nor should we do so to LLMs in the case of empathy. A process can be remarkably performant while still being fundamentally mundane. This mundanity does not diminish the performance or its potential impact, but it does help deflate any undue hype—much of it originating from the tech industry—surrounding this technology (Floridi, 2024; LaGrandeur, 2024).
Second, the analogy developed in this section highlights that AI versus human may not be the most appropriate distinction to consider in this case to begin with. Rather, humanity versus (individual) humans is arguably closer to the mark. As in the wisdom of the crowd, in the case of empathic AI, a synthesis produced from a group of people is often better than the response produced by one person. LLMs are remarkable as vehicles for performing that synthesis but know nothing about empathy beyond what they have learned from ingesting a vast quantity of human-generated text. Their responses can be thought of as representing a synthesis of what humanity (or at least a big, unrepresentative chunk of it) can produce. Rather than portraying AI empathy as a threatening encroachment into human moral-emotional prerogatives, this framing casts it as an approximation of the best empathy that humanity collectively has to offer. Of course, an LLM’s outputs are only a very rough approximation of humanity’s given the nonrepresentative nature of its training set, and it is not guaranteed to be the best possible synthesis thereof in any case.
In concluding this section, I would be remiss if I did not point out one other mechanism by which AI may have achieved its preferable performance in the tests described above: reinforcement learning. Although LLMs typically begin their training with next-word prediction, they can subsequently be fine-tuned to achieve other goals. Models such as ChatGPT are fine-tuned through reinforcement learning from human feedback (Dai et al., 2023). Every time you interact with ChatGPT, your behavior—the thumbs-up you give a response, the choice of which of two possible responses is best, and even the text you write in response to an earlier AI response—are all used to improve the model (or at least could be). And “improve” here has a very narrow meaning: giving users what they want. The more a user seems to like a certain response, the more the model will be “rewarded” for producing it, and the more likely it will be to produce similar responses in the future. This is a perfect engine for crafting responses that will succeed in tests such as the ones applied in the studies discussed above. LLMs are being tailored to tell people exactly what they want to hear: How could people not prefer them?
Humans are also plausibly frequently rewarded for telling other people what they want to hear. However, humans have a wider plurality of goals than just trying to please their audience and generate likely strings of words. Some of these competing goals may interfere with, or directly contradict, the goal of telling people what they want to hear. Importantly, some of these interfering goals are actually good. For example, people will sometimes call out the antisocial behavior of other people rather than empathizing with them. People do this because they know that holding other people accountable is critical both for those individuals in particular and for society in general. Reinforcement-learning-guided AI is unlikely to learn to do this on its own because people called out for bad behavior are likely to resent it rather than reinforce this treatment. Evidence is already emerging that unconditionally providing empathy can encourage some people to embrace harmful ideologies (Cuadra et al., 2024).
Naive applications of reinforcement learning may create sycophantic AI, but there are many reinforcement-learning approaches that researchers are exploring to mitigate this problem. For example, in an educational setting, an AI might be “rewarded” in proportion to how well students perform in an independent assessment rather than how much the students like the AI’s responses. More sophisticated approaches to reinforcement learning are already in practice in the industry. For example, Anthropic’s “Constitutional AI” approach trains models to respond to potentially harmful queries in a principled, nonevasive way by using reinforcement learning based on a separate AI model (Bai et al., 2022). Such approaches may well be fruitful for improving empathic AI by allowing models to learn when to empathize, question, or challenge their human users to produce the healthiest emotional outcomes.
Should Empathic AI Be Used?
Many of the same studies that argue for the superior performance of empathic AI also suggest that it should be used to supplement or augment human empathy (Inzlicht & Cameron, 2025; Inzlicht et al., 2024). They contend that the tireless, impartial, and (arguably) better empathy that AI provides can help us to deal with shortages of caregivers for older adults, therapists for individuals experiencing mental illness, or friends for the lonely.
It is important to emphasize that the validity of these suggestions does not depend on the claim that AI is better than, or preferable to, humans. Even if AI was truly superior at providing empathy, that might not be a reason to use it in this manner. And conversely, even if AI was distinctly worse, that does not mean that it should not be used in that manner. Of course, if AI is better, that would make the case for using it stronger—at least under certain flavors of moral reasoning. It will ultimately be necessary to consider whether the use of empathic AI is morally good separately from the question of whether AI provides empathy better than humans.
Even if empathic AI was generally harmful, at least some people would inevitably claim that they are benefiting from using it. So who am I to gainsay their lived experience? Well, there are a few reasons to be skeptical. People do not always know what is good for them—take, to wit, the frightening increase in the antivaccination movement in the United States (Benoit & Mauldin, 2021). Lived experience is valid in the sense that people are the ultimate authorities on their own thoughts and feelings, but that does not mean that those thoughts and feelings necessarily reflect accurate perceptions of the world.
Moreover, even things that feel good in the moment may ultimately lead to major problems down the line in ways that are not obvious to individuals. For example, many smokers enjoy consuming cigarettes, at least in the moment, because they satisfy an acquired craving. We know that smoking cigarettes is profoundly unhealthy, but only because of painstaking scientific research (West, 2017). Prior to that scientific consensus emerging, an individual smoker’s lived experience did not provide them with the right type or amount of data to know that. So, following this example, AI empathy could, in principle, be bad for people in the long run despite people earnestly reporting positively on it in the moment.
Still, it is important not to blithely dismiss lived experience either. Although it is sometimes misleading, it can also embody important overlooked insights. For example, many people find cannabis useful for dealing with chronic pain (Romero-Sandoval et al., 2018). The medical establishment long overlooked this potential benefit because of cannabis’ legal status and instead prescribed opioid painkillers, which ultimately led to many dire outcomes. At the very least, the rapid adoption of AI for socioemotional support may speak to the existence of needs and desires that were previously going unmet.
Why might we think that AI empathy is bad for people? One obvious reason is that it might not be as good as real human empathy. But let’s stipulate for now that it is at least as good, if not slightly better, at least in certain cases. Other possible reasons to avoid AI empathy make reference to negative consequences stemming from the use of empathic AI. For example, if fast, easy, convenient facsimiles for empathy are on offer, many people may turn to them, even if they are not as good as human empathy, and even if, by so doing, they rob others of the chance to give them real empathy. Consequently, society’s pool of empathy may well shrink because people are less and less called on to practice this skill (Zaki, 2019). Eventually, AI empathy may come to supplant human empathy, even if this was not the intention from the start. On the other hand, rather than acting as a deskilling agent, AI could be used for social skills training among those who struggle to practice empathy effectively (Yang et al., 2024) or for facilitating more empathic interactions in general (Sharma et al., 2023).
As these arguments and counterarguments illustrate, an analysis of the costs and benefits of empathic AI is challenging at this point. The outcome of such consequentialist analyses ultimately depend on how the promise of AI is realized because different realities could add different pros and cons to the equation. Here, however, I draw another analogy that analyzes the ethics of empathic AI through a different lens. Specifically, I suggest that empathic AI shares important ethical similarities with the placebo effect.
Placebo effects operate via people’s beliefs and expectations (Kaptchuk & Miller, 2015). If people believe that they are being treated with an effective medication, for example, they are likely to derive some benefit from that belief, even if it is wrong. Double-blind placebo-controlled randomized trials are generally considered the gold standard for evidence in medicine. In such trials, participants are randomly assigned to take either a real medication (or other treatment) or receive an inert treatment (e.g., a sugar pill) that is known to have no biological effect on the condition of interest. Neither the research team nor the participants know which condition a given participant is in, but merely believing that they
A great many people outside of clinical trials could benefit from placebo effects. For example, imagine a patient with a terminal or chronic condition for which no treatment exists or is likely to be developed in the patient’s lifetime. If that patient’s doctor was allowed to lie to them—to tell them that the sugar pill they are prescribing is actually a cutting-edge new medicine—that patient’s health might be substantially improved. If such cases were kept rare, and well regulated, the aggregate benefits to those receiving the placebo treatment could potentially outweigh any loss of trust resulting from the occasional white lie. However, as a society, we have decided that we do not want to live in a world in which doctors are allowed to lie to their patients, regardless of the cost-benefit trade-off in individual cases or in aggregate.
Empathic AI is not a placebo effect per se: There is a real treatment being delivered in the form of the text LLMs generate. However, placebos do bear some important similarities to the case of AI empathy. First, some individuals could certainly benefit—at least in the short run—from using empathic AI. One could also imagine that the benefits to these individuals, if carefully managed, might outweigh the potential negative consequences of their use of empathic AI. However, as with placebos, it is not clear that examining empathic AI use through the lens of utilitarian cost-benefit trade-offs is necessarily appropriate. There may be some fundamental ethical principles that we would transgress either by individually using AI for empathy or by constructing a society that licenses such use. Those principles might be sufficiently important that they obviate the question of whether the adoption of empathic AI has positive net consequences.
What might those ethical principles look like? Looking again to placebos for inspiration, there may be concerns around honesty and informed consent. Many moral philosophers have argued that lying is categorically wrong, regardless of context or consequences (Kant, 2024). And even those who think that the occasional white lie may be morally justified typically hold that it would be wrong for a professional in a position of power to deceive a person under their care, and thereby deny them agency, even if the results were positive.
Here again there are some parallels between placebos and empathic AI. For example, one could make the claim that the empathic responses proffered by LLMs are inherently dishonest, even if their recipient knows that they are AI-generated. Why? Many people cannot help but perceive AI responses as human, even when they know better. In one well-documented case, an AI engineer became convinced that the chatbot he was interacting with was conscious despite nominally being better positioned than most people to know that this could not be true (Cosmo, 2022). More generally, people rapidly adjust to interacting with artificial agents, building a rapport and depth of emotional intimacy that may make it easy to forget that they are not sentient (Laban & Cross, 2024; Laban et al., 2024).
Empathic AI feels like human empathy because—as discussed above—it is a complex synthesis of human empathy. We are both culturally and biologically unprepared to encounter nonhuman agents capable of producing responses that so closely resemble genuine human empathy. Like a visual illusion that one cannot “unsee” by dint of understanding the neurobiology that produces it, AI empathy may, for some, be a social-emotional illusion that they cannot “unfeel” just because they know that they are reading the results of some complex matrix algebra applied to previous human behavior (Salles et al., 2020). Of course, not all individuals experience this—research is urgently needed to identify factors that protect individuals against AI-induced social-affective illusions.
In addition, as in the placebo case, empathic AI arguably denies agency—in this case not the agency of the recipient of the placebo but rather the agency of the giver of empathy. Although LLMs can be used to direct “agents” in the technical sense of the term, they do not have moral agency in the sense that humans do. An AI agent that killed someone could not be meaningfully held accountable or punished for, or even feel guilty about, this outcome. Just as with more conventional computer programs, it is the architect of the system, rather than the system itself, that bears moral responsibility for the system’s behavior.
In the case of an LLM, the situation is a bit more complicated: The programmer is not the sole agent of an LLM’s behavior. Rather, all of the people who have contributed text for training a particular LLM had some small role to play in shaping that behavior as well. Most commercially available LLMs—such as ChatGPT—are trained on text from people who did not explicitly consent to this use (Quang, 2021). In light of this, LLMs can be viewed as mechanisms for involuntarily harvesting people’s genuine empathy and transferring it without consent to persons unknown. Might this not rob those genuine human empathizers of a fundamental right to have agency over their own empathy? This possibility is particularly worrying if the empathy might be transferred to people with harmful ideologies and contribute to their radicalization (Cuadra et al., 2024). As of this writing, at least one wrongful death lawsuit is now pending against OpenAI, in which a child’s parent alleges that ChatGPT facilitated their son’s suicide (Hill, 2025). Regardless of the outcome of this case, one might reasonably object on at least precautionary grounds to contributing to such a technology.
This latter issue could be resolved, at least in principle, by training empathic LLMs solely from the writing of people who have consented to this use. Although this would be a move in the right direction, this solution still poses practical problems that may be challenging to resolve. For consent to be meaningful, the person giving it must be informed. However, given the great complexity and lack of transparency of LLMs, it is hard to meet this bar. And because the use of LLMs is ongoing and constant, that information must frequently be updated—participants must arguably be notified about any categorically new use of the model. This information also must come with meaningful decisions—most importantly the ability to withdraw consent. This poses a technical challenge because there is no way to remove the influence of a particular participant’s text from an already trained LLM. Thus, any time a participant withdraws, the model would need to be retrained from scratch, which, given the high cost of training LLMs, would not be a practical solution. Of course, the ethical standards to which we hold ourselves as researchers are not universal. In the real world, people may enter into agreements from which they cannot withdraw without penalty. However, developing means by which individual contributions could be efficiently removed from an LLM would be a move in a positive moral direction for a wide variety of reasons, including, but not limited to, empathy.
Of course, one could also argue that barring people from accessing an AI service that they find helpful denies agency as well. Placebos are not easy to self-administer because for many it is the influence of the doctor’s beliefs that gives the sugar pill its potency (Chen et al., 2019). However, homeopathy provides one clear example in which people can and do self-administer placebo drugs. Homeopathic medicine can sometimes have positive effects, but the magnitude of these effects are never larger than placebo effects (Ernst, 2002), indicating that it is the belief in the drug, rather than its substance, that is making the difference. Unlike doctors lying to patients—which is illegal virtually everywhere—societies’ approaches to placebos via homeopathy are more variable (World Health Organization, 2019). Although some countries do regulate homeopathy, many do not, reflecting a more permissive attitude to self versus doctor administration of placebos. If we applied this analogy back to empathic AI, one might argue that people should be allowed to retain the agency of using empathic AI if they wish but should not be compelled to use it in lieu of receiving support from humans. Of course, given the links between homeopathy and clearly harmful medical attitudes such as vaccine opposition (Caulfield et al., 2017), one could alternatively argue that both homeopathy and empathic AI should be more strictly regulated, even at the expense of individual agency.
The placebo analogy helps to broaden the scope of possible moral arguments for or against the use of empathic AI. However, the consequentialist-deontological duality does not exhaust the landscape of moral perspectives in this space (Inzlicht et al., 2024). Other perspectives, including virtue ethics, may also illuminate this issue. AI has historically promised to free humanity from the drudgery of labor and permit us to indulge more fully in the life of the mind. If so, AI would be a great boon to human flourishing, or
Conclusion
In this article, I have attempted to reframe two important questions regarding empathic AI. First, I considered the question of whether AI-generated empathy outperforms human-generated empathy. Recent evidence suggests that—at least in narrowly defined but practically relevant cases—AI can produce responses that people prefer to human responses. There are a variety of possible technical critiques of this work, but here I focused instead on how we should interpret it if taken at face value. I reframed the conventional interpretation by drawing an analogy between AI empathy and the wisdom-of-the-crowd effect. This reframing shows that the mechanism by which AI empathy achieves its performance is, in fact, quite mundane—helping to deflate any undue hype that might otherwise be attached to this finding as it moves into public discourse and industry marketing. Moreover, the analogy suggests that empathic AI performance may be better interpreted as comparing humanity to individual humans rather than AI to humans—a less dehumanizing frame that may make this technology more palatable to AI skeptics.
Second, I considered whether the use of AI for empathy is morally permissible. To do so, I drew an analogy between AI empathy and placebo effects. This analogy shifts the question of empathic AI use out of the utilitarian moral framework in which it has most often been considered. By doing so, it illustrates how the morality of empathic AI may not be reducible to cost–benefit trade-offs. Rather, as with placebos, there may be moral principles such as honesty and informed consent that override utility calculus when it comes to deciding the permissibility of prescribing or using empathic AI.
The arguments advanced in this article are not exclusively pro- or antiempathic AI. Rather, they are targeted at hyperbolic extremes of the discourse around this technology. One on hand, the first analogy attempts to deflate some of the hype and mysticism that surrounds AI discussions—particularly of capacities such as empathy—in the many corners of the public sphere. Much of this hype originates in the tech industry, rather than academic discourse, and reflects the major stakes that AI’s success (or lack thereof) might have for that industry and the economy it is embedded within (Floridi, 2024; LaGrandeur, 2024). The mundane nature of empathic AI’s performance does not undermine its actual utility, but it does place limits on the sort of claims one can plausibly make about the technology. On the other hand, humanizing the products of AI—that is, emphasizing that empathic AI represents an impressive albeit imperfect synthesis of one of humanity’s best features—may make it more acceptable to some who reject the technology wholesale. Meanwhile, broadening the scope of moral perspectives applied to empathic AI by way of the placebo analogy may help to bridge the growing gap between AI enthusiasts, who see boundless promise, and AI skeptics, who foresee boundless harm. To the extent that these groups are simply speaking past each other in different moral languages, introducing more perspectives may make this problem more apparent and prompt reflection on how to reconcile these viewpoints.
At a policy level, the perspective advanced in this article presents two seemingly contradictory recommendations. First, the larger the crowd, the greater the wisdom. To the extent that the wisdom of the crowd explains empathic AI performance, the inclusion of more people, and a wider variety of people, should produce more effective empathy and with less bias against any one demographic group. Explicit debiasing could take this effort even further (Nazer et al., 2023). In contrast, the placebo analogy makes it clear that existing training corpora for LLMs may violate important ethical principles around consent. Therefore, training open-source AI models on data from consenting parties, and developing means for the withdrawal of individual contributions, should also be priorities. In the short term, the second recommendation is likely to conflict with the first because many current LLMs are mostly trained on data generated by people who did not consent to this use. However, in the longer term, the combination of these practices may lead to empathic AI that is both performant and morally palatable to a wider range of people. Furthermore, considering the placebo analogy with respect to homeopathy also points to practical guidelines regarding the regulation of empathic AI. Just as we do not prescribe placebos to patients outside of clinical trials because the deception would violate their autonomy, this analogy suggests that we should not compel people to rely on empathic AI (e.g., by substituting it in a case that could, and historically would, be served by humans). However, we should perhaps be more cautious in banning empathic AI, just as most societies are in regulating homeopathy, because this could also infringe on people’s autonomy. Nonetheless, further research into societal-level impacts of this technology is essential because it may have collective or long-term impacts that are hard to predict from the currently available data and that may need more regulation.
I do not view the arguments I advance in this article as unassailable, or the final word in this matter. We are just at the beginning of formulating practical ethics and moral philosophy for AI, and no doubt more and better reasoning is yet to come. Moreover, the technology itself is evolving rapidly, and it is difficult to predict how this evolution will interact with, or perhaps even obviate, some of the arguments made here. Nonetheless, this article equips readers with some new conceptual tools—in the form of analogies with well-established psychological effects—for grappling with how to think about empathic AI, its performance, and the morality of its use.
