Can AI explain AI? Interactive co-construction of explanations among human and artificial agents

Abstract

This study investigates the potential of using advanced conversational artificial intelligence (AI) to help people understand complex AI systems. In line with conversation-analytic research, we view the participatory role of AI as dynamically unfolding in a situation rather than being predetermined by its architecture. To study user sensemaking of intransparent AI systems, we set up a naturalistic encounter between human participants and two AI systems developed in-house: a reinforcement learning simulation and a GPT-4-based explainer chatbot. Our results reveal that an explainer-AI only truly functions as such when participants actively engage with it as a co-constructive agent. Both the interface’s spatial configuration and the asynchronous temporal nature of the explainer AI – combined with the users’ presuppositions about its role – contribute to the decision whether to treat the AI as a dialogical co-participant in the interaction. Participants establish evidentiality conventions and sensemaking procedures that may diverge from a system’s intended design or function.

Keywords

ChatGPT co-constructed explainability ethnomethodological conversation analysis human-computer interaction multimodality

Get full access to this article

View all access options for this article.

References

Bassett

(2019) The computational therapeutic: Exploring Weizenbaum’s ELIZA as a history of the present. AI & Society 34(4): 803–812.

Button

(1990) Going up a blind alley. In: Luff

Gilbert

Frohlich

(eds) Computers and Conversation. London: Elsevier, pp.67–90.

Elliott

(2022) Making Sense of AI: Our Algorithmic World. Cambridge, UK and Medford, MA: Polity Press.

Garfinkel

(1967) Studies in Ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall.

Garfinkel

Lynch

Livingston

(1981) The work of a discovering science construed with materials from the optically discovered pulsar. Philosophy of the Social Sciences 11(2): 131–158.

Goodwin

(2017) Co-Operative Action. Cambridge: Cambridge University Press.

Han

Ransom

Perfors

, et al. (2024) Inductive reasoning in humans and large language models. Cognitive Systems Research 83: 101155. https://doi.org/10.1016/j.cogsys.2023.101155

Heritage

(2012) Epistemics in action: Action formation and territories of knowledge. Research on Language & Social Interaction 45(1): 1–29.

Moore

Szymanski

Arar

, et al (2018) Studies in Conversational UX Design. Cham: Springer International Publishing.

10.

Reeves

Porcheron

(2023) Conversational AI: Respecifying participation as regulation. In: Housley

Edwards

Beneito-Montagut

, et al (eds) The SAGE Handbook of Digital Society. London: Sage Publications, pp.573–592.

11.

Rohlfing

Cimiano

Scharlau

, et al (2021) Explanation as a social practice: Toward a conceptual framework for the social design of AI systems. IEEE Transactions on Cognitive and Developmental Systems 13(3): 717–728.

12.

Schegloff

(2007) Sequence Organization in Interaction: A Primer in Conversation Analysis. Cambridge and New York, NY: Cambridge University Press.

13.

Suchman

(2007) Human-Machine Reconfigurations: Plans and Situated Actions. Cambridge and New York, NY: Cambridge University Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

2.01 MB