What makes for an explanation of “black box” AI systems such as Deep Nets? We reviewed the pertinent literatures on explanation and derived key ideas. This set the stage for our empirical inquiries, which include conceptual cognitive modeling, the analysis of a corpus of cases of "naturalistic explanation" of computational systems, computational cognitive modeling, and the development of measures for performance evaluation. The purpose of our work is to contribute to the program of research on “Explainable AI.” In this report we focus on our initial synthetic modeling activities and the development of measures for the evaluation of explainability in human-machine work systems.
Get full access to this article
View all access options for this article.
References
1.
AmershiS.ChickeringM.DruckerS.M.LeeB.SimardP.SuhJ. (2015). Modeltracker: Redesigning performance analysis tools for machine learning. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 337–346). New York: Association for Computing Machinery.
2.
BiranO.CottonC. (2017). Explanation and Justification in Machine Learning: A Survey. IJCAI-17 Workshop on Explainable Artificial Intelligence (XAI).
3.
BrézillonP.PomerolJ.-C. (1997). Joint cognitive systems, cooperative systems and decision support systems: A cooperation in context. In Proceedings of the European Conference on Cognitive Science, Manchester (pp. 129–139).
ClanceyW.J. (1986a). From GUIDON to NEOMYCIN and HERACLES in twenty short lessons. AI Magazine, 7(3), 40.
6.
Doshi-VelezF.KimB. (2017). A Roadmap for a Rigorous Science of Interpretability. ArXiv Preprint ArXiv:1702.08608. Retrieved from https://arxiv.org/abs/1702.08608
7.
GoodmanB.FlaxmanS. (2016). European Union regulations on algorithmic decision-making and a “right to explanation.” Presented at the ICML Workshop on Human Interpretability in Machine Learning, New York, NY.
8.
JohnsonH.JohnsonP. (1993). Explanation Facilities and Interactive Systems. In Proceedings of the 1st International Conference on Intelligent User Interfaces (pp. 159–166). New York: Association for Computing Machinery.
9.
KassR.FininT. (1988). The Need for User Models in Generating Expert System Explanation. International Journal of Expert Systems, 1(4), 345–375.
10.
KrullD. S.AndersonC. A. (1997). The process of explanation. Current Directions in Psychological Science, 6(1), 1–5.
LuckyR.W. (2018, January). The mind of neural networks. IEEE Spectrum, p. 24.
15.
LyonsJ.B.ClarkM.A.WagnerA.R.SchuelkeM.J. (2017). Certifiable trust in autonomous systems: Making the intractable tangible. AI Magazine, 38(3), 37–49.
16.
MillerT.HoweP.SonenbergL. (2017). Explainable AI: Beware of Inmates Running the Asylum. In Proceedings of the International joint Conference on Artificial Intelligence (IJCAI-17) Workshop on Explainable Artificial Intelligence (XAI).
17.
MooreJ. D.SwartoutW. R. (1991). A reactive approach to explanation: taking the user’s feedback into account. In ParisC.SwartyoutW.R.MannW.C. (Eds.), Natural language generation in artificial intelligence and computational linguistics (pp. 3–48). New York: Springer.
PirolliP.CardS. (2005). The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of International Conference on Intelligence Analysis (pp. 2–4). Washington, DC: Office of the Assistant Director of Central Intelligence for Analysis and Production.
20.
SacksH.SchegloffE. (1974). A simplest systematics for the organization of turn-taking for conversation. Language, 50, 696–735
21.
ShaftoP.GoodmanN. (2008). Teaching games: Statistical sampling assumptions for learning in pedagogical situations. In Proceedings of the 30th annual conference of the Cognitive Science Society (pp. 1632–1637). Austin, TX: Cognitive Science Society Austin.
22.
TemmingM. (2018, 20January). AI has found an 8-planet system like ours in Keppler data. Science News, p. 12.
23.
ZeilerM. D.KrishnanD.TaylorG. W.FergusR. (2010). Deconvolutional networks. In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2528–2535). New York: IEEE.
