Developing an intelligent virtual agent to stratify people with cognitive complaints: A comparison of human–patient and intelligent virtual agent–patient interaction

Abstract

Previous work on interactions in the memory clinic has shown that conversation analysis can be used to differentiate neurodegenerative dementia from functional memory disorder. Based on this work, a screening system was developed that uses a computerised ‘talking head’ (intelligent virtual agent) and a combination of automatic speech recognition and conversation analysis-informed programming. This system can reliably differentiate patients with functional memory disorder from those with neurodegenerative dementia by analysing the way they respond to questions from either a human doctor or the intelligent virtual agent. However, much of this computerised analysis has relied on simplistic, nonlinguistic phonetic features such as the length of pauses between talk by the two parties.

To gain confidence in automation of the stratification procedure, this paper investigates whether the patients’ responses to questions asked by the intelligent virtual agent are qualitatively similar to those given in response to a doctor. All the participants in this study have a clear functional memory disorder or neurodegenerative dementia diagnosis.

Analyses of patients’ responses to the intelligent virtual agent showed similar, diagnostically relevant sequential features to those found in responses to doctors’ questions. However, since the intelligent virtual agent’s questions are invariant, its use results in more consistent responses across people – regardless of diagnosis – which facilitates automatic speech recognition and makes it easier for a machine to learn patterns. Our analysis also shows why doctors do not always ask the same question in the exact same way to different patients. This sensitivity and adaptation to nuances of conversation may be interactionally helpful; for instance, altering a question may make it easier for patients to understand. While we demonstrate that some of what is said in such interactions is bound to be constructed collaboratively between doctor and patient, doctors could consider ensuring that certain, particularly important and/or relevant questions are asked in as invariant a form as possible to be better able to identify diagnostically relevant differences in patients’ responses.