Developing scalable methods for conversation analytics is essential for health care communication science and quality improvement.
Purpose:
To assess the feasibility of automating the identification of a conversational feature, Connectional Silence, which is associated with important patient outcomes.
Methods:
Using audio recordings from the Palliative Care Communication Research Initiative cohort study, we develop and test an automated measurement pipeline comprising three machine-learning (ML) tools—a random forest algorithm and a custom convolutional neural network that operate in parallel on audio recordings, and subsequently a natural language processing algorithm that uses brief excerpts of automated speech-to-text transcripts.
Results:
Our ML pipeline identified Connectional Silence with an overall sensitivity of 84% and specificity of 92%. For Emotional and Invitational subtypes, we observed sensitivities of 68% and 67%, and specificities of 95% and 97%, respectively.
Conclusion:
These findings support the capacity for coordinated and complementary ML methods to fully automate the identification of Connectional Silence in natural hospital-based clinical conversations.
Get full access to this article
View all access options for this article.
References
1.
CassellEJ. Diagnosing suffering: A perspective. Ann Intern Med, 1999; 131(7):531–534; doi: 10.7326/0003-4819-131-7-199910050-00009
2.
Brown-JohnsonC, SchwartzR, MaitraA, et al.What is clinician presence? A qualitative interview study comparing physician and non-physician insights about practices of human connection. BMJ Open, 2019; 9(11):e030831; doi: 10.1136/bmjopen-2019-030831
3.
ThorneSE, KuoM, ArmstrongE-A, et al.‘Being known’: Patients' perspectives of the dynamics of human connection in cancer care. Psychooncology, 2005; 14(10):887–898; doi: 10.1002/pon.945
4.
Institute of Medicine. Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life. National Academies Press (US): Washington, DC; 2015.
5.
TulskyJA, BeachMC, ButowPN, et al.A research agenda for communication between health care professionals and patients living with serious illness. JAMA Intern Med, 2017; 177(9):1361–1366; doi: 10.1001/jamainternmed.2017.2005
6.
McGrathP, NewellC. The human connection: A case study of spirituality and disability. J Relig Disabil Health, 2004; 8(1–2):89–103; doi: 10.1300/J095v08n01_07
7.
MitchinsonL, DowrickA, BuckC, et al.Missing the human connection: A rapid appraisal of healthcare workers' perceptions and experiences of providing palliative care during the COVID-19 pandemic. Palliat Med, 2021; 35(5):852–861; doi: 10.1177/02692163211004228
8.
BartonTD. Therapeutic jurisprudence, preventive law, and creative problem solving: An essay on harnessing emotion and human connection. Psychol Public Policy Law, 2000; 5(4):921; doi: 10.1037/1076-8971.5.4.921
9.
GramlingCJ, DurieuxBN, ClarfeldLA, et al.Epidemiology of connectional silence in specialist serious illness conversations. Patient Educ Couns, 2022; 105(7):2005–2011.
10.
GramlingR, StratonJ, IngersollLT, et al.Epidemiology of fear, sadness and anger expression in palliative care conversations. J Pain Symptom Manage, 2021; 61(2):246–253.e1; doi: 10.1016/j.jpainsymman.2020.08.017
11.
DurieuxBN, GramlingCJ, ManukyanV, et al.Identifying connectional silence in palliative care consultations: A tandem machine-learning and human coding method. J Palliat Med, 2018; 21(12):1755–1760; doi: 10.1089/jpm.2018.0270
12.
BartelsJ, RodenbachR, CiesinskiK, et al.Eloquent silences: A musical and lexical analysis of conversation between oncologists and their patients. Patient Educ Couns, 2016; 99(10):1584–1594.
13.
BackAL, Bauer-WuSM, RushtonCH, et al.Compassionate silence in the patient–clinician encounter: A contemplative approach. J Palliat Med, 2009; 12(12):1113–1117; doi: 10.1089/jpm.2009.0175
14.
LevittHM. Sounds of silence in psychotherapy: The categorization of clients' pauses. Psychother Res, 2001; 11(3):16.
15.
GramlingR, Gajary-CootsE, StanekS, et al.Design of, and enrollment in, the palliative care communication research initiative: A direct-observation cohort study. BMC Palliat Care, 2015; 14(1):40; doi: 10.1186/s12904-015-0037-8
16.
AllanK, JaszczoltKM. The Cambridge Handbook of Pragmatics. Cambridge University Press: Cambridge, United Kingdom; 2012.
17.
StiversT, EnfieldNJ, BrownP, et al.Universals and cultural variation in turn-taking in conversation. Proc Natl Acad Sci U S A, 2009; 106(26):10587–10592; doi: 10.1073/pnas.0903616106
18.
ManukyanV, DurieuxBN, GramlingCJ, et al.Automated detection of conversational pauses from audio recordings of serious illness conversations in natural hospital settings. J Palliat Med, 2018; 21(12):1724–1728; doi: 10.1089/jpm.2018.0269
19.
BreimanL. Random forests. Mach Learn, 2001; 45(1):5–32; doi: 10.1023/A:1010933404324
20.
LecunY, BottouL, BengioY, et al.Gradient-based learning applied to document recognition. Proc IEEE, 1998; 86(11):2278–2324; doi: 10.1109/5.726791
21.
DevlinJ, ChangM-W, LeeK, et al.BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv, 2019; 2019:arXiv181004805.
RadfordA, KimJW, XuT, et al.Robust speech recognition via large-scale weak supervision. arXiv, 2022; 2022:arcXiv:2212.04356.
24.
StevensSS, VolkmannJ, NewmanEB. A scale for the measurement of the psychological magnitude pitch. J Acoust Soc Am, 1937; 8(3):185–190; doi: 10.1121/1.1915893
25.
WangA, SinghA, MichaelJ, et al.GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv, 2019; 2019:arXiv180407461.
26.
HookerS. Moving beyond “algorithmic bias is a data problem.”. Patterns, 2021; 2(4):100241; doi: 10.1016/j.patter.2021.100241
27.
RossL, DanforthCM, EppsteinMJ, et al.Story arcs in serious illness: Natural language processing features of palliative care conversations. Patient Educ Couns, 2020; 103(4):826–832; doi: 10.1016/j.pec.2019.11.021
MinerAS, FlemingSL, HaqueA, et al.A computational approach to measure the linguistic characteristics of psychotherapy timing, responsiveness, and consistency. NPJ Ment Health Res, 2022; 1(1):1–12; doi: 10.1038/s44184-022-00020-9
30.
AlamF, DanieliM, RiccardiG. Annotating and modeling empathy in spoken conversations. Comput Speech Lang, 2018; 50:40–61; doi: 10.1016/j.csl.2017.12.003
31.
BadshahAM, RahimN, UllahN, et al.Deep features-based speech emotion recognition for smart affective services. Multimed Tools Appl, 2019; 78(5):5571–5589; doi: 10.1007/s11042-017-5292-7
32.
MustaqeemSK. Att-Net: Enhanced emotion recognition system using lightweight self-attention module. Appl Soft Comput, 2021; 102:107101; doi: 10.1016/j.asoc.2021.107101
33.
MustaqeemSK. A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition. Sensors, 2020; 20(1):183; doi: 10.3390/s20010183
34.
NgH-W, NguyenVD, VonikakisV, et al.Deep Learning for Emotion Recognition on Small Datasets Using Transfer Learning. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM: Seattle,Washington; 2015; pp. 443–449.
35.
ChenZ, GibsonJ, ChiuM-C, et al.Automated empathy detection for oncology encounters. arXiv, 2020; 2020:arXiv200700809.
36.
AminiS, HaoB, ZhangL, et al.Automated detection of mild cognitive impairment and dementia from voice recordings: A natural language processing approach. Alzheimers Dement, 2022; alz.12721. [Epub ahead of print]; doi: 10.1002/alz.12721
37.
KoeneckeA, NamA, LakeE, et al.Racial disparities in automated speech recognition. Proc Natl Acad Sci U S A, 2020; 117(14):7684–7689; doi: 10.1073/pnas.1915768117
38.
CohenS, MountBM, BrueraE, et al.Validity of the McGill Quality of Life Questionnaire in the palliative care setting: A multi-centre Canadian study demonstrating the importance of the existential domain. Palliat Med, 1997; 11(1):3–20; doi: 10.1177/026921639701100102
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
