Detecting deception in interpersonal communication is a pivotal issue in social psychology, with significant implications for court and criminal proceedings. In this study, four experiments were designed to compare the performance of natural language processing (NLP) techniques and human judges in detecting deception from linguistic cues in a dataset of 62 transcriptions of video-taped interviews (32 genuine and 30 deceptive). The results showed that machine-learning algorithms significantly outperform naïve (accuracy = 54.7%) and expert judges (accuracy = 59.4%) when trained on features from the reality monitoring (RM) and cognitive load frameworks (accuracy = 69.4%) or on features automatically extracted through NLP techniques (accuracy = 77.3%) but not when trained on the RM criteria alone. This evidence suggests that NLP algorithms, due to their ability to handle complex patterns of linguistic data, might be useful for better disentangling truthful from deceptive narratives, outperforming traditional theoretical models.
AmadoB. G.ArceR.FariñaF. (2015). Undeutsch hypothesis and criteria based content analysis: A meta-analytic review. The European Journal of Psychology Applied to Legal Context, 7(1), 3–12. https://doi.org/10.1016/j.ejpal.2014.11.002
3.
AmadoB. G.ArceR.FarinaF.VilarinoM. (2016). Criteria-based content analysis (CBCA) reality criteria in adults: A meta-analytic review. International Journal of Clinical and Health Psychology, 16(2), 201–210. https://doi.org/10.1016/j.ijchp.2016.01.002
BoydR. L.AshokkumarA.SerajS.PennebakerJ. W. (2022). The Development and Psychometric Properties of LIWC-22 (pp. 1–47). TX: University of Texas at Austin, https://doi.org/10.13140/RG.2.2.23890.43205
6.
CasoL.CavagnisL.VrijA.PalenaN. (2023). Cues to deception: Can complications, common knowledge details, and selfhandicapping strategies discriminate between truths, embedded lies and outright lies in an Italian-speaking sample?Frontiers in Psychology, 14, 1128194. https://doi.org/10.3389/fpsyg.2023.1128194
ConstâncioA. S.TsunodaD. F.SilvaH. d. F. N.SilveiraJ. M. d.CarvalhoD. R. (2023). Deception detection with machine learning: A systematic review and statistical analysis. PLoS One, 18(2), e0281323. https://doi.org/10.1371/journal.pone.0281323
9.
CurciA.LancianoT.BattistaF.GuaragnoS.RibattiR. M. (2019). Accuracy, confidence, and experiential criteria for lie detection through a videotaped interview. Frontiers in Psychiatry, 9, 283. https://doi.org/10.3389/fpsyt.2018.00748
10.
DeebH.VrijA.PalenaN.HypšováP.DibG.LealS.MannS. (2024). Honesty repeats itself: Comparing manual and automated coding on the veracity cues total details and redundancy. Applied Psycholinguistics, 45(5), 934–962. https://doi.org/10.1017/S0142716424000298
ElaadE. (2009). Lie-detection biases among male police interrogators, prisoners, and laypersons. Psychological Reports, 105(3_suppl.), 1047–1056. https://doi.org/10.2466/PR0.105.F.1047-1056
14.
FaulF.ErdfelderE.LangA.-G.BuchnerA. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
15.
GancedoY.FariñaF.SeijoD.VilariñoM.ArceR. (2021). Reality monitoring: A meta-analytical review for forensic practice. European Journal of Psychology Applied to Legal Context, 13(2), 99–110. https://doi.org/10.5093/ejpalc2021a10
16.
HartwigM.BondC. F.Jr (2014). Lie detection from multiple cues: A meta-analysis. Applied Cognitive Psychology, 28(5), 661–676. https://doi.org/10.1002/acp.3052
17.
HauchV.Blandón-GitlinI.MasipJ.SporerS. L. (2015). Are computers effective lie detectors? A meta-analysis of linguistic cues to deception. Personality and Social Psychology Review, 19(4), 307–342. https://doi.org/10.1177/1088868314556539
18.
HauchV.SporerS. L.MasipJ.Blandón-GitlinI. (2017). Can credibility criteria be assessed reliably? A meta-analysis of criteria-based content analysis. Psychological Assessment, 29(6), 819–834. https://doi.org/10.1037/pas0000426
KleinbergB.MozesM.ArntzA.VerschuereB. (2018a). Using named entities for computer-automated verbal deception detection. Journal of Forensic Sciences, 63(3), 714–723. https://doi.org/10.1111/1556-4029.13645
22.
KleinbergB.Van Der ToolenY.VrijA.ArntzA.VerschuereB. (2018b). Automated verbal credibility assessment of intentions: The model statement technique and predictive modeling. Applied Cognitive Psychology, 32(3), 354–366. https://doi.org/10.1002/acp.3407
23.
LaiS.LiuK.HeS.ZhaoJ. (2016). How to generate a good word embedding. IEEE Intelligent Systems, 31(6), 5–14. https://doi.org/10.1109/MIS.2016.45
24.
LancasterG. L.VrijA.HopeL.WallerB. (2013). Sorting the liars from the truth tellers: The benefits of asking unanticipated questions on lie detection. Applied Cognitive Psychology, 27(1), 107–114. https://doi.org/10.1002/acp.2879
25.
LevineT. R. (2014). Truth-default theory (TDT) a theory of human deception and deception detection. Journal of Language and Social Psychology, 33(4), 378–392. https://doi.org/10.1177/0261927X14535916
26.
LoconteR.RussoR.CapuozzoP.PietriniP.SartoriG. (2023). Verbal lie detection using large language models. Scientific Reports, 13(1), 22849. https://doi.org/10.1038/s41598-023-50214-0
27.
MasipJ.SporerS. L.GarridoE.HerreroC. (2005). The detection of deception with the reality monitoring approach: A review of the empirical evidence. Psychology, Crime & Law, 11(1), 99–122. https://doi.org/10.1080/10683160410001726356
28.
MelisG.UrsinoM.ScarpazzaC.ZangrossiA.SartoriG. (2024). Detecting lies in investigative interviews through the analysis of response latencies and error rates to unexpected questions. Scientific Reports, 14(1), 12268. https://doi.org/10.1038/s41598-024-63156-y
29.
MihalceaR.StrapparavaC. (2009, August). The lie detector: Explorations in the automatic recognition of deceptive language. In: Proceedings of the ACL-IJCNLP 2009 conference short papers (pp. 309–312).
30.
MonaroM.CapuozzoP.RagucciF.MaffeiA.CurciA.ScarpazzaC., Angrilli, A., & SartoriG. (2020). Using blink rate to detect deception: A study to validate an automatic blink detector and a new dataset of videos from liars and truth-tellers. In: Human–Computer Interaction. Human Values and Quality of Life: Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part III 22 (pp. 494–509). Springer International Publishing, https://doi.org/10.1007/978-3-030-49065-2_35
MonaroM.GamberiniL.SartoriG. (2017). The detection of faked identity using unexpected questions and mouse dynamics. PLoS One, 12(5), e0177851. https://doi.org/10.1371/journal.pone.0177851
33.
MonaroM.MalderaS.ScarpazzaC.SartoriG.NavarinN. (2022). Detecting deception through facial expressions in a dataset of videotaped interviews: A comparison between human judges and machine learning models. Computers in Human Behavior, 127, 107063. https://doi.org/10.1016/j.chb.2021.107063
34.
MüllerA. C.GuidoS. (2016). Introduction to machine learning with Python: A guide for data scientists. Reilly Media, Inc.
35.
NahariG. (2016). When the long road is the shortcut: A comparison between two coding methods for content-based lie-detection tools. Psychology, Crime & Law, 22(10), 1000–1014. https://doi.org/10.1080/1068316X.2016.1207770
36.
NahariG.VrijA.FisherR. P. (2014). Exploiting liars’ verbal strategies by examining the verifiability of details. Legal and Criminological Psychology, 19(2), 227–239. https://doi.org/10.1111/j.2044-8333.2012.02069.x
37.
NewmanM. L.PennebakerJ. W.BerryD. S.RichardsJ. M. (2003). Lying words: Predicting deception from linguistic styles. Personality and Social Psychology Bulletin, 29(5), 665–675. https://doi.org/10.1177/0146167203029005010
38.
OberladerV. A.NaefgenC.Koppehele-GosselJ.QuintenL.BanseR.SchmidtA. F. (2016). Validity of content-based techniques to distinguish true and fabricated statements: A meta-analysis. Law and Human Behavior, 40(4), 440–457. https://doi.org/10.1037/lhb0000193
39.
OberladerV. A.QuintenL.BanseR.VolbertR.SchmidtA. F.SchönbrodtF. D. (2021). Validity of content-based techniques for credibility assessment—How telling is an extended meta-analysis taking research bias into account?Applied Cognitive Psychology, 35(2), 393–410. https://doi.org/10.1002/acp.3776
40.
OttM.ChoiY.CardieC.HancockJ. T. (2011). Finding deceptive opinion spam by any stretch of the imagination. arXiv preprint arXiv:1107.4557.
41.
PalenaN.CasoL.VrijA.NahariG. (2021). The verifiability approach: A meta-analysis. Journal of Applied Research in Memory and Cognition, 10(1), 155–166. https://doi.org/10.1016/j.jarmac.2020.09.001
42.
PedregosaF.VaroquauxG.GramfortA.MichelV.ThirionB.GriselO., … & DuchesnayÉ. (2011). Scikit-learn: Machine learning in Python. The Journal of Machine Learning Research, 12, 2825–2830.
43.
PennebakerJ. W.FrancisM. E.BoothR. J. (2001). Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates, 71(2001), 2001.
44.
Pérez-RosasV.MihalceaR. (2015). Experiments in open domain deception detection. In Proceedings of the 2015 conference on empirical methods in Natural Language Processing, Lisbon, Portugal, September, 2015 (pp. 1120–1125).
SalminenJ.KandpalC.KamelA. M.JungS. G.JansenB. J. (2022). Creating and detecting fake reviews of online products. Journal of Retailing and Consumer Services, 64, 102771. https://doi.org/10.1016/j.jretconser.2021.102771
47.
Sarzynska-WawerJ.PawlakA.SzymanowskaJ.HanuszK.WawerA. (2023). Truth or lie: Exploring the language of deception. PLoS One, 18(2), e0281179. https://doi.org/10.1371/journal.pone.0281179
48.
SavyR. (2006). Specifiche per la trascrizione ortografica annotata dei testi raccolti. Italiano parlato. Analisi di un dialogo, 1–37.
49.
SchutteM.BogaardG.Mac GiollaE.WarmelinkL.KleinbergB. (2021). Man versus machine: Comparing manual with LIWC coding of perceptual and contextual details for verbal lie detection (preprint), https://doi.org/10.31234/osf.io/cth58
50.
ShroutP. E.FleissJ. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420. https://doi.org/10.1037/0033-2909.86.2.420
51.
Solà-SalesS.AlzettaC.Moret-TatayC.Dell’OrlettaF. (2023). Analysing deception in witness memory through linguistic styles in spontaneous language. Brain Sciences, 13(2), 317. https://doi.org/10.3390/brainsci13020317
52.
SporerS. L. (2004). Reality monitoring and detection of deception. In P.-A. Granhag & L. Strömwall (Eds.), The detection of deception in forensic contexts (pp. 64–101). Cambridge University Press. https://doi.org/10.1017/CBO9780511490071.004
StellerM.KoehnkenG. (1989). Criteria-based content analysis. The Suggestibility of Children’s Recollections. APA PsycTests. https://doi.org/10.1037/t27704-000
55.
StreetC. N.MasipJ. (2015). The source of the truth bias: Heuristic processing?Scandinavian Journal of Psychology, 56(3), 254–263. https://doi.org/10.1111/sjop.12204
56.
StrömwallL. A.BengtssonL.LeanderL.GranhagP. A. (2004). Assessing children’s statements: The impact of a repeated experience on CBCA and RM ratings. Applied Cognitive Psychology, 18(6), 653–668. https://doi.org/10.1002/acp.1021
57.
TomasF.DodierO.DemarchiS. (2022). Computational measures of deceptive language: Prospects and issues. Frontiers in Communication, 7, 792378. https://doi.org/10.3389/fcomm.2022.792378
58.
Van Der ZeeS.PoppeR.HavrileckA.BaillonA. (2022). A personal model of trumpery: Linguistic deception detection in a real-world high-stakes setting. Psychological Science, 33(1), 3–17. https://doi.org/10.1177/09567976211015941
59.
VeriginB. L.MeijerE. H.BogaardG.VrijA. (2019). Lie prevalence, lie characteristics and strategies of self-reported good liars. PLoS One, 14(12), e0225566. https://doi.org/10.1371/journal.pone.0225566
60.
VerschuereB.BogaardG.MeijerE. (2021). Discriminating deceptive from truthful statements using the verifiability approach: A meta-analysis. Applied Cognitive Psychology, 35(2), 374–384. https://doi.org/10.1002/acp.3775
61.
VerschuereB.LinC. C.HuismannS.KleinbergB.WillemseM.MeiE. C. J.MeijerE. (2023). The use-the-best heuristic facilitates deception detection. Nature Human Behaviour, 7(5), 718–728. https://doi.org/10.1038/s41562-023-01556-2
62.
Von EschenbachW. J. (2021). Transparency and the black box problem: Why we do not trust AI. Philosophy & Technology, 34(4), 1607–1622. https://doi.org/10.1007/s13347-018-0330-6
63.
VrijA. (2005). Criteria-based content analysis: A qualitative review of the first 37 studies. Psychology, Public Policy, and Law, 11(1), 3–41. https://doi.org/10.1037/1076-8971.11.1.3
64.
VrijA. (2008). Detecting Lies and Deceit: Pitfalls and Opportunities. John Wiley & Sons.
65.
VrijA.EdwardK.RobertsK. P.BullR. (2000). Detecting deceit via analysis of verbal and nonverbal behavior. Journal of Nonverbal Behavior, 24(4), 239–263. https://doi.org/10.1023/A:1006610329284
66.
VrijA.FisherR. P.BlankH. (2017). A cognitive approach to lie detection: A meta-analysis. Legal and Criminological Psychology, 22(1), 1–21. https://doi.org/10.1111/lcrp.12088
67.
VrijA.FisherR.MannS.LealS. (2008). A cognitive load approach to lie detection. Journal of Investigative Psychology and Offender Profiling, 5(1-2), 39–43. https://doi.org/10.1002/jip.82
68.
VrijA.GranhagP. A.AshkenaziT.GanisG.LealS.FisherR. P. (2022). Verbal lie detection: Its past, present and future. Brain Sciences, 12(12), 1644. https://doi.org/10.3390/brainsci12121644
69.
VrijA.LealS.GranhagP. A.MannS.FisherR. P.HillmanJ.SperryK. (2009). Outsmarting the liars: The benefit of asking unanticipated questions. Law and Human Behavior, 33, 159–166. https://doi.org/10.1007/s10979-008-9143-y
70.
VrijA.MannS.KristenS.FisherR. P. (2007). Cues to deception and ability to detect lies as a function of police interview styles. Law and Human Behavior, 31(5), 499–518. https://doi.org/10.1007/s10979-006-9066-4
71.
WalczykJ. J.IgouF. P.DixonA. P.TcholakianT. (2013). Advancing lie detection by inducing cognitive load on liars: A review of relevant theories and techniques guided by lessons from polygraph-based approaches. Frontiers in Psychology, 4, 14. https://doi.org/10.3389/fpsyg.2013.00014
72.
ZerilliJ.KnottA.MaclaurinJ.GavaghanC. (2019). Transparency in algorithmic and human decision-making: Is there a double standard?Philosophy & Technology, 32, 661–683. https://doi.org/10.1007/s00146-020-00960-w
73.
ZhouL.BurgoonJ. K.NunamakerJ. F.TwitchellD. (2004). Automating linguistics-based cues for detecting deception in text-based asynchronous computer-mediated communications. Group Decision and Negotiation, 13, 81–106. https://doi.org/10.1023/B:GRUP.0000011944.62889.6f
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
