The validity and utility of translated instruments (psychological measures) depend on the quality of their translation, and differences in key linguistic characteristics could introduce bias. Likewise, linguistic differences between instruments designed to measure analogous constructs might contribute to similar instruments possessing dissimilar psychometrics. This article introduces and demonstrates the use of natural language processing (NLP), a subfield of artificial intelligence, to linguistically analyze 13 translations of two psychological measures previously translated into numerous languages. NLP was used to generate estimates reflecting specific linguistic characteristics of test items (emotional tone/intensity, sentiment, valence, arousal, and dominance), which were then compared across translations at both the test- and item-level, as well as between the two instruments. Results revealed that key linguistic characteristics can profoundly vary both within and between tests. Following a discussion of results, the current limitations of this approach are summarized and strategies for advancing this methodology are proposed.
Argüello-GonzálezG.Aquino-EsperanzaJ.SalvadorD.Bretón-RomeroR.Del Río-BermudezC.TelloJ.MenkeS. (2023). Negation recognition in clinical natural language processing using a combination of the NegEx algorithm and a convolutional neural network. BMC Medical Informatics and Decision Making, 23, 216.
2.
BachB.BrownT. A.MulderR. T.Newton-HowesG.SimonsenE.SellbomM. (2021). Development and initial evaluation of the ICD-11 personality disorder severity scale: PDS-ICD-11. Personality and Mental Health, 15(3), 223–236.
3.
BarghJ. A.ChartrandT. L. (2000). The mind in the middle: A practical guide to priming and automaticity research. In ReisH. T.JuddC. M. (Eds.), Handbook of research methods in social and personality psychology (pp. 253–285). Cambridge University Press.
4.
BartzJ. A.LydonJ. E. (2004). Close relationships and the working self-concept: Implicit and explicit effects of priming attachment on agency and communion. Personality and Social Psychology Bulletin, 2004, 1389–1401.
5.
CombaluzierS.GouvernetB.AuvageL.BourgoiseC.MurphyP. (2023). Le modèle alternatif des troubles de la personnalité en population française : évaluation avec des outils brefs. [The alternative model of personality disorders among the French population: Assessment with brief tools]. L’Encephale, 49(5), 496–503.
6.
CottinM.MuzardA.HernándezC.MundtA.BehnA. (in preparation). Validation of the Spanish translation of the LPFS-BF 2.0 in a Chilean community and clinical sample.
7.
CrystalD. (1981). Clinical linguistics. Disorders of human communication (Vol. 3). Springer.
8.
CzarnekG.StillwellD. (2022). Two is better than one: Using a single emotion lexicon can lead to unreliable conclusions. PLoS One, 17(10), e0275910.
9.
ElgendiM.KumarP.BarbicS.HowardN.AbbottD.CichockiA. (2018). Subliminal Priming-State of the Art and Future Perspectives. Behavioral Sciences, 8(6), 54.
10.
GhaliJ. P. E.InuzukaN.ShimaK.MoriyamaK.MutohA. (2023). Automating lexicon generation: A comprehensive review of alternative approaches. Procedia Computer Science, 225, 1142–1150. https://doi.org/10.1016/j.procs.2023.10.102
11.
GrittiE. S.De CarliP.HutsebautJ.SimonelliA.ZimmermannJ. (in preparation). Cross-cultural adaptation of the Personality Functioning Scale- Brief Form 2.0 (LPFS-BF 2.0): Temporal stability and construct validity of the Italian version.
12.
GutiérrezF.AlujaA.RodríguezC.GárrizM.PeriJ. M.GallartS.CalvoN.FerrerM.Gutiérrez-ZotesA.SolerJ.PascualJ. C. (2023). Severity in the ICD-11 personality disorder model: Evaluation in a Spanish mixed sample. Frontiers in Psychiatry, 13, 1015489.
KamC. C. S. (2018). Why do we still have an impoverished understanding of the item wording effect? An empirical examination. Sociological Methods & Research, 47(3), 574–597.
15.
KooT. K.LiM. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163.
Le CorffY.AlujaA.RossiG.LapalmeM.ForgetK.GarcíaL. F.RollandJ. P. (2022). Construct Validity of the Dutch, English, French, and Spanish LPFS-BF 2.0: Measurement Invariance Across Language and Gender and Criterion Validity. Journal of personality disorders, 36(6), 662–679. https://doi.org/10.1521/pedi.2022.36.6.662
18.
LeeK. M.LeeJ.ChungC. Y.AhnS.SungK. H.KimT. W.LeeH. J.ParkM. S. (2012). Pitfalls and important issues in testing reliability using intraclass correlation coefficients in orthopaedic research. Clinics in Orthopedic Surgery, 4(2), 149–155.
19.
MahanyAKhaledHElmitwallyNSAljohaniNGhoniemyS. (2022). Negation and Speculation in NLP: A Survey, Corpora, Methods, and Applications. Applied Sciences, 12(10), 5209. https://doi.org/10.3390/app12105209
20.
MeyerD. E.SchvaneveldtR. W. (1971). Facilitation in recognizing pairs of words: evidence of a dependence between retrieval operations. Journal of Experimental Psychology, 90, 227.
21.
MohammadS. M. (2018a). Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia.
22.
MohammadS. M. (2018b). Word Affect Intensities. In Proceedings of the 11th Edition of the Language Resources and Evaluation Conference (LREC-2018), Miyazaki, Japan.
23.
MohammadS. M. (2020). Practical and ethical considerations in the effective use of emotion and sentiment lexicons. arXiv preprint. https://doi.org/10.48550/arXiv.2011.03492.
24.
MohammadS. M.TurneyP. D. (2013). Crowdsourcing a word-emotion association lexicon. Computational Intelligence, 29(3), 436–465.
25.
MoldenD. C. (2014). Understanding priming effects in social psychology: An overview and integration. Social Cognition, 32(Supplement), 243–249.
MukherjeeP.BadrY.DoppalapudiS.SrinivasanS. M.SangwanR. S.SharmaR. (2021). Effect of negation in sentences on sentiment analysis and polarity detection. Procedia Computer Science, 185, 370–379. https://doi.org/10.1016/j.procs.2021.05.038
28.
NatoliA. P.BachB.BehnA.Cottin ArredondoM.GrittiE. S.HutsebautJ.LambaN.Le CorffY.ZimmermannJ.LapalmeM. (2022). Multi-national evaluation of the measurement invariance of the Level of Personality Functioning Scale—Brief Form 2.0: Comparison of student and community samples across seven countries. Psychological Assessment, 34(12), 1112–1125. https://doi.org/10.1037/pas0001176
29.
NatoliA. P.RodriguezC. M. (2024). A new performance-based measure of personality functioning impairment: Development and preliminary evaluation of reliability and validity. Discover Mental Health, 4(1), 6. https://doi.org/10.1007/s44192-024-00059-4
30.
OitsaluM. L.KreegipuuM.HutsebautJ. (2022). Psychometric evaluation of the Estonian version of the semi-structured interview for personality functioning DSM-5 (STiP-5.1). Borderline Personality Disorder and Emotion Dysregulation, 9(1), 28.
31.
PodsakoffP. M.MacKenzieS. B.LeeJ. Y.PodsakoffN. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879
32.
PortneyL.G.WatkinsM.P. (2000). Foundations of clinical research: Applications to practice (2nd ed.). Prentice Hall Health.
33.
ProutskovaP.GruszczynskiM. (2007). Kripp.boot: An R Package for Performing Bootstrap Replicates of Krippendorff’s alpha on Intercoder Reliability Data. R package version 1.0.0. https://github.com/MikeGruz/kripp.boot
34.
RegmiK.NaidooJ.PilkingtonP. (2010). Understanding the Processes of Translation and Transliteration in Qualitative Research. International Journal of Qualitative Methods, 9, 16.
35.
RevelleW. (2024). psych: Procedures for psychological, psychometric, and personality research. R package version 2.4.6, https://CRAN.R-project.org/package=psych
SorrelM. A.GarcíaL. F.AlujaA.RollandJ. P.RossierJ.RoskamI.AbadF. J. (2021). Cross-cultural measurement invariance in the Personality Inventory for DSM-5. Psychiatry Research, 304, 114134. https://doi.org/10.1016/j.psychres.2021.114134
38.
SpitzerC.MüllerS.KerberA.HutsebautJ.BrählerE.ZimmermannJ. (2021). Die deutsche Version der Level of Personality Functioning Scale-Brief Form 2.0 (LPFS-BF): Faktorenstruktur, konvergente Validität und Normwerte in der Allgemeinbevölkerung. [The German version of the Level of Personality Functioning Scale-Brief Form 2.0 (LPFS-BF): Latent structure, convergent validity and norm values in the general population]. Psychotherapie, Psychosomatik, Medizinische Psychologie, 71(7), 284–293.
39.
StapelD. A.BlantonH. (2004). From seeing to being: Subliminal social comparisons affect implicit and explicit self-evaluations. Journal of Personality and Social Psychology, 87, 468.
40.
StapelD. A.KoomenW. (2000). Distinctness of others, mutability of selves: Their impact on self-evaluations. Journal of Personality and Social Psychology, 79, 1068–1087.
41.
StapelD. A.KoomenW. (2001). I, we, and the effects of others on me: How self-construal level moderates social comparison effects. Journal of Personality and Social Psychology, 80, 766–781.
42.
WeekersL. C.HutsebautJ.KamphuisJ. H. (2019). The level of personality functioning scale-brief form 2.0: Update of a brief instrument for assessing level of personality functioning. Personality and Mental Health, 13(1), 3–14. https://doi.org/10.1002/pmh.1434
43.
WeekersL. C.SellbomM.HutsebautJ.SimonsenS.BachB. (2023). Normative data for the LPFS-BF 2.0 derived from the Danish general population and relationship with psychosocial impairment. Personality and Mental Health, 17(2), 157–164.
44.
ZimmermannJ.MüllerS.BachB.HutsebautJ.HummelenB.FischerF. (2020). A common metric for self-reported severity of personality disorder. Psychopathology, 53(3), 161–171. https://doi.org/10.1159/000507377
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
