This work examined the relationship of the constructs measured by the trust scales developed by Chancey et al. (2017) and Jian et al. (2000) using a multilevel confirmatory factor analysis (CFA).
Background
Modern theories of automation trust have been proposed based on data collected using trust scales. Chancey et al. (2017) adapted Madsen and Gregor’s (2000) trust scale to align with Lee and See’s (2004) human–automation trust framework. In contrast, Jian et al. (2000) developed a scale empirically with trust and distrust as factors. However, it remains unclear whether these two scales measure the same construct.
Method
We analyzed data collected from previous experiments to investigate the relationship between the two trust scales using a multilevel CFA.
Results
Data provided evidence that Jian et al. (2000) and Chancey et al. (2017) automation trust scales are only weakly related. Trust and distrust are found to be distinct factors in Jian et al.’s (2000) scale, whereas performance, process, and purpose are distinct factors in Chancey et al.’s (2017) trust scale.
Conclusion
The analysis suggested that the two scales purporting to measure human–automation trust are only weakly related.
Application
Trust researchers and automation designers may consider using Chancey et al. (2017) and Jian et al. (2000) scales to capture different characteristics of human–automation trust.
BoltonM. L. (2022). Trust is not a virtue: Why we should not trust trust. Ergonomics in Design: The Quarterly of Human Factors Applications, 0(0), Article 106480462211301. https://doi.org/10.1177/10648046221130171
3.
ChanceyE. T. (2016). The effects of alarm system errors on dependence: Moderated mediation of trust with and without risk [Doctoral Dissertation, Old Dominion University]. ProQuest Dissertations & Theses Global.
4.
ChanceyE. T.BlissJ. P.ProapsA. B.MadhavanP. (2015). The role of trust as a mediator between system characteristics and response behaviors. Human Factors, 57(6), 947–958. https://doi.org/10.1177/0018720815582261
5.
ChanceyE. T.BlissJ. P.YamaniY.HandleyH. A. (2017). Trust and the compliance–reliance paradigm: The effects of risk, error bias, and reliability on trust and dependence. Human Factors, 59(3), 333–345. https://doi.org/10.1177/0018720816682648
6.
ChanceyE. T.PolitowiczM. S. (2020). Public trust and acceptance for concepts of remotely operated Urban Air Mobility transportation. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 64(1), 1044–1048. https://doi.org/10.1177/1071181320641251
7.
ChanceyE. T.PolitowiczM. S.Le VieL. (2021). Enabling advanced air mobility operations through appropriate trust in human-autonomy teaming: Foundational research approaches and applications. AIAA Scitech 2021 Forum.
8.
ChenJ. Y.BarnesM. J. (2014). Human–agent teaming for multirobot control: A review of human factors issues. IEEE Transactions on Human-Machine Systems, 44(1), 13–29. https://doi.org/10.1109/thms.2013.2293535
9.
ConchieS. M.TaylorP. J.CharltonA. (2011). Trust and distrust in safety leadership: mirror reflections?Safety Science, 8-9, 1208–1214.
10.
DelbecqA. L.Van de VenA. H.GustafsonD. H. (1975). Group techniques for program planning: A guide to nominal group and Delphi processes. Scott, Foresman.
11.
HancockP. A.BillingsD. R.SchaeferK. E.ChenJ. Y.De VisserE. J.ParasuramanR. (2011). A meta-analysis of factors affecting trust in human-robot interaction. Human Factors, 53(5), 517–527. https://doi.org/10.1177/0018720811417254
12.
HartwichF.WitzlackC.BeggiatoM.KremsJ. F. (2019). The first impression counts–A combined driving simulator and test track study on the development of trust and acceptance of highly automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 65, 522–535. https://doi.org/10.1016/j.trf.2018.05.012
13.
HayesA. F. (2017). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford publications.
14.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
15.
JianJ. Y.BisantzA. M.DruryC. G. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53–71. https://doi.org/10.1207/s15327566ijce0401_04
16.
KarpinskyN. D.ChanceyE. T.PalmerD. B.YamaniY. (2018). Automation trust and attention allocation in multitasking workspace. Applied Ergonomics, 70, 194–201. https://doi.org/10.1016/j.apergo.2018.03.008
17.
KohnS. C.de VisserE. J.WieseE.LeeY.ShawT. H. (2021). Measurement of trust in automation: A narrative review and reference guide. Frontiers in Psychology, 12, Article 604977. https://doi.org/10.3389/fpsyg.2021.604977
18.
LeeJ.YamaniY.LongS. K.UnverrichtJ.ItohM. (2021). Revisiting human- machine trust: A replication study of Muir and Moray (1996) using a simulated pasteurizer plant task. Ergonomics, 64(9), 1–14. https://doi.org/10.1080/00140139.2021.1909752
19.
LeeJ. D.MorayN. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35(10), 1243–1270. https://doi.org/10.1080/00140139208967392
20.
LeeJ. D.MorayN. (1994). Trust, self-confidence, and operators’ adaptation to automation. International Journal of Human-Computer Studies, 40(1), 153–184. https://doi.org/10.1006/ijhc.1994.1007
LewickiR. J.McAllisterD. J.BiesR. J. (1998). Trust and distrust: New relationships and realities. Academy of Management Review, 23(3), 438–458.
23.
LiF.DuncanT. E.HarmerP.AcockA.StoolmillerM. (1998). Analyzing measurement models of latent variables through multilevel confirmatory factor analysis and hierarchical linear modeling approaches. Structural Equation Modeling: A Multidisciplinary Journal, 5(3), 294–306. https://doi.org/10.1080/10705519809540106
24.
LongS. K.LeeJ.YamaniY.UnverrichtJ.ItohM. (2022). Does automation trust evolve from a leap of faith? An analysis using a reprogrammed pasteurizer simulation task. Applied Ergonomics, 100(6), Article 103674. https://doi.org/10.1016/j.apergo.2021.103674
25.
LongS. K.SatoT.MillnerN.LorangerR.MirabelliJ.XuV.YamaniY. (2020). Empirically and theoretically driven scales on automation trust: A multi-level confirmatory factor analysis. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 64(1), 1829–1832. https://doi.org/10.1177/1071181320641440
MadsenM.GregorS. (2000). Measuring human-computer trust. In Proceedings of 11th Australasian Conference on Information Systems (Vol. 53, pp. 6–8). ACM.
28.
MayerR. C.DavisJ. H.SchoormanF. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734. https://doi.org/10.5465/amr.1995.9508080335
29.
MeyerJ.WiczorekR.GunzlerT. (2014). Measures of reliance and compliance in aided visual scanning. Human Factors, 56(5), 840–849. https://doi.org/10.1177/0018720813512865
30.
MooreG. C.BenbasatI. (1991). Development of an instrument to measure the perceptions of adopting an information technology innovation. Information Systems Research, 2(3), 192–222. https://doi.org/10.1287/isre.2.3.192
31.
MorayN.InagakiT.ItohM. (2000). Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. Journal of Experimental Psychology: Applied, 6(1), 44–58. https://doi.org/10.1037//1076-898x.6.1.44
32.
MuirB. M. (1994). Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics, 37(11), 1905–1922. https://doi.org/10.1080/00140139408964957
33.
MuirB. M.MorayN. (1996). Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics, 39(3), 429–460. https://doi.org/10.1080/00140139608964474
34.
NamC. S.LyonsJ. B. (Eds.), (2020). Trust in human-robot interaction. Academic Press.
35.
NeumanS. P. (1994). Generalized scaling of permeabilities: Validation and effect of support scale. Geophysical Research Letters, 21(5), 349–352. https://doi.org/10.1029/94gl00308
36.
ParasuramanR.RileyV. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors: The Journal of the Human Factors and Ergonomics Society, 39(2), 230–253. https://doi.org/10.1518/001872097778543886
37.
PolitowiczM. S.ChanceyE. T.GlaabL. J. (2021). Effects of autonomous sUAS separation methods on subjective workload, situation awareness, and trust. AIAA SciTech.
38.
RaudenbushS. W.BrykeA. S. (2001). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Sage.
39.
RempelJ. K.HolmesJ. G.ZannaM. P. (1985). Trust in close relationships. Journal of Personality and Social Psychology, 49(1), 95–112. https://doi.org/10.1037//0022-3514.49.1.95
40.
RileyV. A. (1994). Human use of automation [Doctoral Dissertation, University of Minnesota]. ProQuest Dissertations Publishing.
41.
RuppM. A.MichaelisJ. R.McConnellD. S.SmitherJ. A. (2016). The impact of technological trust and self-determined motivation on intentions to use wearable fitness technology. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 60(1), 1434–1438. https://doi.org/10.1177/1541931213601329
42.
Santiago- EspadaY.MyerR. R.LatorellaK. A.ComstockJ. R. (2011). The multi- attribute task battery II (MATB- II) software for human performance and workload research: A user’s guide(NASA/TM-2011–217164).
43.
SatoT.IslamS.StillJ. D.ScerboM. W.YamaniY. (2023). Task priority reduces an adverse effect of task load on automation trust in a dynamic multitasking environment. Cognition, Technology & Work, 25(1), 1–13. https://doi.org/10.1007/s10111-022-00717-z
44.
SatoT.JacksonA.YamaniY. (2024) (In press). Number of interrupting events influences response time in multitasking, but not trust in automation. The International Journal of Aerospace Psychology. https://doi.org/10.1080/24721840.2024.2311706
45.
SatoT.LongS. K.YamaniY. (in preparation). The effects of multitasking on visual scanning and trust towards imperfect automation.
46.
SatoT.PolitowiczM. S.IslamS.ChanceyE. T.YamaniY. (2022). Attentional considerations in advanced air mobility operations: Control, manage, or assist?Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 66(1), 28–32. https://doi.org/10.1177/1071181322661184
47.
SatoT.YamaniY.LiechtyM.ChanceyE. T. (2020). Automation trust increases under high-workload multitasking scenarios involving risk. Cognition, Technology & Work, 22(2), 399–407. https://doi.org/10.1007/s10111-019-00580-5
48.
SitkinS. B.RothN. L. (1993). Explaining the limited effectiveness of legalistic “remedies” for trust/distrust. Organization Science, 4(3), 367–392.
49.
SorkinR. D.WoodsD. D. (1985). Systems with human monitors: A signal detection analysis. Human-Computer Interaction, 1(1), 49–75. https://doi.org/10.1207/s15327051hci0101_2
50.
SpainR. D.BustamanteE. A.BlissJ. P. (2008). Towards an empirically developed scale for system trust: Take two. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 52(19), 1335–1339. https://doi.org/10.1177/154193120805201907
51.
YamaniY.HorreyW. J. (2018). A theoretical model of human-automation interaction grounded in resource allocation policy during automated driving. International Journal of Human Factors and Ergonomics, 5(3), 225–239. https://doi.org/10.1504/ijhfe.2018.095912
52.
YamaniY.LongS. K.ItohM. (2020). Human–automation trust to technologies for naïve users amidst and following the COVID-19 pandemic. Human Factors, 62(7), 1087–1094. https://doi.org/10.1177/0018720820948981
53.
YamaniY.McCarleyJ. S. (2016). Workload capacity: A response time–based measure of automation dependence. Human Factors, 58(3), 462–471. https://doi.org/10.1177/0018720815621172
54.
YamaniY.McCarleyJ. S. (2018). Effects of task difficulty and display format on automation usage strategy: A workload capacity analysis. Human Factors, 60(4), 527–537. https://doi.org/10.1177/0018720818759356
55.
YamaniY.SatoT.JacksonA.PolitowiczM.ChanceyE. T. (in press). A theoretical approach to management of limited attentional resources to support the m:N operation in advanced air mobility ecosystem. In LawlessW. F.MittuR.SofgeD.HeshamH. (Eds.), Human-machine shared contexts. Elsevier Science.
