Trusting Automation: Designing for Responsivity and Resilience

Abstract

Objective

This paper reviews recent articles related to human trust in automation to guide research and design for increasingly capable automation in complex work environments.

Background

Two recent trends—the development of increasingly capable automation and the flattening of organizational hierarchies—suggest a reframing of trust in automation is needed.

Method

Many publications related to human trust and human–automation interaction were integrated in this narrative literature review.

Results

Much research has focused on calibrating human trust to promote appropriate reliance on automation. This approach neglects relational aspects of increasingly capable automation and system-level outcomes, such as cooperation and resilience. To address these limitations, we adopt a relational framing of trust based on the decision situation, semiotics, interaction sequence, and strategy. This relational framework stresses that the goal is not to maximize trust, or to even calibrate trust, but to support a process of trusting through automation responsivity.

Conclusion

This framing clarifies why future work on trust in automation should consider not just individual characteristics and how automation influences people, but also how people can influence automation and how interdependent interactions affect trusting automation. In these new technological and organizational contexts that shift human operators to co-operators of automation, automation responsivity and the ability to resolve conflicting goals may be more relevant than reliability and reliance for advancing system design.

Application

A conceptual model comprising four concepts—situation, semiotics, strategy, and sequence—can guide future trust research and design for automation responsivity and more resilient human–automation systems.

Keywords

trust autonomy human–automation interaction adaptive automation intelligent agents intersubjectivity

Get full access to this article

View all access options for this article.

References

Adams

Breazeal

Brooks

R. A.

Scassellati

. (2000). Humanoid robots: A new kind of tool. IEEE Intelligent Systems and Their Applications, 15, 25–31.doi:10.1109/5254.867909

Adler

P. S

. (1997). Work organization: From Taylorism to teamwork. Perspectives on Work, 61–65.

Adler

P. S

. (2001). Market, hierarchy, and trust: The knowledge economy and the future of capitalism. Organization Science, 12, 215–234.doi:10.1287/orsc.12.2.215.10117

Airenti

. (2018). The development of anthropomorphism in interaction: Intersubjectivity, imagination, and theory of mind. Frontiers in Psychology, 9, 1–13.doi:10.3389/fpsyg.2018.02136

Allen

J. E.

Guinn

Horvtz

. (1999). Mixed-initiative interaction. IEEE Intelligent Systems and Their Applications, 14, 14–23.doi:10.1109/5254.796083

Alsaid

Lee

J. D.

Price

. (2020). Moving into the loop: An investigation of drivers' steering behavior in highly automated vehicles. Human Factors: The Journal of the Human Factors and Ergonomics Society, 62, 671–683.doi:10.1177/0018720819850283

31180728

Alvarez-Melis

Jaakkola

. (2017). A causal framework for explaining the predictions of black-box sequence-to-sequence models. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 412–421.doi:10.18653/v1/D17-1042

Amershi

Cakmak

Knox

W. B.

Kulesza

. (2014). Power to the people: The role of humans in interactive machine learning. AI Magazine, 35, 105–120.doi:10.1609/aimag.v35i4.2513

Amershi

Weld

Vorvoreanu

Fourney

Nushi

Collisson

Horvitz

. (2019). Guidelines for human-AI interaction. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–13.doi:10.1145/3290605.3300233

10.

Ananny

Crawford

. (2018). Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society, 20, 973–989.doi:10.1177/1461444816676645

11.

Arrow

. (1974). The limits of organizations (First). Fels Center of Government.

12.

Atakan

S. S.

Bagozzi

R. P.

Yoon

. (2014). Consumer participation in the design and realization stages of production: How self-production shapes consumer evaluations and relationships to products. International Journal of Research in Marketing, 31, 395–408.doi:10.1016/j.ijresmar.2014.05.003

13.

Axelrod

. (1984). The evolution of cooperation. Basic Books.

14.

Babel

Kraus

Miller

Kraus

Wagner

Minker

Baumann

. (2021). Small talk with a robot? The impact of dialog content, talk initiative, and gaze behavior of a social robot on trust, acceptance, and proximity. International Journal of Social Robotics, 6.doi:10.1007/s12369-020-00730-0

15.

Bansal

Nushi

Kamar

Horvitz

Weld

D. S

. (2020). Optimizing AI for teamwork. Retrieved January 25, 2021, fromhttp://arxiv.org/abs/2004.13102

16.

Barnes

Gartland

Stack

. (2004). Old habits die hard: Path dependency and behavioral lock-in. Journal of Economic Issues, 38, 371–377.doi:10.1080/00213624.2004.11506696

17.

Bartlett

C. E.

Cooke

N. J

. (2015). Human-robot teaming in urban search and rescue. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59, 250–254.doi:10.1177/1541931215591051

18.

Bastian

Jetten

Thai

H. A.

Steffens

N. K

. (2018). Shared adversity increases team creativity through fostering supportive interaction. Frontiers in Psychology, 9, 1–10.doi:10.3389/fpsyg.2018.02309

19.

Becchio

Sartori

Castiello

. (2010). Toward you: The social side of actions. Current Directions in Psychological Science, 19, 183–188.doi:10.1177/0963721410370131

20.

Berg

Dickhaut

McCabe

. (1995). Trust, reciprocity, and social history. Games and Economic Behavior, 10, 122–142.doi:10.1006/game.1995.1027

21.

Bertuccelli

L. F.

Cummings

M. L

. (2011). Scenario-based robust scheduling for collaborative human-UAV visual search tasks. 50th IEEE Conference on Decision and Control and European Control Conference, 5702–5707.doi:10.1109/CDC.2011.6160994

22.

Bethel

C. L.

Salomon

Murphy

R. R.

Burke

J. L

. (2007). Survey of psychophysiology measurements applied to human-robot interaction. Proceedings - IEEE International Workshop on Robot and Human Interactive Communication, 732–737.doi:doi.org/10.1109/ROMAN.2007.4415182

23.

Blomqvist

Stahle

. (2004). Trust in technology partnerships. In Huotari

Ivonen

(Eds.), Trust in knowledge management and systems in organizations (pp. 173–199). Idea Group Publishing.doi:10.4018/978-1-59140-126-1.ch008

24.

Bobko

Barelka

A. J.

Hirshfield

L. M

. (2014). The construct of state-level suspicion: A model and research agenda for automated and information technology (IT) contexts. Human Factors, 56, 489–508.doi:10.1177/0018720813497052

24930171

25.

Bowling

Burch

Johanson

Tammelin

. (2015). Heads-up limit hold ’ em poker is solved. Science, 347, 145–149.doi:10.1126/science.1259433

26.

Bradshaw

J. M.

Feltovich

P. J.

Hoffman

R. R.

Klein

Woods

D. D

. (2004). Ten challenges for making automation a “team player” in joint human-agent activity. IEEE Intelligent Systems, 19, 91–95.

27.

Breazeal

Scassellati

. (1999). How to build robots that make friends and influence people. Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289), 2, 858–863.doi:https://doi.org/10.1109/IROS.1999.812787

28.

Brooks

R. A

. (1991). Intelligence without representation. Artificial Intelligence, 47, 139–159.doi:10.1016/0004-3702(91)90053-M

29.

Bryson

J. J.

Diamantis

M. E.

Grant

T. D

. (2017). Of, for, and by the people: The legal lacuna of synthetic persons. Artificial Intelligence and Law, 25, 273–291.doi:10.1007/s10506-017-9214-9

30.

Bryson

J. J.

Theodorou

. (2019). How society can maintain human-centric artificial intelligence. In Marja

T. -N.

Eveliina

(Eds.), Human-Centered Digitalization and Services. Springer. http://www.cs.bath.ac.uk/~jjb/web/publications.html.

31.

Casper

Murphy

R. R

. (2003). Human-robot interactions during the robot-assisted urban search and rescue response at the world Trade center. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 33, 367–385.doi:10.1109/TSMCB.2003.811794

18238185

32.

Cassell

Bickmore

. (2000). External manifestations of trustworthiness in the interface. Communications of the ACM, 43, 50–56.doi:10.1145/355112.355123

33.

Castelfranchi

. (1998). Modelling social action for AI agents. Artificial Intelligence, 103, 157–182.doi:10.1016/S0004-3702(98)00056-3

34.

Castelfranchi

Falcone

. (2000). Trust and control: A dialectic link. Applied Artificial Intelligence, 14, 799–823.doi:10.1080/08839510050127560

35.

Castelfranchi

Falcone

. (2010). Trust and technology. In Trust theory: A socio-cognitive and computational model (pp. 343–357). John Wiley & Sons, Ltd. http://onlinelibrary.wiley.com/doi/.

36.

Celmer

Branaghan

Chiou

. (2018). Trust in branded autonomous vehicles & performance expectations: A theoretical framework. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 62, 1761–1765.doi:10.1177/1541931218621398

37.

Chiou

E. K.

Lee

J. D

. (2015). Beyond reliance and compliance: Human-automation coordination and cooperation. Proceedings of the Human Factors and Ergonomics Society, 195–199.doi:10.1177/0018720816649094

38.

Chiou

E. K.

Lee

J. D

. (2016). Cooperation in human-agent systems to support resilience: A microworld experiment. Human Factors, 58, 846–863.doi:10.1177/0018720816649094

27178676

39.

Chiou

E. K.

Lee

J. D.

. (2019). Negotiated and reciprocal exchange structures in human-agent cooperation. Computers in Human Behavior, 90, 288–297.doi:10.1016/j.chb.2018.08.012

40.

Christoffersen

Woods

D. D

. (2002). How to make automated systems team players. In Salas

(Ed.), Advances in human performance and cognitive engineering research. (Vol. 2, pp. 1–13). JAI Press.

41.

Clark

H. H

. (1996). Using language. Cambridge University Press.

42.

Clark

H. H.

Brennan

S. E

. (1991). Grounding in communication. In Resnick

L. B.

Levine

J. M.

Treasley

S. D.

(Eds.), Perspectives on socially shared cognition (pp. 127–149). American Psychological Association.

43.

Colman

A. M.

Pulford

B. D.

Rose

. (2008). Collective rationality in interactive decisions: Evidence for team reasoning. Acta Psychologica, 128, 387–397.doi:10.1016/j.actpsy.2007.08.003

17868630

44.

Cooke

N. J.

Gorman

J. C.

Myers

C. W.

Duran

J. L

. (2013). Interactive team cognition. Cognitive Science, 37, 255–285.doi:10.1111/cogs.12009

23167661

45.

Cooper

. (2007). Cognitive dissonance: Fifty years of a classic theory. SAGE.

46.

Cramer

Evers

Ramlal

van Someren

Rutledge

Stash

Aroyo

Wielinga

. (2008). The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction, 18, 455–496.doi:10.1007/s11257-008-9051-3

47.

Cramer

H. S. M.

Evers

van Someren

M. W.

Wielinga

B. J

. (2009). Awareness, training and trust in interaction with adaptive spam filters. CHI 09: Proceedings of the 27th International Conference on Human Factors in Computing Systems, 909–912.doi:10.1145/1518701.1518839

48.

Cummings

T. G

. (1978). Self-Regulating work groups: A socio-technical synthesis. Academy of Management Review, 3, 625–634.doi:10.5465/amr.1978.4305900

49.

Dabbish

Kraut

R. E

. (2004). Controlling interruptions. In Proceedings of the 2004 ACM conference on Computer supported cooperative work - CSCW ’04. (Vol. 6, pp. 182–191). ACM Press.

50.

de Souza

C. S

. (2017). Semiotics and human-computer interaction. In Norman

Kirakowski

(Eds.), The Wiley handbook of human computer interaction (pp. 33–49). John Wiley & Sons, Ltd.

51.

de Visser

E. J.

Pak

Shaw

T. H

. (2018). From ‘automation’ to ‘autonomy’: The importance of trust repair in human–machine interaction. Ergonomics, 61, 1409–1427.doi:10.1080/00140139.2018.1457725

29578376

52.

de Visser

E. J.

Parasuraman

. (2011). Adaptive aiding of human-robot teaming: Effects of imperfect automation on performance, trust, and workload. Journal of Cognitive Engineering and Decision Making, 5, 209–231.doi:10.1177/1555343411410160

53.

de Visser

E. J.

Peeters

M. M. M.

Jung

M. F.

Kohn

Shaw

T. H.

Pak

Neerincx

M. A

. (2020). Towards a theory of longitudinal trust calibration in human–robot teams. International Journal of Social Robotics, 12, 459–478.doi:10.1007/s12369-019-00596-x

54.

Defense Science Board . (2012). Task force report: The role of autonomy in DoD systems. Washington, D.C.. Retrieved from http://www.dtic.mil/get-tr-doc/pdf?AD=ADA566864.

55.

Degani

Shafto

Kirlik

. (1999). Modes in human-machine systems: Review, classification, and application. The International Journal of Aviation Psychology, 9, 125–138.

56.

Dietvorst

B. J.

Simmons

J. P.

Massey

. (2014). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology, 143, 1–13.

57.

Dietvorst

B. J.

Simmons

J. P.

Massey

. (2018). Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Science, 64, 1155–1170.doi:10.1287/mnsc.2016.2643

58.

Dohle

Rall

Siegrist

. (2014). I cooked it myself: Preparing food increases liking and consumption. Food Quality and Preference, 33, 14–16.doi:10.1016/j.foodqual.2013.11.001

59.

Domeyer

Dinparastdjadid

Lee

J. D.

Douglas

Alsaid

Price

. (2019). Proxemics and kinesics in automated vehicle–pedestrian communication: Representing ethnographic observations. Transportation Research Record: Journal of the Transportation Research Board, 2673, 70–81.doi:10.1177/0361198119848413

60.

Domeyer

J. E.

Lee

J. D.

Toyoda

. (2020). Vehicle automation–other road user communication and coordination: Theory and mechanisms. IEEE Access, 8, 19860–19872.doi:10.1109/ACCESS.2020.2969233

61.

Dragan

A. D.

Lee

K. C. T.

Srinivasa

S. S

. (2013). Legibility and predictability of robot motion. ACM/IEEE International Conference on Human-Robot Interaction, 1, 301–308.doi:10.1109/HRI.2013.6483603

62.

Dugatkin

L. A.

Mesterton-Gibbonsand

Houston

A. I

. (1992). Beyond the prisoner’s dilemma: Toward models to discriminate among mechanisms of cooperation in nature. Trends in Ecology & Evolution, 7, 202–205.doi:10.1016/0169-5347(92)90074-L

21236008

63.

Duhigg

. (2016). What Google learned from its quest to build the perfect team.The New York Times Magazine. https://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-quest-to-build-the-perfect-team.html

64.

Dzindolet

M. T.

Peterson

S. A.

Pomranky

R. A.

Pierce

L. G.

Beck

H. P

. (2003). The role of trust in automation reliance. International Journal of Human-Computer Studies, 58, 697–718.doi:10.1016/S1071-5819(03)00038-7

65.

Dzindolet

M. T.

Pierce

L. G.

Beck

H. P.

Dawe

L. A

. (2002). The perceived utility of human and automated AIDS in a visual detection task. Human Factors: The Journal of the Human Factors and Ergonomics Society, 44, 79–94.doi:10.1518/0018720024494856

12118875

66.

Eisenberg

Losoya

Spinrad

. (1993). Affect and prosocial responding. In Davidson

R. J.

Scherer

K. R.

Goldsmith

H. H.

(Eds.), Handbook of affective sciences (pp. 787–803). Oxford University Press, Inc.

67.

Emirbayer

Mische

. (1998). What is agency? American Journal of Sociology, 103, 962–1023.doi:10.1086/231294

68.

Endsley

M. R

. (2015). Autonomous horizons. https://www.af.mil/Portals/1/documents/SECAF/AutonomousHorizons.pdf?timestamp=1435068339702

69.

Engström

Bärgman

Nilsson

Seppelt

Markkula

Piccinini

G. B.

Victor

. (2018). Great expectations: A predictive processing account of automobile driving. Theoretical Issues in Ergonomics Science, 19, 156–194.doi:10.1080/1463922X.2017.1306148

70.

Epley

Waytz

Cacioppo

J. T

. (2007). On seeing human: A three-factor theory of anthropomorphism. Psychological Review, 114, 864–886.doi:10.1037/0033-295X.114.4.864

17907867

71.

Executive Office of the President . (2016). Artificial intelligence, automation, and the economy. https://obamawhitehouse.archives.gov/sites/whitehouse.gov/files/documents/Artificial-Intelligence-Automation-Economy.PDF

72.

Fallon

C. K.

Murphy

A. K. G.

Zimmerman

Mueller

S. T

. (2010). The calibration of trust in an automated system: A sensemaking process. 2010 International Symposium on Collaborative Technologies and Systems, 390–395.doi:10.1109/CTS.2010.5478488

73.

Ferrucci

D. A

. (2012). Introduction to “this is Watson”. IBM journal of research and development, 56, 1:1–1:15.doi:10.1147/JRD.2012.2184356

74.

Fickinger

Zhuang

Hadfield-Menell

Russell

. (2020). Multi-principal assistance games. Retrieved January 16, 2021 fromhttp://arxiv.org/abs/2007.09540

75.

Fiore

S. M.

Wiltshire

T. J.

Lobato

E. J. C.

Jentsch

F. G.

Huang

W. H.

Axelrod

. (2013). Toward understanding social cues and signals in human–robot interaction: Effects of robot gaze and proxemic behavior. Frontiers in Psychology, 4, 1–15.doi:10.3389/fpsyg.2013.00859

76.

Flach

. (2017). Supporting productive thinking: The semiotic context for cognitive systems engineering (CSE). Applied Ergonomics, 59, 612–624.doi:10.1016/j.apergo.2015.09.001

26395066

77.

Flach

J. M.

Bennett

K. B.

Jagacinski

R. J.

Mulder

van Paassen

. (2013). The closed-loop dynamics of cognitive work. In Lee

J. D.

Kirlik

(Eds.), The Oxford handbook of cognitive engineering (pp. 1–45). Oxford University Press.

78.

Fong

Nourbakhsh

Dautenhahn

. (2003). A survey of socially interactive robots. Robotics and Autonomous Systems, 42, 143–166.doi:10.1016/S0921-8890(02)00372-X

79.

Frey

C. B.

Osborne

M. A

. (2013). The future of employment: How susceptible are jobs to computerisation? Retrieved from https://www.oxfordmartin.ox.ac.uk/publications/the-future-of-employment/.

80.

Frey

C. B.

Osborne

M. A

. (2017). The future of employment: How susceptible are jobs to computerisation? Technological Forecasting and Social Change, 114, 254–280.doi:10.1016/j.techfore.2016.08.019

81.

Friedland

. (1990). Attribution of control as a determinant of cooperation in exchange interactions. Journal of Applied Social Psychology, 20, 303–320.doi:10.1111/j.1559-1816.1990.tb00413.x

82.

Friedman

Kahn

P. H

. (1992). Human agency and responsible computing. The Journal of Systems and Software, 17, 7–14.

83.

Gallotti

Frith

C. D

. (2013). Social cognition in the we-mode. Trends in Cognitive Sciences, 17, 160–165.doi:10.1016/j.tics.2013.02.002

23499335

84.

Gao

Lee

J. D.

Zhang

. (2006). A dynamic model of interaction between reliance on automation and cooperation in multi-operator multi-automation situations. International Journal of Industrial Ergonomics, 36, 511–526.doi:10.1016/j.ergon.2006.01.013

85.

Garbis

Artman

. (1998). Team communication and coordination as distributed cognition. Proceedings of 9th Conference of Cognitive Ergonomics: Cognition and Cooperation, 151–156.

86.

Gawande

. (2009). The checklist manifesto. Henry Holt and Company, LLC.

87.

Glikson

Woolley

A. W

. (2020). Human trust in artificial intelligence: Review of empirical research. Academy of Management Annals, 14, 627–660.doi:10.5465/annals.2018.0057

88.

Goldstein

D. G.

Johnson

E. J.

Herrmann

Heitmann

. (2008). Nudge your customers toward better choices. Harvard Business Review. https://hbr.org/2008/12/nudge-your-customers-toward-better-choices

89.

Gottman

Swanson

. (2002). A general systems theory of marriage: Nonlinear difference equation modeling of marital interaction. Personality and Social Psychology Review, 6, 326–340.doi:10.1207/S15327957PSPR0604_07

90.

Groom

Nass

. (2007). Can robots be teammates?: Benchmarks in human-robot teams. Interaction studies, 8, 285–301.doi:10.1075/gest.8.3.02str

91.

Groom

Takayama

Ochi

Nass

. (2009). I am my robot: The impact of robot-building and robot form on operators. In Human-robot interaction (pp. 11–16).doi:10.1145/1514095.1514104

92.

Gutzwiller

R. S.

Reeder

. (2017). Human interactive machine learning for trust in teams of autonomous robots. 2017 IEEE Conference on Cognitive and Computational Aspects of Situation Management, CogSIMA 2017, 1–3.doi:10.1109/COGSIMA.2017.7929607

93.

Gutzwiller

R. S.

Reeder

. (2020). Dancing with algorithms: Interaction creates greater preference and trust in machine-learned behavior. Human Factors: The Journal of the Human Factors and Ergonomics Society, 117doi:10.1177/0018720820903893

32048883

94.

Hancock

P. A.

Billings

D. R.

Schaefer

K. E.

Chen

J. Y. C.

De Visser

E. J.

Parasuraman

Szalma

. (2011). A meta-analysis of factors affecting trust in human-robot interaction. Human Factors: The Journal of the Human Factors and Ergonomics Society, 53, 517–527.

95.

Hancock

P. A.

Nourbakhsh

Stewart

. (2019). On the future of transportation in an era of automated and autonomous vehicles. Proceedings of the National Academy of Sciences of the United States of America, 116, 7684–7691.

96.

Heider

Simmel

. (1944). An experimental study of apparent behavior. The American Journal of Psychology, 57, 243–259.doi:10.2307/1416950

97.

Hill

C. W. L

. (1990). Cooperation, opportunism, and the invisible hand: Implications for transaction cost theory. Academy of Management Review, 15, 500–513.doi:10.5465/amr.1990.4309111

98.

Hoc

J. -M

. (2001). Towards a cognitive approach to human-machine cooperation in dynamic situations. International Journal of Human-Computer Studies, 54, 509–540.doi:10.1006/ijhc.2000.0454

99.

Hoegen

Stratou

Lucas

Gratch

. (2015). Comparing behavior towards humans and virtual humans in a social dilemma. Lecture Notes in Computer Science, 5208, 452–460.doi:10.1007/978-3-540-85483-8

100.

Hoff

K. A.

Bashir

. (2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57, 407–434.doi:10.1177/0018720814547570

25875432

101.

Hall

J. L

. (2003). Columbia and Challenger: Organizational failure at NASA. Space Policy, 19, 239–247.doi:10.1016/j.spacepol.2003.08.013

102.

Han

Schulz

H. -J

. (2020). Beyond trust building — Calibrating trust in visual analytics. In 2020 IEEE workshop on trust and expertise in visual analytics (TREX) (pp. 9–15). IEEE.doi:10.1109/TREX51495.2020.00006

103.

Hoffman

R. R.

Johnson

Bradshaw

J. M.

Underbrink

Hoffman

R. R.

Johnson

Bradshaw

J. M

. (2013). Trust in automation. IEEE Intelligent Systems, 28, 84–88.doi:10.1109/MIS.2013.24

104.

Hollnagel

Woods

D. D.

Leveson

. (Eds.). (2006). Resilience engineering: Concepts and precepts: Ashgate Publishing Limited.

105.

Holmes

J. G

. (2004). The benefits of abstract functional analysis in theory construction: The case of interdependence theory. Personality and Social Psychology Review, 8, 146–155.doi:10.1207/s15327957pspr0802_8

15223514

106.

Holmes

J. G.

Rempel

J. K

. (1989). Trust in close relationships. In Hendrick

(Ed.), Close relationships: Review of personality and social psychology. (Vol. 10, pp. 187–220). Sage Publicationsdoi:<br>

107.

Hoonakker

P. L. T.

Pecanac

K. E.

Brown

R. L.

Carayon

. (2017). Virtual collaboration, satisfaction, and trust between nurses in the tele-ICU and ICUs: Results of a multilevel analysis. Journal of Critical Care, 37, 224–229.doi:10.1016/j.jcrc.2016.10.018

27876482

108.

Horvitz

. (1999). Principles of mixed-initiative user interfaces. In Conference on Human Factors in Computing Systems (CHI 99) (pp. 159–166). ACM.

109.

Huang

Cooke

N. J.

Gutzwiller

R. S.

Berman

Chiou

E. K.

Demir

Zhang

. (2020). Distributed dynamic team trust in human, artificial intelligence, and robot teaming. In Trust in human-robot interaction (pp. 301–319). Elsevier.

110.

Hudson

Fogarty

Atkeson

Avrahami

Forlizzi

Kiesler

Lee

Yang

. (2003). Predicting human interruptibility with sensors: A wizard of Oz feasibility study. CHI '03: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 257–264.doi:10.1145/642611.642657

111.

Inbar

Tractinsky

. (2009). The incidental user. Interactions, 16, 56–59.doi:10.1145/1551986.1551998

112.

Israelsen

B. W.

Ahmed

N. R

. (2019). “Dave...I can assure you ...that it’s going to be all right ...” A definition, case for, and survey of algorithmic assurances in human-autonomy trust relationships. ACM Computing Surveys, 51, 1–37.doi:10.1145/3267338

113.

Jian

J. -Y.

Bisantz

A. M.

Drury

C. G

. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4, 53–71.doi:10.1207/S15327566IJCE0401_04

114.

Johnson

E. J.

Goldstein

. (2003). Do defaults save lives? Science, 302, 1338–1339.doi:10.1126/science.1091721

14631022

115.

Johnson

Bradshaw

J. M.

Feltovich

P. J.

Jonker

C. M.

Van Riemsdijk

M. B.

Sierhuis

. (2014). Coactive design: Designing support for interdependence in joint activity. Journal of Human-Robot Interaction, 3, 43.doi:10.5898/JHRI.3.1.Johnson

116.

Jung

M. F.

Martelaro

Hinds

P. J

. (2015). Using robots to moderate team conflict: The case of repairing violations. In ACM/IEEE international conference on human-robot interaction (pp. 229–236).doi:doi.org/10.1145/2696454.2696460

117.

Kazi

T. A.

Stanton

N. A.

Walker

G. H.

Young

M. S

. (2007). Designer driving: Drivers’ conceptual models and level of trust in adaptive cruise control. International Journal of Vehicle Design, 45, 339.doi:10.1504/IJVD.2007.014909

118.

Kelley

H. H

. (1984). The theoretical description of interdependence by means of transition lists. Journal of Personality and Social Psychology, 47, 956–982.doi:10.1037/0022-3514.47.5.956

119.

Kelley

H. H.

Holmes

J. G.

Kerr

N. L.

Reis

H. T.

Rusbult

C. E.

Van Lange

P. A. M

. (2003). An atlas of interpersonal situations. Cambridge University Press.

120.

Khasawneh

Bowling

Jiang

Gramopadhye

. (2003). A model for predicting human trust in automated systems. Proceedings of the 8th Annual International Conference on Industrial Engineering — Theory, Applications and Practice, 216–222.

121.

Kim

P. H.

Dirks

K. T.

Cooper

C. D

. (2009). The repair of trust: A dynamic bilateral perspective and multilevel conceptualization. Academy of Management Review, 34, 401–422.doi:10.5465/amr.2009.40631887

122.

Kirlik

. (1993). Modeling strategic behavior in human-automation interaction: Why an “aid” can (and should) go unused. Human Factors: The Journal of the Human Factors and Ergonomics Society, 35, 221–242.doi:10.1177/001872089303500203

8349287

123.

Klein

Feltovich

P. J.

Bradshaw

J. M.

Woods

D. D

. (2005). Common ground and coordination in joint activity. In Rouse

W. B.

Kenneth

R. B.

(Eds.), Organizational Simulation (pp. 139–184). John Wiley & Songs, Inc. http://csel.eng.ohio-state.edu/woods/distributed/CG final.pdf.

124.

Klein Moon Hoffman . (2006). Making sense of sensemaking 2: A macrocognitive model. IEEE Intelligent Systems, 21, 88–92.doi:10.1109/MIS.2006.100

125.

Knight

. (2013). Smart robots can now work right next to auto workers. MIT Technology Review. http://www.technologyreview.com/news/518661/smart-robots-can-now-work-right-next-to-auto-workers/

126.

Kohn

S. C.

Quinn

Pak

de Visser

E. J.

Shaw

T. H

. (2018). Trust repair strategies with self-driving vehicles: An exploratory study. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 62, 1108–1112.doi:10.1177/1541931218621254

127.

Lasseter

. (1987). Principles of traditional animation applied to 3D computer animation. ACM SIGGRAPH Computer Graphics, 21, 35–44.doi:10.1145/37402.37407

128.

Lee

Moray

. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35, 1243–1270.doi:10.1080/00140139208967392

1516577

129.

Lee

J. D

. (2009). Can technology get your eyes back on the road? Science, 324, 344–346.doi:10.1126/science.1168085

19372419

130.

Lee

J. D

. (2020). Trust in automated, intelligent, and connected vehicles. In Fisher

D. L.

Horrey

W. J.

Lee

J. D.

Regan

M. A.

(Eds.), Handbook of human factors for automated, connected, and intelligent vehicles. Boca Raton.

131.

Lee

J. D.

Hoffman

Hayes

. (2004). Collision warning design to migitate driver distraction. CHI Letters, 6, 65–72.

132.

Lee

J. D.

Kolodge

. (2020). Exploring trust in self-driving vehicles through text analysis. Human Factors: The Journal of the Human Factors and Ergonomics Society, 62, 260–277.doi:10.1177/0018720819872672

31502885

133.

Lee

J. D.

See

K. A

. (2004). Trust in automation: Designing for appropriate reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 46, 50–80.doi:10.1518/hfes.46.1.50.30392

15151155

134.

Lewicki

R. J.

McAllister

D. J.

Bies

R. J

. (1998). Trust and distrust: New relationships and realities. Academy of Management Review, 23, 438–458.doi:10.5465/amr.1998.926620

135.

Liew

T. W.

Tan

S. M.

Jayothisa

. (2013). The effects of peer-like and expert-like pedagogical agents on learners’ agent perceptions, task-related attitudes, and learning achievement. Educational Technology and Society, 16, 275–286.

136.

Loewenstein

Lerner

. (2003). The role of affect in decision making. In Davidson

Scherer

Goldsmith

(Eds.), Handbook of affective science (pp. 619–642). Oxford University Press.

137.

Lowry

P. B.

Zhang

Zhou

. (2010). Effects of culture, social presence, and group composition on trust in technology-supported decision-making groups. Information Systems Journal, 20, 297–315.doi:10.1111/j.1365-2575.2009.00334.x

138.

Lyons

J. B.

Guznov

S. Y

. (2019). Individual differences in human–machine trust: A multi-study look at the perfect automation schema. Theoretical Issues in Ergonomics Science, 20, 440–458.doi:10.1080/1463922X.2018.1491071

139.

Lyons

J. B.

Stokes

C. K

. (2012). Human–Human reliance in the context of automation. Human Factors: The Journal of the Human Factors and Ergonomics Society, 54, 112–121.doi:10.1177/0018720811427034

140.

Madhavan

Wiegmann

D. A

. (2007). Similarities and differences between human–human and human–automation trust: An integrative review. Theoretical Issues in Ergonomics Science, 8, 277–301.doi:10.1080/14639220500337708

141.

Madhavan

Wiegmann

D. A.

Lacson

F. C

. (2006). Automation failures on tasks easily performed by operators undermine trust in automated aids. Human Factors: The Journal of the Human Factors and Ergonomics Society, 48, 241–256.doi:10.1518/001872006777724408

16884046

142.

Madsen

Gregor

. (2000). Measuring human-computer trust. In Proceedings of the 11th Australasian Conference on Information Systems (pp. 6–8).

143.

Malle

B. F.

Scheutz

Austerweil

J. L

. (2017). Networks of social and moral norms in human and robot agents. In Aldinhas Ferreira

M. I.

Silva Sequieira

Tokhi

M. O.

Kadar

Virk

G. S.

(Eds.), A world with robots: International conference on robot ethics: ICRE 2015. (Vol. 84, pp. 3–18). Springer International Publishing.doi:10.1007/978-3-319-46667-5_1

144.

Marin

B. F.

Hunger

Werner

Meila

Schuetz

. (2005). Roles of an intelligent tutor agent in a virtual Society. Proceedings – 2005 Symposium on Applications and the Internet, SAINT’ 2005, 237–244.doi:10.1109/saint.2005.55

145.

Mayer

R. C.

Davis

J. H.

Schoorman

F. D

. (1995). An integrative model of organizational trust. Academy of Management Review, 20, 709–734.doi:10.5465/amr.1995.9508080335

146.

McGuirl

J. M.

Sarter

N. B

. (2006). Supporting trust calibration and the effective use of decision aids by presenting dynamic system confidence information. Human Factors: The Journal of the Human Factors and Ergonomics Society, 48, 656–665.doi:10.1518/001872006779166334

17240714

147.

McNeese

Demir

Chiou

Cooke

N. J.

Yanikian

. (2019). Understanding the role of trust in human-autonomy teaming. Proceedings of the 52nd Hawaii International Conference on System Sciences, 6, 254–263.doi:10.24251/hicss.2019.032

148.

Merritt

S. M.

Lee

Unnerstall

J. L.

Huber

Louis

. (2015). Are well-calibrated users effective users? Associations between calibration of trust and performance on an automation-aided task. Human Factors: The Journal of the Human Factors and Ergonomics Society, 57, 34–47.doi:10.1177/0018720814561675

25790569

149.

Meyer

. (2004). Conceptual issues in the study of dynamic hazard warnings. Human Factors: The Journal of the Human Factors and Ergonomics Society, 46, 196–204.doi:10.1518/hfes.46.2.196.37335

15359670

150.

Meyer

Bitan

. (2002). Why better operators receive worse warnings. Human Factors: The Journal of the Human Factors and Ergonomics Society, 44, 343–353.doi:10.1518/0018720024497754

12502153

151.

Meyerson

Weick

Kramer

. (1996). Swift trust and temporary groups. In Kramer

R. M.

(Ed.), Trust in organizations: Frontiers of theory and research (pp. 166–196). Sage.

152.

Miller

Funk

Goldman

Meisner

Chapman

. (2005). The Playbook™ approach to adaptive automation. Proceedings of the Human Factors and Ergonomics Society ... Annual Meeting, 49, 15–19.doi:10.1177/154193120504900105

153.

Miller

C. A

. (2014). Delegation and transparency: Coordinating interactions so information exchange is no surprise. Lecture Notes in Computer Science, 8525, 191–202.doi:10.1007/978-3-319-07458-0_19

154.

Miller

. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 267, 1–38.doi:10.1016/j.artint.2018.07.007

155.

Mochon

Norton

Ariely

. (2012). Bolstering and restoring feelings of competence via the IKEA effect. International Journal of Research in Marketing, 29, 363–369.doi:10.1016/j.ijresmar.2012.05.001

156.

Molm

L. D.

Collett

J. L.

Schaefer

D. R

. (2006). Conflict and fairness in social exchange. Social Forces, 84, 2331–2352.doi:10.1353/sof.2006.0100

157.

Molm

L. D.

Peterson

Takahashi

. (1999). Power in negotiated and reciprocal exchange. American Sociological Review, 64, 876.doi:10.2307/2657408

158.

Molm

L. D.

Schaefer

D. R.

Collett

J. L

. (2007). The value of reciprocity. Social Psychology Quarterly, 70, 199–217.doi:10.1177/019027250707000208

159.

Molm

L. D.

Takahashi

Peterson

. (2000). Risk and trust in social exchange: An experimental test of a classical proposition. American Journal of Sociology, 105, 1396–1427.doi:10.1086/210434

160.

Montague

. (2012). Understanding active and passive users: The effects of an active user using normal, hard and unreliable technologies on user assessment of trust in technology and co-user. Applied Ergonomics, 43, 702–712.doi:10.1016/j.apergo.2011.11.002

22192788

161.

Montague

Chiou

. (2014). Shared experiences of technology and trust: An experimental study of physiological compliance between active and passive users in technology-mediated collaborative encounters. IEEE Transactions on Human-Machine Systems, 44, 614–624.doi:10.1109/THMS.2014.2325859

162.

Montague

E. N. H.

Winchester

W. W.

Kleiner

B. M

. (2010). Trust in medical technology by patients and health care providers in obstetric work systems. Behaviour & Information Technology, 29, 541–554.doi:10.1080/01449291003752914

20802836

163.

Morris

Antoniades

Took

C. C

. (2017). On making sense of neural networks in road analysis. Proceedings of the International Joint Conference on Neural Networks, 4416–4421.doi:10.1109/IJCNN.2017.7966415

164.

Mutlu

Kanda

Forlizzi

Hodgins

Ishiguro

. (2012). Conversational gaze mechanisms for humanlike robots. ACM Transactions on Interactive Intelligent Systems, 1, 1–33.doi:10.1145/2070719.2070725

165.

Nass

Moon

. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56, 81–103.doi:10.1111/0022-4537.00153

166.

Nass

Steuer

Tauber

. (1994). Computers are social actors. CHI '94: Proceedings of the SIGCHI conference on Human factors in Computing Systems: Celebrating Interdependence, 72–78.doi:10.1145/191666.191703

167.

Neal

D. T.

Wood

Quinn

J. M

. (2006). Habits—A repeat performance. Current Directions in Psychological Science, 15, 198–202.doi:10.1111/j.1467-8721.2006.00435.x

168.

Ostrom

. (2000). Collective action and the evolution of social norms. Journal of Economic Perspectives, 14, 137–158.doi:10.1257/jep.14.3.137

169.

O’Neil

. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Crown Publishing Group.

170.

Parasuraman

Miller

C. A

. (2004). Trust and etiquette in high-criticality automated systems. Communications of the ACM, 47, 51–55.doi:10.1145/975817.975844

171.

Parasuraman

Riley

. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors: The Journal of the Human Factors and Ergonomics Society, 39, 230–253.doi:10.1518/001872097778543886

172.

Parasuraman

Sheridan

T. B.

Wickens

C. D

. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 30, 286–297.doi:10.1109/3468.844354

173.

Parasuraman

Sheridan

T. B.

Wickens

C. D

. (2008). Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs. Journal of Cognitive Engineering and Decision Making, 2, 140–160.doi:10.1518/155534308X284417

174.

Parks

C. D.

Henager

R. F.

Scamahorn

S. D

. (1996). Trust and reactions to messages of intent in social dilemmas. Journal of Conflict Resolution, 40, 134–151.doi:10.1177/0022002796040001007

175.

Petersen

. (2008). Making the analytic review process work: toward a stronger intelligence product. (September 20, 2020). https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/csi-studies/studies/vol49no1/html_files/analytic_review_7.html

176.

Picard

Friston

. (2014). Predictions, perception, and a sense of self. Neurology, 83, 1112–1118.doi:10.1212/WNL.0000000000000798

25128179

177.

Prahl

Van Swol

. (2017). Understanding algorithm aversion: When is advice from automation discounted? Journal of Forecasting, 36, 691–702.doi:10.1002/for.2464

178.

Pritchett

A. R.

Kim

S. Y.

Feigh

K. M

. (2014). Measuring human-automation function allocation. Journal of Cognitive Engineering and Decision Making, 8, 52–77.doi:10.1177/1555343413490166

179.

Rahwan

Cebrian

Obradovich

Bongard

Bonnefon

J. -F.

Breazeal

Crandall

J. W.

Christakis

N. A.

Couzin

I. D.

Jackson

M. O.

Jennings

N. R.

Kamar

Kloumann

I. M.

Larochelle

Lazer

McElreath

Mislove

Parkes

D. C.

Pentland

Alex ‘Sandy’.

Wellman

. . .. (2019). Machine behaviour. Nature, 568, 477–486.doi:10.1038/s41586-019-1138-y

31019318

180.

Ribeiro

M. T.

Singh

Guestrin

. (2016). “Why should I trust you?”. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144.doi:10.1145/2939672.2939778

181.

Riedl

Mohr

P. N.

Kenning

P. H.

Davis

F. D.

Heerkeren

H. R

.Paper 7 (2011). Trusting humans and avatars: Behavioral and neural evidence. In ICIS 2011 Proceedings. https://aisel.aisnet.org/icis2011/proceedings/hci/7.

182.

Riegelsberger

Sasse

M. A.

McCarthy

J. D

. (2003). The researcher’s dilemma: Evaluating trust in computer-mediated communication. International Journal of Human-Computer Studies, 58, 759–781.doi:10.1016/S1071-5819(03)00042-9

183.

Riegelsberger

Sasse

M. A.

McCarthy

J. D

. (2005). The mechanics of trust: A framework for research and design. International Journal of Human-Computer Studies, 62, 381–422.doi:10.1016/j.ijhcs.2005.01.001

184.

Robinette

Wagner

A. R.

Howard

A. A. M

. (2013). Building and maintaining trust between humans and guidance robots in an emergency. In AAAI spring symposium (pp.pp. 78–83). Stanford University.

185.

Rochlin

La Porte

T. R.

Roberts

K. H

. (1987). The self-designing high-reliability organization: Aircraft carrier flight operations at sea. Naval War College review, 40, 76–90.

186.

Rockwell

. (2002). Deception detection: Study of information manipulation through electronic identity theft-email forgery in the us military. Wright-Patterson AFB, OH. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA420820

187.

Romzek

B. S.

Dubnick

M. J

. (1987). Accountability in the public sector: Lessons from the Challenger tragedy. Public Administration Review, 47, 227–238.doi:10.2307/975901

188.

Rudin

. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1, 206–215.doi:10.1038/s42256-019-0048-x

189.

Rusbult

C. E.

Van Lange

P. A. M

. (2003). Interdependence, interaction, and relationships. Annual Review of Psychology, 54, 351–375.doi:10.1146/annurev.psych.54.101601.145059

12415073

190.

Russell

. (2019). Human compatible: Artificial intelligence and the problem of control. Penguin Random House LLC.

191.

Sacha

Senaratne

Kwon

B. C.

Ellis

Keim

D. A

. (2016). The role of uncertainty, awareness, and trust in visual analytics. IEEE Transactions on Visualization and Computer Graphics, 22, 240–249.doi:10.1109/TVCG.2015.2467591

26529704

192.

Sadigh

Landolfi

Sastry

S. S.

Seshia

S. A.

Dragan

A. D

. (2018). Planning for cars that coordinate with people: Leveraging effects on human actions for planning and active information gathering over human internal state. Autonomous Robots, 42, 1405–1426.doi:10.1007/s10514-018-9746-1

193.

Sarter

N. B.

Woods

D. D

. (1995). How in the world did we ever get into that mode? Mode error and awareness in supervisory control. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37, 5–19.doi:10.1518/001872095779049516

194.

Sarter

N. B.

Woods

D. D.

Billings

C. E

. (1997). Automation surprises. In Salvendy

(Ed.), Handbook of human factors & ergonomics (2nd ed.). Wiley.

195.

Schaefer

K. E.

Chen

J. Y. C.

Szalma

J. L.

Hancock

P. A

. (2016). A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human Factors, 58, 377–400.doi:10.1177/0018720816634228

27005902

196.

Schelling

T. C

. (1960). The strategy of conflict. Harvard University Press.

197.

Schilbach

Timmermans

Reddy

Costall

Bente

Schlicht

Vogeley

. (2013). Toward a second-person neuroscience. Behavioral and Brain Sciences, 36, 393–414.doi:10.1017/S0140525X12000660

23883742

198.

Schultz

A. C.

Breazeal

Fong

Kiesler

. (2007). The second International Conference on human-robot interaction. AI magazine, 28, 92–93.doi:aimag.v28i3.2057

199.

Sebanz

Bekkering

Knoblich

. (2006). Joint action: Bodies and minds moving together. Trends in Cognitive Sciences, 10, 70–76.doi:10.1016/j.tics.2005.12.009

16406326

200.

Sebeok

T. A

. (2001). Signs: An introduction to semiotics (2nd ed.). University of Toronto Press Incorporated.

201.

Seemann

. (2009). Joint agency: Intersubjectivity, sense of control, and the feeling of trust. Inquiry, 52, 500–515.doi:10.1080/00201740903302634

202.

Selkowitz

Lakhmani

Chen

J. Y. C.

Boyce

. (2015). The effects of agent transparency on human interaction with an autonomous robotic agent. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59, 806–810.doi:10.1177/1541931215591246

203.

Sheridan

. (2002). Humans and automation: System design and research issues. John Wiley & Sons, Inc.

204.

Sheridan

Verplank

. (1978). Human and computer control of undersea teleoperators. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA057655

205.

Sheridan

T. B

. (2016). Human-robot interaction: Status and challenges. Human Factors, 58, 525–532.doi:10.1177/0018720816644364

27098262

206.

Sheridan

T. B

. (2019). Individual differences in attributes of trust in automation: Measurement and application to system design. Frontiers in Psychology, 10, 1–7.doi:10.3389/fpsyg.2019.01117

207.

Shneiderman

. (2020). Human-centered artificial intelligence: Reliable, safe & trustworthy. Retrieved February 23, 2020, from https://arxiv.org/abs/2002.04087v1

208.

Silver

Schrittwieser

Simonyan

Antonoglou

Huang

Guez

Hubert

Baker

Lai

Bolton

Chen

Lillicrap

Hui

Sifre

van den Driessche

Graepel

Hassabis

Hubert

. (2017). Mastering the game of Go without human knowledge. Nature, 550, 354–359.doi:10.1038/nature24270

29052630

209.

Simpson

J. A

. (2007). Psychological foundations of trust. Current Directions in Psychological Science, 16, 264–268.doi:10.1111/j.1467-8721.2007.00517.x

210.

Smith

E. R.

Conrey

F. R

. (2007). Agent-based modeling: A new approach for theory building in social psychology. Personality and Social Psychology Review, 11, 87–104.doi:10.1177/1088868306294789

18453457

211.

Smith

M. J.

Carayon

Sanders

K. J.

Lim

S. Y.

Legrande

. (1992). Employee stress and health complaints in jobs with and without electronic performance monitoring. Applied Ergonomics, 23, 17–27.doi:10.1016/0003-6870(92)90006-H

15676845

212.

Stephens

R. J.

Woods

D. D.

Branlat

Wears

R. L

. (2011). Colliding dilemmas: Interactions of locally adaptive strategies in a hospital setting. Fourth Symposium on Resilience Engineering, 256–262.doi:10.4000/books.pressesmines.1110

213.

Sweeney

P. J.

Thompson

Blanton

. (2009). Trust and influence in combat: An interdependence model. Journal of Applied Social Psychology, 39, 235–264.doi:10.1111/j.1559-1816.2008.00437.x

214.

Szafir

Mutlu

Fong

. (2015). Communicating directionality in flying robots. Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction - HRI ’15, 2, 19–26.doi:10.1145/2696454.2696475

215.

Takayama

. (2009). Making sense of agentic objects and teleoperation: In-the-moment and reflective perspectives. 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 239–240.doi:10.1145/1514095.1514155

216.

Thibaut

J. W.

Kelley

H. H

. (1959). The social psychology of groups. John Wiley & Sons, Inc.

217.

Tomasello

Carpenter

. (2007). Shared intentionality. Developmental Science, 10, 121–125.doi:10.1111/j.1467-7687.2007.00573.x

17181709

218.

Tomasello

Carpenter

Call

Behne

Moll

. (2005). Understanding and sharing intentions: The origins of cultural cognition. Behavioral and Brain Sciences, 28, 675–691.doi:10.1017/S0140525X05000129

16262930

219.

Tomlinson

E. C.

Mayer

R. C

. (2009). The role of causal attribution dimensions in trust repair. Academy of Management Review, 34, 85–104.doi:10.5465/amr.2009.35713291

220.

Torre

Goslin

White

Zanatto

. (2018). Trust in artificial voices: A “congruency effect” of first impressions and behavioural experience. Proceedings of the Technology, Mind, and Society, 1–6.doi:10.1145/3183654.3183691

221.

Traeger

M. L.

Strohkorb Sebo

Jung

Scassellati

Christakis

N. A

. (2020). Vulnerable robots positively shape human conversational dynamics in a human–robot team. Proceedings of the National Academy of Sciences, 117, 6370–6375.doi:10.1073/pnas.1910402117

222.

Trent

S. A.

Patterson

E. S.

Woods

D. D

. (2007). Challenges for cognition in intelligence analysis. Journal of Cognitive Engineering and Decision Making, 1, 75–97.doi:10.1177/155534340700100104

223.

Unhelkar

Shah

J. A

. (2020). Decision-Making for bidirectional communication in sequential human-robot collaborative tasks. Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, 329–341.doi:10.1145/3319502.3374779

224.

van der Wel

R. P. R. D

. (2015). Me and we: Metacognition and performance evaluation of joint actions. Cognition, 140, 49–59.doi:10.1016/j.cognition.2015.03.011

25880341

225.

Trentin

Perin

Forza

. (2014). Increasing the consumer-perceived benefits of a mass-customization experience through sales-configurator capabilities. Computers in Industry, 65, 693–705.doi:10.1016/j.compind.2014.02.004

226.

Tsai

J. C. -C.

Sebanz

Knoblich

. (2011). The GROOP effect: Groups mimic group actions. Cognition, 118, 135–140.doi:10.1016/j.cognition.2010.10.007

21074148

227.

van Wezel

Cegarra

Hoc

J. -M

. (2011). Allocating functions to human and algorithm in scheduling. In Fransoo

J. C.

Wafler

Wilson

J. R.

(Eds.), Behavioral operations in planning and scheduling (pp. 339–370). Springer.

228.

Verberne

F. M. F.

Ham

Midden

C. J. H

. (2012). Trust in smart systems: Sharing driving goals and giving information to increase trustworthiness and acceptability of smart systems in cars. Human Factors, 54, 799–810.doi:10.1177/0018720812443825

229.

Verberne

F. M. F.

Ham

Midden

C. J. H

. (2015). Trusting a virtual driver that looks, acts, and thinks like you. Human Factors: The Journal of the Human Factors and Ergonomics Society, 57, 895–909.doi:10.1177/0018720815580749

230.

Wagner

A. R.

Arkin

R. C

. (2008). Analyzing social situations for human–robot interaction. Interaction Studies, 9, 277–300.doi:10.1075/is.9.2.07wag

231.

Wagner

A. R.

Arkin

R. C

. (2011). Recognizing situations that demand trust. Proceedings of the 20th International Symposium on Robot and Human Interactive Communication, 7–14.doi:ROMAN.2011.6005228

232.

Wajcman

Rose

. (2011). Constant connectivity: Rethinking interruptions at work. Organization Studies, 32, 941–961.doi:10.1177/0170840611410829

233.

Weiner

. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92, 548–573.doi:10.1037/0033-295X.92.4.548

234.

Wickens

Carswell

. (2006). Information processing. In Salvendy

(Ed.), Handbook of human factors and ergonomics (pp. 111–149). John Wiley & Songs, Inc.

235.

Wickham

R. E.

Knee

C. R

. (2012). Interdependence theory and the actor-partner interdependence model: Where theory and method converge. Personality and Social Psychology Review, 16, 375–393.

236.

Williams

K. D

. (2010). Dyads can be groups (and often are). Small Group Research, 41, 268–274.doi:10.1177/1046496409358619

237.

Winfield

A. F. T.

Jirotka

. (2018). Ethical governance is essential to building trust in robotics and artificial intelligence systems. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 376.doi:10.1098/rsta.2018.0085

238.

Woods

D. D

. (2005). Conflicts between learning and accountability in patient safety. DePaul Law Review, 54, 485–502.

239.

Woods

D. D

. (2015). Four concepts for resilience and the implications for the future of resilience engineering. Reliability Engineering & System Safety, 141, 5–9.doi:10.1016/j.ress.2015.03.018

240.

Woods

D. D

. (2019). Essentials of resilience, revisited. In Handbook on resilience of socio-technical systems (pp. 52–65). Edward Elgar Publishing.

241.

Woodward

Finn

Hausman

. (2020). Learning to interactively learn and assist. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 2535–2543.doi:10.1609/aaai.v34i03.5636

242.

Zhang

Fang

Wei

K.-K.

Chen

. (2010). Exploring the role of psychological safety in promoting the intention to continue sharing knowledge in virtual communities. International Journal of Information Management, 30, 425–436.doi:10.1016/j.ijinfomgt.2010.02.003

243.

Zhang

Liao

Bellamy

R. K. E

. (2020). Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 295–305.doi:10.1145/3351095.3372852

244.

Zhang

Narayanan

Chakraborti

Kambhampati

. (2015). A human factors analysis of proactive support in human-robot teaming. IEEE International Conference on Intelligent Robots and Systems, 3586–3593.doi:IROS.2015.7353878

245.

Zuboff

. (1985). Automate/informate: The two faces of intelligent technology. Organizational Dynamics, 14, 5–18.doi:10.1016/0090-2616(85)90033-6

246.

Zuboff

. (1988). In the Age of the Smart Machine: The Future of Work and Power. Basic Books, Inc.