Failing Consistently and Expectedly: Effects of Error Consistency and Drivers’ Mental Models on Trust in Automated Vehicles

Abstract

This ongoing study investigates how driver expectations and automation error consistency influence trust in Driving Automation Systems (DASs). Drawing on the Expectation Confirmation Theory (ECT), we hypothesize that trust increases when system performance aligns with or exceeds driver expectations on the DASs. Inconsistencies in error patterns may diminish trust. We conducted a 2 × 3 × 3 mixed-design experiment on STISIM Drive with 150 participants, manipulating expectations (high vs. low) and error patterns (no error, consistent, or inconsistent errors). Trust and response times to takeover requests (TORs) were measured three times, one at each drive. We predict higher trust and longer TOR times in low-expectation conditions. As for error patterns, we predict the highest trust to arise from scenarios with no errors and longer TOR times when errors are inconsistent. Our findings are expected to provide insights into enhancing DAS design by aligning system performance with user expectations and promoting error consistency to improve trust and user satisfaction.

Effective and secure utilization of Driving Automation Systems (DASs) requires understanding drivers’ trust and its determinants. This study examines how driver expectations and automation error consistency impact drivers’ trust in DASs.

Drivers’ mental models of the DAS represent their understanding of system purpose and structure (Johnson-Laird, 1983; Norman, 1986). They help people to predict the system’s behaviors (Gentner & Stevens, 1983; Norman, 2014) and form expectations of DAS capabilities. The Expectation Confirmation Theory (ECT) was initially introduced to explain consumers’ post-purchase satisfaction by evaluating the products against their pre-purchase expectations: products meeting or surpassing expectations will lead to satisfaction; otherwise, dissatisfaction ensues (Oliver, 1980). Adopting this theory in the context of DAS, previous research has revealed that trust increases when system performance meets or exceeds drivers’ expectations; otherwise, trust diminishes (Beggiato & Krems, 2013; Victor et al., 2018; Zhang et al., 2020).

Automation errors significantly influence trust. Specifically, takeover requests (TORs) due to system limitations or malfunctions often reduce trust (Hergeth et al., 2015). However, little is known about how the consistency of errors affects trust (Lee et al., 2021; Mishler & Chen, 2023). Error consistency potentially establishes predictability in system behavior, providing drivers with a sense of reliability and helping them form adaptive strategies to readily handle potential TORs. Understanding how consistent errors impact trust will inform the development of more effective DASs.

We recruited 150 participants aged over 18, holding a valid U.S. driver’s license, and having normal or corrected-to-normal vision and hearing. A STISIM driving simulator was employed. Participants underwent a practice drive to familiarize themselves with the driving simulator. Their baseline trust in the DAS in use was measured using Jian et al.’s (2000) questionnaire.

The experiment used a 2 × 3 × 3 mixed design, including between-subjects variables, Expectations (high vs. low), and Error Patterns (no error, consistent, or inconsistent errors). Drivers’ expectations of the DAS were manipulated as either high or low by presenting instructions on system capacities at the beginning of the study, followed by a manipulation check of their mental models. Error patterns were manipulated across three drives, with the order of the drives being randomized across participants. The dependent variables included reaction times in response to the TORs, and trust measured after each drive using the same questionnaire as the baseline trust. Participants explained their changes in DAS trust in response to their expectations of the system’s capabilities and the errors they experienced in a post-task interview.

Here we report our data analysis plan and expected results. Two-way Analyses of Variance (ANOVAs) will be conducted on trust and TOR times separately, with drivers’ expectations and error patterns being the two between-subjects factors.

We predict higher trust and longer TOR times in the low-expectation group, as system performance exceeds the specified low expectations. Error patterns will also show a significant main effect: trust is expected to rank highest with no errors due to the system’s high performance, followed by consistent and then inconsistent errors because consistency renders the errors more predictable. Inconsistent errors are expected to have longer TOR times than consistent errors due to their low predictability.

In conclusion, the goal of this study is to advance the understanding of trust dynamics in DASs. We investigate the impact of drivers’ expectations on trust and the role of mental models in shaping the trust dynamics. Applying ECT to automation, the study emphasizes the importance of aligning system performance with expectations to reinforce trust in automated vehicles.

Practical insights of the current study include strategies to enhance trust. If the expected results are obtained, prioritizing making residual errors consistent can improve predictability and strategy adaptation. Communicating system capabilities with transparency would help align users’ expectations with the system’s actual performance, which would subsequently enhance user satisfaction and perceived reliability in the automated systems.