Using Sensitivity and Bias in Signal Detection Theory to Predict Proportion Correctness: Simulation and Case Study on ADAS Mental Model Evaluation

Abstract

The sensitivity (d’) and bias (c) in signal detection theory can reflect respondents’ objective performance in understanding a system and their bias towards saying yes. Thus, d’ and c can be used as alternatives of proportion correctness (PC) when evaluating drivers’ mental models of advanced driving assistance system (ADAS). The adoption of d’ and c as mental model metrics also allows cross-study comparisons as their values are independent of signal (i.e., ADAS function present) and noise (i.e., ADAS function absent) ratio. However, there is no closed-form solution of the relationships among d’, c, and PC. Hence, using numerical simulations, we extracted an empirical equation that quantifies how d’ and c can estimate PC. The equation was then validated based on participants’ responses from a survey study that targeted towards drivers’ ADAS mental model. The results show that the empirical equation reached a satisfying performance (R² > 0.8) in estimating PC.

Keywords

Mental Model Signal Detection Theory Driving Automation

Get full access to this article

View all access options for this article.

References

Beggiato

Krems

J. F.

(2013). The evolution of mental model, trust and acceptance of adaptive cruise control in relation to initial information. Transportation Research Part F: Traffic Psychology and Behaviour, 18, 47–57.

Beggiato

Pereira

Petzoldt

Krems

(2015). Learning and development of trust, acceptance and the mental model of ACC. A longitudinal on-road study. Transportation Research Part F: Traffic Psychology and Behaviour, 35, 75–84.

DeGuzman

C. A.

Donmez

(2021a). Drivers Still Have Limited Knowledge About Adaptive Cruise Control Even When They Own the System. Transportation Research Record: Journal of the Transportation Research Board, 2675(10), 328–339.

DeGuzman

C. A.

Donmez

(2021b). Knowledge of and trust in advanced driver assistance systems. Accident Analysis & Prevention, 156, 106121.

Dickie

D. A.

Boyle

L. N.

(2009). Drivers’ understanding of adaptive cruise control limitations. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 53(23), 1806–1810. SAGE Publications Sage CA: Los Angeles, CA.

Green

D. M.

Swets

J. A.

(1966). Signal detection theory and psychophysics (Vol. 1). Wiley New York.

Larsson

A. F. L.

(2012). Driver usage and understanding of adaptive cruise control. Applied Ergonomics, 43(3), 501–506.

McDonald

Carney

McGehee

D. V.

(2018). Vehicle owners’ experiences with and reactions to advanced driver assistance systems.

National Highway Traffic Safety Administration. (2022). Summary Report: Standing General Order on Crash Reporting for Level 2 Advanced Driver Assistance Systems. Retrieved from https://www.nhtsa.gov/sites/nhtsa.gov/files/2022-06/ADAS-L2-SGO-Report-June-2022.pdf

10.

Parasuraman

Masalonis

A. J.

Hancock

P. A.

(2000). Fuzzy signal detection theory: Basic postulates and formulas for analyzing human and machine performance. Human Factors, 42(4), 636–659.

11.

R Core Development Team, R Core Team, & Team, R. D. C. (2019). R: A Language and Environment for Statistical Computing. Vienna, Austria.

12.

Rossi

Gastaldi

Biondi

Orsini

De Cet

Mulatti

(2020). A Driving Simulator Study Exploring the Effect of Different Mental Models on ADAS System Effectiveness.

13.

Stanislaw

Todorov

(1999). Calculation of signal detection theory measures. Behavior Research Methods, Instruments, & Computers, 31(1), 137–149.