Estimating User’s Preferred Response Bias in an Automated Diagnostic Aid

Abstract

Operators often perform a target detection task better when assisted by a diagnostic aid of conservative response bias than when assisted by an aid of liberal bias. Little is known, however, about the factors that might influence the preferred level of automation response bias across task contexts. The present study used a psychophysical staircase technique to assess the influence of signal frequency and automation sensitivity on users’ preferences. Subjects performed a signal detection task with the assistance of an automated aid, and were allowed to adjust the aid’s response criterion across trials. A staircase procedure gauged the subjects’ preferred level of automation response bias across factorially-manipulated levels of signal rate (25% vs. 75%) and aid sensitivity (d’ = 1.5 vs. 3.0). The preferred level of bias was generally conservative, but varied as an interaction of signal rate and aid’s sensitivity.