(Some) Benefits in Operator Decisions to Use AI After Experiencing Optimal Outcomes

Abstract

Objective

The current study aimed to explore the impacts of experiencing superior behaviors—accumulating large amounts of evidence and high automation use rates—on subsequent evidence accumulation rates and adaptable (discretionary) automation use decisions in a dynamic decision-making task.

Background

Operators prefer to choose when to engage automated support systems but seldom use them appropriately. They also do not typically collect enough evidence to optimize their decision making. This creates suboptimal performance that could benefit from training better behaviors.

Method

Participants collected evidence about movement patterns of ships while assisted by a machine learning aid. They were initially required to collect high levels of evidence and use the aid as a form of hands-on training. Then, they chose how much evidence to collect and when to engage the aid.

Results

When given the choice, operators collected less evidence and used the automation less often than had been required during training, but improved their performance compared to unaided trials.

Conclusion

Providing operators with early experience of superior behavioral strategies can improve their subsequent decisions. This is a promising direction for achieving human–automation team synergy.

Applications

Short exposures to optimal behaviors may be a feasible training approach to improve human–automation interactions in contexts where operators want decisional freedom in their interactions.