Evaluation of a Remote Data Collection Method to Study Human-Automation Interaction and Workload

Abstract

Technological advances have increased the automation of Uncrewed Aerial Vehicles, allowing human operators to manage multiple vehicles at a high-level without the need to understand low-level system behaviors.Previous laboratory studies have explored the relationship between reliability, trust, use of automation, andthe effects of number of vehicles under supervision on subjective workload. Due to limitations resulting fromthe COVID-19 pandemic, in-person laboratory studies are not always possible. Therefore, this work aimed to investigate if remote data collection alternatives, such as Amazon’s Mechanical Turk, can provide comparativeresults as those obtained in laboratorysettings. A study was conducted in the context of small droneoperations. As expected, higher reliability led to higher trust ratings and the inclusion of more vehicles ledto higher workload. In contrast, reliability unexpectedly had no significant effect on intention to use theautomation. Though these results were encouraging, several limitations were identified