Effects of Automation Type on Human Performance in Proofreading Tasks

Abstract

Automation is ubiquitous and indispensable in modern working environments. It is adopted and used in not only advanced industrial- and technology-oriented operations, but also ordinary home or office computational functions. In general, automated systems aim to improve overall work efficiency and productivity of labor-intensive tasks by decreasing the risk of errors, and cognitive and physical workloads. The systems offer the support for diverse decision-making processes as well. However, the benefits of automation are not consistently achieved and depend on the types and features of automation (Onnasch, Wickens, Li, & Manzey, 2014; Parasuraman, Sheridan, & Wickens, 2000). Possible negative side effects have been reported. Sometimes, automation may lead to multi-tasking environments, which allows operators to be distractive with several tasks. It ultimately prolongs task completion time and causes to neglect monitoring and follow-up steps of the pre-processing tasks (Endsley, 1996). Furthermore, the operators who excessively depend on automation are easily deteriorated in skill acquisition, which is necessary for the emergency or manual operations. Thus, inconsistent performance in automation is a major issue in successful adoption and trust in automation (Jeong, Park, Park, & Lee, 2017). This paper presents an experimental study that investigates the main features and causes of the inconsistency in task performance in different types of automation. Automated proofreading tasks were used in this study, which is one of the most common types of automation we experience in daily life. Based on the similar algorithm of the auto-correct function in Microsoft Word, a custom-built program of five proofreading tasks, including one non-automated and four automated proofreading tasks, were developed using Visual Studio 2015 C#. In the non-automated task used as a reference for individual difference, participants were asked to manually find a typographical error in a sentence. In the automated tasks, auto-correcting functions are provided in two levels (i.e., low and high) of automation and two statuses (i.e., routine and failure of automation). The type of automation is defined as the combinations of a status and a level. Participants identified typographical errors by only an underlined word at the low-level automation, whereas an underlined word with a possible substituting word was given at the high-level. Additionally, in the routine automation status, a correct substituting word is provided. On the other hand, a grammatically incorrect word is given in the failed automation status. Nineteen participants (11 females and 8 males; age mean = 33.8, standard deviation = 19.1) took part in this study. Results of statistical analyses show a clear advantage in high-routine automation, in terms of both task completion time and accuracy. While task performances of high & routine automation types are quite obvious in both task completion time and accuracy, those in the failed automation types are mixed and indistinguishable. Different levels and statues of failed automation do not much influence task performance. Moreover, task completion time and mental demand are strongly correlated, and the accuracy rate and perceived trust show a strong positive correlation. The approaches and outcomes of the current study can provide some insights into the human-automation interaction systems that support human performance and safety, such as in-vehicle warning systems and automated vehicle controls.