Abstract
Extant levels of automation (LoAs) taxonomies describe variations in function allocations between the driver and the driving automation system (DAS) from a technical perspective. However, these taxonomies miss important human factors issues and when design decisions are based on them, the resulting interaction design leaves users confused. Therefore, the aim of this paper is to describe how users perceive different DASs by eliciting insights from an empirical driving study facilitating a Wizard-of-Oz approach, where 20 participants were interviewed after experiencing systems on two different LoAs under real driving conditions. The findings show that participants talked about the DAS by describing different relationships and dependencies between three different elements: the context (traffic conditions, road types), the vehicle (abilities, limitations, vehicle operations), and the driver (control, attentional demand, interaction with displays and controls, operation of vehicle), each with associated aspects that indicate what users identify as relevant when describing a vehicle with automated systems. Based on these findings, a conceptual model is proposed by which designers can differentiate LoAs from a human-centric perspective and that can aid in the development of design guidelines for driving automation.
Keywords
Introduction
Automation is defined as the “automatically controlled operation of an apparatus, process, or system by mechanical or electronic devices that takes the place of human labour” (Merriam-Webster.com, 2011). Previously, it has been used in professional domains such as the aviation and processing industries, where a range of automation from simple on-off control to multivariable high-level algorithms has been applied to everything from GPS route planning, to flight management systems, to large industrial control room systems. However, as automation becomes increasingly capable, it also becomes more immersed in our everyday lives; that is, it is moving from professional domains into domains where the users are, to a greater extent, novice. Over time, many definitions of the different types and levels of automation (LoAs) have been proposed (for an overview, see the literature review by Vagia et al. (2016), where they present the developed taxonomies for LoAs since the 1950s). This increased automation reveals new challenges for human–machine interaction, calling for categorizations and taxonomies for the design of automated systems.
A prime example of a domain where LoAs are being continuously developed is that of vehicle automation, where the driving task is increasingly shared between the driver and the vehicle. Several organizations and consortiums have developed taxonomies that describe levels of driving automation, for example, the Federal Highway Research Institute (BASt), National Highway Traffic Safety Administration, and SAE International. The most prominent, SAE’s taxonomy J3016, was developed with the aim of creating a common understanding of the different LoA, and to provide a definition for the classification of driving automation, in order to support regulations. The taxonomy describes variations in task allocations between driver and the driving automation system (DAS). The classification ranges from Level 0 “No Driving Automation” to Level 5 “Full Driving Automation,” where each level entails specific functional requirements for the vehicle and consequently also a specific role for the driver. For example, in Level 1 “Driver Assistance” and Level 2 “Partial Driving Automation,” the vehicle is regarded as a support system, leaving the driver in control at all times, while on Level 3 “Conditional Driving Automation,” the vehicle takes over but the driver remains in supervisory control, and in Level 4 “High Driving Automation,” the driver is relieved of the driving task and responsibility under certain conditions and for a certain period of time, making him/her a passenger in the driver seat, with no need to supervise the vehicle (SAE International, 2018). As the driving task is increasingly shared between driver and vehicle, interaction between the driver and the system can end in situations where system mode is falsely classified by the driver, leading to erroneous actions and mode confusion (Sarter et al., 1997). One reason for confusion over the automation levels and modes of interaction may be that the relationship and the allocation of responsibility between human and automation system are not clearly defined (Flemisch et al., 2012; Sheridan, 2011; Vagia et al., 2016). Hence, a key factor for a successful development of AVs is that the users understand the mode and their own responsibility in the interaction. In this paper, we refer to “system” as a combination of various components or clusters of functions, in order to support longitudinal vehicle motion (direction of travel) and lateral vehicle motion (position of the vehicle in the lane). A “mode,” on the other hand, is regarded as the specific level of automation executed by the system (Degani et al., 1999; Sarter & Woods, 1995), that is, driving mode of a specific DAS.
Problematically from a user perspective, Yang et al. (2017) argue that the existing LoA taxonomies are technology centered and presented from the perspective of vehicle technology development or policy development. Seppelt et al. (2018) also point out that the classifications are written by engineers for engineers. This has also been identified by SAE, which is why they have added a visual chart to simplify and clarify the “Levels of Driving Automation” for consumers (SAE International, 2018). Having technology-centered taxonomies may affect interaction between user and vehicle in two ways. First, even if not intended for the users, the taxonomies can affect the way users understand the system. Abraham et al. (2017) showed that the naming of the DAS influences the users’ perception of the system and the degrees of responsibility that are expected. Similarly, a survey by Seppelt et al. (2018) found that the taxonomy proposed by SAE created confusion about the users’ role and responsibility. It was found that users were able to differentiate the extreme ends of the automation levels, that is, driver assistance and full automation, but had problems with differentiating the levels in between. This ambiguity may lead to misuse or even disuse of the system, mitigating potential benefits that automation support can offer (Parasuraman et al., 2000). Second, Smith (2018) argues that the established taxonomies affect how designers think about the system, leading to design decisions based on technology-centered taxonomies. Therefore, having a technology-centered taxonomy may lead to design decisions that will miss important user perspectives, such as how users perceive and understand the systems. Previous research has pointed out issues related to existing taxonomies and some have proposed initial improvements, for example, a more simplistic communication tool using a binary framing such as “driving” and “riding” (Seppelt et al., 2018) and identification of system naming that is less ambiguous, leading to a clearer understanding about functionality (Abraham et al., 2017).
However, more research is needed to develop a human-centric taxonomy that addresses the current gap regarding, on the one hand, the lack of a design tool and, on the other hand, a communication tool for users. To the best of our knowledge, there has not yet been any naturalistic study, under real driving conditions, of how users of the DAS understand and describe the different modes of operation, that is to say, the LoA. Therefore, the aim of this paper is first to describe how users perceive and understand different LoAs by eliciting insights from an empirical on-road observation study, where the participants experienced two different LoAs, and second, based on that to propose a categorization by which designers can differentiate the LoAs from a human-centric perspective.
Method
The paper is based on insights from an empirical on-road study that took place in the San Francisco Bay area in the United States in June 2019. In the study, the participants experienced two different modes (LoA), a Level 2 partial automation system and a Level 4 high automation system (SAE International, 2018), in a Wizard-of-Oz (WOz) car.
Equipment
Vehicle
The WOz vehicle is a Volvo XC90 platform-based prototype, which was modified to enable testing of the two LoAs. The modified platform suggests to the driver that the car is taking over certain or all parts of the driving task, simulating a realistic usage experience for the participants. The simulation is possible through the installation of a steering wheel, instrument cluster, and pedals in the rear seat so that the vehicle can be driven from that position. The installation is hidden and cannot be seen by the participants sitting in the front seat. The vehicle was modified according to all relevant road permission standards and was audited and approved for road testing by the local authorities, making it possible to investigate users’ experience of automated driving in a real driving context.
Systems
During the test sessions, the two systems were mimicked by two test leaders sitting in the WOz vehicle. The Level 2 system was capable of automatically adjusting the vehicle’s speed with regard to other objects moving in front, or maintaining a set speed, as well as offering lane-keeping assistance, however not offering any advanced steering capability on its own. This mode could be activated at any time by the driver and was regarded as an assistive system, offering partial automation. Even though the system offered lateral and longitudinal support, it left the driver in full control and with responsibility over the driving task. However, if the vehicle sensors registered that the driver was disengaging or not keeping their hands on the steering wheel, they were prompted with a visual and auditory warning, reminding them of their responsibility to execute the driving task, leading to deactivation of the system if the driver did not engage within 10 s, returning to manual driving mode.
The Level 4 system, on the other hand, was an unsupervised driving mode and operated within a specified operational design domain (ODD)/under certain conditions as defined by the SAE driving automation levels (SAE International, 2018). For the study, it was decided to limit the context of this system to a congested traffic scenario, since this was a daily occurrence in the region where the study was conducted. For the mode to be available, it was necessary to have dense traffic with cars in front of and around the vehicle. If such a situation was encountered, the system would be available for activation by the driver. If the driver chose to do so, the system would take over the driving task completely, for as long as the conditions were met. If this was no longer the case, for example if the congestion disappeared, the driver was asked to take back control through a visual and auditory cue, with a 1 min period of preparation time, to give the driver enough time to get back into the loop if engaged with other tasks during the time the system was active. If the driver would not engage in the required time, the system would be able to continue operation under the specified conditions or perform a fallback and transition to a minimal risk maneuver. While the Level 4 system was engaged, the driver was free to do any task of their choice as the system was operating in a high automation mode, which did not require driver involvement or supervision, and would therefore, leave the driver to be a passenger in the driver’s seat.
Table 1 gives an overview of the information the participants received over the two systems. The information was handed to the participants in written form during the introduction to the test, as well as in the car before the driving sessions. The introduction of the systems was held similar to what a new car owner would get at a car dealership when picking up their car. This decision to provide the drivers with that level of information was made in order to provide a high level of realism, as the aim of the study was to investigate the users’ understanding of a vehicle offering several LoAs and usually intense driver training is not provided, when receiving a vehicle.
Description of Systems and What Information the Participants Got About Them.
Participants
The study involved 20 participants, 11 female and nine male, whose ages ranged from 22 to 62 years (Mean = 42, SD = 14). They were recruited and reimbursed through a local agency, which received screener and exclusion criteria. All participants had to be holders of a valid driver’s license, and they had to drive a car equipped with Common Cruise Control (driver assistance system that maintains a steady set speed) and an automatic gearbox. All the participants were frequent drivers, and all but one commuted to work by car on a daily basis. Seven of the participants commuted less than 30 min, nine participants between 30 min and 1 hr, and three participants drove more than 1 hr to work, one way. None of the participants had jobs related to the automotive or tech industries, with education and health care being the most common occupation sectors, followed by construction, social work, retail, and accounting.
Study Design and Procedure
During the drive, all 20 participants experienced both modes. However, while the participants could choose to engage with the Level 2 system whenever they preferred, they could only use the Level 4 system when they received a prompt, indicating that the necessary traffic congestion conditions were fulfilled, and the system was ready.
The study consisted of three phases during which data were collected, applying a mixed methods approach. This comprised personal interviews and on-road observations, and the interviews were analyzed for this paper.
Phase 1
Before every session, the participants were provided with a description of the session and had the chance to ask questions, before providing their informed consent according to the General Data Protection Regulations (GDPR) and agreeing to be audio- and video-recorded during the interviews.
After introducing the participants to the study, a 20-min interview was held to gain general information, such as prior experiences with the DAS and their expectations of vehicles with highly automated systems. Next, the participants were introduced to the driving scenario and received written and verbal information about the two modes of the vehicle, explaining the respective systems’ capabilities (driving context, road types, and traffic conditions in which each system is operational) and how to interact with them (activation and deactivation of each system, information and displays, responsibility during modes, safety instructions). It is worth mentioning that it was explained to the participants that they are responsible when driving with the Level 2 system, and that they can relax and are not responsible when the Level 4 system is engaged. All the information and instructions were communicated first in an interview room and a second time when sitting in the car, where the participants could also try out the interaction and inspect the displays. They were encouraged to test and try out the systems as much as they wanted during the drive and asked to think aloud so that their thought processes during their interaction with the system could be mapped.
Phase 2
The participants got a few minutes to acquaint themselves with the car before the drive started. The drive took approximately 90 min and was conducted on highways (U.S. highway 101: six-lane partially controlled-access road, speed limit = 70 mph, LOS E-F; Interstate 280: eight-lane fully controlled-access road, speed limit = 70 mph, LOS C-D) and urban areas (State Route-84: four-lane, speed limit 25–50mph, LOS B-C) and under regular traffic conditions in the San Francisco Bay area (Figure 1).
Route for the observation study in the San Francisco Bay area.
A route with several occasions of slow or halted traffic as well as a section with free-flowing traffic was needed to meet the two different conditions for the system’s ODD. A round trip on different road types was therefore chosen based on collected traffic information and several test drives. All drives were conducted during rush hour traffic, both in the morning and in the evening, and aimed to observe the users during their interaction with the two modes. Figure 2 shows how much time each participant used the Level 2 and Level 4 systems, as well as drove manually. The data of participant 12 have been removed due to distortion of the signal.
Exposure to Level 2 and level 4 system, and manual driving. Ordered by total amount of time using the Level 2 and 4 system.
Phase 3
After the drive, the session was concluded with an interview that focused on the participants’ understanding of and comparison between the systems. The participants were specifically asked to describe and explain the different systems they had experienced during the drive. In particular, they were asked to describe the two modes and how they perceived the differences. The participants were not aware of the WOz approach since the system was introduced as a prototype. However, after the session, it was disclosed to them that they had experienced a simulation and had an opportunity to ask additional questions.
Coding and Analysis
As a first step, interviews were transcribed verbatim. From the transcribed material, statements where participants described either of the two modes (“LoAs”) were extracted. A thematic analysis was then conducted using an inductive coding approach (Boyatzis, 1998), in order to explore how the participants described the two levels. Two researchers (1st and 2nd author) read through the material and color-coded recurring patterns in the statements, which the participants used to explain how the systems worked. The seven main aspects were then reviewed and refined until full agreement was reached between the researchers. The code “driving context,” for example, consisted of quotes regarding the context in which the system should be used, such as “will more or less function [in] light stop-and-go traffic” and “…was mostly for lower speeds […] when there’s congestion on the freeways.” Of the seven main aspects, two were divided into two sub-aspects each, since it became evident from the analysis that participants referred to the different aspects to various degrees and with different descriptions depending on which mode they talked about. A more in-depth analysis was conducted in order to identify any difference in the description and understanding of the two modes of operation, which resulted in a categorization of the identified aspects into the three elements. These elements are context, vehicle, and driver, and they constitute three different layers which entail the drivers’ understanding of the LoAs.
Findings
The results of the analysis revealed that the participants talked about the vehicle modes through different elements: the context, the vehicle, and the driver. In addition to the three major elements, seven recurring aspects could be discerned in the thematic analysis: Driver Tasks, Vehicle Operations, Abilities and Limitations of the Systems, the Attentional Demand Required from the Driver, Control of the Vehicle, and Driving Context. The Driver Tasks aspect was divided into two sub-aspects, namely Interaction with Displays and Controls and Operation of Vehicle, and the Driving Context was divided into Traffic Conditions and Road Types. A definition of each main aspect and its associated sub-aspects as well as the categorization into the three different elements is provided in Table 2.
Summary of the Different Aspects Identified From Descriptions by the Participants About the Systems.
The element that was talked about the most was driver (approximately half of the statements) followed by vehicle and last context. Concerning the driver, the participants talked about all aspects and sub-aspects to the same extent. Regarding the vehicle, half of the times the participants talked about which driving tasks that are performed by the systems and when they talked about the context, they mostly mentioned traffic conditions. See Figure 3 for an overview of how much each element and aspect were mentioned.
TreeMap showing the distribution of statements regarding the elements and aspects. Color indicates elements and gradients indicate associated aspects and sub-aspects (lightest) within an element.
Context
The context manifests through the aspects described by the Driving Context. This includes environmental conditions, which influence the system’s performance. The participants described road types such as freeways or traffic conditions that define how and when the vehicle performs. This element could further include weather and infrastructural conditions that were not apparent during this study but that may have greater relevance in other geographical locations.
Driving context
The driving context was mentioned by many participants as an important decider in whether or not the two modes could be activated. Here, there was a clear distinction between the traffic situation, for example, free flowing traffic versus congestion, and different road types as preconditions for use of the systems. The Level 2 system was perceived to be “available right away, you could use it upon leaving wherever you are.” Thus, participants concluded that it was available under all conditions and on all road types. However, most participants stated that the Level 2 system was “more for a freeway situation […] free-flowing traffic,” but some participants were under the impression that it “will more or less function [in] light stop-and-go traffic.” For the Level 4 system, descriptions of suitable road types were absent, but participants described the traffic conditions it needed as “more congested rush hour traffic” and “bumper to bumper” or “slower, commute traffic.” However, the similarities between the systems and the contexts in which they work seemed to confuse some of the participants, as they stated that “[the Level 2 system] was mostly for lower speeds […] when there’s congestion on the freeways. For [the Level 4 system] it is probably better suited for cruising speeds. Both of them will keep the car in the lane, but one seems better for highway cruising vs. the other one for commutes.”
Recalling the system description, the Level 2 system was meant to be an assistive system, which is operational on all road types and will work in free-flowing traffic as well as in congested situations, while the Level 4 system operates only under certain conditions, which in this case was chosen to be a congested traffic situation, requiring a scenario that leaves the vehicle boxed in between other vehicles. This highlights that the driving context as envisaged by the designers is not understood by the users in the intended manner.
Vehicle
Vehicle Operations, Limitations, and Abilities are all aspects that describe the performance and activities that the vehicle performs or that the users ascribe to it. The first aspect clustered in this category is Vehicle Operations, which refers to parts of the driving task that the vehicle takes over, for instance acceleration, deceleration, and steering. In addition, there are the perceived Limitations of the automated systems, as well as perceived Abilities that the users ascribe to the vehicle, such as keeping the vehicle in the lane, keeping the vehicle’s position in the queue, or a smart and predictive characteristic of the vehicle. These aspects are not influenced by or in interaction with the driver, which is why these aspects are vehicle specific.
Vehicle operations
The participants talked about Vehicle Operations that were conducted by the vehicle. These driving-related tasks were described as “speed up, slow down, and brake.” Both systems were perceived to conduct the operations “according to how traffic flows.” One difference was that the participants described the Level 2 system as also taking over steering, while the Level 4 system was perceived as only accelerating and decelerating. This difference in perception is very relevant as the Level 2 system has no steering ability, even though participants correctly identified that the car can “self-correct in the lane,” which is explained by the lane-keeping assistance the Level 2 system provides. However, only the Level 4 system has full steering abilities, which was not discussed or mentioned by the participants.
Abilities
In relation to Vehicle Operations, the participants described certain higher-level Abilities of the vehicle, beyond those of the driving task. The participants described the vehicle as “smart” and explained the predictive capabilities of sensors, which were regarded as the “eyes of the car.” Further, they stated that they perceived the Level 4 system as being able to understand the traffic situation and they thought it had “good predictive capabilities on how to engage with other vehicles.” This ability was perceived to keep the driver safe, as “it is just watching [out] for you to [not] get hit.” Even though the Level 2 system was also perceived as smart, it appeared that the Level 4 system was perceived as more capable, as the car was perceived to “watch over you” when driving with the Level 4 system, while the Level 2 system would support and react to the situation at hand.
Limitations
When talking about the Limitations of each of the two modes, the respondents were more ambiguous, and it appeared that the systems’ capabilities and constraints were not clear. Some participants stated that “there were no clear-cut directions as to what each function would do,” which indicates that there were uncertainties about the capabilities and limitations of each mode. The participants understood that “in either feature it couldn’t switch lanes.” On the other hand, the participants identified that the vehicle “wouldn’t notice lights or signs” when driving with the Level 2 system but thought that it could with the Level 4 system, which it is not able to do. Furthermore, some of the participants seemed to be under the impression that the Level 2 system needed “a vehicle in front of and on the side [of the car]” to be able to work, but did not mention any such conditions or limitations for the Level 4 system. As the Level 4 system was meant to work in congested traffic, these were the limitations, but the Level 2 system did not have any such restrictions at all. However, it is interesting that when speaking about the Level 4 system, the participants did not identify many limitations overall.
To summarize, there was uncertainty and confusion among the participants, especially when describing characteristics and features of the systems. In particular, the Vehicle Operations and Limitations seemed to be misunderstood. It led to confusion about whether the Level 4 system works at low speeds and in congested traffic situations, or if it is some sort of highway pilot. This finding was supported by the descriptions relating to the Context. Even though most participants identified the Level 2 system as working in all traffic situations and on all road types, and the Level 4 system as working in congested traffic situations, there was uncertainty as to the road types and speeds at which the Level 4 system was operational. Furthermore, the participants described the vehicle as smart and supportive, but particularly the Level 4 system was perceived as looking ahead and as having some type of situation awareness, with the participants ascribing higher-level abilities to the vehicle. These aspects, however, were especially important as they seemed to evoke a feeling of trust and a sense that the participants could rely on the system to keep them safe, while the Level 2 system was not ascribed with those abilities as it was seen as merely reacting to situations and supporting the driver.
Driver
The driver-specific element includes the Driver Tasks, Control, and Attentional Demand aspects, as they describe and consider actions that the driver has to perform when interacting with the vehicle. While the Driver Tasks, including operation of vehicle and interaction with displays and controls, concern direct interaction with the vehicle and the execution of the driving tasks, the Control and Attentional Demand aspects are constructs that describe the demands those tasks impose on the driver. Hence, this element describes more abstract constructs, such as the driver’s responsibility and the possibility of engaging in other tasks than driving. However, this element also includes operational activities such as interaction with the vehicle during the driving task. This can be the driving activity itself, but also connected tasks such as activation and deactivation of the DAS.
Driver tasks
Operation of the vehicle and direct interaction with the displays and controls could be differentiated from the participants’ description about the Driver Tasks. When operating the vehicle with the Level 2 system activated, participants described that they needed to drive themselves and to pay attention to the traffic, for instance, and also had to be aware of what the car was doing. This driving mode was regarded merely as an assistant to the driver, as evidenced by descriptions such as “I still have to drive and steer the car where it is going” and “It is more as though it is assisting you.” The aspect of interaction with the Level 2 system was mainly described in terms of controlling the system: “The [Level 2 system] would work when you press the button in the middle” and “When you press the plus (+) and minus (-) button, and it would accelerate and decelerate.” On the other hand, interaction with the Level 4 system was only mentioned in terms such as “You put it in self-driving mode with these buttons, and you see this graphic, and you can completely sit back.” Seemingly, the participants agreed that interaction with the Level 4 system was limited to activation or deactivation of the mode and that there was “… almost no need for human input,” while the Level 2 system required more than just activation, with one participant stating that “… there needs to be consistent human interaction” with the vehicle when the system is engaged.
Attentional demand
The differences discernible in relation to Driver Tasks carry over to similar distinctions related to Attentional Demand—the participants did not only understand the difference in the interaction required to operate the two modes, but also the difference in attention required to be placed on the driving task for each system. Their perception of the Level 2 system was that “it kind of helps you, but you still have to pay attention.” This meant that they were unable to completely relax when driving with this system as “you still have to drive” and “I have to be alert all the time.”
In contrast, when the Level 4 system was active, participants stated that they could “let go of the steering wheel and take the foot off the brake.” They felt they could then relax and engage in secondary tasks as exemplified by one participant: “You can really just relax and do other things,” like “you can check your email, you can raise your level of productivity.” However, the drivers also felt that they could “take time to pay attention to yourself and your thoughts and be able to get a little bit prepared for wherever you are going.” This shows that the drivers felt that the Level 4 system was relieving them of strenuous physical and mental activities, compared with the Level 2 system, which was perceived as a support, but not supportive enough to significantly lower the workload.
Control
Regarding Control allocation while driving with the activated systems, the descriptions differed here too. When using the Level 2 system, participants regarded the driver as being in control of the vehicle, or the driver was perceived as receiving assistance from, or sharing control with, the vehicle. This control allocation was described as “sharing it, not giving all to a machine.” As a consequence of the perceived shared control, several participants revealed confusion regarding the extent of control allocation when the Level 2 system was active, with one participant stating that “it’s kind of ambiguous to me exactly how much responsibility it’s going to take.” The Level 4 system was described as taking over control completely from the driver and requiring very little human input. One participant described the difference by stating that with the Level 2 system they were sharing control while when driving with the Level 4 system they felt like a passenger in the car, and that “you can completely sit back.” The modes were compared with the following statement “with the [Level 2 system], and the adaptive speeds, and turning assist, it still felt like I was in control of the vehicle, while in the [Level 4 system] I felt like then I am not in control of the vehicle.” The variety of statements shows that control allocation was something that caused confusion, especially when driving with low automation such as the Level 2 system, with some participants even stating “it just felt confusing to me to share [the control].”
In summary, a few aspects regarding the drivers’ activities in the car were perceived by the participants as intended by the system designers. All participants understood that the Level 2 system is an assistive system that needs constant engagement, while the Level 4 system does not need any human input after activation. Furthermore, the participants were clear that the Level 2 system needs constant attention and that they were supposed to supervise the system, whereas when driving with the Level 4 system they were free to divert their attention to other tasks than the driving task, leaving them free to tend to activities such as reading, texting, watching videos, or socializing. These findings imply that the Driver Tasks and the Attentional Demand were understood as intended and that there was no confusion about what the driver’s role in the car was during engagement with the two systems. However, regarding Control allocation, there was confusion about who is in control when driving with the Level 2 system. While the Level 4 system was clearly understood as taking over control and leaving the driver to travel a passenger, the Level 2 system was perceived as ambiguous who was in control or as sharing control with the driver. This finding is critical, both for the design of the system and for what is communicated to the driver regarding responsibility over the driving task, as the driver should be in control at all times when driving with an assistive system and not feel overpowered by the vehicle.
Discussion and Implications
Even though existing frameworks for describing different levels of automated driving have benefits (Endsley, 2018; Kaber, 2018) they have also been criticized for being based on detailed technical and functional taxonomies (e.g., Seppelt et al., 2018; Yang et al., 2017) or as having too narrow a function allocation approach (e.g., Lee, 2018). Several authors have therefore argued the need for a more human-centric taxonomy of automation types (e.g., Kaber & Endsley, 2004; Jamieson and Skraaning (2018); Seppelt et al., 2018). For example Homans et al. (2020) conclude, based on a survey, that users’ understanding of automation levels is not in line with the taxonomies, and in another study Seppelt et al. (2018) found that consumers had low to moderate understanding of different types of automation and that the proposed taxonomy produced confusion regarding responsibilities. Acknowledging these issues, attempts have been made to decrease users’ confusion, for example by investigating how the names of the DAS are associated with different LoAs (Abraham et al., 2017), by rephrasing the descriptions of responsibilities (e.g., Yang et al., 2017) or by framing the driver’s responsibility in terms of “driving” and “riding” only (Seppelt et al., 2018). The addition to SAE J3016 (SAE International, 2019) aims to describe the LoAs through simpler language and a graphical representation, exemplifying what the driver has to do and what the system does in each automation level. Even though the human role is now more clearly defined, the chart still exemplifies which actor is responsible for which task and fails to address a more detailed explanation of the driver’s understanding of the driving automation. Therefore, the additional explanations do not provide a sufficiently detailed description of the vehicle’s capabilities and limitations or of the driver’s responsibilities.
The conclusions referred to are based on studies of how users understand descriptions of systems and LoAs. The purpose of the reported study was therefore to investigate in greater detail how users actually perceive and understand different LoAs in a real vehicle, in a real traffic context. Based on the findings, a categorization of aspects was proposed (Table 1) and used to describe the participants’ understanding of the differences between the Level 2 system and the Level 4 system. This shows that the participants noted differences between the two systems, that they attributed higher abilities to the Level 4 system than it in fact had, overestimating the capabilities within the ODD (e.g., some participants believed it could change lanes). It is difficult to know the underlying reasons for the overestimation of the capabilities, but it is possible to assume that it stems from a combination of expectations and preconceptions based on earlier experiences and information received through media about automated vehicles. Issues regarding control and responsibility were also noted including a critique of the Level 2 system, as it was ambiguous to the driver who was in charge of the driving task. This is an indicator that when presented with a Level 3 system, drivers might experience even more uncertainty or confusion over their responsibility, as they are handing over the driving task to the vehicle and taking on a monitoring role. Considering that there is already confusion when interacting with both a Level 2 and a Level 4 system, which are very different in terms of functionality and responsibility, one can assume that the confusion will be notably higher when dealing with a Level 2 together with a Level 3 system. This has also been indicated in an on-road study conducted by Banks et al. (2018), who found that the drivers’ faulty understanding led to complacency and over-trust into the system. Fundamentally, however, the participants’ perception and understanding of the different systems were neither based on the notion of levels, something also recognized by Banks et al. (2018), nor were they merely regarded as a division between “drive” and “don’t drive” or between when the driver was in charge and when not. Thus, even though the driver and the vehicle play an important role, as defined in the SAE taxonomy, the results of this study show that the users’ understanding of the interaction with the DAS is more complex.
The proposed new categorization shows that users of vehicles with assisted and automated drive do not understand or perceive the differences in terms of how the systems are described in the existing taxonomies (compare Homans et al., 2020; Seppelt et al., 2018). Rather, the findings imply a continuous consideration of the interdependency between aspects related to traffic conditions, perception of the abilities and limitations of the activated system, and responsibility—in other words aspects related to context (conditions), vehicle (performance), and driver (responsibility). We suggest that this interdependency can be described as a layered structure, where the different layers are in interplay with each other and influence the users’ interaction with the system. On the highest level is the driving context, which affects how the driver perceives the performance of the vehicle. The vehicle performance will determine how the drivers understand their own responsibility. If they understand who is in charge of the driving task during engagement with the different LoAs, this will in turn, at the lowest layer, shape the drivers’ understanding of how and when they can interact with the vehicle. With the assumption that drivers make sense of and interact with the system in this way, it could be useful to also represent the interaction and explain a system accordingly (Figure 4).
Conceptual model of the drivers’ understanding of driving automation systems.
Practical Implications for the Design of Automated Driving Systems
Another critique of the levels of automation has been that existing descriptions and taxonomies do not provide any basis for design (e.g., Kaber, 2018; Vagia et al., 2016). Based on the findings and the proposed structure, users’ interactions with the autonomous vehicle are proposed to be guided by three main questions: (1) When can I use the system? (2) What does the vehicle do? (3) What do I do? The introduction of these questions into the design process can help the designer reduce the uncertainty that users seem to have regarding automation. The questions can serve as a starting point when designing either the functionality of the system or the information and feedback that the system should provide to the user. However, it is then crucial to consider not only vehicle functionality and driver responsibility but also the different aspects that influence users’ perceptions and understanding. It is relevant to take account of the different aspects when designing the system, but it is perhaps equally important to communicate to the driver as an introduction to the modes, or even during driver interaction with the system. Likewise, it is important to consider the layers. On the one hand, if the context of use (i.e., the ODD) for the system is set, one could start from the context layer and continue by defining the performance of the vehicle and responsibility of the user. On the other hand, if the intended responsibility of the driver is defined, one could instead work “upwards,” defining functionalities and the context in which the system could be used.
Below, we introduce one example of how the questions can guide practitioners during the design of the DAS assuming a Level 4 automation. A highway pilot is used as a concrete example. Applying the model, designers and developers can use the identified questions and ask themselves the questions the users will ask when interacting with the systems. Following those, the model can be used as a decision support to cover the relevant elements and aspects that users identify during the interaction with a DAS. Table 3 illustrates how the model is put to use to specify what information the users need to be provided with in order to understand the system’s capabilities and limitations, as well as which design implications are connected to the decisions made in the development.
Example of How to Use the Model on the Concrete Example of a Level 4 Highway Pilot.
Returning to the case of the highway pilot, the designers are presented with the ODD, here the Context, as the starting point of their investigation. Starting from there, technical limitations will have direct implications for the context specific characteristics, and in turn the context will dictate the abilities that the system has to fulfill. On the driver-specific level, the description of their responsibilities and what interaction is needed at a given point need to be clarified. Considering that one starts from a user perspective, where the goal is to free the user from the driving task during long distance drives, the designers start from the bottom first deciding on the Control and the Attentional Demand that is required from the user. After that, the designers and developers can work their way up and decide which functionalities the Vehicle must provide and under which Contexts the systems are available. However, no matter the decisions made for the development of the systems, it must be ensured that the vehicle communicates information on all aspects and sub-aspects to answer the users questions and support their understanding. How the designers and developers choose to communicate these solutions is up to each specific case and technical considerations that have to be met. The communication to the user can, for example, be done through the use of the driving behavior of the vehicle, information through displays, smart agents guiding the driver, or multimodal messages.
The categorization shows the need to involve different competences in the DAS design process. The different layers indicate that users understand the interaction with automated driving systems as different elements, that is to say driver, vehicle, and Environment. Having only a functional (vehicle) view or a user-centric (driver) view is not enough to design an automated driving system, addressing the drivers’ needs. The design of the DAS needs to consider all the different elements and the connected questions, in order to reflect what the users are looking for when trying to make sense of the interaction with the system. When designing a DAS, this means that software developers, interaction designers, and strategists need to work hand in hand in order to address user needs and create value for the user.
Conclusion
This study has identified several aspects that users use to describe different LoA, reflecting their understanding of a Level 2 and a Level 4 system. Furthermore, different elements could be derived from the system descriptions, indicating that user understanding does not only encompass a single level regarding the function allocation, but that there is a layered structure, represented in a conceptual model. Therefore, this paper concludes that the identified elements can aid in the development of design guidelines for DAS, by adding a new perspective on how users understand their interaction with the systems, its capabilities and restraints, as well as their role as driver.
Further work is required to address these aspects in future taxonomies and guidelines. However, the proposed structure can contribute to the development of more user-centered taxonomies, which can be utilized by designers and developers for the improvement of DAS.
Footnotes
Acknowledgments
The authors want to thank all the participants who offered valuable insights and contributed to enjoyable sessions. Furthermore, the authors want to thank their colleagues at Volvo Car Corporation who supported and made this study possible, specifically Annie Rydström and Mattias Söderholm Mullaart who supported us with the extraction of the vehicle data. Funding: Author 1 is thankful for the grant from Sweden’s Innovation Agency VINNOVA (grant no. 2017–01946.). Author 2 gratefully acknowledges the financial support provided by Chalmers University of Technology.
Author Biographies
Fjollë Novakazi is an industrial PhD candidate at Volvo Cars, and affiliated to the Division of Design & Human Factors at Chalmers University of Technology. Her research concerns how users use and understand driving automation. The main focus is to describe and understand the phenomenon of mode confusion in the interaction between driver and driving automation system. She approaches her studies through empirical mixed-methods research, facilitating surveys, in-depth interviews, naturalistic studies, and on-road observations.
Mikael Johansson is a PhD candidate at the Division of Design & Human Factors at Chalmers University of Technology. He received his MSc in industrial design engineering from Chalmers University of Technology in 2015. His research focuses on users’ understanding of automated vehicles, particularly the development of mental models in the interaction with automated systems.
Helena Strömberg is a senior lecturer at the Division of Design & Human Factors at Chalmers University of Technology. She studies the interplay between people, technology, and behavior through the lens of design. The main application area is transport, encompassing everything from increasing cycling, to developing mobility-as-a-service, to the future use and role of new transport technology such as autonomous vehicles.
MariAnne Karlsson is a professor (Chair) in Human–Technology Systems and Head of the Division of Design & Human Factors at Chalmers University of Technology. Her research concerns the relation between people and technology in a broad sense. A particular focus is how users understand, interact with, and use different types of driving support systems. In addressing the issue, she has used different methodological approaches including surveys, focus group interviews, simulator studies, as well as field evaluations.