Abstract
The rapid development of driving automation systems (DAS) in the automotive industry aims to support drivers by automating longitudinal and lateral vehicle control. As vehicle complexity increases, it is crucial that drivers comprehend their responsibilities and the limitations of these systems. This work investigates the role of the driver’s perception for the understanding of DAS by cross-analysing four empirical studies. Study I investigated DAS usage across different driving contexts via an online survey conducted in Germany, Spain, China, and the United States. Study II explored contextual DAS usage and the factors influencing drivers’ understanding through a Naturalistic Driving Study (NDS), followed by in-depth interviews. Study III employed a Wizard-of-Oz on-road driving study to simulate a vehicle offering Level 2 and Level 4 DAS, paired with pre- and post-driving interviews. Study IV following up used a Wizard-of-Oz on-road driving study to simulate Level 2 and Level 3 DAS and subsequent in-depth interviews. The findings from these studies allowed the identification of aspects constituting a driver’s understanding and factors influencing their perception of DAS. The identified aspects and factors were consolidated into a unified conceptual model, describing the process of how perception shapes the driver’s mental model of a driving automation system.
Introduction
The development of automated vehicles (AVs) has garnered significant attention in various sectors, including industry, academia, and the general public because of their argued potential to revolutionise transportation. However, the widespread availability of fully automated vehicles capable of operating under all situations is unlikely to occur over the next few decades, despite the rapid evolution of technology. This has resulted in a transitional phase for drivers, where the vehicle and the driver share the responsibility and control over the driving task. During this transitional phase, there is a need for clear guidelines and regulations to ensure the safe integration of driving automation systems into vehicles.
Driving automation systems (DAS) can be conceptualised as a compilation of active safety technologies designed to assist the driver. Therefore, DAS are widely recognised as effective measures for tackling issues related to traffic safety (Kyriakidis et al., 2015; NHTSA, 2017). However, drivers find themselves in the situation where the capabilities and limitations of such systems vary greatly, as they become increasingly complex with the introduction of vehicles offering several levels of automation (LoA). According to the definition provided by SAE International (2021), the Levels of Driving Automation range from Level 0 (No Driving Automation) to Level 5 (Full Driving Automation). Vehicles classified as Level 0 to Level 2 automation are fitted with driver assistance systems that possess varying capabilities and functions. For example, Level 1 vehicles are equipped with adaptive cruise control (ACC) and Level 2 vehicles have both lane-keeping and adaptive cruise control functionalities (Marcano et al., 2020). Vehicles equipped with Level 3 and Level 4 DAS can operate without any input or intervention from the driver in specified driving contexts. These are termed operational design domains (ODDs), that is, specific environment, scenarios, and situations in which the automated system is intended to function without human intervention. Level 5 DAS can fully automate the vehicle in all ODDs and does not require any driver input.
A number of studies have investigated different factors that impact drivers’ safe utilisation of DAS. Many conclude that the utilisation of automated systems is heavily influenced by the driver’s mental model of the capabilities and limitations of these systems (Ahlström et al., 2018; Ahmed & Ghasemzadeh, 2018; Papazikou et al., 2017; Zhai et al., 2018). However, it has been shown manifold that a significant number of drivers lack awareness or possess incomplete comprehension about the constraints associated with the DAS in their personal vehicles (Boelhouwer et al., 2020; McDonald et al., 2018; Regan et al., 2020). Therefore, for these technologies to improve traffic safety as well as enhance the driving experience and comfort (Vasile et al., 2023), DAS must be designed so that drivers accept them, understand their capabilities and limitations, and use them appropriately, without abusing them or becoming overly reliant on them.
However, the implementation of several levels of automation (i.e., driving modes) in vehicles has brought about a level of intricacy that poses considerable difficulties for drivers in understanding the capabilities of a DAS and their own role when interacting with such systems (Endsley, 2017). The availability of several driving modes, which are offered depending on different ODDs, can result in a state of confusion about what the system’s and what the driver’s responsibility is at a given time. The state where a driver is not aware of what driving mode or automation level is currently active is also known as mode confusion (Sarter & Woods, 1995). A simulator study by Feldhütter and colleagues (2019), the phenomenon of mode confusion in the interaction with automated vehicles, found that mode confusion between two driving modes is especially an issue in transitions. Some work contends that the user interface (UI) is essential in addressing this issue (cf., Banks et al., 2018; Carsten & Martens, 2018). A wide range of studies has investigated how transitions effects in automated driving (cf. Zhang et al., 2019). A literature study conducted by Kim and colleagues (2021) reviewed the effects of the UI in transitions and found that there are many conflicting results as to if the UI can have a positive effect on the driver’s interaction with DAS. These findings highlight that there are other factors that need to be investigated.
Therefore, it can be deduced that one of the elements crucial for the safe utilisation of DAS is the driver’s mental model of the vehicle’s driving modes. Failure to acquire this knowledge can result in an inability to formulate effective usage strategies and lead to hazardous situations (Banks et al., 2018; Wilson et al., 2020). Thus, in order to create a positive driving experience and prevent the misuse of DAS due to overreliance and misaligned mental models, designers must carefully consider the implications of various variables, such as cognitive, behavioural, and contextual aspects, and how they influence the driver’s perception of and consequent interaction with DAS.
Mental models and perception
Within the automotive research community, mental models (cf., Rouse & Morris, 1986) have been recognised as a significant element in ensuring the safe utilisation of DAS. Mental models, although subject to development and evolution through experience, primarily consist of static knowledge related to the product or system. This includes its prominent features, operational mechanisms, interrelationships among different components, and the anticipated behaviour of its components when exposed to various external conditions (Endsley & Garland, 2000).
Different investigations address different aspects of the topic. For instance, how do drivers construct a mental representation of a DAS operation? What is an adequate mental representation that facilitates safe utilisation? In what ways might human–machine Interface design contribute to the formation and development of a mental model? How do faulty mental models emerge, and how can they be avoided? How can driver training support the development of a sufficient mental model in the interaction with DAS?
However, our comprehension of the world is based on the information we perceive through our senses from the environment. In other words, perception is the process by which objects, events, and relationships become phenomenally here, now, and real (Hochberg, 1956). In contemporary research, there is a prevailing consensus that the existing theories pertaining to perception do not offer comprehensive explanations for the intricate processes involved. Consequently, it can be inferred that the field of study pertaining to perception is an ongoing endeavour. To date, there are two opposing views concerning the concept: (i) “bottom-up” theory of perception (Gibson, 1972), describing this process as a direct interpretation of incoming stimuli, and (ii) “top-down” theory of perception (Gregory, 1974), describing the integration of higher cognitive information, such as previous experiences and knowledge.
Other work suggests that all information is processed and interpreted based on a person’s “perceptual sets” (Allport, 1955), which are previous experiences, beliefs, and context (Oliver, 1977). Thus, perception is the result of a combined bottom-up and top-down process, and hence, that categorises what is perceived from the world in patterns, guided by our needs, expectations, and beliefs. In other words, perception is the process of receiving sensory information, and then organising it into patterns, and recognising or interpreting its meaning, in order to plan and execute an action. According to this interpretation, one can infer that top-down and bottom-up processing occur simultaneously during the perception process and that attention is guided by either of them at any given moment, depending on the strength of the stimulus or the individual’s objectives.
The irony of driving automation lies in the perception of increased capability of such systems, which leads to a range of human factors issues (e.g., overreliance and reduced situation awareness), and the introduction of several driving modes into one vehicle seems to exacerbate the existing problems (e.g., Feldhütter et al., 2019). It follows that the more driving modes are introduced into the vehicle, the less likely it is for the driver to understand the intricate system states fully, as the increased complexity makes it difficult for drivers to understand the various driving modes the vehicle offers, and thus, it becomes more challenging to grasp how the vehicle will behave in different situations. This contributes to ambiguity surrounding automation levels in vehicles, reducing the driver’s overall understanding of the vehicle’s capabilities and their own responsibilities (cf. Sarter & Woods, 1995; Seppelt & Lee, 2015; Vagia et al., 2016). Multiple studies indicate that the driver’s understanding of the capabilities and limitations of driving DAS, their awareness of the available and currently active driving modes, and ability to maintain the appropriate level of engagement and intervention in crucial scenarios is a matter of worry (Jenness et al., 2008; Larsson, 2012). These concerns are widely based on evidence about driver misconception of the relevant functions, or over-confidence in the systems’ capabilities and limitations. A survey conducted by Seppelt and colleagues (2018) revealed that the taxonomy proposed by SAE International (2021) led to confusion regarding the roles and responsibilities of drivers, for example, that drivers could distinguish between driver assistance and full automation but had difficulty distinguishing between the levels in between. Similarly, Abraham et al. (2017) demonstrated that the design of DAS affects the driver’s perception of the system and the expected levels of accountability. Furthermore, Homans and colleagues (2020) concluded based on a survey that drivers’ understanding of automation levels does not align with the taxonomies. This ambiguity may result in misuse or even non-use of the system, thereby diminishing the potential benefits that automation support can provide (Parasuraman et al., 2000).
Some research streams, in search of explanations for the observed human factor issues, have indicated that the existing design of DAS does not sufficiently consider the driver’s mental model of interacting with a DAS (e.g., Beggiato & Krems, 2013; McDonald et al., 2018; Novakazi et al., 2021). While ongoing attempts are being made to address these enquiries and others of a similar nature, some has long come to recognise that addressing the mental model during the design of a DAS can aid drivers in their understanding the vehicle’s capabilities and limitations (Dickie & Boyle, 2009; Larsson, 2012); information and feedback received from the vehicle is relevant for drivers to be able to develop a mental model (Blömacher et al., 2020; Boelhouwer et al., 2020; Seppelt & Lee, 2019); and that the ability to anticipate potential hazards and make effective decisions and responses on the road is impacted by the ability to form a sufficient mental model (Beggiato et al., 2015; Beggiato & Krems, 2013) and that a more comprehensive mental model affects the interaction with DAS positively (Gaspar et al., 2020). Hence, many works have investigated how exposure and use of a system impacts the driver’s mental model of it (Carney et al., 2022; Jenness et al., 2019), and a lot of focus has been put on how training and what different training methods can aid the development of a sufficient mental model for the calibration of trust and better utilisation of a DAS (Gaspar et al., 2021; Merriman et al., 2023, 2023b; Singer et al., 2024).
However, drivers struggle to understand feedback received from the vehicle about the different mode changes or are confused about information received from the vehicle, and what they are supposed to do (Monsaingeon et al., 2021), and other work shows that car manuals, which include information about the operation of the systems, are not successful in training drivers due to the provided information being too technical and abstract and therefore go underused (Harms et al., 2020; Viktorová & Sucha, 2018). As a result, drivers often encounter challenges when attempting to put the information into practical application (McDonald et al., 2018; Oviedo-Trespalacios et al., 2021).
These results suggest that there may be more variables, which have not been already taken into account, that influence the development of a mental model, that is, the role of driver’s perception in mental model formation. However, the effect of a priori information provided to the driver and previous knowledge of and experiences with the systems is little researched, even though it appears of big importance for the formation of mental models. To address some of these challenges, Biondi (2023) proposes developing clear and distinct naming conventions, training of users at the point of sale, and driver self-education as solutions that can help the user develop a mental image and points out that the mental model formation should better match and sustain the system’s actual design. Other work has explored how the driver–automation collaboration can be explored in different ways in order to reveal practical requirements and design considerations to support the matching of a mental model (cf. Lee et al., 2023). The match of the drivers’ mental model to the system design has long been acknowledged as a key aspect to the implementation of DAS. Lee and See (2004) established that a mismatch between the expected and actual system behaviour can lead to distrust. Kraus and colleagues (2020) found that provided information before the interaction with a system affects trust and is highly dependent on the level of previous knowledge. Furthermore, it has been found that incomplete or mismatched mental models affect the situation awareness (Endsley, 2000), as well as the ability to understand limitations of a system and anticipate system behaviour and the driver’s own responsibility for the driving task in those situations (Blömacher et al., 2018). Furthermore, research suggests that driver’s acceptance and attitude towards driving automation is also affected by the mental model (Beggiato & Krems, 2013).
Thus, it can be inferred that irrespective of whether there have been previous experiences or if it is the first encounter with a DAS, the driver’s pre-existing knowledge of DAS will impact their interaction with such systems. This mental predisposition or readiness to perceive sensory stimuli in a particular way based on previous experiences, expectations, beliefs, and context is known to be influenced by “perceptual sets” (Allport, 1955). Perceptual sets regulate choices between competing alternative activities and, therefore, influence the outcomes of the perception process. This means, at a fundamental level, that perceptual sets influence what in the available sensory data we perceive and what we ignore and thus constitute our existent knowledge of the world. This is also in line with expectation theory (Oliver, 1977), which asserts that a person will act in a particular manner based on their personal frame of reference, motivations, and interests, resulting in that decisions are based on the individual’s estimation of how closely the anticipated outcomes of a given behaviour correspond to the desired outcomes. The lack of correlation between expectations and the actual state of the real condition of the situation might lead to an adverse outcome than initial intentions.
In the realm of driving automation this can result in situations such as the fatal Tesla Model S accident which occurred on 7th of May 2016 (National Transportation Safety Board, 2017). Data extracted from the Tesla vehicle revealed that it was being operated in autopilot mode, and that the system failed to issue any forwards collision warnings or intervene by reducing the velocity of the vehicle. Further, it was shown that the driver did not attempt to intervene in any form either. The results of the investigation did not identify any design defects of the vehicle that may have contributed to the collision, however deducted that the incident was primarily caused by human error, that is, the driver being distracted from the driving task for an extended period, which was described in the report as “inattention due to overreliance” (National Transportation Safety Board, 2017). Historically, human error has been used to classify and predict issues during interaction with complex systems (c.f., Embrey, 1986; Stanton et al., 2009). However, it has long been argued by different researchers that it is essential to human errors should not be the result of an investigation, but rather the starting point for finding an explanation for incidents (Dekker, 2017), and that is important to understand the reason for the actions or inactions of the involved persons within the context of local rationality (Reason, 1990), in order to understand the underlying reasoning for their actions. As previously stated, the answers are likely to be found in the driver’s perception of the capabilities and limitations of DASs. Nevertheless, there is currently a scarcity of research examining the role of driver’s perception on the subsequent use of these systems.
Human-centric perspectives on driving automation
Earlier research efforts have investigated a range of variables that have been deemed important for the driver’s interaction with a DAS. These studies have most frequently investigated the topic of safety and take-over requests, trust and complacency, acceptance of and attitude towards automated vehicles, situation awareness, workload and stress, and drowsiness and fatigue, among other factors. For a comprehensive summary, please refer to the literature review conducted by Frison and colleagues (2020). The review highlights that existing research often focuses on a limited range of variables and methodologies, without considering the interplay between variables in the dynamic driving environment or triangulating data for a deeper understanding of driver–automation interaction.
Over the years, different approaches have been sought to identify variables relevant to the driver’s interaction with DASs. For example, various works have attempted to describe the driver’s interaction with a vehicle through behavioural models. Michon (1985) conducted a review of driver behaviour models and identified four different types of models along two dimensions: (i) behavioural models, representing behaviour, versus (ii) psychological models, representing cognitive processes, versus (iii) taxonomic models, representing an inventory of facts and their relationships, versus (iv) functional models, containing components which interact dynamically. Based on his analysis, he remarks on the absence of driver-related factors such as cognitive functions, beliefs, and emotions in most models, as they are behavioural-functional and concentrate on specific characteristics of the driving task and driver behaviours. Thus, they do not answer the question as to why the driver behaves in a certain way. He emphasises the need for additional research in the area of cognitive processes to gain a better understanding of driver behaviour and driver motivation (Michon, 1985). To date, this circumstance has not changed.
Building on Stanton and Young’s (2000) psychological model of driving automation, Heikoop and colleagues (2015) proposed a consensus-based model describing interrelations between psychological constructs. Their model, based on a limited literature search, includes nine variables (mental model, situation awareness, attention, trust, mental workload, stress, feedback, task demands, and fatigue) and is critiqued for construct proliferation. The authors acknowledged the need to extend the model with additional psychological constructs and empirically investigate these variables and their interrelations.
Consequently, numerous authors call for a more human-centric classification of automation levels (cf. Kaber, 2017; Roth et al., 2018) than provided by the currently prominent Levels of Driving Automation provided by the Society of Automotive Engineers (SAE International, 2021). While there are advantages to utilising established taxonomies that categorise levels of automation (cf. Endsley, 2017), studies show that the driver’s mental models do not align with the technically driven taxonomy (cf. Novakazi et al., 2021; Seppelt et al., 2018; Yang et al., 2017). Some have proposed improvements, and attempts have been made to reduce driver confusion, including investigating how the names of DAS that are associated with different levels of automation (Abraham et al., 2017), rephrasing the descriptions of responsibilities (e.g., Yang et al., 2017), and framing the driver’s responsibility in terms of “driving” and “riding” (Seppelt et al., 2018). In light of this realisation, SAE has added a visual chart to the initial taxonomy to simplify and clarify the “Levels of Driving Automation” for consumers (SAE International, 2021). The addition describes the levels of automation through simpler language and graphical representation, illustrating what the driver must do and what the system does. However, despite the fact that the human role has been clarified, the taxonomy continues to illustrate which actor is responsible for which task and does not account for human perception of the system. Further, studies show that the SAE taxonomy is not only ambiguous to drivers but also to practitioners and researchers, who struggle to find a unanimous interpretation of the provided LoA (for an overview, see Parker et al., 2023). Other researchers argue that the extant levels of driving automation taxonomies are technology-centred and presented from the perspective of vehicle technology development or policy development (Yang et al., 2017), which is problematic as this approach may impact the design of the driver–vehicle interaction and might not address the ways drivers comprehend such systems. Smith (2017) further argues that the established taxonomies influence the way designers think about the system, resulting in design decisions driven by technological instead of driver-centric viewpoints.
The aforementioned approaches collectively indicate an underlying issue: current research efforts lack a cognitive and holistic approach to the driver’s understanding of DAS.
In an effort to address this, several studies have examined the difficulties associated with designing DAS and have reached the consensus that numerous difficulties may emerge in the initial phases of development. These challenges are primarily attributed to the adoption of technology-centric taxonomies, which tend to overlook the human driver and prioritise task-allocation strategies (cf. Seppelt et al., 2018; Smith, 2018; Yang et al., 2017), as well as the lack of variety in methodologies, and no triangulation of data. Moreover, it has been established through additional research that the user’s understanding of automation levels does not align with the existing taxonomies (Homans et al., 2020).
Seppelt and Victor (2020) developed a conceptual model describing the influence of a driver’s mental model on automation reliance. The authors argue that drivers initially construct general mental models from various sources like manuals and marketing materials, gaining a basic understanding of system purpose and operating rules. However, these general models may lack the specificity needed for real-time interaction with varying environmental conditions. Applied mental models develop through experience with the system, which in turn will influence the general mental model. Therefore, assessing both general and applied mental models, and particularly identifying mismatches between them, is essential for understanding driver reliance decisions and automation use. While the model points out that different information sources influence driver’s reliance on DASs, it lacks a comprehensive description of the different aspects constituting a driver’s understanding and the factors influencing mental model development. Therefore, a more nuanced understanding of the specific components of driver mental models and the factors that shape their development is necessary.
Aim
The existing research has placed little attention on the effects of perception on the construction of mental models, and the effect this can have on the interaction with a DAS. To date, the research body offers only scattered insights, focusing on specific components that affect a driver’s mental model of a DAS. Novakazi and colleagues (2021) conducted empirical on-road observation and in-depth interviews pre- and post-driving in order to investigate drivers’ understanding of vehicles offering several levels of automation. Their findings revealed a nuanced understanding, demonstrating that drivers’ mental models are not solely based on a single automation level or task-allocation perspective. Instead, they identified three key levels: Context, Vehicle, and Driver, each comprising various aspects that influence driver perception and interaction with the automated system.
Thus, the here presented manuscript builds upon this foundational work, aiming to explore and investigate the different factors that impact drivers’ perception and subsequently perception’s role in mental model development, in order to guide the development of driving automation from a human-centric perspective.
Overview of the Studies, Describing Data Collection Method, Assessed Level of Automation (According to SAE International, 2021), Number of Participants, and Context.
Method
The present research has been based on the utilisation of inductive and exploratory methodologies, more specifically a sequential mixed-methods research design (Creswell, 2014), employing online surveys, naturalistic driving studies, driving studies paired with observations, in-depth interviews, and think-aloud protocols to gather empirical observations. The synthesis of these observations has been instrumental in identifying recurring patterns and developing a comprehensive understanding of drivers’ behaviours and potential challenges for drivers when utilising DAS.
Summary of studies
To ensure consistency across datasets, each of the selected studies investigated driver understanding of DAS in different driving contexts and with different levels of automation. The section that follows provides a summary of the four study designs, including a description of the data collection method(s) and a brief summary of the key insights. An additional overview presenting details can be found in Table 1.
As all of the here presented studies involved human participants, a paramount ethical consideration revolved around ensuring the well-being, confidentiality, and informed consent of the individuals involved. To ensure the voluntary participation and guaranteeing that the data collected is utilised responsibly and ethically, efforts were made for each of the conducted studies to adhere to the guidelines provided by the Swedish Research Council (2017). Additionally, the retrieval, storage, and processing of the collected data rigorously adhered to the General Data Protection Regulations (GDPRs) of the European Union.
Study I
The aim of this study was to investigate the application of Level 1 and Level 2 DAS, in a variety of driving contexts. The survey was administered to 2000 drivers in Germany (DE), Spain (ES), the United States (US), and China (CHN) and covered their usage and experience with Level 1 and Level 2 DAS. The ages of the respondents ranged from 18 to over 65. Among the respondents, 27.1% indicated an annual mileage of 5001–10,000 km, 35.9% drove 10,001–20,000 km annually, 16.9% reported 20,001–30,000 km per year, and 6.7% said they drove above 30,000 km annually.
The driving contexts in all countries were mainly urban areas, followed by motorways or highways, and then the countryside, with a fairly even distribution between these, with the exception of China, where the majority of participants drove primarily in urban areas, with close to no driving in the other driving contexts.
The responses were collected via an online survey which was sent out via email by a third party with access to the various markets. There were a total of 36 questions, and it took participants between 10 and 15 minutes to complete the survey. The majority of the questions were Likert-type (Likert, 1932), scenario-based statements querying the participants on their use of the DAS in their personal vehicles in different driving contexts, for example, road types, weather and light conditions, road conditions, and the drivers’ physical and mental condition.
Key insights
The study identified the driving context, encompassing both situational factors, such as road types, traffic conditions, weather, and time of day, and the driver’s physical or mental state, including boredom, monotony, and tiredness, as crucial for understanding the capabilities and limitations of DAS. Regardless of the automation level, knowing the ODD is essential. However, despite DAS not being designed for all traffic, weather, and road conditions, the study found that participants often used these systems outside the intended ODD.
Study II
The aim of this study was to acquire a more in-depth understanding of the drivers’ motivations for using Level 1 and Level 2 DAS, as well as the drivers’ understanding of the capabilities and limitations of these systems. Building upon Study I, the contextual effects on the utilisation of the systems were also investigated. The Naturalistic Driving (ND) 1 Study was conducted over a period of 7 months, collecting data from 132 vehicles, and followed up with in-depth interviews. The results from the NDS were categorised into driver behaviour and system performance and measured in order to evaluate them independently and investigate their relationship. To evaluate driving contexts, the evaluation incorporated vehicle speed, driving distance, time of day, GPS data, wiper sensor status, road conditions data, and other sensor data. This analysis allowed for the identification of the driving conditions (e.g., road, traffic, and weather) in which the driver utilised the systems or decided to disable them. Subsequently, using in-depth, semi-structured interviews, an investigation and validation of the quantitative data were conducted to investigate the individual driver experience and understanding of the systems. The interview study consisted of 12 participants, 2 females and 10 males, with an age range of 31–62 years (Mean 52.4, SD = 9.0). The participants who had recently purchased a new car and were novice users of a Level 2 system were recruited via an email newsletter. All participants commuted every day, with five participants accounting for an annual mileage of more than 30,000 km, four participants driving between 20,001 km and 30,000 km per year, and three covering between 10,001 km and 20,000 km. The interview covered four sections: Contextual Information, System Usage and Scenarios, Perception and System Experience, and Information Display and Controls. Accordingly, a series of open-ended questions were formulated based on the four major themes. The structure of and questions in the interview were based on the preliminary findings of Study I and the learnings from the ND Study.
Key insights
The findings of the study found the driving context, including road types, traffic conditions, weather, time of day, and trip types, is fundamental for understanding the capabilities and limitations of DASs regardless of automation level. Personal conditions like tiredness, stress, and monotony of driving also play a role. Understanding the vehicle’s capabilities and limitations is crucial. The perceived capabilities influence how capable drivers believe the systems are and their willingness to use them, which is context dependent. Preconceptions and previous experiences with similar systems significantly impact acceptance and usage. Feedback on automation status is relevant but can sometimes misguide drivers if their perceptions of system performance contradict the feedback. Trust in DAS is closely linked to the driving context and learning experience, with varying levels of trust depending on the situation.
Study III
The aim of this study was to gather insights into first-time users’ perception and understanding of a vehicle that offers several levels of automation, specifically Level 2 and Level 4.
This was achieved by gathering insights from an empirical road study conducted in the San Francisco Bay Area, California, United States, in June 2019, with 20 participants: 11 females and 9 males ranging in age from 22 to 62 years (Mean = 42, SD = 14). During the study, participants encountered two different driving modes (levels of automation) in a Wizard-of-Oz 2 (WOz) vehicle: a Level 2 partial automation system and a Level 4 high automation system (SAE International, 2021). The setup facilitated the elicitation of insights by means of observation during the driving sessions, collection of think-aloud data, as well as in-depth semi-structured interviews before and after the driving sessions.
Key insights
This study revealed the driving context, including road types, traffic conditions, weather, time of day, and trip types, is essential for understanding when to use DASs. Personal conditions like monotony, tiredness, and stress also play a role. Understanding the vehicle’s abilities is crucial, as drivers’ perceptions of system performance influence how capable they believe the systems are. Less human involvement leads to higher perceived capability and trust in the system’s situational and predictive abilities. Trust is closely linked to the learning experience and is affected by how systems handle different driving situations. Previous experiences with similar systems serve as benchmarks for understanding new systems. Preconceptions, influenced by media, social circles, and legislation, affect the willingness to use unknown systems. Information about the vehicle’s capabilities and intentions is received through multiple sensory channels.
Study IV
The aim of this study was to analyse the dynamics of human–machine interaction in a vehicle that incorporated several levels of automation, specifically Level 2 and Level 3. This was accomplished by implementing an empirical road study conducted in Gothenburg, Sweden, in September 2022, with 16 participants (7 female and 9 male), ranging in age from 23 to 70 years old (M = 44, SD = 13.48). During the study, participants encountered two different modes (levels of automation) in a Wizard-of-Oz (WOz) vehicle: a Level 2 partial automation system and a Level 3 high automation system (SAE International, 2021). Furthermore, the study utilised an A/B UI test in order to compare the influence of different UI elements on the driver’s understanding and interaction with each of the systems. Data was collected through recordings of participants’ behaviour and think-aloud comments. Additionally, in-depth semi-structured interviews were conducted after the driving sessions to further enhance the insights gathered. The results were then contrasted and compared, with the end goal of gaining insights into the factors that influence drivers’ perception of a vehicle offering several levels of automation.
Key insights
The study found that the driving context, particularly road types and traffic conditions, is essential for understanding when to use DASs. The vehicle’s abilities are crucial for drivers’ perceptions of system performance influence their views on the system’s capabilities and limitations. Less human involvement leads to higher perceived capability and trust in the system’s situational and predictive abilities. Trust is closely linked to the learning experience and is calibrated by observing system performance and information received from the vehicle. Previous experiences with similar systems shape expectations and interactions with DAS. Many drivers struggled to understand the purpose of the two automation levels/driving modes, seeing themselves as always responsible for supervision and monitoring. Preconceptions, influenced by media, social circles, and legislation, affect the willingness to use DASs. Legislative status and social discourse on automated vehicle technologies can inspire distrust in the vehicle’s capabilities.
The following sections describe the process of how the results from all four empirical studies (Studies I–IV) were analysed during the cross-study analysis using a framework analysis approach, and synthesising the collected knowledge was turned into a model.
Cross-study analysis
The empirical studies were followed by a cross-study analysis that synthesised the results of the four empirical studies into a summary of the process by which the driver’s perception influences their understanding of DAS. The cross-study analysis primarily depends on the driver’s verbal accounts and observed behaviours while using a driving automation system, as well as their descriptions of the systems and their usage gathered during the conducted interviews.
The “Framework Approach” (Ritchie & Spencer, 2002) was utilised to approach the large and complex dataset. The framework approach, also referred to as framework analysis, is an inherently comparative form of thematic analysis that uses an organised structure of inductively and deductively derived themes (i.e., thematic framework) to conduct cross-study analyses by combining data description and abstraction (Ritchie & Spencer, 2002; Spencer et al., 2014). These methods facilitate the identification of similarities and differences in qualitative data prior to focusing on the relationships between the various data components and attempting to draw descriptive and/or explanatory conclusions. The defining characteristic of framework analysis is that it follows explicit steps to generate a structured output in a matrix or table (cases, codes, and summarised data), which is then used to systematically reduce the data (Ritchie & Spencer, 2002).
The data analysis was divided into five stages in accordance with the framework methodology: 1. Data Familiarisation: Immersion into the collected data and becoming familiar with the raw information. The purpose of this step was to gain a deeper understanding of the data’s contents and context. 2. Framework Identification: Identifying an appropriate analytical framework. A first step in the framework identification was the mapping of data into concept maps, allowing to organise the data. The purpose of this step was to establish a structured approach for categorising and understanding the large amount of collected data and its nuances. 3. Indexing: Systematic coding and labelling of segments in the data set. The codes represent themes, which were later categorised into different aspects and factors, enabling to identify recurring patterns across all the four studies and data types, allowing to triangulate the data. 4. Charting: In this step, the indexed data was condensed into matrixes, summarising and facilitating an overview of the identified categories, which aided in the identification of relationships and hierarchies. 5. Mapping and Interpretation: Finally, the matrixes were analysed deeper, and its contents compared in order to identify overarching themes with regard to the driver’s perception or understanding. The purpose of this step was to interpret and synthesise the data into tables, organised by the different categories and typologies.
The final stage was conducted iteratively and yielded a combination of key learnings that summarise the factors describing the driver’s perception and the aspects comprising the driver’s understanding of DASs. In addition, the structured approach generated typologies and enabled the mapping of connections between and within the identified aspects.
This ultimately led to the development of a model illustrating the process of how perception shapes understanding, including the identified factors of perception, and aspects of understanding associated with DASs. This model developed through an iterative process, and by conducting a workshop with colleagues who are experts in the area of automation and cognition and scrutinising the iterated ideas in discussions and thought experiments.
The result was a unified descriptive model, incorporating all identified factors, describing how user’s perception shapes, through a top-down and bottom-up process, the understanding of a DAS.
Findings
The findings from the cross-study analysis revealed that, irrespective of the degree of automation, users of DAS understand the systems by reference to three distinct components: the Context, the Vehicle, and the Driver. Further, the findings present 11 recurring characteristics that constitute the driver’s understanding of an automated driving system. The various aspects and connected sub-aspects have been found to collectively form the elements that contribute to a driver’s understanding, or mental model, of a DAS. Moreover, a total of six factors that exert an influence on the driver’s perception were found, ultimately impacting and shaping the driver’s understanding. These factors were further categorised as perceptual sets and sensory information, which subsequently modify and/or impact the driver’s mental representation of the DAS. The reported findings are subject to drivers’ perceptions of DAS and their assumptions about it, and may therefore not correspond with the actual capabilities or specifications of the vehicle and its ODDs. Thus, the following sections will detail the components of the driver’s perception and understanding of DASs, illustrating how perception shapes understanding by presenting a unified descriptive model that delineates the various interacting aspects of understanding and factors of perception.
The driver’s understanding
The findings indicate that regardless of the level of automation, drivers of such systems discussed driving modes by reference to three distinct elements: the Context, the Vehicle, and the Driver. In addition, the thematic analysis identified 11 recurring aspects: Driving Context, Personal Condition, Vehicle Operations, Comfort, Safety, Abilities, Limitations, Driver Tasks, Attentional Demand, Engagement in Other Tasks, and Authority. According to the findings, the identified aspects, along with their sub-aspects, constitute the driver’s knowledge of a DAS. The proposed classification suggests that the driver’s understanding consists of a layered structure in which the different elements (Context, Vehicle, and Driver) interact.
Context
Context. Description of Aspects and Sub-Aspects.
Vehicle
Vehicle. Description of Aspects and Sub-Aspects.
Driver
Driver. Description of Aspects and Sub-Aspects.
The driver’s perception
The cross-study analysis of the data from the four studies (I–IV) identified six factors that influence how drivers perceive driving automation during usage. The six factors are Preconceptions, Previous Experiences, Perceived Safety, Trust, Vehicle Behaviour, and Information Sources, which have been further split into different aspects. These factors together with their respective aspects describe how a driver perceives driving automation in the moment of use, but also aspects influencing the driver’s perception prior to use, which in turn influences their understanding and therefore usage strategies. The driver develops from this a mental representation of the system, describing when it can be used, what tasks it will take over, and what responsibilities the driver has when engaging with the system. The identified factors have been categorised further into two groups: Perceptual Sets and Sensory Information.
Perceptual sets
Preconceptions, Previous Experiences, Perceived Safety, and Trust belong to the category of top-down processing factors, that is, Perceptual Sets (Allport, 1955), as they comprise contextual information. In the case of a driver interacting with an automated driving system, the driver’s perception will be influenced by preconceived notions, previous interactions with other DAS (not exclusive to a specific level of automation), perceived safety, and trust in the system’s capabilities. Thus, the driver’s perception of the system is influenced by what they expect to perceive.
Preconceptions
Preconceived ideas or assumptions that are present before the use of a DAS are organised under Preconceptions. This factor influences the driver’s understanding significantly, as it typically harbours the driver’s ideas about the system and their anticipations of it, be they current ones or those concerning its unrealised potential.
Perceptual Sets (Top-Down Processing Factors): Preconceptions.
Previous experiences
Previous experiences include the driver’s experienced situations and learning process with the current system, but also those involving previously encountered DAS. This factor also incorporates knowledge obtained through social discourse, for example, topics debated in the news and on social media, marketing campaigns, or discussions in social circles.
Perceptual Sets (Top-Down Processing Factors): Previous Experiences.
Perceived safety
The factor Perceived Safety incorporates a range of aspects describing the driver’s subjective assessment of how safe it is to use the system. Aspects like the system’s performance and its consistent and predictable behaviour influence whether the driver assesses the system to be safe, but also their awareness of the system’s technological and legislative situation influences the driver’s perception, and their willingness to risk usage.
Perceptual Sets (Top-Down Processing Factors): Perceived Safety.
Trust
Under Trust, aspects that describe trust at different levels of abstraction are found. On a basic level, a driver’s appreciation of a brand will influence whether, even before engaging with it, the driver will trust the system. Although, drivers will also choose when to trust the system and to what extent.
Perceptual Sets (Top-Down Processing Factors): Trust.
Sensory information
The factors Vehicle Behaviour and Information Sources are classified as bottom-up processing factors because they pertain to sensory data in terms of environmental stimuli and occur in real-time. The vehicle’s behaviour is perceived in real-time via several sensory channels such as visual, auditory, and haptic feedback. This means that the driver considers the vehicle’s driving behaviour as information from the vehicle to the driver, which will inform their evaluation of the vehicle’s capabilities, for example, in terms of comfort, reliability, and even the vehicle’s ability to communicate.
Vehicle Behaviour
Sensory Information (Bottom-Up Processing Factors): Vehicle Behaviour.
Information sources
The factor Information Sources concerns all input that the driver perceives through their senses. This includes multimodal feedback stemming from visual, auditory, and tactile sources, but also kinaesthetic aspects like the perceived motion of the vehicle.
Sensory Information (Bottom-Up Processing Factors): Information Sources.
The process of how perception shapes understanding
The results of the cross-study analysis suggest interdependencies between the driver’s understanding and their perception of the driving automation in their vehicle. This process is presented as a descriptive model of how perception shapes the understanding of a DAS (Figure 1). In the context of the presented work, the term “perception” refers to the cognitive process by which individuals assess and interpret information they receive from their surroundings; and the term “understanding” refers to the capacity to construct a mental representation or mental model that facilitates the interaction with a DAS. Perception's role in mental model development. Descriptive model of the process of how the driver’s perception shapes their understanding of driving automation systems.
The illustrated process cycle in Figure 1 can be characterised as continuous – a feedback loop that integrates the information received through a top-down and bottom-up process of information as well as a feedback loop updating the driver’s mental representation of the DAS.
The process can be conceptually divided into three distinct parts: (i) the mental representation, (ii) the perception during engagement with a DAS, and (iii) the dynamic process through which perception shapes understanding.
Mental model
This element of the process entails the driver’s understanding of the system’s characteristics and interaction strategies. It consists of all the aspects and components that comprise the driver’s understanding and can be considered the starting point for all interactions. This part of the model represents a static image of the driver’s mental representation of the driving automation. This image remains unchanged unless new information is received about the system.
Perception during use
This block is a mirror of the driver’s mental representation of the system. This part of the model demonstrates how the interaction is affected by the perceptual sets (top-down processing of information) and sensory information (bottom-up processing of information) they receive while driving, operating the vehicle in real-time, and during interaction with a DAS, or when the driver is presented with information about a DAS (e.g., reading about it or talking to someone about a DAS). Here, the driver’s existing knowledge and received information about the DAS is synthesised into a mental representation during use, enabling the driver to assess an interaction or plan and execute an action.
Shaping understanding
This part of the process is a feedback loop that connects the driver’s perception of the system during use with their mental representation of the system. The perception of the system during use is continuously evaluated and the results of that interaction and evaluation have the capacity to change the driver’s mental representation based on what they perceive while using the DAS. This can lead to the reinforcement of existing knowledge about a DAS, or when encountering new information, the evaluation of such information and subsequent revision of the driver’s understanding of the DAS.
Furthermore, the results from the cross-study analysis suggest that the driver’s understanding of a DAS is structured in layers (Context, Vehicle, Driver) that are in continuous interaction (illustrated by the arrows in the model). This means that, for example, the driver’s understanding of when they can use the system, and what the vehicle does, will affect their assessment of what they are supposed to do. Likewise, if the driver believes they are allowed to let go of control over the driving task, this will influence their expectations of what the vehicle will do. Each aspect contributes to the overall understanding of how the system operates and how it affects the driver’s role. Thus, when a driver is asked to explain how the DAS in their vehicle functions, they will recall the knowledge organised in their mental representation. Similarly, in the moment of using the system, the driver will ask themselves: “When can I use the system?”, “What does the vehicle do?”, and “What do I do (now)?”. During use however, their mental representation of the DAS is accessed and facilitates their interaction with the encountered system. As long as the system works as expected by the driver, their mental representation will be confirmed by what is perceived during use. However, should the driver encounter a new scenario, this may prompt them to reassess their assumptions about the system and update their mental representation and all the connected aspects.
Discussion
The aim of the presented research was to investigate the factors that impact the driver’s perception and consequent understanding of driving automation systems. The works contribution lies in the identification of the aspects that shape the driver’s understanding of DAS, as well as the factors that influence the driver’s perception of such systems. Additionally, the work integrates the discovered aspects constituting understanding, and factors influencing perception, into a unified conceptual model that describes the process by which the driver’s understanding of a DAS is shaped by their perception.
The present work represents the first attempt to thoroughly investigate and develop a model elucidating the process shaping the driver’s understanding of DASs through their perception of the DAS. Consequently, by providing a comprehensive and unified overview, this work addresses a gap in the existing literature: the lack of a holistic understanding of how the driver’s perception influences their mental model of DAS. Several key aspects set the proposed model (Figure 1) apart.
In contrast to prior studies undertaken in the field of driving automation, the present model and its associated aspects and factors presented in this work provide a holistic examination of variables pertinent to the driver’s interaction with DAS. Furthermore, the present model is firmly grounded in empirical evidence derived from drivers in real-world driving environments. In order to enhance the model’s robustness, data from four distinct empirical studies (Studies I–IV) was triangulated through a structured analysis approach. Further, the conducted studies utilised multiple methodological approaches (e.g., surveys, in-depth interviews, and field observations) capturing a wide array of driver behaviours and impressions, from first-time drivers and long-term drivers, over prolonged periods. As a result, these studies provided valuable insights into how the driver’s understanding of DAS is shaped through their perception. Through the triangulation of various data points, a comprehensive list of aspects shaping the driver’s understanding and factors influencing their perception was identified, allowing for a nuanced approach to the driver’s interaction with DAS. Finally, the variables that were identified were systematically classified, enabling the author to analyse patterns and relationships. The comprehensive categorisation presented illuminates the intricate interplay between several variables, providing useful insights into the perception and understanding of DAS by drivers – which is independent of the categorisation into levels of automation.
The proposed model is notable for its attention to the driver’s perception and consequent understanding of DASs, as it thoroughly addresses the criticisms put forward by other researchers. These criticisms revolve around the lack of varied and empirical methodologies and data triangulation, as well as the neglect of a wider range of factors and their interrelation (cf. de Winter et al., 2021; Frison et al., 2020; Heikoop et al., 2015; Stanton & Young, 2000), in the attempt to identify a holistic driver-centric perspective.
However, ever since the introduction of automation, it has been widely discussed in the research community that humans are not cut out for supervising automated systems (cf. Bainbridge, 1983; Sheridan, 1995). Different work has shown that drivers become bored, fatigued, and unalert (Körber et al., 2015), and lose track of what the automation is doing, or even what the surrounding circumstances are (de Winter et al., 2014; Louw & Merat, 2017). Further, they may not understand what their tasks and responsibilities are, or even simply forget which driving mode the automation is engaged in (Novakazi et al., 2021; Wilson et al., 2020).
Therefore, the successful implementation of DASs relies on effective cooperation between the driver and the vehicle. This requires designers to view them as a joint cognitive system, wherein both must work in cooperation to guarantee a safe and enjoyable driving experience, and thus they must take a more human-centric approach to the design of DASs. Hence, designers must take into account that the driver’s perception extends beyond the system’s functionalities and their own tasks. And while the HMI can play an important role in communicating crucial information about needed actions from the driver (Banks et al., 2018; Carsten & Martens, 2018), the driver’s understanding of the DAS is not just impacted during its utilisation or solely by the HMI. Rather, their knowledge is shaped by various factors and influenced by variables that extend outside the realm of user interface design. This implies that designers’ understanding of the system, including its limitations, capabilities, and feedback, may not align with the driver’s perception, rendering the use of technology-centred taxonomies problematic as they do not account for a comprehensive view of the complex interaction between the driver and the vehicle. Until now, no other work has attempted to provide such a holistic understanding of driver interaction with DAS, comprehensively listing, describing, and showing the interrelationships between the diverse aspects that influence driver’s perception and its role for mental model development.
The here presented model is based on broad and deep empirical evidence describing the driver’s perception and consequent understanding of DAS, and addresses one of the most important needs of designers and developers of such systems – understanding and applying the driver’s perspective. Designers can apply the specified questions: When can I use the systems?, 2. What does the vehicle do?, and 3. What do/should I do? and ask themselves questions that drivers will ask when interacting with the systems while working on design solutions. In addition, the model defines which factors influence the driver’s perception of the system, its capabilities and limitations, and, consequently, how they understand its utilisation. The stated questions are further supported by the comprehensive set of factors influencing driver’s perception and constituting their understanding, which enable designers to address driver’s requirements in a structured way.
This approach recognises that variables such as the driver’s experience, emotions, and context influence their perception and behaviour. By integrating these variables into the design methodology, a human-centric and effective interface can be developed that more closely corresponds to the requirements and expectations of the driver. Thus, the model can be used as a design aid to include the relevant aspects that drivers identify during their interaction with a DAS, as well as the factors affecting them.
In summary, the model highlights that the driver’s perspective on driving automation differs fundamentally from the technological perspective that currently guides the design and development of these systems. In order to improve the user experience, we must reframe our perspective in order to design products that align with the driver’s mental models and facilitate their development of a sufficient understanding of the DAS.
Perception’s role in understanding interactions with DAS
To provide further clarification of how the driver’s perception influences their understanding of a DAS, a hypothetical scenario will be introduced, exemplifying how the different aspects shaping the driver’s understanding and factors influencing their perception are affected through the use of a DAS and their encounters with different driving situations.
If a person has a system in their personal car that supports the dynamic driving task (DDT) by (i) keeping the vehicle at a set speed, (ii) at a set distance from other moving vehicles, as well as (iii) taking over the steering, as long as there are (iv) visible lane markings, the driver would make sense of the automated system’s use by searching for answers to the aforementioned questions that correspond to the aspects of each layer in their understanding. They would do this in order to make sense of the information received from the vehicle or to understand the interaction required.
In this scenario, a driver is travelling to work from a residential location, passing through countryside and some highway segments into an urbanised area. When they are driving, they will wonder “When can I use the system?”. In this instance, that would be on well-established roads with good lane markings (Driving Context). The driver might not use the system in the suburbs, but they may attempt to use it in the countryside and on the highway. However, the driver could also deduce the wrong specifics, leading them to create a faulty understanding of the DAS and its capabilities and limitations. Considering the same system has certain inherent technical limitations, such as the system performance being unstable when driving on roads with high curvature (Limitations), as are often experienced in the countryside. If the driver further assumes that all that is needed are clear lane markings, they might assume that they can use the DAS on any type of road, while the manufacturer has primarily intended it to be used on highways, where road infrastructure is more controlled and streamlined. This could lead to a situation where the driver experiences the DAS as acting in an unstable way, and may even leave the road, impacting what the driver assumes the benefits of using the system are (Comfort and Safety).
Next, they will ask the question “What does the vehicle do?”. The answer is that it maintains the predetermined speed and distance from other moving vehicles, as well as assisting with the steering (Vehicle Operations). Finally, when utilising the system, they might ask “What do I do (now)?” which, depending on the level of the steering assistance, may imply that the driver does not need to steer or control the speed (Driver Tasks), but must supervise the system (Authority). In such cases, the driver must remain attentive and ready to take over control of the vehicle, if necessary (Attentional Demand), assuming that the vehicle automation has another inherent limitation. While the DAS is able to detect and keep a safe distance to other moving objects, it is not able to do this with objects that are stationary (Limitations). If the driver does not understand this important limitation (which is often only mentioned in a manual), they might wrongly assume that the vehicle will come to a halt when approaching a red light, and that it does not need their intervention. However, since the system is not capable of processing that an object is stationary, it would require the driver in those moments to act by braking and resuming full control over the driving task (Driver Tasks and Authority). Otherwise, this situation could well result in a crash.
This demonstrates that the answers to these questions are reflected through the different aspects and are interrelated with and inform each other. In this way, a comprehensive understanding of the automation while in use is created by the driver. However, the driver’s understanding and consequent use of such systems might not always be the one the designers and developers of such systems intended, which can be explained by introducing the driver’s perception into the interplay. For example, the driver’s preconceptions about a DAS could influence their perception and understanding while using the DAS in a way that might obscure the intended use of the system. In the case of our hypothetical system, the driver does not need to steer but still has to pay attention and supervise the system.
To illustrate, the system performs the driving task to a very high degree of satisfaction according to the driver’s perception (System Performance). This might lead the driver to perceive the system as more capable than it is and assume that they do not need to pay attention (Purpose of the System). Their willingness to relinquish control and even take up other activities would indicate a high Level of Trust. While previous experiences can influence the driver’s expectations as to what the systems are capable of, also other sources like news articles or a chat among friends (Social Discourse) can influence their expectations and consequent use of these systems. These expectations can sometimes be misinformed and lead to negative experiences and unexpected interactions (Learning Experience). For example, the driver might use the DAS in a way that the designers did not intend. These experiences can result in confusion about what the system is doing, which can cause frustration and mistrust, leading the driver to reject the systems (Perceived Safety) – or in the worst case to fatal incidents.
To illustrate further, the hypothetical system is able to keep a set speed and safe distance to other road users. While the technical specification in itself only describes the task the vehicle takes over, the vehicle’s behaviour when the driver is using the systems will influence the driver’s judgement about its performance. For example, if a system keeps a set distance to other road users and the driver perceives it to accelerate and decelerate too fast in relation to the other vehicles, it may be judged as too aggressive in its Driving Style and thus uncomfortable to use. Other communication from the system to the driver falls under Information Sources that are perceived through the in-vehicle user interface(s). This can include visual, auditory, and haptic cues and feedback. For instance, the system might run into limitations and not be able to steer for the driver any longer. It might then send a take-over request (TOR) through a visual and auditory prompt in the in-vehicle interfaces, for example, displays and speakers. If the driver does not act on the prompt, the vehicle might send a warning by intensifying the signal and adding vibration in the seat, in order to get the driver’s attention. All these are ways that the driver can receive information about what the vehicle is doing or what is expected of the driver, and this in turn, will influence their perception of the system’s capabilities and limitations.
Implications for the design of driving automation
Finding an adequate framework that explains the problem area and guides design solutions is a significant challenge for designers, especially in the context of driving automation. In order to enhance system design and the user experience, it is crucial that designers have tools to support their design decisions when developing solutions for DAS. The absence of such support or frameworks can result in solutions that are not understood by users or do not serve their needs. The model presented here seeks to support designers by providing the necessary insights, adding to the existing taxonomies and reshaping the way we communicate the capabilities and limitations of technical systems to drivers.
The process depicted in the model (Figure 1) illustrates that the driver’s perception and understanding of DAS are influenced by various factors. For example, the driver’s preconceptions about the purpose and capabilities of the system can shape their perception and understanding during usage. Additionally, more abstract ideas, such as previous experiences and social discourse, play a significant role in shaping the driver’s expectations and subsequent use of such systems. This has important implications for the design of DAS. Although the levels of automation may be well-defined within the industry, the drivers of vehicles with such capabilities lack an understanding of the associated expectations of them as drivers and their responsibilities (cf. McDonald et al., 2018; Novakazi et al., 2021; Wilson et al., 2020). However, the inherent intricacies underlying the allocation of responsibility for the driving task at all times, and the limitations of vehicles offering such capabilities fall under the expertise of professionals. Another challenge that drivers face is that, while the taxonomies provide general guidance on technical specifications surrounding the vehicle’s capabilities for each level, the implementation of these systems is not standardised, and manufacturers provide different solutions with different feedback and interaction strategies. This has also been shown through an online survey conducted by Kim et al. (2024b), highlighting the discrepancies between driver’s expectations and the actual logic of mode transitions in commercial vehicles. Further, a widely recognised paradox in the realm of automation pertains to the phenomenon wherein the increasing proficiency of a system in being automated leads to a decrease in the motivation for the human operator to sustain their attention, especially with increased vehicle performance (Strauch, 2017).
The challenge stands: An integrated automation, when built into a system, operates without direct user control. This lack of control can lead to users being unaware of the presence of automated functions or the current status of the system. The presence of concealed operations in built-in automation runs the risk of elicit frustration among human operators, may lead to a lack of awareness about the inherent system states (Endsley, 2016), and in certain instances, may even result in hazardous situations. This was exemplified in the case of the Boeing 737 MAX incident, wherein a malfunctioning sensor initiated a sequence of events that led to a state of perplexity and ultimately culminated in a tragic accident (AAIB, 2022).
The airline industry, where most of this research has gathered its findings, has tried to address the aforementioned challenges since the 1970s and introduced additional training, educating pilots about the changes to their role when flying with an automation (Kanki et al., 2010; Sarter & Woods, 1997), as well as educating them about the technical and functional aspects of the automation system (Casner, 2013). Through these efforts, the airline industry has managed to provide pilots with a more complete understanding of the systems and significantly reduced the number of such events (Casner & Hutchins, 2019).
Consequently, the introduction of automated technologies into personal vehicles appears to reintroduce similar challenges. However, one significant obstacle is the fact that the general population possesses a lesser understanding of driving automation, and its many levels compared to proficient operators, such as pilots, and training possibilities for DAS are lacking (Merriman et al., 2021). In addition, the various automation taxonomies and different approaches proposed over the years often serve a specific domain, and sometimes one domain applies different taxonomies (Vagia et al., 2016). For instance, the taxonomies created by Parasuraman et al. (2000) and Endsley and Kaber (1999) were based on the knowledge gathered in the aviation sector but were considered a basis for the introduction of automation into the automotive sector. However, there is a disparity between the levels provided in each of the automation descriptions, showing that a unification of approaches is not an easy task, and different stakeholders regard different aspects as suitable for their respective approaches and domains. This and therefore the usefulness of LoA in their status quo for the design of technical systems has often been criticised (c.f., Kaber, 2017; Wickens, 2018).
Unsurprisingly, the introduction of levels of automation into vehicles presents a new, but similar, set of challenges for designers, developers, and regulators of such systems. Therefore, it is crucial for designers to consider the influence of the driver’s perception on their understanding of the DAS when designing the system’s functionalities and capabilities. Clear communication and education about the system’s purpose and limitations can help align the driver’s understanding with the intended use of the system and possibly bridge the gaps between different manufacturers’ solutions by taking the driver’s perspective. As illustrated by the hypothetical scenario, while there are factors that designers and developers can and must consider during the design of a DAS, there is a range of factors that one cannot directly influence but nonetheless has to account for. These external factors include unpredictable weather conditions, road infrastructure, and the behaviours of other road users, as well as information that is not published by the manufacturers themselves. Despite not being directly controllable, designers and developers must anticipate that these variables will influence the driver’s interaction with the system – often not in the intended ways. Hence, for designers and developers of such systems, it is imperative to comprehend the impact of these factors on the driver’s perception in order to develop DAS that are in line with the users’ mental models and facilitate safe and effective interactions.
In conclusion, the model illustrates the cognitive process by which perception influences the driver’s understanding and subsequent interaction with driving automation, through which the model aims to support a holistic perspective on the driver’s understanding and subsequently on the design of DAS.
Reflections and limitations
The facilitation of empirical research is of utmost importance in order to gain a comprehensive understanding of complex phenomena (Dawadi et al., 2021). Further, recognising the end-users of DAS as valuable sources of information, innovation, and adaptability while striving to create and enhance DASs is a choice that is critical to the success of the designed solution (Beggiato et al., 2015; Buckley et al., 2018), and thus should be seen as the focus of attention. Numerous research endeavours pertaining to the assessment of DASs primarily rely on simulator studies. While these studies provide valuable insights into the utilisation of such systems (Jamson & Jamson, 2010), their limited realism undermines the ecological validation of the results. Consequently, these findings fail to provide genuine insights into the user’s strategies and perception of the driving automation, as the presence of a safety net in simulated environments obscures the true experiences and challenges encountered during real-world driving scenarios (Blaauw, 1982; Godley et al., 2002; Fisher et al., 2011).
Thus, the decision to analyse the here presented four studies, relying on empirical data that the author has collected themselves, enabled an examination of the driver’s utilisation, perception, and understanding of driving automation technologies, hence providing valuable insights. The integration of quantitative and qualitative data yielded a new perspective of the subject matter compared to relying just on one method, as the different approaches typically possess distinct characteristics and explore different facets of the same issue. Subsequently, this prompted a more comprehensive examination employing a hybrid approach of quantitative and qualitative methodologies by utilising observational techniques paired with in-depth interviews and triangulating the qualitative data with quantitative data points. This was done to acquire a broader and deeper understanding of the driver’s perception and consequent understanding of DAS.
Although the data were coded and analysed by a single researcher, the framework approach employed offers a rigorous and transparent process that mitigates potential biases through its detailed documentation and structured approach. This framework, while relying on the expertise of the individual conducting the analysis, provides a clear audit trail and enhances the replicability of the research. Furthermore, the interpretations were reviewed with colleagues during discussions and workshops through several iterations to enhance the validity of the findings. Critically, the consistency of identified patterns across multiple studies and analyses within this research project, including the triangulation of multiple data sources, further strengthens the validity of the interpretations, as similar themes and trends emerged independently.
While the four empirical studies that were analysed and presented here offered significant empirical insights, they also highlight the need for further exploration of the process of how perceptions shape understanding. Future research incorporating additional data collection and analysis could identify and integrate further aspects, leading to a more refined and robust model. It is likely that additional, unobserved factors play a role in shaping drivers understanding and that aspects that constitute their understanding have not been observed. Future research could broaden the scope to identify and incorporate any additional aspects of understanding and factors influencing the driver’s perception into the model.
Conclusions
This research investigated the influence of driver perception on mental model formation for DASs, addressing critical gaps in the literature. Specifically, this work makes four key contributions: (i) it identified a comprehensive set of factors influencing both driver perception of DASs and the resulting mental models; (ii) it developed a unified conceptual model explaining how perception shapes understanding; (iii) it directly addresses criticisms of the LoA framework by incorporating the driver’s perspective, using empirical data from end-users; and (iv) it grounds its findings in rigorous empirical research and data triangulation, moving beyond the limitations of literature reviews and simulations.
The resulting model provides a holistic understanding of driver interaction with DASs, highlighting the need for designers to adopt driver-centric design perceptive. This perspective ensures that the system conforms to the driver’s mental model, rather than the development of design solutions that are influenced by technical classifications. Such considerations are crucial for the safe and effective integration of DAS into vehicles, as it supports driver’s development of an appropriate mental model and adequate interaction with such systems that does not deviate from intended use.
Future research should prioritise the translation of this theoretical framework into practical design guidelines for practitioners in the industry. By embracing a human-centric methodology, emerging design solutions of DAS can more accurately mirror the mental models of drivers, hence enhancing safety and inherently intuitive interactions with these complex systems.
Footnotes
Authors’ Note
The research was conducted at Chalmers University of Technology; however, the manuscript was completed during the author’s tenure at Örebro University.
Acknowledgements
I want to extend sincere gratitude to all the participants who generously shared their valuable insights, making the research sessions both enjoyable and enlightening. I want to express gratitude to Sweden’s Innovation Agency VINNOVA (grant number. 2017–01946) for providing funding for Studies I–III and to Volvo Cars for facilitating Study IV. Special thanks are also due to my colleagues at Volvo Cars for their various support, whether through discussions, prototyping, or technical assistance. I am especially grateful to all collaborators and discussion partners who contributed to the presented research studies, with particular appreciation for MariAnne Karlsson from Chalmers University of Technology, whose guidance advanced my research. The views and opinions presented are solely my own and may not necessarily represent the views of any collaborating institutions.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the VINNOVA (2017–01946).
