Assessing the physiological effect of non-driving-related task performance in conditionally automated driving systems: A systematic review and meta-analysis protocol

Abstract

Background

Level 3 automated driving systems involve the continuous performance of the driving task by artificial intelligence within set environmental conditions, such as a straight highway. The driver's role in Level 3 is to resume responsibility of the driving task in response to any departure from these conditions. As automation increases, a driver's attention may divert towards non-driving-related tasks (NDRTs), making transitions of control between the system and user more challenging. Safety features such as physiological monitoring thus become important with increasing vehicle automation. However, to date there has been no attempt to synthesise the evidence for the effect of NDRT engagement on drivers’ physiological responses in Level 3 automation.

Methods

A comprehensive search of the electronic databases MEDLINE, EMBASE, Web of Science, PsycINFO, and IEEE Explore will be conducted. Empirical studies assessing the effect of NDRT engagement on at least one physiological parameter during Level 3 automation, in comparison with a control group or baseline condition will be included. Screening will take place in two stages, and the process will be outlined within a PRISMA flow diagram. Relevant physiological data will be extracted from studies and analysed using a series of meta-analyses by outcome. A risk of bias assessment will also be completed on the sample.

Conclusion

This review will be the first to appraise the evidence for the physiological effect of NDRT engagement during Level 3 automation, and will have implications for future empirical research and the development of driver state monitoring systems.

Keywords

Automated driving physiology psychophysiology driver behaviour advanced driver assistance systems task performance

Introduction

Automated driving systems (ADS) represent an innovative means of leveraging artificial intelligence for the assistance or performance of driving-related tasks. In the context of ADS, the extent of a system's automation can be categorised across a number of levels, ranging from Level 0 (no automation) to Level 5 (full automation).¹ Level 3, or conditional automation, is the highest level currently available at the consumer level and can be defined as the continuous performance of all driving-related tasks within a specified operational design domain (ODD). In Level 3 automation, when the system encounters a scenario outside of its ODD (e.g. missing lane markings), a takeover request is issued to the user, who must then resume control of the vehicle. As such, while the user can delegate all driving-related tasks to the system in Level 3, they must remain ready to resume control.

While the handover of responsibility of driving-related tasks to artificial intelligence affords greater opportunities to promote driver safety and wellbeing, in Level 3 automation, interactions related to transitions of control can be problematic. This is because as the level of vehicle automation increases, the driver is increasingly removed from the system control loop, making the transition back to manual driving, when needed, very difficult.^2,3 During prolonged periods of automated driving, drivers may experience what is referred to as ‘passive fatigue’ by May and Baldwin⁴ due to the monotonous nature of simply monitoring the road without driving. As fatigue increases, drivers may even experience a state of drowsiness.⁵ They may also become distracted from monitoring the road as their attention shifts towards non-driving-related tasks (NDRTs).

An NRDT can be defined as any auxiliary task which does not directly or indirectly assist in the performance of the driving task. Drawing on findings from real-world observational research, users of conditionally ADS display a greater propensity to engage in NDRTs.⁶ Hecht et al.⁷ found that drivers free to perform NDRTs during a 1-hour period of Level 3 automated driving most commonly engaged in mobile phone use (75% of their sample), reading (60%) or browsing a tablet (50%). NDRTs are commonly used to induce psychological and cognitive states in the driving environment (e.g. stress, cognitive load) in a laboratory setting. The freedom to engage in NDRTs is seen as a major opportunity afforded by automated driving.⁸

A number of recent reviews have found evidence that engaging in an NDRT results in worse takeover performance, particularly when there are overlapping resource demands between the NDRT and driving task – that is, when the NDRT has a visual or a manual component.^9–12 According to the out-of-the-loop performance problem,¹³ as the level of vehicle automation increases, drivers display a diminished capability to respond to sudden or urgent takeover requests from the system. Being ‘out of the loop’ is characterised by a situation where the user is not actively involved in the control of a system and is not monitoring the environment. This leads to decrements in the user's situation awareness. A classic definition of situation awareness is “knowing what's going on so you can figure out what to do”.¹⁴ Being out of the loop also results in a delay in the user's reaction time to critical events, particularly as they take up NDRTs during automated driving.¹⁵ De Winter et al.¹⁶ found in their review that NDRT engagement during automated driving can result in poorer situation awareness than during manual driving. This understandably could have serious implications for user safety during conditionally automated driving, when the driver is assumed to be ready to take over as needed.

Malleable attentional resources theory¹⁷ suggests NDRTs may actually mitigate increasing fatigue due to automation. This theory proposes that attentional resources shrink to meet situational demands, and so performing an NDRT could combat such a shrinkage and prevent the onset of fatigue. In their review, De Winter et al.¹⁸ found that a monotonous automated drive led to slower reaction times compared with manual driving when drivers are experiencing a state of drowsiness. Miller et al.¹⁹ also suggested that NDRTs could be used to reduce drowsiness. It is therefore possible that NDRT performance during automated driving could lead to conservation of attentional resources (and consequently, one's ability to respond) in certain scenarios.

In addition to simply engaging in an NDRT during automated driving, the modality of the NDRT seems to have an effect on driver performance. Wandtner et al.²⁰ conducted a driving simulator study with several critical takeover scenarios and found that when drivers were performing an NDRT that was both visual and manual, performance degraded the most. As driving is also both a visual and manual task, the authors concluded that NDRTs with resource demands that overlap with the driving task can result in interference with driving performance. In their review, Zhang et al.¹⁰ found that longer takeover times were associated with performing a visual NDRT – a similar finding was observed in reviews by Merlhiot et al.¹¹ and McDonald et al.¹² with respect to handheld tasks. Wickens’ multiple resource theory²¹ accounts for how performance interference is greater among two tasks with competing resources, compared with two tasks that differ in some way. In the context of automated driving, the idea that some NDRTs interfere with driving more than others could have implications for safety.

Meinlschmidt et al.²² conducted a systematic review to examine the effect of ADS engagement on drivers’ physiological responses. The authors found that increased electrodermal and masseter electromyography activity and decreased heart rate were observed when ADS was engaged. However, these findings were based on only four studies (N = 194 participants). As recent technological advancements have made it possible to study driver performance in immersive, high-fidelity driving simulators with greater ease, there is a pool of recent research which has yet to be synthesised.^22–25 Furthermore, it will soon become a requirement for all ADS to be equipped with a driver monitoring system that can track the psychological or cognitive state of the driver and make an assessment about the driver's fitness to drive.²⁶ This will rely heavily on the collection of physiological data; hence these measures are of particular interest.

It is clear from previous systematic reviews in this area^9–12 that NDRT engagement can have consequences for takeover performance, and there is some evidence to suggest this effect may vary depending on specific NDRT attributes. However, this is the first systematic review that will directly attempt to synthesise the evidence concerning the effect of NDRTs on drivers’ physiological measures. This work is of particular importance to the future of conditionally automated driving, in which assessments will be made about a driver's readiness to drive through physiological data collection. This review will also expand on other reviews in this area that examined the effect of mental states during automated driving^11,27 by looking at physiological measures, which can be used as an index of underlying mental states.

Objectives

The central objective of the present systematic review and meta-analysis is to examine the effect of NDRT engagement on drivers’ physiological responses in Level 3 ADS. The secondary objective of this review, guided by Wickens’ multiple resource theory, is to examine the effect of visual NDRTs versus nonvisual NDRTs, and manual NDRTs versus nonmanual NDRTs, on drivers’ physiological responses.

Methods

The present systematic review and meta-analysis was submitted for registration to the International Prospective Register of Systematic Reviews (PROSPERO) on July 1^st, 2022 and was registered on July 12^th, 2022. The methods chosen for this review were guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols (PRISMA-P).²⁸ Any amendments to the protocol for this review will be documented in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)²⁹ statement.

Information sources and search strategy

The electronic databases MEDLINE (Ovid), EMBASE (Elsevier), Web of Science (Core Collection), PsycINFO (Ovid), Compendex (Elsevier) will be searched for records for the purpose of this review. These databases were selected in consultation with a librarian information specialist. To minimise publication bias, the electronic databases Compendex and PsycExtra will be used to search for grey literature. The databases will be queried using the search strategy developed for this review (see Appendix A), and the following filters will be applied: English language only, and records existing from January 1^st, 2012 to the date of search (July 29^th, 2022). Forward and backward citation searches will be conducted with the sample of eligible studies obtained from the database and grey literature search. Where a full-text article is not accessible, study authors will be contacted to request the full-text. A cut-off period of 2 weeks will be provided, whereby the full-text will be considered inaccessible if there is no author response 2 weeks after it is requested. A follow-up database search will be conducted on November 1^st, 2022, using the identical search strategy and filters applied during the initial search, to ensure that all relevant and up-to-date records and included in this review. Given that this is a particularly novel and fast-progressing area of research, it was decided that a follow-up search was warranted.

The search strategy for this review was developed by identifying search terms used in Meinlschmidt et al.,¹⁶ upon which this review is expanding by specifically examining NDRT engagement and synthesising a larger pool of more recent data. A librarian information specialist was also consulted with to develop the search strategy. Three concept blocks were constructed: search terms related to automobile driving, terms related to automation, and terms related to physiological activity. Where possible, search terms were mapped on to relevant subject headings and supplemented with free terms. Subject headings were chosen by conducting a scoping search to identify headings used by studies that would be eligible for inclusion in this review. The search terms within each concept block were combined using the Boolean operator OR, and the three concept blocks were then combined using the AND operator. Appendix A contains a list of all search terms and syntax that will be used in each database for this review.

Eligibility criteria

The eligibility criteria for this review were developed in accordance with the patient, intervention, comparison, outcome, studies (PICOS) model.³⁰

Population

Participants from eligible studies will include adults aged 18 years or older, holding a full driving license at the time of the study. Studies that recruit participants that either do not hold a full driving license at the time of participation, or are below 18 years of age, will be excluded from the review.

Interventions

Studies which use either an on-road vehicle, fixed-base driving simulator, or fixed-base off-road vehicle with an audio/visual display, capable of performing or simulating Level 3 (conditional automation) will be included. Studies which use a vehicle or simulator that can perform another level of automation (Level 1, 2, 4 or 5), and do not include a vehicle or simulator capable of performing Level 3 automation will be excluded. Studies that use manual driving only will be excluded. Studies must also use at least one type of NDRT in addition to the primary driving task. The categorisation of NDRTs for this review will be based largely on a recent overview by Naujoks et al.,³¹ but broadly speaking, they can be categorised as being either visual or nonvisual in terms of their modality of presentation, and manual or nonmanual in terms of task input. Studies which use Level 3 automation but do not include an NDRT will be excluded. Studies which include only additional tasks that relate to the primary driving task (e.g. pressing a button to toggle vehicle automation on/off) will be excluded.

Comparators

Studies which include either a within-subjects baseline condition, or a between-subjects comparison group with no NDRT involvement will be included. This is a necessary criterion for inclusion, as the presence of such a condition provides a reference value against which the effect of the NDRT can be compared. Studies which do not feature either a within-subjects or between-subjects comparator will be excluded. Studies which compare the differential effects of various NDRTs, but do not feature either a within-subjects or between-subjects comparator, will be excluded.

Outcomes

Eligible studies will measure at least one peripheral physiological parameter that is directly measurable by psychophysiological monitoring equipment. Appendix B contains a list of potential physiological measures that we expect to see in included studies. Studies which measure biochemical parameters only (e.g. cortisol, interleukin 6, C-reactive protein) will be excluded. Studies which use self-report measures only will be excluded. Eligible studies must measure at least one peripheral physiological parameter both during NDRT engagement and during either a within-subjects baseline or between-subjects comparison condition.

Types of study

Empirical studies that use either a vehicle or driving simulator capable of performing Level 3 (conditional) automation and employ at least one type of NDRT will be included. Eligible studies will compare the effect of NDRT involvement on physiological responses with either a within-subjects baseline condition, or a between-subjects comparison group. Grey literature (conference proceedings, unpublished dissertations) will be included. Articles which use qualitative methods only (such as interviews or focus groups) will be excluded. Editorials, theoretical papers, and review papers will be excluded. Studies not published in English will be excluded. Studies published prior to January 1^st, 2012 will be excluded.

Article screening

The results of the electronic database search will be merged and deduplicated in EndNote 20 citation management software. Following deduplication, the results will be exported to Rayyan for screening. Article screening will take place in two stages. In stage 1, one independent reviewer (RC) will assess the titles and abstracts of the total deduplicated results, and a second reviewer (LR) will assess a random sample (as recommended by Shoukri³²) of 20% to check for inter-observer agreement. In stage 2, RC will screen the full-text articles of all records deemed to have met the eligibility criteria at stage 1. LR will again screen a random sample of 20% of the full-text articles for this sample. If there is disagreement between RC and LR at either stage regarding whether a record meets the eligibility criteria, a third reviewer (MM) will be consulted with, and their decision will determine that study's eligibility. The results of the screening process will be presented in a PRISMA flow diagram, along with a rationale for exclusion of articles at each stage. Cohen's kappa statistic will be calculated at stage 1 and stage 2 to assess the inter-observer agreement among authors.

Data extraction

One reviewer (RC) will extract relevant data from the final sample of eligible studies, using a pre-determined electronic data extraction form that will be managed using Microsoft Word (see Appendix C). The data that will be extracted will comprise: (i) bibliographic information (study title, authors, year of publication, country of origin), (ii) study characteristics (study design, theoretical background/conceptual models mentioned), (iii) participant characteristics (age, gender, level of experience with ADS), (iv) information about the experiment (the type of vehicle or simulator used, vehicle speed, the content, modality and duration of NDRT employed, and environmental factors such as traffic density or weather condition), (v) outcome measures (type of physiological outcomes, method of measurement), (vi) main findings (results with respect to the effect of NDRT engagement on physiological outcome measures) and (vi) limitations acknowledged by the author(s).

Critical appraisal

For the purpose of critical appraisal of the final sample of included studies in the review, the risk of bias in nonrandomised studies of interventions (ROBINS-I) tool³³ will be used. The ROBINS-I tool assesses risk of bias across seven domains, which are broadly categorised into pre-intervention domains (bias due to confounding, bias in selection of participants), at intervention (bias in classification of interventions) and post-intervention domains (bias due to deviations from intended interventions, bias due to missing data, bias in outcome measurement, bias in selection of reporting of results). A six-step process is taken for study assessment. The first steps involve specifying the research question, the outcome and result being addressed. The assessor must then examine whether and how confounders were addressed with respect to the results. Subsequently, they must address a series of signalling questions for each domain provided by Sterne et al.³³ Finally, the assessor will formulate a risk of bias judgement for each domain and make an overall judgement of the risk of bias (low risk, moderate risk, serious risk, critical risk, or no information).

Data synthesis and analysis

To address the central objective of this review, the findings with respect to the effect of NDRT engagement on drivers’ physiological responses in Level 3 ADS will be synthesised in a series of meta-analyses. The data will be meta-analysed by outcome measure in R using the meta package.³⁴ The meta-analyses will assess the effect of NDRT engagement on physiological parameters, in comparison to either a within-subjects baseline condition, or a between-subjects control condition.

Risk ratio (RR), mean difference (MD) or standardised mean difference (SMD) between conditions will be calculated for each measure across all included studies in R using extracted data from included studies and expressed at the 95% confidence interval. Where critical statistical information is not available in-text, but results are presented in a graph format, the online package WebPlotDigitizer will be used to extract statistical information from the graph. This online package has previously demonstrated high levels of intercoder reliability and validity.³⁵

The level of heterogeneity between-studies will be assessed using the Higgins I² statistic. In the presence of moderate heterogeneity (Higgins I² > 50%), random-effects model will be used, otherwise a fixed-effects model will be used. In the case of high heterogeneity (Higgins I² > 75%) a narrative synthesis of the data will be conducted. The results of a series of meta-analyses will be presented using forest maps stratified by the type of physiological measure assessed in included studies. For the purpose of exploratory analysis, a series of meta-analyses will be conducted to examine the effect of NDRT modality (visual NDRTs vs nonvisual NDRTs, manual NDRTs vs nonmanual NDRTs) on drivers’ physiological responses.

Results

The results of the database search will be presented in a PRISMA flow diagram, which will include the number of articles excluded at each stage of the screening process, and the reasons for exclusion. The results of the data extraction from the final sample of included studies will first be presented in a table of results, and key study characteristics will be reported in-text. This descriptive information will include information on the population, study design and outcome measures of the final sample of included studies. Following this, the results of the meta-analysis will be presented, followed by the results of the risk of bias assessment. It is anticipated that NDRT performance will lead to greater physiological activation, compared with when no NDRT is performed. Additionally, in line with Wicken's multiple resource theory, we expect that NDRTs with resource demands which overlap with the driving task (i.e. manual and visual tasks) will lead to greater physiological activation, compared with NDRTs with resource demands that do not overlap with the driving task.

Conclusion

In summary, the present systematic review and meta-analysis will be the first to critically assess the effect of NDRT engagement on drivers’ physiological responses in Level 3 ADS. While previous reviews have shown that engaging in NDRTs can have a negative effect on driver performance metrics, far less is known concerning how secondary tasks impact the physiological state of the driver. This information is central to the development of driver monitoring systems which can continuously track the psychological or cognitive state of the user, and issue pre-emptive warnings when a target state reaches critical levels. The findings of the review are likely to provide several insights for researchers in this area. For one, it will elucidate the extent to which task performance affects driver arousal levels during conditionally automated driving. Such information will be helpful in efforts to measure complex constructs such as stress, fatigue and cognitive load in the in-cabin setting.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076231174782 - Supplemental material for Assessing the physiological effect of non-driving-related task performance in conditionally automated driving systems: A systematic review and meta-analysis protocol

Supplemental material, sj-docx-1-dhj-10.1177_20552076231174782 for Assessing the physiological effect of non-driving-related task performance in conditionally automated driving systems: A systematic review and meta-analysis protocol by Rory Coyne, Leona Ryan, Mohamed Moustafa, Alan F Smeaton, Peter Corcoran and Jane C Walsh in DIGITAL HEALTH

Footnotes

Acknowledgments

The authors thank Rosie Dunne, Research Support Librarian at the James Hardiman Library, University of Galway, Ireland, for her support in refining our search strategy.

Contributorship

RC selected the methodological and theoretical underpinning of this research, devised the search strategy and data extraction form, registered the review with critical revisions by LR, JW and MM, and prepared the manuscript. RC will complete the searches, and conduct the screening, data extraction and critical appraisal with LR. JW, AS and PC critically revised the manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

As the present research is a review of primary research, ethical approval is not required.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was conducted with the financial support of the Science Foundation Ireland Centre for Research Training in Digitally-Enhanced Reality (d-real) under Grant No. 18/CRT/6224. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.

Guarantor

RC.

Informed consent

As the present research is a review of primary research, informed consent is not required.

Systematic review registration: PROSPERO 2022: CRD42022330827.

ORCID iDs

Rory Coyne

Alan F Smeaton

Supplemental material

Supplemental material for this article is available online.

References

Society for Automated Engineers. Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles, https://webstore.ansi.org/Standards/SAE/SAE30162021?source=blog&_ga=2.4954803.1695274797.1651152407-929243452.1649261025 (2002, accessed 22 August 2022).

Neubauer

Matthews

Langheim

, et al. Fatigue and voluntary utilization automation in simulated driving. Hum Factors 2012; 54: 734–746.

Vogelpohl

Kühn

Hummel

, et al. Asleep at the automated wheel—sleepiness and fatigue during highly automated driving. Accid Anal Prev 2019; 126: 70–84.

May

Baldwin

. Driver fatigue: the importance of identifying causal factors of fatigue when considering detection and countermeasure technologies. Transp Res F: Traffic Psychol Behav 2009; 12: 218–224.

Radlmayr

Feldhütter

Frey

, et al. Drowsiness and fatigue in conditionally automated driving – towards an integrative framework. In: Proceedings of the Human Factors and Ergonomics Society Europe. 2019.

Omozik

Yang

Kuntermann

, et al. How long does it take to relax? Observation of driver behavior during real-world conditionally automated driving. In Driving Assessment Conference 2019, June 26, 2019 (Vol. 10, No. 2019). University of Iowa.

Hecht

Feldhütter

Draeger

, et al. What do you do? An analysis of non-driving related activities during a 60 minute conditionally automated highway drive. In International Conference on Human Interaction and Emerging Technologies, August 22, 2019. Springer, Cham, pp. 28–34.

König

Neumayr

. Users’ resistance towards radical innovations: the case of the self-driving car. Transp Res F: Traffic Psychol Behav 2017; 44: 42–52.

Weaver

DeLucia

. A systematic review and meta-analysis of takeover performance during conditionally automated driving. Hum Factors 2020; 64: 1227–1260. DOI: 10.1177/0018720820976476.

10.

Zhang

De Winter

Varotto

, et al. Determinants of take-over time from automated driving: a meta-analysis of 129 studies. Transp Res F: Traffic Psychol Behav 2019; 64: 285–307.

11.

Merlhiot

Bueno

. How drowsiness and distraction can interfere with take-over performance: a systematic and meta-analysis review. Accid Anal Prev 2022; 170: 106536.

12.

McDonald

Alambeigi

Engström

, et al. Toward computational simulations of behavior during automated driving takeovers: a review of the empirical and modeling literatures. Hum Factors 2019; 61: 642–688.

13.

Endsley

Kiris

. The out-of-the-loop performance problem and level of control in automation. Hum Factors 1995; 37: 381–394.

14.

Adam

. Fighter cockpits of the future. In [1993 Proceedings] AIAA/IEEE Digital Avionics Systems Conference, October 25, 2005. IEEE, pp. 318–23.

15.

Louw

Madigan

Carsten

, et al. Were they in the loop during automated driving? Links between visual attention and crash potential. Inj Prev 2017; 23: 281–286.

16.

De Winter

Happee

Martens

, et al. Effects of adaptive cruise control and highly automated driving on workload and situation awareness: a review of the empirical evidence. Transp Res F: Traffic Psychol Behav 2014; 27: 196–217.

17.

Young

Stanton

. Attention and automation: new perspectives on mental underload and performance. Theor Issues Ergon Sci 2002; 3: 178–194.

18.

De Winter

Happee

Martens

19.

Miller

Sun

Johns

, et al. Distraction becomes engagement in automated driving. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2015 September (Vol. 59, No. 1, pp. 1676–80). Los Angeles, CA: Sage Publications.

20.

Wandtner

Schömig

Schmidt

. Effects of non-driving related task modalities on takeover performance in highly automated driving. Hum Factors 2018; 60: 870–881.

21.

Wickens

. Multiple resources and mental workload. Hum Factors 2008; 50: 449–455.

22.

Meinlschmidt

Stalujanis

Tegethoff

. The psychobiology of using automated driving systems: a systematic review and integrative model. Psychoneuroendocrinology 2019; 105: 51–63.

23.

Wandtner

Schömig

Schmidt

. Secondary task engagement and disengagement in the context of highly automated driving. Transp Res F: Traffic Psychol Behav 2018; 58: 253–263.

24.

Meteier

De Salis

Capallera

, et al. Relevant physiological indicators for assessing workload in conditionally automated driving, through three-class classification and regression. Front Comput Sci 2022; 3: 139.

25.

Müller

Fernandes-Estrela

Hetfleisch

, et al. Effects of non-driving related tasks on mental workload and take-over times during conditional automated driving. Eur Transp Res Rev 2021; 13: 1–5.

26.

Regulation (EU) 2019/2144 of the European Parliament and of the Council of 27 November 2019 on type-approval requirements for motor vehicles and their trailers, and systems, components and separate technical units intended for such vehicles, as regards their general safety and the protection of vehicle occupants and vulnerable road users, amending Regulation (EU) 2018/858 of the European Parliament and of the Council and repealing Regulations (EC) No 78/2009, (EC) No 79/2009 and (EC) No 661/2009 of the European Parliament and of the Council and Commission Regulations (EC) No 631/2009, (EU) No 406/2010, (EU) No 672/2010, (EU) No 1003/2010, (EU) No 1005/2010, (EU) No 1008/2010, (EU) No 1009/2010, (EU) No 19/2011, (EU) No 109/2011, (EU) No 458/2011, (EU) No 65/2012, (EU) No 130/2012, (EU) No 347/2012, (EU) No 351/2012, (EU) No 1230/2012 and (EU) 2015/166.

27.

Huang

Petersen

Pitts

. The impact of mental states on semi-autonomous driving takeover performance: a systematic review. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, December 2020 (Vol. 64, No. 1, pp. 1372–6). Los Angeles, CA: SAGE Publications.

28.

Moher

Shamseer

Clarke

, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015; 4: 1–9.

29.

Moher

Liberati

Tetzlaff

, et al. PRISMA Group*. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009; 151: 264–269.

30.

Eriksen

Frandsen

. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. J Med Libr Assoc 2018; 106: 420.

31.

Naujoks

Befelein

Wiedemann

, et al. A review of non-driving-related tasks used in studies on automated driving. In Advances in Human Aspects of Transportation: Proceedings of the AHFE 2017 International Conference on Human Factors in Transportation, July 17–21, 2017, The Westin Bonaventure Hotel, Los Angeles, CA, USA 8 2018 (pp. 525–37). Springer International Publishing.

32.

Shoukri

. Measures of interobserver agreement and reliability. Boca Raton, FL: CRC Press, 2010.

33.

Sterne

Hernán

Reeves

, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J 2016; 355: i4919.

34.

Schwarzer

. Meta: an R package for meta-analysis. R News 2007; 7: 40–45.

35.

Drevon

Fursa

Malcolm

. Intercoder reliability and validity of WebPlotDigitizer in extracting graphed data. Behav Modif 2017; 41: 323–339.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB