Conceptual development of an advanced air traffic controller workstation based on objective workload monitoring and augmented reality

Abstract

NextGen, SESAR, and other international projects intend to develop the coming air traffic management (ATM) environment to cope with the present problems, including e.g. capacity, efficiency, safety, and environmental impact. These projects also introduce considerable changes in the role of air traffic controllers (ATCOs), and shift some of the present tasks from ground to on-board (pilots) and diversify automation. Due to the consequential effects of these developments, (i) the models describing the present operational circumstances, (ii) the working conditions, and (iii) the working environment (e.g. human–machine interactions, task-work and mental loads, cognitive aspects, situation awareness, decision support) must be reconsidered and adapted to the future role of operators. This paper aims to describe a new approach to model the ATCOs’ working processes, and also introduces a new concept for the ATCOs’ future working environment. The developed model covers the situation awareness and decision making processes, including the subjective aspects of the ATCOs decision making based on their workload, mental condition, knowledge, or practice. The proposed new working environment contains special information displays, the decision support system and the ATCOs workload and mental condition measurement / monitoring system, based on objective, on-line, continuous, and nonintrusive measurement techniques. Some of the core systems developed in collaboration with HungaroControl were exhibited at the World ATM congress in 2014 and 2015.

Keywords

Air traffic management air traffic controller workload mental condition workstation decision support

Get full access to this article

View all access options for this article.

References

EUROCONTROL. Challenges of air transport 2030. Survey of experts’ view. EUROCONTROL - European Organisation for the Safety of Air Navigation. Bretigny-sur-Orge, 2009, p.64.

National Aeronautics and Space Administration. NASA aeronautics blueprint: Towards a bold new era in aviation, Washington, D.C.: NASA, 2002, pp. 39–39.

ACARE. European Aeronautics: Vision 2020. Report of the group of personalities. ACARE - Advisory Council for Aeronautics Research in Europe. Brussels, 2001, p.26.

Flightpath 2050, Europe’s Vision for Aviation. Report of the High Level Group on Aviation Research, Directorate-General for Research and Innovation, Directorate General for Mobility and Transport, European Commission, 2011, p.28.

ACARE. Strategic Research and Innovation Agenda (SRIA) vol.1. ACARE - Advisory Council for Aeronautics Research in Europe, 2011, p.152.

Holmes B. Transformation in Transportation Systems of the 21st century. Invited lecture, ICAS (International Council of Aeronautical Sciences) Congress, Yokohama, 2004, pp.1–16., http://icas.org/ICAS_ARCHIVE/ICAS2004/PAPERS/600.PDF (accessed 10 March 2015).

SESAR - Single European Sky ATM Research. SESAR Joint undertaking. 2000, http://www.sesarju.eu/.

European air traffic management master plan. SESAR, European Commission, 2009, http://ec.europa.eu/transport/modes/air/single_european_sky/doc/european_atm_master_plan.pdf (accessed 10 March 2015).

FAA: Fact Sheets - NextGen. FAA - Federal Aviation Administration, 2007, http://www.faa.gov/news/fact_sheets/news_story.cfm?newsid=8145.

10.

Joint Planning and Development Office (JPDO). Concept of operations for the Next Generation Air Transportation System Version 2.0, Washington D.C.: JPDO, 2007, pp. 219–219.

11.

CARATS - Collaborative Actions for Renovation of Air Traffic System. Long-term vision for the future air traffic system. Changes to intelligent air traffic systems. Study Group for the Future Air Traffic Systems. 2010. p.56., http://www.mlit.go.jp/common/000128185.pdf (accessed 10 March 2015).

12.

SIRIUS Brasil - Impulsionando o Desenvolvimento do ATM Nacional A aviocao do future jo comecou. 2013. http://www.decea.gov.br/sirius/ (accessed 10 March 2015).

13.

Dirickson YMR, Vismari LF and Camargo JB Jr. A methodological approach to safety analysis of automated tasks of TWR ATCO. Escola Politécnica, Universidade de São Paulo, 2014, p.23. http://sitraer2014.pcs.usp.br/wp-content/uploads/Apresentacoes/18-11-2014/Session3/01-Yuri-Maximilain-Rottner-Dirickson.pdf (accessed 10 March 2015).

14.

ICAO - International Civil Aviation Organization. 2013-2028 Global Air Navigation Capacity & Efficiency Plan. 2014–2016 Triennium Edition, Montreal, 2014. p.137. http://www.icao.int/Meetings/anconf12/Documents/DraftDoc 9750.GANP.en.pdf (accessed 10 March 2015).

15.

ICAO. Working document for the Aviation System Block Upgrades. The framework for global harmonization. ICAO (International Civil Aviation Organisation), 2014, p.392. http://www.icao.int/Meetings/anconf12/Pages/Aviation-System-Block-Upgrades.aspx (accessed 10 March 2015).

16.

Metzger

Parasuraman

. The role of the air traffic controller in future air traffic management: An empirical study of active control versus passive monitoring. Human Factor 2001; 43: 519–528.

17.

Langan-Fox J,Canty J and Sankey M. Human factors issues in air traffic control under free flight. In: Proceedings of the 45th annual human factors and ergonomics society of Australia conference 2009, Melbourne, Australia, Human Factors and Ergonomics Society of Australia Inc. (HFESA), 2009, pp.37–44.

18.

Merwe

van de Oprins

EAPB

Erikson

DJF

. The influence of automation support on performance, workload and situation awareness of Air Traffic Controllers. Int J Aviat Psychol 2012; 22: 120–143.

19.

Endsley

. Toward a theory of situation awareness in dynamic systems. Human Factors 1995; 37: 32–64.

20.

Rasmussen

. Information processing and human-machine interaction: An approach to cognitive engineering, New York: Elsevier Sciences Inc., 1986, pp. 230–230.

21.

Kallus KW, Barbarino M and van Damme D. Model of the cognitive aspects of air traffic control. EUROCONTROL, 1997, p.62. https://www.eurocontrol.int/sites/default/files/content/documents/nm/safety/safety-model-of-the-cognitive-aspects-of-air-traffic-control-1997.pdf (accessed 10 March 2015).

22.

Hilburn

Jorna

PGAM

Workload and air traffic control. In: Hancock

Desmond

(eds). Stress workload and fatigue: Theory, research and practice, Hillsdale, NJ: Erlbaum, 2001, pp. 384–394.

23.

Niessen

Leuchter

Exferth

A psychological model of air traffic control and its implementation. In: Riter

Young

(eds). Proceedings of the second European conference on cognitive modelling (ECCM-98), Nottingham: Nottingham University Press, 1998, pp. 104–111. http://www.researchgate.net/profile/Cornelia_Niessen/publication/245013060_A_Psychological_Model_of_Air_Traffic_Control_and_Its_Implementation/links/00b7d52963a8d68587000000.pdf (accessed 10 March 2015).

24.

Model for Task and Job Descriptions of Air Traffic Controllers. EUROCONTROL, HUM.ET1.ST01.1000-REP-01, 1996. p.36, https://www.eurocontrol.int/sites/default/files/content/documents/nm/safety/safety-model-for-task-and-job-descriptions-of-air-traffic-controllers.pdf (accessed 10 March 2015).

25.

Keinrath C, Pouw F, Wise JA, et al. iFly: Human factors in autonomous aircraft operations (Airborne Self-Separation Environment). In: The 28th conference of the European Association for Aviation Psychology, 27–31 October, Valencia, Spain (pdf) p.6. http://ifly.nlr.nl/documents/P2.1%20iFly_EAAP08_nf.pdf (accessed 10 March 2015).

26.

Wise

A modest proposal for future systems. In: Wise

Hopkin

Smith

(eds). Automation and system issues in air traffic control, Berlin: Springer, 1991, pp. 529–533.

27.

Joint Planning and Development Office (JPDO). Concept of operations for the Next Generation Air Transportation System Version 3.2. Washington D.C.: JPDO, 2011, p.107.

28.

Kopardekar P, Bilimoria KD and Sridhar B. Initial concepts for dynamic airspace configuration. In: 7th AIAA aviation technology, integration and operations conference (ATIO), Belfast, Northern Ireland, AIAA 2007-7763, 2007, p.12. http://www.aviationsystemsdivision.arc.nasa.gov/publications/strategic/Kopardekar_AIAA_ATIO_2007_1.pdf (accessed 10 March 2015).

29.

Rouse

Morris

. On looking into the black box: Prospects and limits in the search for mental models. Psychol Bull 1986; 100: 349–363.

30.

Dominguez C. Can situational awareness be defined? In: M Vidulich, C Dominguez, E Vogel and G McMillan (eds) Situation awareness: Papers and annotated bibliography. Armstrong Laboratory, Human System Center, ref. AL/CF-TR-1994-0085. Wright-Petterson AFB, Ohio, USA, 1994, pp.5–16.

31.

Endsley

. Measurement of situation awareness in dynamic systems. Human Factors 1995; 37: 65–84.

32.

Jeannot E, Kelly C and Thompson D. The development of situation awareness measures in ATM systems. EUROCONTROL, Brussels, 2003, p.88. http://www.eurocontrol.int/humanfactors/docs/HF35-HRS-HSP-005-REP-01withsig.pdf.

33.

Lüdtke

Osterloh

J-P

Mioch

Capability test for a digital cognitive flight crew model. In: Duffy

(ed.). Advances in applied digital human modeling, New York: CRC Press, Taylor and Francis Group, 2011, pp. 119–128.

34.

Millot P and Boy GA. Human-machine cooperation: A solution for life-critical systems? In: Proceedings of the 2012 IEA world congress on ergonomics, Recife, Brazil, 2012, pp.4452–4459.

35.

Endsley

Kiris

. Situation awareness global assessment technique (SAGAT) TRACON air traffic control version user’s guide, Lubbock: Texas Tech University, 1995.

36.

Hart

Staveland

. Development of NASA-TLX (task load index): Results of empirical and theoretical research. Adv Psychol 1998; 52: 139–183.

37.

Taylor, R. M. Situation awareness rating technique (SART): The development of a tool for aircrew systems design. In: Situational awareness in aerospace operations. France: Neuilly sur-Seine, NATO-AGARD-CP-478, 1990, pp.1–17.

38.

Dehn

. Assessing the impact of automation on the air traffic controller: The shape questionnaires. ATC Quart 2008; 16: 127–146.

39.

Lee PU, Sheridan T, Poage JL, et al. Identification and characterization of key human performance issues and research in the next generation air transportation system (NextGen). NASA, NASA/CR-2010-216390, 2010, p.230.

40.

Dörner

. Wissen, Emotionen und Handlungsregulation oder die Vernunft der Gefühle. Zeitschrift für Psychologie 1993; 201: 167–202.

41.

Delahaye D and Puechmorel S. Air traffic complexity: towards intrinsic metrix. In: 3rd USA/Europe air traffic management R&D seminar, 13–16 June 2000, Napoli, p. 11. http://www.atmseminar.org/seminarContent/seminar3/papers/p_012_AMSMP.pdf (accessed 5 March 2015).

42.

Kopardekar

Schwartz

Magyarits

. Airspace complexity measurement: An air traffic control simulation analysis. Int J Ind Eng 2009; 16: 61–70.

43.

Isaacson DR and Robinson III JE. A knowledge-based conflict resolution algorithm for terminal area air traffic control advisory generation. In: AIAA guidance, navigation, and control conference, Montreal, Canada, NASA 2001-4116, August 2001, p.11.

44.

Sanchez J and Zakrzewski E. The impact of ATC automation on a controller’s visual attention. In: IEEE proceedings of the digital avionics and surveillance communications (DASC) conference, Orlando, FL, USA, 2009, pp.5.B.5-1–5.B.5-7.

45.

Sanchez J, Krumbholz CJ and Silva MO. Controller-automation interaction in NextGen: A new paradigm. The MITRE Corporation, 2010, p.21. https://www.mitre.org/sites/default/files/pdf/09_4925.pdf (accessed 5 March 2015).

46.

Kasyanov

. Subjective analysis, Kiev: National Aviation University, 2007, pp. 512–512.

47.

Rohacs

Kasyanov

. Pilot subjective decisions in aircraft active control system. J Theoret Appl Mech 2011; 49: 175–186.

48.

Rohacs J. Subjective aspects of the less-skilled pilots, performance, safety and well-being in aviation. In: Proceedings of the 29th conference of the European association for aviation psychology (eds A Droog and M Heese), Budapest, Hungary, 20–24 September 2010, pp. 153–159.

49.

Rohacs

Subjective factors in flight safety. In: Agarwal

(ed). Recent advances in aircraft technology, Intech, 2012, Rijeka, Croatia (ISBN 978-953-51-0150-5), 263–286.

50.

Bentham J. An introduction to the principles of morals and legislation, 1781. Republished for example by Batoche Books Kitchenber, 2000, p.416. http://socserv.mcmaster.ca/econ/ugcm/3ll3/bentham/morals.pdf (accessed 22 August 2015).

51.

Edwards

. The theory of decision making. Psychol Bull 1954; 51: 380–417.

52.

PPLANE - The personal plane project, EU FP7 project, http://www.pplane-project.org/ (accessed 10 March 2015).

53.

Aoyama H, Iida H and Shiomi K. An expression of air traffic controller’s workload by recognition-primed decision model. In: 27th international congress of the aeronautical sciences, Nice, France, 19–24 September 2010, p.7.

54.

Prandini M, Putta V and Hu J. Air traffic complexity in advanced automated Air Traffic Management systems. In: 9th innovative research workshop and exhibition. EUROCONTROL, 2011, pp.3–10. https://engineering.purdue.edu/∼jianghai/Publication/INO2010_ATM_Complexity.pdf (accessed 22 August 2015).

55.

Hangmuller M, Rank E and Kubin G. Evaluation of the human voice for indications of workload induced stress in the aviation environment. In: EUROCONTROL, Brussels, 2006, p.89.

56.

Metzger

Parasuraman

. Automation in future air traffic management: effects of decision aid reliability on controller performance and mental workload. Human Factors 2005; 47: 35–49.

57.

Vogt

Hagemann

Kastner

. The impact of workload on heart rate and blood pressure in en-route and tower air traffic control. J Psychophysiol 2006; 20: 297–314.

58.

Dasari D, Crowe C, Ling C, et al. EEG pattern analysis for physiological indicators of mental fatigue in simulated air traffic control tasks. Proceedings of the human factors and ergonomics society annual meeting, (ISSN 1541-9312) September 2010 vol. 54 no. 3, September 2010, pp.205–209.

59.

Shou G and Ding L. Frontal theta EEG dynamics in a real-world air traffic control task. In: Proceedings of 35th Annual Conference of the IEEE Engineering in Medicine and Biology Society, Osaka, Japan, 3–7 July, 2013, pp.5594–5597. http://www.ncbi.nlm.nih.gov/pubmed/24111005#.

60.

Kingsley-Jones S. Combining multimodal touch and haptic response. http://www.electronicproducts.com/Sensors_and_Transducers/Sensors_and_Transducers/Combining_multimodal_touch_and_haptic_response.aspx (2013, accessed 5 January 2013).

61.

Measure Heart Rate through Your Skin with the Tiny Pulse Sensor. Launch, 2011, http://launch.co/blog/measure-heart-rate-through-your-skin-with-the-tiny-pulse-sen.html (accessed 5 January 2013).

62.

Penzo M. Introduction to eyetracking: Seeing through your users’ eyes. Uxmatters - Insight and inspiration for the user experience community, 2005. http://www.uxmatters.com/mt/archives/2005/12/introduction-to-eyetracking-seeing-through-your-users-eyes.php (accessed 13 May 2006).

63.

Ruff J. Information overload: Causes, symptoms and solutions. Learning Innovations Laboratories, Harvard Graduate School of Education, 2002. http://www.newsmaster.be/flow/dw/ciel/2011/aout11/infooverloadbrief.pdf (accessed 10 March 2015).

64.

Jankovics I. Pilot's workload measurement in flight simulator laboratory. In: Rohacs J (ed.) Proceedings of the TVL: Second workshop on transport, vehicle and logistics (organized by the PhD Schools of the Faculty of Transportation Engineering and Vehicle Engineering), BME, Budapest, 2012, pp.1–11. Paper KJK2012-2-J3.

65.

Nagy A, Szabo A and Rohacs J. Monitoring system for in-flight, continuous measurement of pilot’s mental state. In: AIRTEC 1st aerospace sensors conference, Frankfurt, Germany, 7–8 November 2012, pp.1–9.

66.

Jankovics I, Nagy A and Rohacs D. Developing the air traffic controllers' decision supporting system. In: Peter T (ed.) Innováció és fenntartható felszíni közlekedés, IFFK 2014. Magyar Mérnökakadémia. Budapest, Magyarország, 2014, pp.26–28.