This research provides a current benchmark of the cognitive skills and cognitive processes needed for flight path management (FPM) in current commercial air transportation flight operations. While some cognitive skills for aviation have been identified, it remains unclear which skills are most pertinent for different phases of flight, for different tasks, across different aircraft types, and during operational complexity. Further, there is concern that flight deck automation may contribute to cognitive skill degradation. Two expert pilots participated in cognitive walkthroughs to establish a current benchmark of the cognitive skills and cognitive processes needed for FPM. The tasks involved seven different phases of flight and two different aircraft; the results from two phases are reported—Preflight Briefing and Initial Climb. The findings indicate nineteen cognitive skills, and three metacognitive skills are used by pilots for FPM. In addition, the cognitive process models needed for FPM are all very similar, regardless of the aircraft type, task, phase of flight, or increased operational complexity. These results provide a foundation for future efforts on cognitive skill degradation, training of FPM cognitive skills, and may be used to inform the design of new automated systems to support pilot cognition.
AbbottK.McKenneyD.RailsbackP. (2013). Operational use of flight path management systems–final report of the performance-based operations aviation rulemaking committee. Commercial aviation safety team flight deck automation working group. FAA.
2.
AdamsJ. A.HumphreyC. M.GoodrichM. A.CooperJ. L.MorseB. S.EnghC.RasmussenN. (2009). Cognitive task analysis for developing unmanned aerial vehicle wilderness search support. Journal of Cognitive Engineering and Decision Making, 3(1), 1–26. https://doi.org/10.1518/155534309X431926
ArgoteL.BeckmanS. L.EppleD. (1990). The persistence and transfer of learning in industrial settings. In KleinD. A. (Ed.), The strategic management of intellectual capital (pp. 189–209). https://doi.org/10.1287/mnsc.36.2.140
5.
BaniyaS.PottsL. (2022). Experts, knowledge making, and disaster across twitter. In Proceedings of the 40th ACM international conference on design of communication (pp. 9–14). Association for Computing Machinery. https://doi.org/10.1145/3513130.3558972.
6.
BillingsC. E. (1991) Human-centered aircraft automation: A concept and guidelines (Vol. 103885). National Aeronautics and Space Administration, Ames Research Center.
7.
BillmanD.MumawR.FearyM. (2020). A model of monitoring as sensemaking: Application to flight path management and pilot training. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 64, pp. 244–248). Sage Publications. https://doi.org/10.1177/1071181320641058
8.
BlandfordA.FurnissD. (2006). DiCoT: A methodology for applying distributed cognition to the design of teamworking systems. In Interactive systems. Design, specification, and verification: 12th international workshop, DSVIS 2005, Newcastle upon Tyne, UK, July 13–15, 2005. Revised papers (Vol. 12, pp. 26–38). Springer Berlin Heidelberg. https://doi.org/10.1007/11752707_3
9.
BrewerW. F. (1987). Schemas versus mental models in human memory. In MorrisP. (Ed.), Modeling cognition (pp. 187–197). John Wiley and Sons.
10.
CainR. E.Jr. (2001). The relationships of metacognition, self-efficacy, and educational and/or flight experience to situation awareness in aviation students. University of Missouri-Columbia.
11.
CarrollJ. B. (1993). Human cognitive abilities: A survey of factor-analytic studies (No. 1). Cambridge University Press.
12.
CasnerS. M.GevenR. W.ReckerM. P.SchoolerJ. W. (2014). The retention of manual flying skills in the automated cockpit. Human Factors, 56(8), 1506–1516. https://doi.org/10.1177/0018720814535628
13.
ChandraD. C.SparkoA. L.KendraA.KochanJ. (2020). Operational complexity in performance-based navigation (PBN) arrival and approach instrument flight procedures (IFP) (No. DOT-VNTSC-FAA-20-02). John A. Volpe National Transportation Systems Center (US).
14.
CompanionM. A.CorsoG. M.KassS. J.HerschlerD. A. (1990). Situational awareness: An analysis and preliminary model of the cognitive process (IST-TR-89 – 5). University of Central Florida, Institute for Simulation and Training.
15.
CookeN. J.GormanJ. C.MyersC.DuranJ. (2012). Theoretical underpinning of interactive team cognition. In MyersC.DuranI. (Eds.), Theories of team cognition: Cross-disciplinary perspectives (p. 187).
16.
CrandallB.KleinG.KleinG. A.HoffmanR. R. (2006). Working minds: A practitioner’s guide to cognitive task analysis. MIT Press.
17.
CuevasH. M.AguiarM. D. (2021). Metacognitive situation awareness in aviation. In 33rd international symposium on aviation psychology (p. 450). Wright State University.
18.
De BoerR.DekkerS. (2017). Models of automation surprise: Results of a field survey in aviation. Safety Now, 3(3), 20. https://doi.org/10.3390/safety3030020
19.
DekkerS. W.LundstromJ. (2006). From threat and error management (TEM) to resilience. Human Factors and Aerospace Safety, 6(3), 261.
20.
DorneichM. C.DudleyR.Letsu-DakeE.RogersW.WhitlowS. D.DillardM. C.NelsonE. (2017). Interaction of automation visibility and information quality in flight deck information automation. IEEE Transactions on Human-Machine Systems, 47(6), 915–926. https://doi.org/10.1109/THMS.2017.2717939
21.
DoumanisI.EconomouD. (2019). Using cognitive walkthrough and hybrid prototyping to gather user requirements in early design virtual reality prototypes. In Immersive learning research network: 5th international conference, iLRN 2019, London, UK, June 23–27, 2019, proceedings 5 (pp. 234–246). Springer International Publishing. https://doi.org/10.1007/978-3-030-23089-0_18
22.
DudleyR.DorneichM. C.Letsu-DakeE.RogersW.WhitlowS. D.DillardM.NelsonE. (2014). Characterization of information automation on the flight deck. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 58, pp. 295–299). Sage Publications. https://doi.org/10.1177/1541931214581061
23.
EbbatsonM. (2009). The loss of manual flying skills in pilots of highly automated airliners. Doctoral Dissertation). Cranfield University. https://hdl.handle.net/1826/3484.
24.
EbbatsonM.HarrisD.HuddlestoneJ.SearsR. (2010). The relationship between manual handling performance and recent flying experience in air transport pilots. Ergonomics, 53(2), 268–277. https://doi.org/10.1080/00140130903342349
25.
EdenJ. U. (2007). Distributed cognitive walkthrough (DCW) a walkthrough-style usability evaluation method based on theories of distributed cognition. In Proceedings of the 6th ACM SIGCHI conference on Creativity and cognition. Association for Computing Machinery.
26.
ElmW. C.PotterS. S.GualtieriJ. W.EasterJ. R.RothE. M. (2003). Applied cognitive work analysis: A pragmatic methodology for designing revolutionary cognitive affordances. In Handbook of cognitive task design (pp. 357–382). CRC Press.
27.
EndsleyM. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 32–64. https://doi.org/10.1518/001872095779049543
FornetteM. P.BourgyM.JollansJ. Y.RoumesC.DarsesF. (2014). Enhancing management of complex and unforeseen situations among pilots: New trends in cognitive-adaptation training. In Advances in Aviation Psychology (pp. 229–246). Routledge.
33.
GentnerD.StevensA. L. (Eds.), (2014). Mental models. Psychology Press.
34.
GoreB. F.HooeyB. L.HaanN.BakowskiD. L.MahlstedtE. (2011). A methodical approach for developing valid human performance models of flight deck operations. In International conference on human centered design (pp. 379–388). Springer. https://doi.org/10.1007/978-3-642-21753-1_43
35.
HardyD. J.ParasuramanR. (1997). Cognition and flight performance in older pilots. Journal of Experimental Psychology: Applied, 3(4), 313–348. https://doi.org/10.1037/1076-898X.3.4.313
36.
HaslbeckA.HoermannH. J. (2016). Flying the needles: Flight deck automation erodes fine-motor flying skills among airline pilots. Human Factors, 58(4), 533–545. https://doi.org/10.1177/0018720816640394
37.
HelanderM. G.SkinnarsÖ. (2000). Use of cognitive walkthrough for evaluation of cockpit design. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 44, pp. 616–619). Sage Publications. https://doi.org/10.1177/154193120004400619
38.
HutchinsE. (1995a). Cognition in the wild (No. 1995). MIT press.
HutchinsE.KlausenT. (1996). Distributed cognition in an airline cockpit. In Cognition and communication at work (pp. 15–34). University of California.
41.
InoueS.FurutaK.NakataK.KannoT.AoyamaH.BrownM. (2012). Cognitive process modelling of controllers in en route air traffic control. Ergonomics, 55(4), 450–464. https://doi.org/10.1080/00140139.2011.647093
42.
IrwinW.KellyT. (2021). Airline pilot situation awareness: Presenting a conceptual model for meta-cognition, reflection and education. Aviation, 25(1), 50–64. https://doi.org/10.3846/aviation.2021.14209
43.
JadhavD.BhutkarG.MehtaV. (2013). Usability evaluation of messenger applications for Android phones using cognitive walkthrough. In Proceedings of the 11th asia Pacific conference on computer human interaction (pp. 9–18). Association for Computing Machinery. https://doi.org/10.1145/2525194.2525202.
44.
KhajoueiR.Zahiri EsfahaniM.JahaniY. (2017). Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems. Journal of the American Medical Informatics Association: JAMIA, 24(e1), e55–e60. https://doi.org/10.1093/jamia/ocw100
45.
KleinG. (2011). Streetlights and shadows: Searching for the keys to adaptive decision making. MIT Press/Bradford Books.
KovesdiC. (2022). Addressing function allocation for the digital transformation of existing nuclear power plants. Human Factors in Energy: Oil, Gas, Nuclear and Electric Power, 54, 49. https://doi.org/10.54941/ahfe1002220.
48.
KrchD.KreutzerJ. S.DeLucaJ.CaplanB. (2011). Cognitive processing. In Encyclopedia of clinical neuropsychology (p. 627). Springer.
49.
KushnirukA. W.MonkmanH.TudenD.BellwoodP.BoryckiE. M. (2015). Integrating heuristic evaluation with cognitive walkthrough: Development of a hybrid usability inspection method. In Driving quality in informatics: Fulfilling the promise (pp. 221–225). Los Press.
50.
LachterJ.BattisteV.MatessaM.DaoQ. V.KoteskeyR.JohnsonW. W. (2014). Toward single pilot operations: The impact of the loss of non-verbal communication on the flight deck. In Proceedings of the international conference on human-computer interaction in aerospace (pp. 1–8). Association for Computing Machinery. https://doi.org/10.1145/2669592.2669695.
51.
Leal de Oliveira RibeiroS. (2007). Distributed cognition at cockpit: A preliminary study. In 2007 international symposium on aviation psychology (p. 588). Wright State University.
52.
LiniS.FavierP. A.ServantieX.VallespirB.HourlierS. (2011). Developing ASAP (anticipation support for aeronautical planning). In 16th international symposium on aviation psychology 2011 (pp. 103–105). Wright State University.
LinternG.MotavalliA. (2018). Healthcare information systems: The cognitive challenge. BMC Medical Informatics and Decision Making, 63(3), 3–10. https://doi.org/10.1186/s12911-018-0584-z.
55.
MahatodyT.SagarM.KolskiC. (2010). State of the art on the cognitive walkthrough method, its variants and evolutions. International Journal of Human-Computer Interaction, 26(8), 741–785. https://doi.org/10.1080/10447311003781409
56.
MayerR. E. (1998). Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science, 26(1/2), 49–63. https://doi.org/10.1023/a:1003088013286
57.
McKayE. (2002). Cognitive skill acquisition through a meta-knowledge processing model. Interactive Learning Environments, 10(3), 263–291. https://doi.org/10.1076/ilee.10.3.263.8766
58.
MogfordR. H.GuttmanJ. A.MorrowS. L.KopardekarP. (1995). The complexity construct in air traffic control: A review and synthesis of the literature. FAA.
59.
MumawR. J.SwatzlerD.RothE. M.ThomasW. A. (1994). Cognitive skill training for nuclear power plant operational decision making (No. NUREG/CR-6126). Division of Systems Research; Westinghouse Electric Corp.
60.
NaikarN. (2017). Cognitive work analysis: An influential legacy extending beyond human factors and engineering. Applied Ergonomics, 59(Pt B), 528–540. https://doi.org/10.1016/j.apergo.2016.06.001
61.
NingW.Goodman-DeaneJ.ClarksonP. J. (2019). Addressing cognitive challenges in design–a review on existing approaches. In Proceedings of the design society: International conference on engineering design (Vol. 1, pp. 2775–2784). Cambridge University Press.
62.
NormanD. A. (2014). Some observations on mental models. In GentnerD.StevensA. L. (Eds.), Mental models (2nd ed., pp. 7–14). Psychology Press.
63.
O’BrienK. S.O’HareD. (2007). Situational awareness ability and cognitive skills training in a complex real-world task. Ergonomics, 50(7), 1064–1091. https://doi.org/10.1080/00140130701276640
64.
OddenT. O. B.RussR. S. (2019). Defining sensemaking: Bringing clarity to a fragmented theoretical construct. Science Education, 103(1), 187–205. https://doi.org/10.1002/sce.21452
OrasanuJ. M. (2017). 12 Shared problem models and flight crew performance. In JohnsonN.McDonaldN. (Eds.), Aviation psychology in practice (p. 255). Routledge.
67.
PantiwatiY. (2013). Authentic assessment for improving cognitive skill, critical-creative thinking and meta-cognitive awareness 2013. Journal of Education and Practice, 4(14). https://www.iiste.org.
68.
ParasuramanR.MolloyR.SinghI. L. (1993). Performance consequences of automation-induced ‘complacency’. The International Journal of Aviation Psychology, 3(1), 1–23. https://doi.org/10.1207/s15327108ijap0301_1
69.
ParkM. (2018). Innovative assessment of aviation English in a virtual world: Windows into cognitive and metacognitive strategies. ReCALL, 30(2), 196–213. https://doi.org/10.1017/s0958344017000362
70.
PirolliP.CardS. (2005). The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, (Vol. 5, pp. 2–4).
71.
PolsonP.SmithN. (1999). The cockpit cognitive walkthrough. In Proceedings of the tenth symposium on aviation psychology (pp. 427–432). Wright State University.
72.
PolsonP. G.LewisC.RiemanJ.WhartonC. (1992). Cognitive walkthroughs: A method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies, 36(5), 741–773. https://doi.org/10.1016/0020-7373(92)90039-n
73.
ProphetW. W. (1976). Long-term retention of flying skills: A review of the literature (No. HUMRRO-FR-ED (P)-76-35). Assistant Chief of Staff Studies and Analyses (Air Force) Washington DC Command Control and Recon Div.
74.
RankinA.WoltjerR.FieldJ. (2016). Sensemaking following surprise in the cockpit—a re-framing problem. Cognition, Technology & Work, 18(4), 623–642. https://doi.org/10.1007/s10111-016-0390-2
75.
ReysR. E.BestgenB. J. (1981). Teaching and assessing computational estimation skills. The Elementary School Journal, 82(2), 117–127. https://doi.org/10.1086/461246
76.
RitterS.AndersonJ. R.KoedingerK. R.CorbettA. (2007). Cognitive Tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14(2), 249–255. https://doi.org/10.3758/BF03194060
77.
RothE.KleinD.ErnstK. (2021). Aviation decision making and situation awareness study: Decision making literature review. Contract Report USAARL-TECH-CR–2022-17. Applied Decision Science.
78.
SarterN.WickensC.MumawR.KimballS.MarshR.NikolicM.XuW. (2003). Modern flight deck automation: Pilots’ mental model and monitoring patterns and performance. In Proceedings of the international symposium on aviation psychology. Dayton.
79.
SchraagenJ. M.KleinG.HoffmanR. R. (2008). The macrocognition framework of naturalistic decision making (pp. 3–25). Ashgate Publishing Limited.
80.
SilvaS. S.HansmanR. J. (2015). Divergence between flight crew mental model and aircraft system state in auto-throttle mode confusion accident and incident cases. Journal of Cognitive Engineering and Decision Making, 9(4), 312–328. https://doi.org/10.1177/1555343415597344
81.
StaggersN.NorcioA. F. (1993). Mental models: Concepts for human-computer interaction research. International Journal of Man-Machine Studies, 38(4), 587–605. https://doi.org/10.1006/imms.1993.1028
82.
StevensS. M.GoldsmithT. E. (2007). Skill decay on takeoffs as a result of varying degrees of expectancy. In 2007 international symposium on aviation psychology (p. 687). Wright State University.
TaloneA.FincannonT.SchusterD.JentschF.HudsonI. (2013). Comparing physical and virtual simulation use in UGV research: Lessons learned from HRI research with two test beds. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 57, pp. 2017–2021). Sage Publications.
85.
ThomasM. J.PetrilliR. M.DawsonD. (2004). An exploratory study of error detection processes during normal line operations. Doctoral Dissertation. European Association for Aviation Psychology.
86.
Van BenthemK. D. (2015). Identifying latent cognitive constructs in a comprehensive model of aviation outcomes: The role of the dynamic mental model for pilots (Doctoral Dissertation). Carleton University.
VolzK. M.DorneichM. C. (2020). Evaluation of cognitive skill degradation in flight planning. Journal of Cognitive Engineering and Decision Making, 14(4), 263–287. https://doi.org/10.1177/1555343420962897
89.
WeiJ.SalvendyG. (2004). The cognitive task analysis methods for job and task design: Review and reappraisal. Behaviour & Information Technology, 23(4), 273–299. https://doi.org/10.1080/01449290410001673036
90.
WestbrookL. (2006). Mental models: A theoretical overview and preliminary study. Journal of Information Science, 32(6), 563–579. https://doi.org/10.1177/0165551506068134
91.
WhartonC. (1994). The cognitive walkthrough method: A practitioner’s guide. In Usability inspection methods (pp. 105–140). Association for Computing Machinery.
92.
WojcikD. Z.MoulinC. J. A.FernandezA. (2021). Assessment of metacognition in aviation pilot students during simulated flight training of a demanding maneuver. Applied Ergonomics, 95, 103427. https://doi.org/10.1016/j.apergo.2021.103427.
93.
YeL.VarelasM.GuajardoR. (2011). Subject-matter experts in urban schools: Journeys of enacted identities in science and mathematics classrooms. Urban Education, 46(4), 845–879. https://doi.org/10.1177/0042085911399930
94.
ZhangP.SoergelD. (2014). Towards a comprehensive model of the cognitive process and mechanisms of individual sensemaking. Journal of the Association for Information Science and Technology, 65(9), 1733–1756. https://doi.org/10.1002/asi.23125
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
