Cognitive Task Analysis of Air Traffic Control Instruction to Identify Rule-Based Measures of Student Simulator Performance

Abstract

This study is the first cognitive task analysis of the air traffic controller student evaluation process. The goal was to identify cognitive and behavioral rule-based measures for assessing student simulator performance. A knowledge structure analysis of experienced instructor's sorting of assessment concepts was followed by a protocol analysis of instructor's simulator training sessions with students. The data were translated into IF-THEN cognitive and/or behavioral assessment rules for use in an automated evaluation or intelligent tutoring system. Traditional summary measures of performance activate the rules, providing real-time diagnosis of student performance. This event-driven approach determines not only that some action is incorrect but also what is missing from the student's knowledge or skill base.

Get full access to this article

View all access options for this article.

References

Boone

J.O.

Steen

J.A.

(1981). A comparison between over-the-shoulder and computer-derived measurement procedures in assessing student performance in radar air traffic control. Aviation, Space, and Environmental Medicine, 52, 583–589

Eastman

R.W.

(1984). Investigation of airmen's conceptual understanding of a complex task. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA

Ericsson

K.A.

Simon

H.A.

(1993). Protocol analysis: Verbal reports as data. (2d ed.). Cambridge, MA: MIT Press

Redding

R.E.

(1995). Cognitive task analysis for instructional design: Applications in distance education. Distance Education, 16(1), 88–106

Redding

R.E.

Seamster

T.L.

(1995). Cognitive task analysis for human resource management in aviation: Personnel selection, training and evaluation. In Johnston

McDonald

Fuller

(Eds.), Aviation psychology: Training and selection, (pp. 170–175). Aldershot, UK: Avebury Technical

Redding

R.E.

Seamster

T.L.

(1994). Cognitive task analysis in air traffic controller and aviation crew training. In Johnston

McDonald

Fuller

(Eds.), Aviation psychology in practice, (pp. 190–222). Aldershot, UK: Avebury Technical

Ryder

J.M.

Redding

R.E.

(1993). Integrating cognitive task analysis into Instructional Systems Development. Educational Training Research & Development, 41(2), 75–96

Seamster

T.L.

Redding

R.E.

Cannon

J.R.

Ryder

J.A.

Purcell

J.A.

(1993). Cognitive task analysis of expertise in air traffic control. Intern'l. J. of Aviation Psychology, 3(4), 257–283

Vreuls

Obermayer

R.W.

(1985). Human-system performance measurement in training simulators. Human Factors, 27, 241–250