Line Operational Evaluation (LOE) Air Carrier Scenario Based Evaluation

Abstract

This paper presents a methodology for developing Line Operational Evaluations (LOEs) which assess crew and individual pilot proficiency in the FAA's Advanced Qualification Program (AQP). Under AQP, Crew Resource Management (CRM) skills are trained and assessed along with technical maneuvers in the issuing of an Airmen type certification. The environment for assessing this technical and CRM proficiency is the LOE which must be designed and implemented under strict adherence to the requirements of AQP. The paper will discuss the five major steps in the design process, discuss the issues associated with scenario based measurement of pilot proficiency, and the application of statistical analysis to improve evaluator standardization and the reliability and validity of data.

Get full access to this article

View all access options for this article.

References

Hamman Seamster Edens , (1995) The Development and results of the Line Operational Evaluation in the Advanced Qualification Program. In Proceedings of the Eighth International Symposium on Aviation Psychology. Columbus, Ohio: Ohio State University.

Hays

W. L.

(1981) Statistics. New York: Holt Rinehart and Winston.

Holt

R. W.

Meiman

Seamster

T. L.

(1996) Evaluation of aircraft pilot team performance. In Proceedings of the 40th Annual Meeting of the Human Factors Society, Vol. 1 (pp. 44–48). Santa Monica, CA: The Human Factors Society.

Law

J. R.

Sherman

P. J.

(1995) Do raters agree? Assessing inter-rater agreement in the evaluation of air Crew Resource Management skills. In Proceedings of the Eighth International Symposium on Aviation Psychology. Columbus, Ohio: Ohio State University.

Williams

D. M.

Holt

R. W.

Davis

D. A. Boehm

(1997) Training for inter-rater reliability: baselines and benchmarks. In Proceedings of the Ninth International Symposium on Aviation Psychology. Columbus, Ohio: Ohio State University.