Measurement and assessment of operator performance of complex tasks with decisional and psychomotor components of which only ultimate outcome measures are well defined provide difficult methodological challenges. One such task is the practice of air-to-air combat skills in simulated combat environments such as the Air Force Air Combat Maneuvering Instrumentation (ACMI) or ground-based visual flight simulators. Measurement and assessment of pilot performance in these environments are especially challenging because of the special cognitive and psychomotor skills involved in the freeplay and gamesmanship of two or more competing pilots. A review of the existing approaches to automated aircrew performance measurement was performed. Although the most common measurement models, using positional advantage or disadvantage, provide an adequate performance metric, other measures, such as control manipulation and the management of kinetic and potential energy, must be added to provide a refined performance measurement algorithm.
Get full access to this article
View all access options for this article.
References
1.
Breidenbach, S. T., Ciavarelli, A. P., Sievers, R., and Lilienthal, M. (1985). Measurement methods and metrics for aircrew assessment during close-in air-to-air combat (NAVAIRSYSCOM N00019-81-C-0098). Orlando: Naval Training Systems Center.
2.
Burgin, G. H., Fogel, L. J., and Phelps, J. P. (1975). An adaptive maneuvering logic computer program for the simulation of one-on-one air-to-air combat (NASA CR-2582). Washington, DC: National Aeronautics and Space Administration.
3.
Carter, V. E. (1976). Development of automated performance measures for introductory air combat maneuvers. In Experiments to evaluate advanced flight simulation for air combat pilot training (NOR 76-52). Hawthorne, CA: Northrop Corporation.
4.
Carter, V. E. (1977). Development of automated performance measures for introductory air combat maneuvers. In Proceedings of the Human Factors Society 21st Annual Meeting (pp. 435–439). Santa Monica, CA: Human Factors Society.
5.
Ciavarelli, A. P., and Brictson, C. A. (1978). Air Combat Maneuvering Range (ACMR): Has operational performance measurement entered a golden age? In Proceedings of the Human Factors Society 22nd Annual Meeting (pp. 365–368). Santa Monica, CA: Human Factors Society.
6.
Ciavarelli, A. P., Pettigrew, K. W., and Brictson, C. A. (1980). Development of a computer-based air combat maneuvering range debrief system. La Jolla, CA: Dunlap and Associates, Inc.
7.
Ciavarelli, A. P., Williams, A. M., and Krasovec, F. (1980). Operational performance measures for air combat: Development and application. In Proceedings of the Human Factors Society 24th Annual Meeting (pp. 560–563). Santa Monica, CA: Human Factors Society.
8.
deBerg, O. H. (1977). A scoring algorithm for assessing air-to-air combat performance: A factor analysis (Report 0590-77). Air Command and Staff College, Air University.
9.
deLeon, P. (1977). The peacetime evaluation of the pilot skill factor in air-to-air combat (Rand Report R-2070-PR). Santa Monica, CA: Rand Corporation.
10.
Eubanks, J. L., and Killeen, P. R. (1983). An application of signal detection theory to air combat training. Human Factors, 25, 449–456.
11.
Jenkins, D. H. (1981, June). Simulator training effectiveness evaluation (Final Report TAC Projcct 79Y-001F). Nellis AFB, NV: Tactical Fighter Weapons Center.
12.
Kelly, MJ., Wooldridge, A. L., Hennessey, R. T., Vreuls, D., Barnebey, S. F., Cotton, J. C., and Recd, J. C. (1979). Air combat maneuvering performance measurement (AFHRL-TR-79-3). Williams Air Force Base, AZ: Air Force Human Resources Laboratory.
13.
McGuinness, J., Forbes, J. M., and Rhoads, J. E. (1983). Design of an air combat maneuvering performance measurement system for the Simulator for Air-to-Air Combat and the Air Combat Maneuvering Instrumentation Range. Alexandria, VA: Person-System Integration, Ltd.
14.
Mixon, T. R., and Moroncy, W. F. (1982). An annotated bibliography of objective pilot performance measurement (NAVTRAEQUIPCEN IH-330). Orlando: Naval Training Equipment Center.
15.
Moore, S. B., Madison, W. G., Sepp, G. D., Stracener, J. T., and Coward, R. E. (1979). Air combat training: Good stick index validation (AFHRL-TR-79-15). Williams AFB, AZ: Air Forcc Human Resources Laboratory.
16.
Nunn, W. R., and Oberle, R. A. (1976). Evaluating air combat maneuvering engagements (CNS 1077). Arlington, VA: Center for Naval Analysis.
17.
Oberle, R. A., and Narone, S. F. (1979). Project overview: Air Combat Maneuvering Range Readiness Estimation System (Technical Report CRC 355, Vol. I). Arlington, VA: Center for Naval Analysis.
18.
Polzella, D. J., and Reid, G. B. (1987). Multidimensional scaling analysis of simulated air combat maneuvering performance data. In Proceedings of the Fourth International Symposium on Aviation Psychology. Columbus: Ohio State University.
19.
Pruitt, V. R., and Moroney, W. P. (1980). Energy maneuverability displays for air combat training (SAE 801185). Warrendale, PA: Society of Automotive Engineers.
20.
Semple, C. A., Cotton, J. C., and Sullivan, D. J. (1981). Aircrew training devices: Instructional support features (AFHRL-TR-80-58). Brooks AFB, TX: Air Force Human Resources Laboratory.
21.
Simpson, W. R. (1976). Development of a time variant figure-of-merit for use in analysis of air combat maneuvering engagements (TM 76-1 SA). Patuxent River, MD: Naval Air Test Center.
22.
Simpson, W. R., and Oberle, R. A. (1977). The numerical analysis of air combat engagements dominated by maneuvering performance (TM 77-2 SA). Patuxent River, MD: Naval Air Test Center.
23.
Vreuls, D., and Obermayer, R. W. (1985). Human-system performance measurement in training simulators. Human Factors, 27, 241–250.
24.
Vreuls, D., Obermayer, R. W., Wooldridge, A. L., and Kelly, M. J. (1984). Performance measurement guidelines for research. Thousand Oaks, CA: Vreuls Research Corporation.
25.
Waag, W. L., Eddowes, E. E., Fuller, J. H., and Fuller, R. R. (1975). ASUPT automated objective performance measurement system. (AFHRL TR-75-3). Williams AFB, AZ: Air Force Human Resources Laboratory.
26.
Wooldridge, A. L., Kelly, M. J., Vreuls, D., Obermayer, R. W., Nelson, W. H., and Norman, D. A. (1982). Air combat maneuvering performance measurement state space analysis (AFHRL-TR-82-15). Williams AFB, AZ: Air Force Human Resources Laboratory.
