This paper offers an improved methodology for systematic development and validation of a performance measurement system. Application of this methodology is presented in the context of a measurement system designed to assess aircrew performance during air-to-air combat training conducted on the Navy's Tactical Aircrew Combat Training System. Procedures for identifying the purpose and appropriate application of measures, formatting operationally useful training feedback displays, and conducting reliability and validity tests are presented.
Get full access to this article
View all access options for this article.
References
1.
BensonJ.ClarkF. (1982). A guide for instrument development and validation. American Journal for Occupational Therapy, 36, 12, 789–800.
2.
BloomB. (Ed.) (1972). Taxonomy of educational objectives: The classification of educational goals, Handbook I: Cognitive domain. New York: David McKay.
3.
BreidenbachS.T.CiavarelliA.P.SieversR.LilienthalM. (1986). Measurement methods and metrics for aircrew assessment during close-in air-to-air combat (NAVAIRSYSCOM N00019-81-C-0098). Orlando: Naval Training Systems Center.
4.
BreidenbachS.T.FrankL.H. (1984). The use of graphic performance feedback in pilot carrier landing training. Training Technology Journal, 1, 2.
5.
CiavarelliA.P. (1982). Methodology to assess in-flight performance for air-to-air combat training. In Proceedings of the 4th Interservice/Industry Training Equipment Conference: Volume I. Washington, D.C.: National Security Industrial Association.
6.
CiavarelliA.P.PettigrewK.W.BrictsonC.A. (1981a). Development of a computer-based air combat maneuvering range debrief system (NAVTRAEQUIPCEN 78-C-0136-1). Orlando: Naval Training Equipment Center.
7.
CiavarelliA.P.WilliamsA.M.PettigrewK.W. (1981b). Performance assessment and appraisal system (PAAS) (NAVTRAEQUIPCEN 80-D-0011-1). Orlando: Naval Training Equipment Center.
8.
GagneR.M.BriggsL.J. (1979). Principles of instructional design (2nd ed.). New York: Holt, Rinehart and Winston.
HarrowA. (1972). Taxonomy of psychomotor domain. New York: David McKay.
11.
KrathwholD.BloomB.BertramM. (1972). Taxonomy of educational objectives: The classification of educational goals, Handbook II: Affective domain. New York: David McKay.
12.
KressG.BreidenbachS.T. (1983). A performance assessment and appraisal system for air combat training. Orlando: Naval Training Equipment Center.
13.
LaneN.E. (1986). Issues in performance measurement for military aviation with applications to air combat maneuvering (Essex Report No. 86-3). Orlando: Essex Corporation.
14.
LivingstonS.A. (1971). The reliability of criterion-referenced measures. Paper presented at the annual meeting of the American Educational Research Association.
15.
LovettH.T. (1977). Criterion-referenced reliability estimated by ANOVA. Educational and Psychological Measurement, 37, 21–29.
16.
McGuinnessJ.ForbesJ.M.RhoadsJ.E. (1983). Design of an air combat maneuvering performance measurement system for the simulator for air-to-air combat and the air combat maneuvering instrumentation range. Alexandria, VA: Person-System Integration, Ltd.
17.
OberleR.A. (1974). An air combat maneuver conversion model (Report CRC 274). Washington, D.C.: Office of Naval Research.
18.
PedhazurE.J. (1982). Multiple regression in behavioral research: Explanation and prediction (2nd ed.). New York: Holt, Rinehart and Winston.
SimpsonW.R. (1976). Development of a time-variant figure of merit for use in analysis of air combat maneuvering engagements (TM 76-1 SA). Patuxent River, MD: Naval Air Test Center.
21.
SwezeyR.W. (1978). Aspects of criterion-referenced measurement in performance evaluation. Human Factors, 20, 2, 169–178.
22.
VreulsD.WooldridgeL. (1977). Aircrew performance measurement. In PopeJ.MeisterD. (Eds.), Proceedings of symposium on productivity enhancement and personnel assessment in Navy systems. San Diego: Navy Personnel Research and Development Center.
23.
WooldridgeL.KellyM.J.ObermayerR.W.VreulsD.NelsonW.H.NormanD.A. (1982). Air combat maneuvering performance measurement state space analysis (NAVTRAEQUIPCEN III-342/AFHRL-TR-82-15). Brooks AFB, TX: HQ Air Force Human Resources Laboratory.
