Analyzing Training Device Effectiveness in Cases Where Test Data is Inconclusive

Abstract

Budgetary and time constraints continue to turn the trainer toward the use of simulative devices in designing training programs. Sophisticated devices engineered to simulate often complex man-machine interfaces usually are major investments for the Government Agency or Corporation who use them. How can the user ensure that the device is, in fact, worth the investment? The obvious answer to the question is to conduct prototype testing using representative students to measure the degree of Training Transfer. Testing requires a further substantial investment, however, in terms of both time and money. In many cases, proper test data is just not available. Can device effectiveness be measured when marginal test data or no test data at all exists? The authors, through four successive Cost and Training Effectiveness Analyses commissioned by the United States Army, have attempted to make training effectiveness assessments in just such cases. Through the application of a behavioral predictive model known as TRAINVICE (Wheaton, 1976), they have been moderately successful in determining device utility in terms of potential training transfer. This paper specifically discusses the most recent study, that of the Ground/Vehicular Laser Locator Designator and its associated prototype trainer. The paper describes the test data available at the beginning of the study, discusses the strengths and weaknesses of the TRAINVICE Model as applied in the study, and contributes some thoughts pertaining to the future directions that studies of this nature might take.