Quantifying Operator Preference during Human Factors Test and Evaluation

Abstract

The present paper will describe the construction and pilot testing of a human factors evaluation instrument. The instrument was constructed using psychometric procedures generally applied to development of attitude scales. The goal of the instrument was the quantification of operator preference in helicopter design within four major areas of human factors consideration: handling qualities, comfort/discomfort, human engineering design and safety. Each area had a common scale against which 10 specific parameters were evaluated. The ten items within each area were chosen using system operators' expertise, human factors standards, human factors experimental literature and the experience of the authors. The instrument was validated using a sample of 16 aviators during the conduct of an actual operational test. The results of these validation studies indicated a test-retest reliability of .85 (P < .001) and an inter-rater reliability of .93 (P < .001). Use of the instrument did provide statistically significant differences among aircraft candidates under various operational test conditions as measured within each of the four indices within the instrument.

It is concluded that the instrument provides an effective method of quantifying the preference of operational aviators. The scrupulous development process using inputs from experienced aviators, experts in aviation field test design, and experts in test construction has provided a high measure of construct validity to the instrument.