The Influence of Rating Method on Knowledge Structures

Abstract

Pathfinder network scaling has been used widely to assess knowledge acquisition and inform interface design. While a large body of research agrees on the validity of this technique to develop knowledge structures, the rating task becomes cumbersome when users are faced with a large number of concepts to relate. Thus, a new rating method was created to help users determine the relationships between concepts. This new interface allows subjects to judge proximity by arranging concepts in visual space on a target. Traditionally, subjects were presented the concepts in pairs and made ratings on Likert Scales. Students (N = 88) at the Air Force Academy used both methods to rate the similarity of basic flying concepts. Results showed that the new method was more sensitive to differences in knowledge structures between experienced and inexperienced pilots (p < .01). Experienced subjects also preferred the new target method compared to the traditional rating method. The relative efficiency of the two methods is discussed.