Modeling and Diagnosing Domain Knowledge Using Latent Semantic Indexing

Abstract

Latent Semantic Indexing is a variant of factor analysis that has been used to automate grading of free text, model human vocabulary acquisition, and improve information retrieval. We present new applications of LSI: modeling knowledge of a specific domain problem and diagnosing deficiencies in the knowledge of groups and individuals. LSI and multidimensional scaling were used to represent the arguments made by 31 Navy officers concerning the intent of a suspect aircraft in a high-fidelity simulation. From the MDS dimensions, elements of a mental model of the tactical problem were defined. Officers with greater experience reliably generated more arguments regarding one element of this model than did less experienced officers. The potential for automatically diagnosing the knowledge state of individuals was tested by using LSI and discriminant function analysis to categorize arguments into model elements. Arguments were classified with 86% precision at 88% recall for the element that discriminated on expertise, and 84% precision at 77% recall over all elements. The implications for automating knowledge modeling and diagnosing knowledge are discussed.