Cognitive Capacity Usage in Responding to Test Items

Abstract

Procedures for validating the complexity level of test items include empirical examinations of item characteristics and subjective judgments of expert raters. These methods have yielded mixed and inconsistent results. This study employs a technique derived from cognitive psychology research in the examination of levels of item complexity. Using a microcomputer to administer test items, response time to a secondary task was used as an index of the level of item complexity. A series of three studies are reported that (1) employ the technique on items of marked discrepancy in complexity level and (2) contrast the sensitivity of the technique to more traditional analyses of level of item complexity. The results suggest that the technique has promise as a mechanism for validating or establishing level of item complexity.