Item Homogeneity in Verbal Tests: A Rasch Analysis of Amthauer'S Verbal Tests

Abstract

The dichotomous Rasch model was applied to the verbal subtests of the Intelligence Structure Test Battery (IST) developed by Amthauer (1973). Three tests were used: (a) Word-grouping, (b)Word Analogies, and (c) Word-pairing. The first two are similar to tests which have been used by Thurstone (1938) in his search for the "Primary Mental Abilities." The tests were administered with a time-limit. Hence, not all items were completed in the time given. The Rasch model held for neither of the tests, when all items were included in the analysis. Importantly, van den Wollenberg's 02statistic was highly significant, which may indicate multidimensionality of the items. A thorough analysis based on subsets of items made quite clear that the following five factors should be taken into account to understand the poor model fit: (1) ambiguity in the formulation of the items; (2) the use of specific knowledge to solve the problems; (3) the use of distractors by which the subject may easily be tempted to resort to responses which seem to be correct, but are actually incorrect; (4) the use of specific problem solving strategies, which appear in some of the items, but not in all;(5) positional effects, in which some alternatives attract the attention more than others. The factors as such do not violate the Rasch model. The Rasch model may only be violated when such a factor plays a role in some of the items, but not in all. In order to establish item homogeneity one should take into consideration the following rule: if any factor is used to increase the difficulty of the items, then that factor should be used in all items.