Applying Item Response Theory Methods to Examine the Impact of Different Response Formats

Abstract

In aptitude and achievement tests, different response formats are usually used. A fundamental distinction must be made between the class of multiple-choice formats and the constructed response formats. Previous studies have examined the impact of different response formats applying traditional statistical approaches, but these influences can also be studied using methods of item response theory to deal with incomplete data. Response formats can influence item attributes in two ways: different response formats could cause items to measure different latent traits or they could contribute differently to item difficulty. In contrast to previous research, the present study examines the impact of response formats on item attributes of a language awareness test applying different item response theory models. Results indicate that although the language awareness test contains items with different response formats, only one latent trait is measured; no format-specific dimensions were found. Response formats do, however, have a distinct impact on the difficulty of the items. In addition to the effects of the three administered item types, a fourth component that makes items more difficult was identified.