Rescuing Computerized Testing by Breaking Zipf's Law

Abstract

For many reasons, both economic and psychometric, computerized tests are typically administered continuously throughout the year rather than at only a few dates spread out over the year. Continuous test administration can yield serious security problems, since examinees could remember some of the items administered and communicate them to other potential examinees. In this essay I argue that because of the nonlinear relationship between item usage and item security, this problem cannot be resolved merely by increasing the size of the item pool. I found that, with the then current methods of exposure control, in order for an item pool's security to increase linearly the size of the pool must increase exponentially. Such a strategy is not economically feasible. In addition to offering evidence of the failure of obtaining a secure item pool merely through increases in the item pool size I provide two alternative strategies with a much better chance of success.