Reaching for the Brass Ring of Psychometric Test Standards: Commenting on Slaney,Storey,and Barnes

Abstract

Slaney, Storey, and Barnes (2011), in delineating guidelines for data-driven psychometric test evaluation, advocate modern test theory for extending a prescriptive framework proposed in Slaney and Maraun (2008). Here, with an emphasis on noncognitive tests, we begin with a historical review of psychological testing and the developments of testing theory up to current standards and practices. We argue that although the efforts of Slaney et al. are commendable and may be superior in a field in which classical test theory still dominates, they may fall upon deaf ears for two reasons. First, present psychology training is negligent in keeping up with recent developments in statistics and measurement. Second, the practical advantages of modern test theory over classical approaches have not yet been sufficiently demonstrated to test users. This is not to say modern theory cannot produce better tests, rather, when we consider that the ultimate criterion has been and will be predictive validity, old, unsophisticated, classically developed tests still seem to perform satisfactorily. The prescriptive guidelines put forth in Slaney et al. are therefore better understood as aspirational targets, as recommendations, as challenges for tomorrow's generations of test users and developers, and less like psychometric law.