Scale Validation via Quantifying Item Validity Using the D m Index

Abstract

While most validity indices are based on total test scores, this paper describes a method for quantifying the construct validity of items. The approach is based on the item selection technique originally described by Piazza in 1980. Unfortunately, Piazza's P² index suffers from some substantial limitations. The D_m coefficient provides an alternative which can be used for item selection and provides a validity index for a set of items. The index is similar to that of traditional criterion-related validity indices. Criterion-related validity is used to demonstrate the accuracy of hypothesized relations of the measure with outcome variables of interest in research and practice. This method may be useful when the sample of items or persons is small, rendering more traditional approaches such as factor analysis or item response theory inappropriate. An example of how to use the technique is provided.