Moving the Needle on Evaluating Multivariate Screening Accuracy

Abstract

The assessment of screening accuracy and setting of cut points for a universal screener have traditionally been evaluated using logistic regression analysis. This analytic technique has been frequently used to evaluate the trade-offs in correct classification with misidentification of individuals who are at risk of performing poorly on a later outcome. Although useful statistically, coefficients from a multiple logistic regression can be difficult to explain to practitioners as it pertains to classification decisions. Moreover, classifications based on multivariate assessments are challenging to understand how performance on one assessment compensates for performance on another. The purpose of this article is to demonstrate and compare the use of logistic regression with classification and regression tree (CART) models in the identification of students who are at risk of reading comprehension difficulties. Data consisted of 986 Grade 1 students and 887 Grade 2 students who were administered a screening assessment at the middle of the school year as well as the 10th edition of the Stanford Achievement Test. Results indicated that CART performs comparably with logistic regression and may assist researchers and practitioners in explaining classification rules to parents and educators.