Selecting Students at Risk of Academic Difficulties

Abstract

This paper aims to translate for practitioners the principles and methods for evaluating screening measures in education, including benchmark goals and cut points, from our technical manuscript “Evaluation of Diagnostic Systems: The Selection of Students at Risk of Academic Difficulties” (this issue). We offer a brief description of procedures developed over the past 50 years including the receiver operating characteristic (ROC) curves, the area under the ROC curve as a general measure of screener accuracy, and approaches to selecting a specific cut score to indicate risk. We also provide reporting standards to help practitioners evaluate research on screeners supported by best practices and to encourage researchers to attend to key reporting principles, such as using confidence bounds as estimates of precision. We then discuss examples from the literature and emphasize the imprecision of statistical estimates from small samples. Screeners and diagnostic tests, developed and evaluated with care and implemented consistently in schools, can improve educators’ decisions about resource allocation and ultimately improve the delivery of supports to students.