A Two-Stage Scoring Method to Enhance Accuracy of Performance Level Classification

Abstract

Many tests classify each examinee into one of multiple performance levels on the basis of a combination of multiple-choice (MC) and constructed-response (CR) items. This study introduces a two-stage scoring method that identifies examinees whose MC scores place them near a cut point, advising scorers on which examinees will be most affected by score discrepancies. The idea that additional attention be given to such examinees' CR responses is explored from both psychometric and practical perspectives. Because more precision is achieved for examinees near the cut points, the method has potential to enhance the percentage of examinees classified correctly. In an empirical investigation of two operational assessments, the two-stage scoring approach successfully predicted which examinees would ultimately attain scores near the border between two performance levels.