A Validation of Ebel's Method for Performance Standard Setting through its Application with Comparison Approaches to a Selected Criterion-Referenced Test

Abstract

This paper seeks to describe the research and development into performance standard setting processes used by the U.S. Navy's Strategic Weapon System Training Program for developing Criterion-Referenced System Achievement Tests (C-R SAT) for selected Naval Enlisted Classifications (NECs) or job titles in the Strategic Weapon System (SWS). The design of the C-R SAT involves identification of essential skills and knowledge for each NEC. This process entails planned workshops involving selected senior fleet and supervisory personnel drawn from the NEC. Following this test development process each C-R SAT is then subjected to a performance standard setting process. As all performance standard setting methodologies are subjective, the decision was made to validate the Ebel process and its outputs. Two alternative performance standard setting methodologies were applied to data generated from the two versions (watch stander and supervisor) of a recently administered TRIDENT Technician C-R SAT.