A Preliminary Investigation of Two Procedures for Setting Examination Standards

Abstract

Two standard setting procedures were employed by two groups of judges to set pass-fail levels for comparable samples of a nationally administered examination. These procedures were both designed to set standards in relation to the minimally qualified examinee. The study was undertaken to determine whether similar standards would be set for the same examination content when determined by different groups of judges, and whether the two procedures employed would result in similar standards for comparable samples of test content. In addition, the extent to which group consensus judgments might differ from individual judgments was also investigated. The results suggest that different groups of judges do set similar examination standards when using the same procedure, and that the average of individual judgments does not differ significantly from group consensus judgments. Significant differences were found, however, between the standards set by the two procedures employed. This finding was observed for both groups. The nature of these differences is described, and their implications for setting examination standards are discussed.