A sow’s ear purse: Developing an integrated placement test in an under-resourced English for Academic Purposes program

Abstract

Integrated assessment remains comparatively underused in local placement testing contexts despite attracting increasing attention in the language testing community. Two practical difficulties have been noted: First, rating integrated tasks can be time-consuming, and second, integrated assessments may not offer sufficient diagnostic information about receptive language skills. This Brief Report describes a validation study of an integrated placement test designed to maximize efficiency by minimizing rating bottlenecks that arise from attending to copying, source details, and citation conventions, while at the same time maximizing information about receptive skills by triangulating data from analytic subscores, multiple integrated tasks, and a handful of selected-response items. Internal validation measures of the resulting placement test, including rater agreement metrics and many-facet Rasch measurement, indicate that the test meaningfully distinguishes between test-takers of different ability levels and exhibits strong rater consistency. Diagnostic intake checklists indicate that the test supports student learning and results in few misclassifications, with similar accuracy for productive and receptive skills. These innovations on the placement test were accomplished without perceptibly lengthening testing or rating time. Despite some continuing friction points, the test illustrates the potential of adapting integrated assessment to accommodate local resource constraints.