Evaluating the Reliability of Name Writing Rubric Scoring

Abstract

Early literacy screening tools are often a critical component of schools’ intervention frameworks. As the use of screening tools becomes more prevalent for identifying students who may be at risk for learning difficulties and in need of additional instruction, it is important to continuously evaluate teachers’ implementation of these screening tools and to improve efforts to support their implementation. Measures of name writing as a part of a screening tool can pose specific implementation challenges due to the subjectivity that rubric scoring allows. As such, this study aimed to evaluate the reliability of a rubric used by pre-kindergarten teachers to score a name writing task, as part of a larger screening tool, in the context of their own classrooms, and to examine scoring patterns between researchers and teachers on the name writing rubric. Results revealed that researchers had higher interrater reliability among themselves than the reliability between researchers and teachers. Although there was not a substantial difference between the two groups, it is worthwhile to examine the implementation of screening tool scoring systems in the field to ensure proper identification of and intervention for students who may be at risk for learning difficulties. Implications for researchers and teachers are discussed.