This article describes the development of the Early Elementary Writing Rubric (EEWR), an analytic assessment designed to measure kindergarten and first-grade writing and inform educators’ instruction. Crocker and Algina’s (1986) approach to instrument development and validation was used as a guide to create and refine the writing measure. Study 1 describes the development of the 10-item measure (response scale ranges from 0 = Beginning of Kindergarten to 5 = End of First Grade). Educators participated in focus groups, expert panel review, cognitive interviews, and pretesting as part of the instrument development process. Study 2 evaluates measurement quality in terms of score reliability and validity. Data from writing samples produced by 634 students in kindergarten and first-grade classrooms were collected during pilot testing. An exploratory factor analysis was conducted to evaluate the psychometric properties of the EEWR. A one-factor model fit the data for all writing genres and all scoring elements were retained with loadings ranging from 0.49 to 0.92. Internal consistency reliability was high and ranged from .89 to .91. Interrater reliability between the researcher and participants varied from poor to good and means ranged from 52% to 72%. First-grade students received higher scores than kindergartners on all 10 scoring elements. The EEWR holds promise as an acceptable, useful, and psychometrically sound measure of early writing. Further iterative development is needed to fully investigate its ability to accurately identify the present level of student performance and to determine sensitivity to developmental and instruction gains.
AllenA. A.JungP. G.PochA. L.BrandesD.ShinJ.LembkeE. S.McMasterK. L. (2020). Technical adequacy of curriculum-based measures in writing in grades 1–3. Reading & Writing Quarterly, 36(6), 563–587. https://doi.org/10.1080/10573569.2019.1689211
2.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing.
3.
ApplebeeA. (2000). Alternative models of writing development. Perspectives on Writing Research, Theory, and Practice, 90–110.
4.
BeckI. L.McKeownM. G.KucanL. (2013). Bringing words to life: Robust vocabulary instruction. Guilford Press.
5.
BondieR. S.DahnkeC.ZushoA. (2019). How does changing “one-size-fits-all” to differentiated instruction affect teaching?Review of Research in Education, 43(1), 336–362. https://doi.org/10.3102/0091732X18821130
6.
BurginJ.HughesG. D. (2009). Credibly assessing reading and writing abilities for both elementary student and program assessment. Assessing Writing, 14(1), 25–37. https://doi.org/10.1016/j.asw.2008.12.001
7.
CohenJ. (1988). Statistical Power Analysis for the Behavioral Sciences. Routledge Academic.
8.
CokerD. L.RitcheyK. D. (2010). Curriculum-based measurement of writing in kindergarten and first grade: An investigation of production and qualitative scores. Exceptional Children, 76(2), 175–193.
9.
CokerD. L.RitcheyK. D. (2014). Universal screening for writing risk in kindergarten. Assessment for Effective Intervention, 39(4), 245–256. https://doi.org/10.1177/1534508413502389
10.
ConwayJ. M.JakoR. A.GoodmanD. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80(5), 565–579.
11.
CrockerL. M.AlginaJ. (1986). Introduction to classical and modern test theory. Holt, Rinehart, and Winston.
12.
CurranP. J.WestS. G.FinchJ. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16–29. https://doi.org/10.1037/1082-989X.1.1.16
13.
EspinC. A.WeissenburgerJ. W.BensonB. J. (2004). Assessing the writing performance of students in special education. Exceptionality, 12(1), 55–66.
FreyB. B. (Ed.). (2018). The SAGE encyclopedia of educational research, measurement, and evaluation. Sage Publications.
16.
GansleK. A.VanDerHeydenA. M.NoellG. H.ResetarJ. L.WilliamsK. L. (2006). The technical adequacy of curriculum-based and rating-based measures of written expression for elementary school students. School Psychology Review, 35(3), 435–450.
17.
Hall-MillsS.ApelK. (2015). Linguistic feature development across grades and genre in elementary writing. Language, Speech, and Hearing Services in Schools, 46(3), 242–255. https://doi.org/10.1044/2015_LSHSS-14-0043
18.
HayesA. F.KrippendorffK. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77–89.
19.
HensonR. K.RobertsJ. K. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393–416.
20.
HuotB. (2002). Toward a new discourse of assessment for the college writing classroom. College English, 65(2), 163–180.
21.
James-BurdumyS.MansfieldW.DekeJ.CareyN.Lugo-GilJ.HersheyA.. . . FaddisB. (2009). Effectiveness of selected supplemental reading comprehension interventions: Impacts on a first cohort of fifth-grade students. NCEE 2009-4032. National Center for Education Evaluation and Regional Assistance.
22.
JefferyJ. V. (2011). Subjectivity, intentionality, and manufactured moves: Teachers’ perceptions of voice in the evaluation of secondary students’ writing. Research in the Teaching of English, 46(1), 92–127.
23.
JeongH. (2017). Narrative and expository genre effects on students, raters, and performance criteria. Assessing Writing, 31, 113–125. https://doi.org/10.1016/j.asw.2016.08.006
24.
JuelC.GriffithP. L.GoughP. B. (1986). Acquisition of literacy: A longitudinal study of children in first and second grade. Journal of Educational Psychology, 78(4), 243–255.
25.
KaminskiR. A.GoodR. H.III. (1998). Assessing early literacy skills in a problem-solving model: Dynamic indicators of basic early literacy skills. In ShinnM. R. (Ed.), The Guilford school practitioner series. Advanced applications of curriculum-based measurement (pp. 113–142). Guilford Press.
26.
Keller-MargulisM. A.OchsS.ReidE. K.FaithE. L.SchandingG. T.Jr. (2019). Validity and diagnostic accuracy of early written expression screeners in kindergarten. Journal of Psychoeducational Assessment, 37(5), 1–14. https://doi.org/10.1177/0734282918769978
27.
KimY. S.Al OtaibaS.FolsomJ. S.GreulichL.PuranikC. (2014). Evaluating the dimensionality of first-grade written composition. Journal of Speech, Language, and Hearing Research, 57(1), 199–211. https://doi.org/10.1044/1092-4388(2013/12-0152)
28.
KimY. S.Al OtaibaS.PuranikC.FolsomJ. S.GreulichL.WagnerR. K. (2011). Componential skills of beginning writing: An exploratory study. Learning and Individual Differences, 21(5), 517–525. https://doi.org/10.1016/j.lindif.2011.06.004
29.
KimY. S.Al OtaibaS.WanzekJ.GatlinB. (2015). Toward an understanding of dimensions, predictors, and the gender gap in written composition. Journal of Educational Psychology, 107(1), 79–95. https://doi.org/10.1037/a0037210
30.
KruegerR. A. (1994). Focus groups: A practical guide for applied research (2nd ed.). SAGE.
31.
LamR. (2018). Portfolio assessment for the teaching and learning of writing. Springer.
32.
LandisJ. R.KochG. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.
33.
LembkeE. S.GarmanC.DenoS. L.SteckerP. M. (2010). One elementary school’s implementation of response to intervention (RTI). Reading & Writing Quarterly, 26(4), 361–373. https://doi.org/10.1080/10573569.2010.500266
LoniganC. J.MilburnT. F. (2017). Identifying the dimensionality of oral language skills of children with typical development in preschool through fifth grade. Journal of Speech, Language, and Hearing Research, 60(8), 2185–2198. https://doi.org/10.1044/2017_JSLHR-L-15-0402
36.
LynnM. R. (1986). Determination and quantification of content validity. Nursing Research, 35, 382–385.
McCraeR. R.KurtzJ. E.YamagataS.TerraccianoA. (2011). Internal consistency, retest reliability, and their implications for personality scale validity. Personality and Social Psychology Review, 15(1), 28–50. https://doi.org/10.1177/1088868310366253
40.
McKennaM.GoldsteinH.SotoX.ChengK.TroiaG.FerronJ. (2021). Supplemental intervention improves writing performance of first grade students: A single case experimental design evaluation. Journal of Educational Research, 114(3), 278–293. https://doi.org/10.1080/00220671.2021.1923450
41.
McMasterK. L.EspinC. (2007). Technical features of curriculum-based measurement in writing: A literature review. The Journal of Special Education, 41(2), 68–84.
42.
McMasterK. L.ShinJ.EspinC. A.JungP. G.WaymanM. M.DenoS. L. (2017). Monitoring elementary students’ writing progress using curriculum-based measures: Grade and gender differences. Reading and Writing, 30, 2069–2091. https://doi.org/10.1007/s11145-017-9766-9
43.
MessickS. (1975). The standard problem: Meaning and values in measurement and evaluation. American Psychologist, 30(10), 955–966.
44.
MuthenL. K.MuthenB. O. (2019). Mplus [computer software]. Muthén & Muthén.
45.
National Center for Education Statistics. (2012). The nation’s report card: Writing 2011. Institute of Education Sciences, U.S. Department of Education.
OlinghouseN. G. (2008). Student-and instruction-level predictors of narrative writing in third-grade students. Reading and Writing, 21(1–2), 3–26.
49.
PayanA. M.Keller-MargulisM.BurridgeA. B.McQuillinS. D.HassettK. S. (2019). Assessing teacher usability of written expression curriculum-based measurement. Assessment for Effective Intervention, 45(1), 51–64. https://doi.org/10.1177/1534508418781007
50.
PhilippakosZ. A. T.FitzPatrickE. (2018). A proposed tiered model of assessment in writing Instruction: Supporting all student-writers. Insights into Learning Disabilities, 15(2), 149–173.
51.
PuranikC.DuncanM.LiH.YingG. (2020). Exploring the dimensionality of kindergarten written composition. Reading and Writing, 33, 2481–2510. https://doi.org/10.1007/s11145-020-10053-1
RomigJ. E.TherrienW. J.LloydJ. W. (2017). Meta-analysis of criterion validity for curriculum-based measurement in written language. The Journal of Special Education, 51(2), 72–82. https://doi.org/10.1177/0022466916670637
54.
SturmJ. M.CaliK.NelsonN. W.StaskowskiM. (2012). The Developmental Writing Scale: A new progress monitoring tool for beginning writers. Topics in Language Disorders, 32(4), 297–318. https://doi.org/10.1097/TLD.0b013e318272159e
55.
TaylorR. L. (2006). Assessment of exceptional students: Educational and psychological procedures (7th ed.). Pearson Education.
WagnerR. K.PuranikC. S.FoormanB.FosterE.TschinkelE.KantorP. T. (2011). Modeling the development of written language. Reading and Writing, 24, 203–220. https://doi.org/10.1007/s11145-010-9266-7
58.
WhiteK. M. (2013). Associations between teacher–child relationships and children’s writing in kindergarten and first grade. Early Childhood Research Quarterly, 28(1), 166–176. https://doi.org/10.1016/j.ecresq.2012.05.004
59.
WillisG. B. (2004). Cognitive interviewing revisited: A useful technique, in theory. In PresserS.RothgebJ. M.CouperM. P.LesslerJ. T.MartinE.MartinJ.SingeE. (Eds.), Methods for testing and evaluating survey questionnaires (pp. 23–43). Wiley.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
