To adequately monitor progress in writing, a test must show stability and demonstrate a performance profile over time that parallels those of accepted criterion measures. This study investigated the technical adequacy of seven objective indexes of writing quality in monitoring the progress of 36 middle school (Grades 6–8) students with mild disabilities over a 6-month period. The stability of each index was assessed. Holistic ratings of the same writing samples and the Test of Written Language served as validation criteria. Three indexes were moderately correlated with holistic ratings, but were not sufficiently stable over time. Direct, objective writing assessment must he used with caution for progress monitoring, given our present lack of precision in measuring the complex task of writing.
Get full access to this article
View all access options for this article.
References
1.
BarenbaumE.NewcomerP.NodineB. (1987). Children's ability to write stories as a function of variation in task, age, and developmental level. Learning Disability Quarterly, 10, 175–188.
2.
BockR. D. (1975). Multivariate statistical methods in behavioral research. New York: McGraw-Hill.
3.
BridgeC. A.HiebertE. H. (1985). A comparison of classroom writing practices, teachers' perceptions of their writing instruction, and textbook recommendations on writing practices. Elementary School Journal, 86, 155–172.
4.
CharneyD. (1984). The validity of using holistic scoring to evaluate writing: A critical overview. Research in the Teaching of English, 18, 65–81.
5.
ConlanG. (1978). How the essay in the College Board English Composition Test is scored. An introduction to the reading for readers. Princeton, NJ: Educational Testing Service.
6.
CronbachL. J. (1971). Test validation. In ThorndikeR. L. (Ed.), Educational measurement (2nd ed.) (pp. 443–507). Washington, DC: American Council on Education.
7.
CTB/McGraw Hill. (1985). California Achievement Tests, Form E. Monterey, CA: CTB/McGraw Hill.
8.
D'AgostinoR. B.TietjenG. L. (1973). Approaches to the null distribution of b1. Biometrika, 60, 169–173.
9.
DenoS.MarstonD.MirkinP. (1982). Valid measurement procedures for continuous evaluation of written expression. Exceptional Children, 48, 368–371.
10.
EnglertC. S.RaphaelT. E.FearK. L.AndersonL. M. (1988). Students' metacognitive knowledge about how to write informational texts. Learning Disability Quarterly, 11, 18–46.
11.
GorellR. (1983). How to make Mulligan stew: Process and product again. College Composition & Communication, 34, 272–277.
12.
GrobeC. (1981). Syntactic maturity, mechanics, and vocabulary as predictors of quality ratings. Research in the Teaching of English, 15, 75–85.
13.
HammillD.LarsenS. (1983). Test of written language. Austin, TX: Pro-Ed.
14.
HammillD.LarsenS. (1988). Test of written language-revised. Austin, TX: Pro-Ed.
15.
HasbrouckJ. (1989). Training manual for direct, objective scoring of writing samples. (Resource Consultant Training Program Module No. 2). Eugene: University of Oregon.
16.
HuntK. W. (1965). Grammatical structures written at 3 grade levels. (Research Report No. 3). Champaign, IL: National Council of Teachers of English.
17.
HuntK. W. (1977). Early blooming and late blooming syntactic structures. In CooperC.OdellL. (Eds.), Evaluating writing (pp. 91–104). Buffalo: National Council of Teachers of English, State University of New York at Buffalo.
18.
IsaacsonS. (1985). Assessing written language skills. In SimonC. S. (Ed.), Communication skills and classroom success: Assessment methodologies for language-learning disabled students (pp. 403–424). San Diego: College-Hill Press.
19.
LeinhardtG.ZigmondN.CooleyW. W. (1980, April). Reading instruction and its effects. Paper presented at the annual meeting of the American Educational Research Association, Boston.
20.
LynchE. M.JonesS. D. (1989) Process and product: A review of the research on LD children's writing skills. Learning Disability Quarterly, 12, 74–86.
21.
MarstonD.DenoS. (1981). The reliability of simple, direct measures of written expression. (Research Report No. 50). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.
22.
McCollyW. (1970). What does educational research say about the judging of writing ability?Journal of Educational Research, 64, 147–156.
23.
MishlerC.HoganT. (1982). Holistic scoring of essays. Diagnostique, 8, 4–16.
24.
MoranM. R. (1987). Options for written language assessment. Focus on Exceptional Children, 19(5), 1–10.
25.
MossP. A.ColeN. S.KhampalikitC. (1982). A comparison of procedures to assess written language skills at grades 4, 7, and 10. Journal of Educational Measurement, 19, 37–47.
26.
NodineB. F.BarenbaumE.NewcomerP. (1985). Story composition by learning disabled, reading disabled, and normal children, Learning Disability Quarterly, 8, 167–179.
27.
NoldE.FreedmanS. (1977). An analysis of readers' responses to essays. Research in the Teaching of English, 11, 164–174.
28.
NormallyJ. (1978). Psychometric theory. New York: McGraw-Hill.
29.
ParkerR.TindalG. (1988). A curriculum evaluation strategy to guide district-level basal text adoption decision-making. Special Services in the Schools, 5(1/2), 33–66.
30.
Phelps-GunnT.Phelps-TerasakiD. (1982). Written language instruction: Theory and remediation. Rockville, MD: Aspen Systems.
31.
RafothB. S.RubinD. L. (1984). The impact of content and mechanics on judgments of writing quality. Written Communication, 1(4), 446–458.
32.
RoitM.McKenzieR. (1985). Disorders of written communication: An instructional priority for LD students. Journal of Learning Disabilities, 19, 258–260.
33.
SabersD. L.FeldtL. S.ReschlyD. J. (1988). Appropriate and inappropriate use of estimated true scores for normative comparisons. The Journal of Special Education, 22(3), 358–366.
34.
ScardamaliaM.BereiterC. (1986). Research on written composition. In WittrockM. C. (Ed.), Handbook of research on teaching (3rd ed.) (pp. 778–803). New York: Macmillan.
35.
SchenckS. J. (1981). The diagnostic/instructional link in individualized education programs. Journal of Special Education, 14(3), 337–345.
36.
SimmonsJ. (1984). The one-to-one method of teaching composition. College Composition & Communication, 35, 222–229.
37.
SpandelV. (1981). Classroom applications for writing assessment: A teacher's handbook. Portland, OR: Northwest Regional Educational Laboratory.
38.
SpandelV.StigginsR. J. (1980). Direct measures of writing skill: Issues and applications. Portland, OR: Northwest Regional Educational Laboratory.
39.
StevensJ. (1986). Applied multivariate statistics for the social sciences. Hillsdale, NJ: Lawrence Erlbaum.
40.
StewartS. R. (1985). Development of written language proficiency: Methods for teaching text structure. In SimonC. S. (Ed.), Communication skills and classroom success: Therapy methodologies for learning-language disabled students (pp. 341–364). San Diego: College-Hill Press.
41.
StigginsR. J. (1982). A comparison of direct and indirect writing assessment methods. Research in the Teaching of English, 16(2), 101–114.
42.
ThomasC. C.EnglertC. S.GreggS. (1987). An analysis of errors and strategies in the expository writing of learning disabled students. Remedial and Special Education, 8, 21–30.
43.
TindalG. (1989). Evaluating the effectiveness of educational programs at the system level using CBM. In ShinnM. (Ed.), Applications of curriculum-based measurement to the development of programs for mildly handicapped students. New York: Guilford Press.
44.
TindalG.ParkerR. (1988). Direct observation in special education classrooms: Concurrent use of two instruments and their validation. The Journal of Special Education, 21(2), 43–59.
45.
TindalG.ParkerR. (1989). Assessment of written expression for students in compensatory and special education programs. The Journal of Special Education, 23(2), 169–184.
46.
VealR. L.HudsonS. A. (1983). Direct and indirect measures for large-scale evaluation of writing. Research in the Teaching of English, 17, 290–296.
47.
VideenJ.DenoS.MarstonD. (1982). Correct word sequences: A valid indicator of proficiency in written expression (Research Report No. 84). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.
48.
WhiteE. (1984). Holisticism. College Composition & Communication, 35, 400–409.
49.
WoodcockR. W. (1973). Woodcock reading mastery tests. Circle Pines, MN: American Guidance Service.
