Research into the technical adequacy of statewide alternate assessments is limited. In this study, the authors analyzed 2 years of data from one state's alternate assessment in written language in an attempt to validate current test score interpretations. More than 1,000 students were included in each year. Findings support the test's technical adequacy on two major dimensions: (a) strong convergent and discriminant evidence for tasks on the assessment and (b) evidence that a collection of writing subtasks contribute unique information to the assessment of writing among students in special education. Implications include the use of the data (a) on subtasks to inform instruction in written language and (b) for accountability purposes.
Get full access to this article
View all access options for this article.
References
1.
Abrams, L.M., Pedulla, J.J., & Madaus, G.F. (2003). Views from the classroom: Tteachers' opinions of statewide testing programs. Theory Into Practice, 42, 18—29.
2.
Berninger, V.W. (2000). Development of language by hand: Its connections with language by ear, mouth, and eye. Topics in Language Disorders , 20, 65—84.
3.
Berninger, V., Cartwright, A., Yates, C., Swanson, H.L., & Abbott, R. (1994). Developmental skills related to writing and reading acquisition in the intermediate grades: Shared and unique variance. Reading and Writing.: An Interdisciplinary Journal, 6, 161—196.
4.
Browder, D., Flowers, C., Ahlgrim-Delzell, L., Karvonen, M., Spooner, F., & Algozzine, R. (2004). The alignment of alternate assessment content with academic and functional curricula. The Journal of Special Education, 37, 211—223.
5.
Browder, D.M., Spooner, F., Algozzine, R., Ahlgrim-Delzell, L., Flowers, C., & Karvonen, M. (2003). What we know and need to know about alternate assessment. Exceptional Children, 70, 45—62.
6.
Chaney, S. (2004, November 28). Thirty-four area schools miss “adequate progress” mark. Colorado SpringsGazette, pp. B1B4, .
7.
Colorado Department of Education. (2003). What you should know about the Colorado Student Assessment Program Alternate (CSAPA). Retrieved March 1, 2005, from http://www.cde.state.co.us/cdesped/StuDis-Sub2.asp
8.
Cronbach, L.J., & Meehl, P. (1955). Construct validity in psychological tests. Psychological Bulletin,52, 281—302.
9.
Dabney, M. (2004). August 24). Indiana schools expand special-needs programs under No Child Left Behind law. Indianapolis Star. Retrieved March, 4, 2005. from Newspaper Source database, http://search.epnet.com/login.aspx?direct=true&db=nfh&an=2W64200078691 >
10.
DeBard, R., & Kubow, P.K. (2002). From compliance to commitment: The need for constituent discourse in implementing test policy. Educational Policy , 16, 387—405.
11.
Englert, C.S. (1990). Unraveling the mysteries of writing through strategy instruction. In T. E. Scruggs & B. Wong (Eds.), Intervention research in learning disabilities (pp. 180—223). New York: Springer Verlag.
12.
Englert, C.S., & Raphael, T. (1988). Constructing well-formed prose: Process, structure and metacognitive knowledge. Exceptional Children, 54, 513—520.
13.
Ford, A., Davern, L., & Schnorr, R. (2001). Learners with significant disabilities. Remedial and Special Education, 22, 214—222.
14.
Gordon, B., Engelhard, G., Gabrielson, S., & Bernknopf, S. (1996). Conceptual issues in equating performance assessments: Lessons from writing assessment. Journal of Research and Development in Education, 29, 81—88.
15.
Graham, S. (1999). Handwriting and spelling instruction for students with learning disabilities: A review. Learning Disability Quarterly, 22, 78—98.
16.
Graham, S., & Harris, K.R. (1997). It can be taught, but does it develop naturally: Myths and realities in writing instruction. School Psychology Review, 26, 414—424.
17.
Graham, S., Harris, K.R., MacArthur, C., & Schwartz, S. (1991). Writing and writing instruction for students with learning disabilities: Review of a research program. Learning Disability Quarterly, 14, 89—114.
18.
Hayes, J.R. (1996). A new framework for understanding cognition and affect in writing. In C. M. Levy & S. Ransdell (Eds.), The science of writing: Theories, methods, individual di ferences, and applications (pp. 1— 27). Mahwah, NJ: Erlbaum.
19.
Individuals with Disabilities Education Act Amendments of 1997, 20 U.S.C. § 1400 et seq.
20.
Individuals with Disabilities Education Improvement Act of 2004), 20 USC § 1400 et seq.
21.
Johnson, E., & Arnold, N. (2004). Validating an alternate assessment. Remedial and Special Education, 25, 266—275.
22.
Klein, S.P., McCaffrey, D., Stecher, B., & Koretz, D. (1995). The reliability of mathematics portfolio scores: Lessons from the Vermont experience. Applied Measurement in Education, 8, 243—260.
23.
Kleinert, H., Kearns, J., & Kennedy, S. (1997). Accountability for all students: Kentucky's alternate portfolio system for students with moderate and severe cognitive disabilities . The Journal of the Association for Persons with Severe Handicaps , 22, 88—101.
24.
Koretz, D., & Barron, S.I. (1998). The validity of gains in scores on the Kentucky Instructional Results Information system (KIRIS). Santa Monica, CA: Rand Institute on Education and Training .
25.
Koretz, D.M., & Barton, K. (2003). Assessing students with disabilities: Issues and evidence (CSE Technical Report 587). Los Angeles: University of California, Center for the Study of Evaluation.
26.
Lembke, E., Deno, S.L., & Hall, K. (2003). Identifying an indicator of growth in early writing proficiency for elementary school students. Assessment for E fective Intervention, 28(3&4), 23—35.
27.
McCutchen, D. (1988). “Functional automaticity” in children's writing: A problem of metacognitive control. Written Communication, 5(3), 306—324.
28.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. The American Psychologist, 50, 741—749.
29.
Miller, M.D., & Crocker, L. (1990). Validation methods for direct writing assessment . Applied Measurement in Education, 3, 285—296.
30.
No Child Left Behind Act of 2001 (2002). February 3, 2005, from http://education.umn.edu/nceo/TopicAreas/AlternateAssessments/alt_assess.topic.htm
31.
Norussis, M.J. (1985). SPSS-X advanced statistics guide. New York: McGraw-Hill.
32.
Olson, L. (2004). January 8). Enveloping expectations: Federal law demands that schools teach the same content to children they wrote off a quarter-century ago. Education Week, 23(17), 8—20.
33.
Oregon Department of Education. (2001). Statewide assessment decision: What is the level of student's current curriculum/instruction? Retrieved March 20, 2005 from http://www.ode.state.or.us/pubs/forms/iep/
34.
Oregon Department of Education. (2002a). Extended writing administration. Salem: Author.
35.
Oregon Department of Education. (2002b). Oregon statewide assessment 2001— 2002 (District Participation Report: State Data). Salem: Author.
36.
Oregon Department of Education. (2003a). Oregon statewide extended assessments: Interpretive guide for individual student reports and class roster. Salem: Author.
37.
Oregon Department of Education. (2003b). Part B annual performance report. Retrieved March 20, 2005, from http://www.ode.state.or.us/pubs/sped/publications.aspx
38.
Quenemoen, R., Thompson, S., & Thurlow, M. (2003). Measuring academic achievement of students with significant cognitive disabilities: Building understanding of alternate assessment scoring criteria (Synthesis Report 50). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved March 2, 2004, from http://education.umn.edu/NCEO/OnlinePubs/Synthesis50.html
39.
Rivera, T.C., & Ganaden, M.F. (2001). The development and validation of a classroom environment scale for Filipinos. Retrieved March 7, 2005, from http://www.dilnet.upd.edu.ph/~ismed/online/articles/dev/factor.htm
40.
Roach, A.T., Elliott, S.N., & Webb, N.L. (2005). Alignment of an alternate assessment with state academic standards: Evidence for the content validity of the Wisconsin alternate assessment. The Journal of Special Education, 38, 218—231.
41.
Roeber, E. (2002). Setting standards on alternate assessments (Synthesis Report 42). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved January 20, 2005, from http://education.umn.edu/NCEO/OnlinePubs/Synthesis42.html
42.
Stevens, J.P. (2002). Applied multivariate statistics for the social sciences. Mahwah, NJ: Erlbaum.
43.
Thompson, S., & Thurlow, M. (2003). 2003State special education outcomes: Marching on. Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved January 18, 2005, from http://education.umn.edu/NCEO/OnlinePubs/2003StateReport.html
44.
Tindal, G., McDonald, M., Tedesco, M., Glasgow, A., Almond, P., Crawford, L., et al. (2003). Alternate assessments in reading and math: Development and validation for students with significant disabilities. Exceptional Children, 69, 481—494.
45.
Tindal, G., & Parker, R. (1989). Assessment of written expression for students in compensatory and special education programs. The Journal of Special Education, 23, 169—183.
46.
Troia, G.A., & Graham, S. (2003). Effective writing instruction across the grades: What every educational consultant should know. Journal of Educational and Psychological Consultation, 14, 75—89.
47.
Vogler, K. (2002). The impact of high-stakes, state-mandated student performance assessment on teachers' instructional practices. Education, 123(1), 39—51.
