Generalizability (G) theory was used with a sample of 37 third-grade students to assess the variability in words correct per minute (WCPM) scores caused by student skill and pas sage variability. Reliability-like coefficients and the SEM based on a specific number of assess ments using different combinations of passages demonstrated how manipulating probe vari ability could reduce measurement error. Results showed that 81 % of the variance was due to student skill, 10% was due to passage or probe variability, and 9% was due to unac counted sources of error. Reliability-like coeffi cients ranged from .81 to .99, and SEMs ranged from 18 to 4 WCPM depending on the number of probes given. When passage vari ability was controlled, SEMs were decreased and ranged from 12 to 4 WCPM. Results indi cated that WCPM scores yield high reliability- like coefficients but also have a large SEM that can be reduced by administering multiple alternate passages. Discussion focuses on con ducting research designed to identify more equivalent passages in order to reduce erro neous relative and absolute decisions.
Get full access to this article
View all access options for this article.
References
1.
Ardoin, S.P., Witt, J.C., Suldo, S.M., Connell, J.E., Koenig, J.L., Resetar, J.L., Slider, N.J., & Williams, K.L. (2004). Examining the incremental benefits of administering a maze and three versus one curriculum-based measurement reading probes when conducting universal screening. School Psychology Review, 33, 218-233.
2.
Brennan, R.L. (2001). Generalizability theory. New York: Springer-Verlag.
3.
Deno, S.L. (1986). Formative evaluation of individual student programs: A new role for school psychologists. School Psychology Review , 15, 358-374.
4.
Deno, S.L. (1989). Curriculum-based measurement and special education services: A fundamental and direct relationship. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children (pp. 1-17). New York: Guilford Press.
5.
Deno, S.L., & Mirkin, P. (1977). Data-based program modification: A manual. Reston, VA: Council for Exceptional Children.
6.
Derr-Minneci, T.F., & Shapiro, E.S. (1992). Validating curriculum-based measurement in reading from a behavioral perspective. School Psychology Quarterly , 7, 2-16.
7.
Fuchs, L.S., & Deno, S.L. (1994). Must instructionally useful performance assessment be based in the curriculum ? Exceptional Children, 61, 15-24.
8.
Good, R.H., & Jefferson, G. (1998). Contemporary perspectives on curriculum-based measurement validity. in M. R. Shinn (Ed.), Advanced applications of curriculum-based measurement (pp. 61-88). New York; Guilford Press.
9.
Good, R.H., & Kaminski, R.A. (2002). DIBELS oral reading fluency passages for first through third grades (Technical Report No. 10). Eugene, OR: University of Oregon.
10.
Good, R.H., Kaminski, R.A., & Dill, S. (2002). DIBELS oral reading fluency. In R. H. Good & R. A. Kaminski (Eds.), Dynamic Indicators of Basic Early Literacy Skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement. Available: http://dibels.uoregon.edu.
11.
Gresham, F., & Noell, G.H. (1998). Functional analysis assessment as a corner-stone for noncategorical special education. In D. J. Reschly, W. D. Tilly, & J. P. Grimes (Eds.), Functional and noncategorical identification and intervention in special education (pp. 39-64). Des Moines, IA: Iowa Department of Education.
12.
Gresham, F.M., & Witt, J.C. (1997). Utility of intelligence tests for treatment planning, classification, and placement decisions: Recent empirical findings and future directions. School Psychology Quarterly, 12, 249-267.
13.
Heartland Area Education Agency. (2002). Program manual for special education. Johnston, IA: Author.
14.
Hintze, J.M., & Christ, T.J. (2004). An examination of variability as a function of passage variance in cbm progress monitoring. School Psychology Review, 33, 204-217.
15.
Hintze, J.M., Owen, S.V., Shapiro, E.S., & Daly, E.J. (2000). Generalizability of oral reading fluency measures: Application of g theory to curriculum-based measurement. School Psychology Quarterly, 15, 52-68.
16.
Powell-Smith, K.A., & Bradley-Klug, K.L. (2001). Another look at the "c" in cbm: Does it really matter if curriculum-based measurement reading probes are "curriculum-based?" Psychology in the Schools, 38, 299-312.
17.
Reschly, D. J., Tilly, W. D., & Grimes, J. P. (Eds.). (1998). Functional and noncategorical identification and intervention in special education. Des Moines, IA: Iowa Department of Education.
18.
Reschly, D.J., & Ysseldyke, J.E. (2002). Paradigm shift: The past is not the future. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp. 1-20). Washington, DC: National Association of School Psychologists.
19.
Sattler, J.M. (2001). Assessment of children: Cognitive applications (4th ed,). San Diego, CA: Author.
20.
Shavelson, R.J., & Webb, N.M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
21.
Shinn, M. R. (Ed.). (1989). Curriculum-based measurement: Assessing special children. New York: Guilford Press.
22.
Shinn, M. R. (Ed.). (1998). Advanced applications of curriculum-based measurement. New York: Guilford Press.
23.
Shinn, M.R., & Bamonto, S. (1989). Advanced applications of curriculum-based measurement: "Big ideas" and avoiding confusion. In M. R. Shinn, (Ed.), Advanced applications of curriculum-based measurement (pp. 1-31). New York: Guilford Press.
24.
Shinn, M.R., Good, R.H., & Parker, C. (1998). Noncategorical special education services with students with severe achievement deficits. In D. J. Reschly, W. D. Tilly, & J. P. Grimes, (Eds.), Functional and noncategorical identification and intervention in special education (pp. 65-84). Des Moines, IA: Iowa Department of Education.
25.
Tawney, J.W., & Gast, D.L. (1984). Single subject research in special education. Columbus, OH: Charles E. Merrill.
26.
Tindal, G., Germann, G., & Deno, S.L. (1983). Descriptive research on the Pine County norms: A compilation of findings. (Research Report No. 132). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.
27.
Tindal, G., Marston, D., & Deno, S.L. (1983). The reliability of direct and repeated measurement. (Research Report No. 109). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.
28.
Ysseldyke, J., & Marston, D. (1998). Origins of categorical special education services and a rationale for changing them. In D. J. Reschly, W. D. Tilly, & J. P. Grimes (Eds.), Functional and noncategorical identification and intervention in special education (pp. 15-38). Des Moines, IA: Iowa Department of Education.
