The purpose of the present investigation was to develop curriculum-valid mathematics survey tests based on an analysis of district adopted basal mathematics curriculum material. We describe a taxonomy of curriculum analysis that was beneficial, in part, in item creation and test construction. Test items were selected to accurately represent the curriculum. The survey test was piloted with a group of 240 students each in Grades 3, 4, and 5 of six schools. A second, parallel test was developed and administered to the same group approximately six months later. Test results were reported to teachers as student and item information. Implications for future assessment research and practice involving students with learning difficulties are also discussed.
Get full access to this article
View all access options for this article.
References
1.
Airasian, P.W., & Madaus, G.F. (1983). Linking testing and instruction: Policy issues . Journal of Educational Measurement, 20, 103-118.
2.
Alexander, L., & James, H.T. (1987). The nation's report card. Cambridge, MA: National Academy of Education.
3.
Berk, R.A. (1984a). Conducting the item analysis. In R.A. Berk (Ed.), A guide to criterion-referenced test construction (pp. 97-143), Baltimore, MD: The Johns Hopkins University Press.
4.
Berk, R.A. (1984b). Selecting the index of reliability. In R.A. Berk (Ed.), A guide to criterion-referenced test construction (pp. 231-266). Baltimore, MD: The Johns Hopkins University Press.
5.
Bloom, B.S., Hastings, J.T., & Madaus, G.F. (1971). Handbook of formative and summative evaluation of student learning. NYMcGraw-Hill.
6.
Bormuth, J.R. (1970). On the theory of achievement test items. Chicago: University of Chicago Press.
7.
Carey, L.M. (1994). Measuring and evaluating school learning (2nd. ed.). Boston: Allyn & Bacon.
8.
Christenson, S.L., Ysseldyke, J.E., & Thurlow, M.L. (1989). Critical instructional factors for students with mild handicaps: An integrative review. Remedial and Special Education, 10(5), 21-31.
9.
Cole, N.S., & Nitko, A.J. (1981). Instrumentation and bias: Issues in selecting measures for educational evaluations. In R.A. Berk (Ed.), Educational evaluation methodology: The state of the art. Baltimore, MD: Johns Hopkins University Press.
10.
Cone, J.D. (1987). Intervention planning using adaptive behavior instruments. The Journal of Special Education, 21, 127-148.
11.
Cooley, W.W., & Leinhardt, G. (1980). The instructional dimensions study. Educational Evaluation and Policy Analysis, 2, 7-25.
12.
Denmark, T., & Kepner, H.S. Jr. (1980). Basic skills in mathetics: A survey . Journal for Research in Mathematics Education, 11(2), 104-123.
Frederiksen, N.The real test bias: Influences of testing on teaching and learning. American Psychologist, 39, 193-202.
16.
Freeman, D.J., Kuhs, T.M., Porter, A.C., Floden, R.F., Schmidt, W.H., & Schwille, J.R. (1983). Do textbooks and tests define a national curriculum in elementary school mathematics? The Elementary School Journal. 83(5), 501-513.
17.
Fuchs, L.S., Fuchs, D., & Hamlett, C.L. (1994). Strengthening the connection between assessment and instructional planning with expert systems. Exceptional Children, 61, 138-146.
18.
Fuchs, L.S., Fuchs, D., Hamlett, C.L., & Stecker, P.M. (1990). The role of skills analysis in curriculum-based measurement in math. School Psychology Review, 19(1), 6-22.
19.
Fuchs, L.S., Fuchs, D., Hamlett, C.L., & Stecker, P.M. (1991). Effects of curriculum-based measurement and consultation on teacher planning and student achievement in math operations. American Journal of Educational Research, 28(3), 617-641.
20.
Fuchs, L.S., Fuchs, D., Hamlett, C.L., Thompson, A., Roberts, P.H., Kubek, P., & Stecker, P. (1994). Technical features of a mathematics concepts and applications curriculum-based measurement system. Diagnostique , 19(4), 23-49.
21.
Fuchs, L.S., Fuchs, D., Hamlett, C.L., Walz, L.,& Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22(1), 27-48.
22.
Glennon, V.J., & Wilson, J.W. (1972). Diagnostic-prescriptive teaching. In The slow learner in mathematics, National Council of Teachers of Mathematics Yearbook, 34 (pp. 282-318), Washington, DC: National Council of Teachers of Mathematics.
23.
Gersten, R., Keating, T., & Irvin, L.K. (1995). The burden of proof: Validity as improvement of instructional practice. Exceptional Children, 61(6), 510-519.
24.
Gronlund, N.E. (1988). How to construct achievement tests. Englewood Cliffs, NJ: Prentice-Hall.
Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer-Nijhoff .
27.
Hays, W.L. (1981). Statistics. NYHolt, Reinhart and Winston.
28.
Hively, W., Patterson, H.L., & Page, S.A. (1968). A "universe defined" system of arithmetic achievement tests. Journal of Educational Measurement, 5, 275-290.
29.
Howell, K.W., Fox, S.L. & Morehead, M.K. (1993). Curriculum-based evaluation: Teaching and decision making (2nd ed.). Pacific Grove, CA: Brooks/Cole.
30.
Jones, L.V. (1988). School achievement trends in mathematics and science, and what can be done to improve them. In E.Z. Rothkopf (Ed.), Review of Research in Education (pp.307-341). Washington, DC: American Educational Research Association .
31.
Kameenui, E.J., & Griffin, C.C. (1989). The national crisis in verbal problem solving in mathematics: A proposal for examining the role of basal mathematics programs . The Elementary School Journal, 89(5), 575-593.
32.
Kane, M.T. (1986). The role of reliability in criterion-referenced tests. Journal of Educational Measurement, 23(3), 221-224.
33.
Kingsbury, G.G. (1990). Adapting adaptive testing: Using the MicroCAT Testing System in a Local School District. Educational Measurement: Issues and Practice, 9(2), 3-6.
34.
Kuhs, T., Schmidt, W., Porter, A., Floden, R., Freeman, D., & Schwille, J. (1979). A taxonomy for classifying elementary school mathematics content (Research Series No. 4). East Lansing: Institute for Reasearch on Teaching, Michigan State University.
35.
Linn, R.L., (1983). Testing and instruction: Links and distinctions . Journal of Educational Measurement, 20, 179-189.
36.
Linn, R.L. (1994). Performance assessment: Policy promises and technical measurement standards. Educational Researcher,23,(9), 4-14.
37.
Linn, R.L., & Gronlund, N.E. (1995). Measurement and assessment in teaching (7th ed.). Englewood Cliffs, NJ: Merrill.
38.
Mayer, R.E., Sims, V., & Tajika, H. (1995). A comparison of how textbooks teach mathematical problem solving in Japan and the United States. American Educational Research Journal,32,(2), 442-460.
39.
Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H.I. Braun (Eds.), Test validity (pp. 33-46). Hillsdale, NJ: Lawrence Erlbaum.
40.
Millman, J. (1980). Computer-based item generation. In R.A. Berk (Ed.), Criterion-referenced measurement: The state of the art (pp. 78-96). Baltimore: Johns Hopkins University Press.
41.
Murnana, R.J., & Raizen, S.A. (1988). Inproving indicators of the quality of science and mathematics education in grades K-12. Washington, DC : National Academy Press.
42.
Nitko, A.J. (1980). Distinguishing the many varieties of criterion-referenced tests. Review of Educational Research, 50(3), 461-485.
43.
Nitko, A.J. (1989). Designing tests that are integrated with instruction . In R.L. Linn (Ed.), Educational Measurement (3rd ed., pp. 447-474), NY: Macmillan.
44.
Olsen, J.B., Cox, A., Price, C., Strozeski, M., & Vela, I. (1990). Development, implementation, and validation of a computerized test for statewide assessment. Educational Measurement: Issues and Practice, 9(2), 7-10.
45.
Osburn, H.G. (1968). Item sampling for achievement testing. Educational and Psychological Measurement, 28, 95-104.
46.
Popham, W.J. (1976). Normative data for criterion-referenced tests ? Phi Delta Kappan, 57, 593-594.
47.
Popham, W.J. (1984). Specifying the domain of content or behaviors . In R.A. Berk (Ed.), A guide to criterion-referenced test construction (pp. 29-48). Baltimore, MD: Johns Hopkins University Press.
48.
Porter, A. (1989). A curriculum out of balance: The case of elementary school mathematics. Educational Researcher, 18(5), 9-15.
49.
Porter, A.C.Schmidt, W.H.Floden, R.E. & Freeman, D.J. (1978). Practical significance in program evaluation. American Educational Research Journal, 15, 529-539.
50.
Rand, W.M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66, 846-850.
51.
Roid, G.H. (1984). Generating the test items. In R.A. Berk (Ed.), Criterion-referenced measurement: The state of the art (pp. 49-77). Baltimore, MD : Johns Hopkins University Press.
52.
Roid, G.H. (1989). Item writing and item banking by microcomputer: An update. Educational Measurement: Issues and practices, 8(3), 17-20.
53.
Roid, G., & Haladyna, T. (1982). A technology for test item writing. NYAcademic Press.
54.
Romberg, T.A., & Carpenter, T.P. (1986). Research on teaching and learning mathematics: Two disciplines of scientific inquiry (pp. 850-873). In M.C. Wittrock (Ed.), Handbook of research on teaching (3rd ed.). NY: Macmillan.
55.
Salmon-Cox, L. (1981). Teachers and standardized achievement tests: What's really happening? Phi Delta Kappan, 62, 631-634.
56.
Salvia, J., & Ysseldyke, J.E. (1995). Assessment in special and remedial education (6th ed.). Boston, MA: Houghton Mifflin.
57.
Schmidt, W.H. (1983). Content bias in achievement tests. Journal of Educational Measurement, 20, 165-178.
58.
Shinn, M.R., Habedank, L., Rodden-Nord, K., & Knutson, N. (1993). Using curriculum-based measurement to identify potential candidates for reintegration into general education. The Journal of Special Education, 27(2), 202-221.
59.
Silbert, J., Carnine, D., & Stein, M. (1990). Direct instruction mathematics (2nd ed.). Columbus, OH: Merrill.
60.
Stephens, W. (1982). Mathematics knowledge and school work: A case study of the teaching of developing mathematical processes (Project Report) . Madison: Wisconsin Center for Education Research.
61.
Stone, C.A. (1989). Testing software review: MicroCAT Version 3.0 . Educational Measurement: Issues and practice, 8(3), 33-38.
62.
Tindal, G. (1989). Evaluating the effectiveness of educational programs at the systems level using curriculum-based measurement. In M. Shinn (Ed.), Curriculum-based measurement. NYThe Guilford Press.
63.
Tindal, G., Fuchs, L.S., Fuchs, D., Shinn, M.R., Deno, S.L., & Germann, G. (1985). Empirical validation of criterion-referenced tests . Journal of Educational Research, 78(4), 203-209.
Tyson, H., & Woodward, A. (1989). Why students aren't learning very much form textbooks . Educational Leadership, 47, 14-17.
66.
Walker, D., & Schaffarzick, J. (1974). Comparing curricula. Review of Educational Research, 44, 83-111.
67.
Webster, R.E., McInnis, E.D., & Craver, L. (1986). Curriculum biasing effects in standardized and criterion-referenced achievement tests. Psychology in the Schools, 23(2), 205-213.
68.
Whinnery, K.W., & Fuchs, L.S. (1992). Implementing effective teaching strategies with learning disabled students through curriculum-based measurement. Learning Disabilities Research and Practice, 7(1), 25-30.
69.
Williams, R.G., & Haladyna, T.M. (1982). Logical operations for generating intended questions (LOGIQ): A typology for higher level test items . In G. Roid & T. Haladyna (Eds.), A technology for test item writing (pp. 161-185). NY: Academic Press.
70.
Willis, J.A. (1990). Learning outcome testing program: Standardized classroom testing in West Virginia through item banking, test generation, and curricular management software. Educational Measurement Issues and Practice, 9(2), 11-14.
71.
Willoughby, S.S., Bereiter, C., Hilton, P., & Rubenstein, J.H. (1987). Real mathematics. Peru, IL: Open Court Publishing.
72.
Woodward, A., & Elliott, D.L. (1990). Textbook use and teacher professionalism. In D.L. Elliott & A. Woodward (Eds.), Textbooks and schooling in the United States (89th Yearbook of the National Society for the Study of Education, Part 1, pp. 178-193). Chicago: National Society for the Study of Education.
