At present, it is unclear whether math curriculum-based measurement (M-CBM) procedures provide a dependable measure of student progress in math computation because support for its technical properties is based largely upon a body of correlational research. Recent investigations into the dependability of M-CBM scores have found that evaluating probe design procedures is necessary to estimate the contribution of probe-related factors to variance in M-CBM scores. To extend this research, the present study compared a set of commercially available probes (i.e., AIMSweb) with an experimental set that was constructed to enhance consistency in item content across alternative forms. Two probes were randomly selected from each set and administered to 43 students in fifth grade. Generalizability analyses indicated that the experimental probes generated scores with less error variance than those of the AIMSweb probes. Follow-up analyses indicated that fewer administrations of the experimental probes were necessary to attain adequate reliability for relative decisions (i.e., >.80) and high reliability for absolute decisions (i.e., >.90). Results support prior studies and suggest that error variance is associated with M-CBM probe content and development procedures.
AllinderR. M.FuchsL. S.FuchsD.HamlettC. L. (1992). Effects of summer break on math and spelling performance as a function of grade level. The Elementary School Journal, 92, 451–460. Retrieved from http://dx.doi.org/10.1086/461701
2.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
3.
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.
4.
BatscheG. M.CurtisM. J.DormanC.CastilloJ. M.PorterL. J. (2007). The Florida problem-solving/response to intervention model: Implementing a statewide initiative. In JimersonS. R.BurnsM. K.VanDerHeydenA. M. (Eds.), Handbook of response to intervention (pp. 378–395). New York, NY: Springer. Retrieved from http://dx.doi.org/10.1007/978-0-387-49053-3_28
5.
BettsJ. (2010). Expanding school psychology’s horizons for understanding and conducting research: Commentary on the special series on statistical analysis. Journal of School Psychology, 48, 1–4. Retrieved from http://dx.doi.org/10.1016/j.jsp.2009.11.001
6.
BettsJ.PickartM.HeistadD. (2009). An investigation of the psychometric evidence of CBM-R passage equivalence: Utility of readability statistics and equating for alternate forms. Journal of School Psychology, 47, 1–17. Retrieved from http://dx.doi.org/10.1016/j.jsp.2008.09.001
BrickerD.YovanoffP.CaptB.AllenD. (2003). Use of a curriculum-based measure to corroborate eligibility decisions. Journal of Early Intervention, 26, 20–30. Retrieved from http://dx.doi.org/10.1177/105381510302600102
9.
BrieschA. M.SwaminathanH.WelshM.ChafouleasS. M. (2014). Generalizability theory: A practical guide to study design, implementation, and interpretation. Journal of School Psychology, 52, 13–35. Retrieved from http://dx.doi.org/10.1016/j.jsp.2013.11.008
10.
BurnsM. K. (2012). Assessment research and school psychology: Introduction to the special series. School Psychology Review, 41, 243–245.
11.
ChristT. J.ArdoinS. P. (2009). Curriculum-based measurement of oral reading: Passage equivalence and probe-set development. Journal of School Psychology, 47, 55–75. Retrieved from http://dx.doi.org/10.1016/j.jsp.2008.09.004
12.
ChristT. J.Johnson-GrosK. N.HintzeJ. M. (2005). An examination of alternate assessment durations when assessing multiple-skill computational fluency: The generalizability and dependability of curriculum-based outcomes within the context of educational decisions. Psychology in the Schools, 42, 615–622. Retrieved from http://dx.doi.org/10.1002/pits.20107
13.
ChristT. J.ScullinS.TolbizeA.JibanC. L. (2008). Implications of recent research: Curriculum-based measurement of math computation. Assessment for Effective Intervention, 33, 198–205. Retrieved from http://dx.doi.org/10.1177/1534508407313480
14.
ChristT. J.ViningO. (2006). Curriculum-based measurement procedures to develop multiple-skill math computation probes: Evaluation of random and stratified stimulus-set arrangements. School Psychology Review, 35, 387–400.
15.
CooneyJ. B.SwansonH. L.LaddS. F. (1988). Acquisition of mental multiplication skill: Evidence for the transition between counting and retrieval strategies. Cognition and Instruction, 5, 323–345. Retrieved from http://dx.doi.org/10.1207/s1532690xci0504_5
16.
CrickJ. E.BrennanR. L. (1983). Manual for GENOVA: A generalized analysis of variance system. Iowa City, IA: The American College Testing Program.
17.
CrockerL.AlginaJ. (1986). Introduction to classical and modern test theory. Orlando, FL: Holt, Rinehart, and Winston.
DenoS. L.MirkinP. K. (1977). Data-based program modification: A manual. Reston, VA: Council for Exceptional Children.
20.
DirksE.SpyerG.van LieshoutE. C.de SonnevilleL. (2008). Prevalence of combined reading and arithmetic disabilities. Journal of Learning Disabilities, 41, 460–473. Retrieved from http://dx.doi.org/10.1177/0022219408321128
21.
DowningS. M. (2006). Twelve steps for effective test development. In DowningS. M.HaladynaT. M. (Eds.), Handbook of test development (pp. 3–25). Mahwah, NJ: Lawrence Erlbaum.
22.
FoegenA.JibanC.DenoS. L. (2007). Progress monitoring measures in math: A review of the literature. Journal of Special Education, 41, 121–139. Retrieved from http://dx.doi.org/10.1177/00224669070410020101
23.
FoegenA.OlsonJ. R.Impecoven-LindL. (2008). Developing progress monitoring measures for secondary math: An illustration in algebra. Assessment for Effective Intervention, 33, 240–249. Retrieved from http://dx.doi.org/10.1177/1534508407313489
24.
FuchsL. S. (2003). Assessing intervention responsiveness: Conceptual and technical issues. Learning Disabilities Research & Practice, 18, 172–186. Retrieved from http://dx.doi.org/10.1111/1540-5826.00073
25.
FuchsL. S. (2004). The past, present, and future of curriculum based measurement research. School Psychology Review, 33, 188–192.
26.
FuchsL. S.DenoS. L. (1991). Paradigmatic distinctions between instructionally relevant measurement models. Exceptional Children, 57, 488–500.
27.
FuchsL. S.FuchsD. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53, 199–208.
28.
FuchsL. S.FuchsD.ComptonD. L.BryantJ. D.HamlettC. L.SeethalerP. M. (2007). Math screening and progress monitoring at first grade: Implications for responsiveness to intervention. Exceptional Children, 73, 311–330. Retrieved from http://dx.doi.org/10.1177/001440290707300303
29.
FuchsL. S.FuchsD.HamlettC. L.WalzL.GermannG. (1993). Formative evaluation of academic progress: How much growth can we expect?School Psychology Review, 22, 27–48.
30.
FuchsL. S.FuchsD.PhillipsN.HamlettC.KarnsK. (1995). Acquisition and transfer effects of classwide peer-assisted learning strategies in math for students with varying learning histories. School Psychology Review, 24, 604–620.
31.
FuchsL. S.FuchsD.PowellS. R.SeethalerP. M.CirinoP. T.FletcherJ. M. (2008). Intensive intervention for students with math disabilities: Seven principles of effective practice. Learning Disability Quarterly, 31, 79–92.
GearyD. C.HoardM. K.NugentL.BaileyD. H. (2012). Mathematical cognition deficits in children with learning disabilities and persistent low achievement: A five-year prospective study. Journal of Educational Psychology, 104, 206–233. Retrieved from http://dx.doi.org/10.1037/a0025398
35.
GerstenR.BeckmannS.ClarkeB.FoegenA.MarchL.StarJ. R.WitzelB. (2009). Assisting students struggling with mathematics: Response to Intervention (RtI) for elementary and middle schools (Practice Guide Report). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
36.
HalfonN.HoutrowA.LarsonK.NewacheckP. W. (2012). The changing landscape of disability in childhood. The Future of Children, 22, 13–42. Retrieved from http://dx.doi.org/10.1353/foc.2012.0004
37.
HauserM. D.ChomskyN.FitchW. T. (2002). The faculty of language: What is it, who has it, and how did it evolve?Science, 298, 1569–1579. Retrieved from http://dx.doi.org/10.1126/science.298.5598.1569
38.
HintzeJ. M.ChristT. J.KellerL. A. (2002). The generalizability of CBM survey-level math assessments: Just how many samples do we need?School Psychology Review, 31, 514–528.
39.
HospJ. L. (2012). Using assessment data to make decisions about teaching and learning. In HarrisK. R.GrahamS.UrdanT. (Eds.), APA educational psychology handbook: Application to learning and teaching (Vol. 3, pp. 87–109). Washington, DC: American Psychological Association.
40.
Keller-MargulisM.ShapiroE. S.HintzeJ. M. (2008). Long-term diagnostic accuracy of curriculum-based measures in reading and math. School Psychology Review, 37, 374–390.
41.
LeiP.SmithM.SuenH. (2007). The use of generalizability theory to estimate data reliability in single-subject observational research. Psychology in the Schools, 44, 433–439. Retrieved from http://dx.doi.org/10.1002/pits.20235
42.
LembkeE.SteckerP. (2007). Curriculum-based measurement in math: An evidence-based formative assessment procedure. Portsmouth, NH: RMC Research Corporation, Center on Instruction.
43.
LinnR. L. (2006). The standards for educational and psychological testing: Guidance in test development. In DowningS. M.HaladynaT. M. (Eds.), Handbook of test development (pp. 27–38). Mahwah, NJ: Lawrence Erlbaum.
44.
MarstonD. B. (1989). A curriculum-based measurement approach to assessing academic performance: What it is and why do it. In ShinnM. R. (Ed.), Curriculum-based measurement: Assessing special children (pp. 18–78). New York, NY: Guilford Press.
45.
MazzoccoM. M. M. (2007). Defining and differentiating mathematical learning disabilities and difficulties. In BerchD. B.MazzoccoM. M. M. (Eds.), Why is math so hard for some children? The nature and origins of mathematical learning difficulties and disabilities (pp. 7–27). Baltimore, MD: Paul H. Brookes.
46.
National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common core state standards for math. Washington, DC: Author.
47.
SeethalerP. M.FuchsL. S. (2011). Using curriculum-based measurement to monitor kindergarteners’ math development. Assessment for Effective Intervention, 36, 219–229. Retrieved from http://dx.doi.org/10.1177/1534508411413566
48.
ShalevR. S. (2007). Prevalence of developmental dyscalculia. In BerchD. B.MazzoccoM. M. M. (Eds.), Why is math so hard for some children? The nature and origins of mathematical learning difficulties and disabilities (pp. 49–60). Baltimore, MD: Paul H. Brookes.
49.
ShapiroE. S. (2008). From research to practice: Promoting academic competence for underserved students. School Psychology Review, 37, 46–51.
50.
ShapiroE. S.EdwardsL.ZigmondN. (2005). Progress monitoring of math among students with learning disabilities. Assessment for Effective Intervention, 30, 15–32. Retrieved from http://dx.doi.org/10.1177/073724770503000203
51.
ShinnM. R. (1989). Curriculum-based measurement: Assessing special children. New York, NY: Guilford Press.
52.
ShinnM. R. (2004). Administration and scoring of math computation curriculum-based measurement (M-CBM) and math fact probes for use with AIMSweb. Available from www.aimsweb.com
SteckerP. M.FuchsL. S. (2000). Effecting superior achievement using curriculum-based measurement: The importance of individual progress monitoring. Learning Disabilities Research & Practice, 15, 128–134. Retrieved from http://dx.doi.org/10.1207/SLDRP1503_2
55.
ThurberR. S.ShinnM. R.SmolkowskiK. (2002). What is measured in math tests? Construct validity of curriculum-based math measures. School Psychology Review, 31, 498–513.
56.
VanDerHeydenA. M. (2011). Technical adequacy of response to intervention decisions. Exceptional Children, 77, 335–350. Retrieved from http://dx.doi.org/10.1177/001440291107700305
57.
VanDerHeydenA. M.BurnsM. K. (2005). Using curriculum-based assessment and curriculum-based measurement to guide elementary math instruction: Effect on individual and group accountability scores. Assessment for Effective Intervention, 30, 15–31. Retrieved from http://dx.doi.org/10.1177/073724770503000302
58.
VanderwoodM. L.LinklaterD.HealyK. (2008). Predictive accuracy of nonsense word fluency for English language learners. School Psychology Review, 37, 5.
59.
VolpeR. J.BrieschA. M. (2012). An investigation of the generalizability and dependability of single item and multiple item Direct Behavior Rating scales. School Psychology Review, 41, 246–261.
60.
YsseldykeJ. E.VanderwoodM. L.ShrinerJ. (1997). Changes over the past decade in special education referral to placement probability: An incredibly reliable practice. Assessment for Effective Intervention, 23, 193–201.
