Conceptual Replications of the Critical Content Monitoring General Outcome Measure in Science Content

Abstract

We conducted conceptual replications for an online-administered general outcome measure known as critical content monitoring in science. Totally, 547 Grades 4 to 6 students from a rural Louisiana school district participated. Research questions addressed criterion validity, diagnostic accuracy, growth, and social validity concerns. Criterion measures included science and reading/literacy subtests from a state accountability test and a nationally standardized test. Findings indicated moderate to strong concurrent and predictive criterion validity correlations for the overall scores, fair diagnostic accuracy statistics for fall and winter and moderate for spring benchmark scores, variable growth scores across grades, and favorable social validity comments from teachers. Study limitations were listed. Research and practice implications were discussed.

Keywords

content assessment general outcome measurement science English language arts

Get full access to this article

View all access options for this article.

References

Alexander

(n.d.). Understanding vocabulary. Retrieved from http://www.scholastic.com/teachers/article/understanding-vocabulary

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Baker

S. K.

Kame’enui

E. J.

Simmons

D. C.

Simonsen

(2007). Characteristics of students with diverse learning and curricular needs. In Coyne

M. D.

Kame’enui

E. J.

Carnine

D. W.

(Eds.), Effective teaching strategies that accommodate diverse learners (3rd ed., pp. 23–43). Upper Saddle River, NJ: Pearson.

Beck

I. L.

McKeown

M. G.

Kucan

(2002). Bringing words to life: Robust vocabulary instruction. New York, NY: Guilford.

Bell

B.A.

Ene

Smiley

Shonenberger

J.A

. (2013). A multilevel primer using SAS® PROC MIXED. SAS Global Forum 2013 Proceedings. http://support.sas.com/resources/papers/proceedings13/433-2013.pdf

Borsuk

E. R.

(2010). Examination of an administrator-read vocabulary-matching measure as an indicator of science achievement. Assessment for Effective Intervention, 35, 168–177. doi:10.1177/1534508410372081

Boudreaux-Johnson

Mooney

Lastrapes

R. E.

(2017). An evaluation of close reading with at-risk fourth-grade students in science content. Journal of At-Risk Issues, 20(1), 27–35.

Brown

Kappes

(2012). Implementing the common core state standards: A primer on “close reading of text.” Washington, DC: The Aspen Institute Education and Society Program. Retrieved from http://www.aspeninstitute.org/publications/implementing-common-core-state-standards-primer-close-reading-text

Busch

T. W.

Espin

C. A.

(2003). Using curriculum-based measurement to prevent failure and assess learning in the content areas. Assessment for Effective Intervention, 28(3-4), 49–58.

10.

Carney

R. N.

(n.d.). Mental Measurements Yearbook. Review of the Stanford Achievement Test-Tenth Edition. Retrieved from http://buros.org/mental-measurements-yearbook

11.

Coyne

M. D.

Cook

B. G.

Therrien

W. J.

(2016). Recommendations for replication research in special education: A framework of systematic, conceptual replications. Remedial and Special Education, 27, 244–253. doi:10.1177/0741932516648463

12.

Deno

S. L.

(1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219–232.

13.

Deno

S. L.

Fuchs

L. S.

(1987). Developing curriculum-based measurement systems for data-based special education problem solving. Focus on Exceptional Children, 19, 1–16.

14.

Deno

S. L.

Mirkin

Chiang

(1982). Identifying valid measures of reading. Exceptional Children, 49, 36–45.

15.

Espin

C. A.

Busch

T. W.

Lembke

E. S.

Hampton

D. D.

Seo

Zukowski

B. A.

(2013). Curriculum-based measurement in science learning: Vocabulary-matching as an indicator of performance and progress. Assessment for Effective Intervention, 38, 203–213. doi:10.1177/1534508413489724

16.

Espin

C. A.

Busch

T. W.

Shin

Kruschwitz

. (2001). Curriculum-based measurement in the content areas: Validity of vocabulary matching as an indicator of performance in social studies. Learning Disabilities Research & Practice, 16, 142-151. doi:10.1111/0938-8982.00015

17.

Espin

C. A.

Deno

S. L.

(1994–1995). Curriculum-based measures for secondary students: Utility and task specificity of text-based reading and vocabulary measures for predicting performance on content-area tasks. Diagnostique, 20, 121–142.

18.

Espin

C. A.

Foegen

(1996). Validity of three general outcome measures for predicting secondary students’ performance on content-area tasks. Exceptional Children, 62, 497–514.

19.

Espin

C. A.

Shin

Busch

T. W.

(2005). Curriculum-based measurement in the content areas: Vocabulary matching as an indicator of progress in social studies learning. Journal of Learning Disabilities, 38, 353–363. doi:10.1177/00222194050380041301

20.

Forstmeier

Maercker

(2007). Comparison of two diagnostic systems for complicated grief. Journal of Affective Disorders, 99, 203–211.

21.

Fuchs

L. S.

(2017). Critique of the national evaluation of response to intervention: A case for simpler frameworks. Exceptional Children, 83, 255–268. doi:10.1177/0014402917693580

22.

Fuchs

L. S.

(2004). The past, present, and future of curriculum-based measurement research. School Psychology Review, 33, 188–192.

23.

Fuchs

L. S.

(2017). Curriculum-based measurement as the emerging alternative: Three decades later. Learning Disabilities Research & Practice, 32, 5–7. doi:10.1111/ldrp.12127

24.

Fuchs

L. S.

Deno

S. L.

(1991). Paradigmatic distinctions between instructionally relevant measurement models. Exceptional Children, 57, 488–500.

25.

Fuchs

L. S.

Fuchs

Compton

D. L.

(2010). Rethinking response to intervention at middle and high school. School Psychology Review, 39, 22–28.

26.

Fuchs

L. S.

Fuchs

Zumeta

R. O.

(2008). A curricular-sampling approach to progress monitoring. Assessment for Effective Intervention, 33, 225–233. doi:10.1177/1534508407313484

27.

Hosp

J. L.

Ford

J. W.

(2014). Investigating the relation of a modified sentence verification task of science knowledge with the science test of the Iowa Assessments (Internal Iowa Measurement Research Foundation report, Unpublished).

28.

Johnson

E. S.

Semmelroth

Allison

Fritsch

(2013). The technical properties of science content maze passages for middle school students. Assessment for Effective Instruction, 38, 214–223. doi:10.1177/1534508413489337

29.

Kilgus

S. P.

Riley-Tillman

T. C.

Chafouleas

S. M.

Christ

T. J.

Welsh

M. E.

(2014). Direct behavior rating as a school-based behavior universal screener: Replication across sites. Journal of School Psychology, 52, 63–82.

30.

Lastrapes

R. E.

Mooney

. (2017). Sentence verification technique’s potential in measuring science achievement in a rural school district: A replication and extension. Manuscript submitted for publication.

31.

Louisiana Department of Education. (n.d.-a). iLEAP 2014 operational technical summary. Baton Rouge: Author.

32.

Louisiana Department of Education. (n.d.-b). LEAP/iLEAP interpretive guide, grades 3-8, science and social studies, spring 2015. Baton Rouge: Author.

33.

Louisiana Department of Education. (n.d.-c). LEAP interpretive guide, grades 3-8, ELA and math, spring 2016. Baton Rouge: Author.

34.

Louisiana Department of Education. (n.d.-d). LEAP 2014 operational technical summary. Baton Rouge: Author.

35.

Marston

D. B.

(1989). A curriculum-based measurement approach to assessing academic performance: What is it and why do it. In Shinn

M. R.

(Ed.), Curriculum-based measurement: Assessing special children (pp. 18–78). New York, NY: Guilford.

36.

Mooney

Lastrapes

R. E

. (2016). The benchmarking capacity of a general outcome measure of academic language in science and social studies. Assessment for Effective Intervention, 41, 209-219. doi:10.1177/1534508415624648

37.

Mooney

Lastrapes

R. E.

Marcotte

A. M.

Matthews

. (2016). Validity of two general outcome measures of science and social studies achievement. Specialusis Ugdymas/ Special Education, 34(1), 145-188

38.

Mooney

McCarter

K. S.

Russo

R. J.

Blackwood

D. L

. (2013). Examining an online content general outcome measure: Technical features of the static score. Assessment for Effective Intervention, 38(4), 249–260. doi: 10.1177/1534508413488794

39.

Mooney

McCarter

K. S.

Russo

R. J.

Blackwood

D. L

. (2014). The structure of an online assessment of science and social studies content: Testing optional formats of a general outcome measure. Social Welfare Interdisciplinary Approach, 4(1), 81–93.

40.

Mooney

McCarter

K. S.

Schraven

Callicoatte

(2013). Additional performance and progress validity findings targeting the content-focused vocabulary matching. Exceptional Children, 80, 85–100.

41.

Mooney

McCarter

K. S.

Schraven

Haydel

(2010). The relationship between content area general outcome measurement and statewide testing in world history. Assessment for Effective Intervention, 35, 148–158. doi:10.1177/1534508409346052

42.

Nagy

Townsend

(2012). Words as tools: Learning academic vocabulary as language acquisition. Reading Research Quarterly, 47, 91–108. doi:10.1002/RRQ.011

43.

National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards for English language arts and literacy in history/social studies, science, and technical subjects. Washington, DC: Authors. Retrieved from http://www.corestandards.org/other-resources/key-shifts-in-english-language-arts/

44.

Pearson Education. (n.d.). Stanford Achievement Test Series-Tenth Edition. Retrieved from https://www.pearsonassessments.com/learningassessments/products/100000563/stanford-achievement-test-series-tenth-edition-abbreviated-battery.html

45.

Royer

J. M.

Hastings

C. N.

Hook

(1979). A sentence verification technique for measuring reading comprehension. Journal of Literacy Research, 11, 355–363.

46.

Shek

D. T.

(2011). Longitudinal data analyses using linear mixed models in SPSS: Concepts, procedures and illustrations. The Scientific World Journal, 11, 42–76.

47.

Silberglitt

Hintze

J. M.

(2007). How much growth can we expect? A conditional analysis of R-CBM growth rates by level of performance. Exceptional Children, 74, 71–84.

48.

Townsend

(2015). Who’s using the language? Supporting middle school students with content area academic language. Journal of Adolescent & Adult Literacy, 58, 376–387. doi:10.1002/jaal.374

49.

Townsend

Filippini

Collins

Biancarosa

(2012). Evidence for the importance of academic word knowledge for the academic achievement of diverse middle school students. The Elementary School Journal, 112, 497–518.

50.

Travers

J. C.

Cook

B. G.

Therrien

W. J.

Coyne

M. D.

(2016). Replication research and special education. Remedial and Special Education, 37, 195–204.

51.

Vannest

K. J.

Parker

Dyer

(2011). Progress monitoring in Grade 5 science for low achievers. The Journal of Special Education, 44, 221–233. doi:10.1177/0022466909343121

52.

Wayman

Wallace

Wiley

H. I.

Tichá

Espin

C. A.

(2007). Literature synthesis on curriculum-based measurement in reading. The Journal of Special Education, 41, 85–12.