The Classroom Strategies Assessment System (CSAS) is a multi-rater, multi-method (direct observation and rating scale methodology) assessment of teachers’ use of research-based instructional and behavior management strategies. The present study investigated the association between teacher self-report and school administrator ratings using the CSAS Teacher (CSAS-T) and Observer (CSAS-O) Forms in 15 high-poverty U.S. charter schools. The CSAS-T and CSAS-O were designed to be used concurrently as a valid formative assessment of teacher practice. Findings include small, but statistically significant correlations between the CSAS-T and CSAS-O. Analysis of a multi-trait–multi-method (MTMM) matrix found teachers and observers to be measuring different constructs. No mean score differences were found between teacher self-reported instruction and behavior management strategy use compared with school administrators’ observed ratings. Furthermore, school administrators and teachers have similar ratings of overall effectiveness, with the majority of teachers in the sample being rated at or above effective. Overall, findings offer support for using the CSAS-O and CSAS-T for guiding professional development conversations.
Arnold-BerkovitsI.KurzA.ReddyL. A. (2019). Teacher log of students’ opportunity to learn and classroom observation: An initial investigation of convergence. Educational Assessment, Evaluation and Accountability, 31, 97–119. https://doi.org/10.1007/s11092-018-9288-2
2.
BoydD.GrossmanP.HammernessK.LankfordH.LoebS.RonfeldtM.WyckoffJ. (2012). Recruiting effective math teachers, evidence from New York City. American Educational Research Journal, 49, 1008–1047. https://doi.org/10.3102/0002831211434579
3.
CampbellD. T.FiskeD. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. https://doi.org/10.1037/h0046016
4.
CohenD. K.PeurachD. J.GlazerJ. L.GatesK. E.GoldinS. (2014). Improvement by design: The promise of better schools. University of Chicago Press.
5.
CohenJ. (1992). A power primer. Psychological Bulletin, 112, 155–159.
6.
CoutinhoS.Wiemer-HastingsK.SkowronskiJ. J.BrittM. A. (2005). Metacognition, need for cognition and use of explanations during ongoing learning and problem solving. Learning and Individual Differences, 15, 321–337. https://doi.org/10.1016/j.lindif.2005.06.001
7.
DesimoneL. M.SmithT. M.FrisvoldD. E. (2010). Survey measures of classroom instruction: Comparing student and teacher reports. Educational Policy, 24(2), 267–329. https://doi.org/10.1177/0895904808330173
8.
DudekC. M.ReddyL. A.LekwaA. (2018). Measuring teach practices to inform student achievement in high poverty schools: A predictive validity study. Journal of Contemporary School Psychology, 23, 290–303. https://doi.org/10.1007/s40688-018-0196-8
9.
Every Student Succeeds Act of 2015, Pub. L. No. 114-95 § 114 Stat. 1177 (2015-2016). Retrieved from: https://www.ed.gov/essa?src=rn
10.
FabianoG. A.ReddyL. A.DudekC. M. (2018). Teacher coaching supported by formative assessment for improving classroom practices. School Psychology Quarterly, 33(2), 293–304. https://doi.org/10.1037/spq0000223
11.
Feeney-KettlerK.A.KettlerR.DembitzerL. (2009). Application of a multi-trait, multi-method matrix to a social, emotional, and behavioral secreening. Journal of Applied School Psychology, 35, 20–36. https://doi.org/10.1080/15377903.2018.146884
12.
FordT. G.Van SickleM. E.ClarkL. V.Fazio-BrunsonM.SchweenD. C. (2017). Teacher self-efficacy, professional commitment, and high-stakes teacher evaluation policy in Louisiana. Educational Policy, 31, 202–248. https://doi.org/10.1177/0895904815586855
13.
FosterS.RollefsonM.DoksumT.NoonanD.RobinsonG.TeichJ. (2005). School mental health services in the United States, 2002–2003 [DHHS Pub. No. (SMA) 05-4068]. Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental Health Services Administration.
14.
GableR. A.HesterP. H.RockM. L.HughesK. G. (2009). Back to basics: Rules, praise, ignoring and reprimands revisited. Intervention in School and Clinic, 44, 195–205. https://doi.org/10.1177/1053451208328831
15.
GoddardY. L.KimM. (2018). Examining connections between teacher perceptions of collaboration, differentiated instruction, and teacher efficacy. Teachers College Record, 120(1), 1–24. https://doi.org/10.1177/016146811812000102
16.
HairE.HalleT.Terry-HumenE.LavelleB.CalkinsJ. (2006). Children’s school readiness in the ECLS-K: Predictions to academic, health, and social outcomes in first grade. Early Childhood Research Quarterly, 21, 431–454. https://doi.org/10.1016/j.ecresq.2006.09.005
HattieJ. (2009). Visible learning: A synthesis of meta-analyses relating to achievement. Routledge.
19.
HermanJ.KleinD.AbediJ. (2000). Assessing students’ opportunity to learn: Teacher and student perspectives. Educational Measurement: Issues and Practice, 194(4), 16–24. https://doi.org/10.1111/j.1745-3992.2000.tb00042.x
20.
IsoreM. (2009). Teacher evaluation: Current practices in OECD countries and a literature review (OECD Education Working Papers No. 23). OECD. https://doi.org/10.1787/223283631428
21.
JanusM.DukuE. (2007). The school entry gap: Socioeconomic, family, and health factors associated with children’s school readiness to learn. Early Education and Development, 18, 375–403. https://doi.org/10.1080/10409280701610796a
22.
KraftM. A.GilmourA. F. (2017). Revisiting the widget effect: Teacher evaluation reforms and the distribution of teacher effectiveness. Educational Researcher, 46, 234–249. https://doi.org/10.3102/0013189X17718797
23.
LekwaA. J.ReddyL. A.ShernoffE. S. (2019). Measuring teacher practices and student academic engagement: A convergent validity study. School Psychology Quarterly, 34, 109–118. https://doi.org/10.1037/spq0000268
McCaffreyD.HamiltonK.StecherB.KleinS.BugliariD.RobynA. (2001). Interaction among instructional practices, curriculum, and student achievement: The case of standards-based high school mathematics. Journal for Research in Mathematics Education, 22, 493–517. https://doi.org/10.2307/749803
26.
MeyerG. J.FinnS. E.EydeL. D.KayG. G.MorelandK. L.DiesR. R.. . . ReedG. M. (2001). Psychological testing and psychological assessment. American Psychologist, 56(2), 128–165. https://doi.org/10.1037/0003-066X.56.2.128
27.
NelsonP. M.ReddyL. A.DudekC. M.LekwaA. (2016). Observer and student ratings of the class environment: A preliminary investigation of convergence. School Psychology Quarterly, 32(4), 465–479. https://doi.org/10.1037/spq0000176
28.
PearsK. C.HealeyC. V.FisherP. A.BraunD.GillC.ConteH. M.NewmanJ.TicerS. (2014). Immediate Effects of a Program to Promote School Readiness in Low-Income Children: Results of a Pilot Study. Education & Treatment of Children, 37(3), 431–460. https://doi.org/10.1353/etc.2014.0021
29.
PollardA.AndersonJ.MaddockM.SwatfieldS.WarinJ.WarwickP. (2008). Reflective teaching: Evidence informed professional practice (3rd ed.). Continuum.
30.
ReddyL. A.DudekC. M. (2014). Teacher progress monitoring of instructional and behavioral management practices: An evidence-based approach to improving classroom practices. International Journal of School & Educational Psychology, 2, 71–84. https://doi.org/10.1080/21683603.2013.876951
31.
ReddyL. A.DudekC. M.RualoA. J.FabianoG. A. (2016). Concurrent validity of the Classroom Strategies Scale–Teacher form: A preliminary investigation. Educational Assessment, 21, 267–277. https://doi.org/10.1080/10627197.2016.1236675
32.
ReddyL. A.DudekC. M.ShernoffE. S. (2016). Teacher formative assessment: The missing link in response to intervention. In JimersonS.BurnsM.VanDerHeydenA. (Eds.), Handbook of response to intervention (pp. 607–623). Springer. https://doi.org/10.1007/978-1-4899-7568-3_34
33.
ReddyL. A.FabianoG.DudekC. M.HsuL. (2013a). Development and construct validation of the Classroom Strategies Scale–Observer Form. School Psychology Quarterly, 28, 317–341. https://doi.org/10.1037/spq0000043
34.
ReddyL. A.FabianoG.DudekC. M.HsuL. (2013b). Predictive validity of the Classroom Strategies Scale–Observer Form on statewide testing. School Psychology Quarterly, 28, 301–316. https://doi.org/10.1037/spq0000041
35.
ReddyL. A.HuaA. N.DudekC. M.KettlerR.Arnold-BerkovitzI.LekwaA.CrouseK.KurzA.HuJ. (2021). The relationship between school administrator and teacher ratings of classroom practices and student achievement in high-poverty schools. Assessment for Effective Intervention, 46(2), 87–98. https://doi.org/10.1177/1534508419862863
36.
ReddyL. A.KettlerR. J.KurzA. (2015). School-wide educator evaluation for improving school capacity and student achievement in high-poverty schools: Year 1 of the school system improvement project. Journal of Educational and Psychological Consultation, 25, 90–108. https://doi.org/10.1080/10474412.2014.929961
37.
ReddyL. A.LekwaA.DudekC.KettlerR.HuaA. (2020). Evaluation of teacher practices and student achievement in high-poverty schools. Journal of Psychoeducational Assessment, 38(7), 816–830. https://doi.org/10.1177/0734282920913394
38.
ReddyL. A.ShernoffE.LekwaA. (2021). A randomized controlled trial of instructional coaching in high-poverty schools: Examining teacher practices and student outcomes. Journal of School Psychology, 86, 151–168. https://doi.org/10.1016/j.jsp.2021.04.001
39.
TaieS.GoldringR. (2020). Characteristics of Public and Private Elementary and Secondary School Teachers in the United States: Results from the 2017-18 National Teacher and Principal Survey. First Look. NCES 2020-142. National Center for Education Statistics.
40.
TomlinsonC. A. (2000). Differentiation of instruction in the elementary grades (ED443572 2000-08-00). ERIC Digest. https://www.eric.ed.gov
