The transition to new assessments aligned to the Common Core State Standards was a significant turning point in the standards’ implementation. Concerns about the transition led districts to suspend the use of value-added scores for evaluating teachers, but changes to other measures, such as classroom observations, were rare. Using data from the Washington, DC Public Schools, I evaluate the effect of the assessment transition on teachers’ practice. I find substantial declines in instructional practice, ranging from 13% to 20% of a standard deviation, for teachers in tested grades and subjects when the new exam was introduced. These results suggest that policymakers should consider the ramifications of testing changes on a wider array of teaching outcomes than value-added scores alone.
AdnotM. (2016). Effects of incentives and feedback on instructional practice: Evidence from the District of Columbia Public Schools’ IMPACT teacher evaluation system [Working paper].
3.
AjayiL. (2016). High school teachers’ perspectives on the English language arts Common Core State Standards: An exploratory study. Educational Research for Policy and Practice, 15(1), 1–25. https://doi.org/10.1007/s10671-015-9174-3
4.
AllenJ. P.PiantaR. C.GregoryA.MikamiA. Y.LunJ. (2011). An interaction-based approach to enhancing secondary school instruction and student achievement. Science, 333(6045), 1034–1037. https://doi.org/10.1126/science.1207998
5.
AtteberryA.LoebS.WyckoffJ. (2015). Do first impressions matter? Predicting early career teacher effectiveness. AERA Open, 1(4), 1–23. https://doi.org/10.1177/2332858415607834
BackesB.CowanJ. (2019). Is the pen mightier than the keyboard? The effect of online testing on measured student achievement. Economics of Education Review, 68(1), 89–103. https://doi.org/10.1016/j.econedurev.2018.12.007
8.
BackesB.CowanJ.GoldhaberD.KoedelC.MillerL.XuZ. (2018). The common core conundrum: To what extent should I worry that changes to assessments and standards will affect test-based measures of teacher performance?Economics of Education Review, 62(1), 48–65. https://doi.org/10.1016/j.econedurev.2017.10.004
9.
BastianK. C.FortnerC. K. (2020). Is less more? Subject-area specialization and outcomes in elementary schools. Education Finance and Policy, 15(2), 357–382. https://doi.org/10.1162/edfp_a_00278
10.
BellC.JamesJ.TaylorE.WyckoffJ. (2021). Measuring improvement in true teaching performance using classroom observations [Working paper].
11.
BerlinR.CohenJ. (2020). The convergence of emotionally supportive learning environments and college and career ready mathematical engagement in upper elementary classrooms. AERA Open, 6(3), 1–20. https://doi.org/10.1177/233285 8420957612
BlazarD. (2015). Grade assignments and the teacher pipeline: A low-cost lever to improve student achievement?Educational Researcher, 44(4), 213–227. https://doi.org/10.3102/0013189X15580944
14.
BormanG. D.HewesG. M.OvermanL. T.BrownS. (2003). Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research, 73(2), 125–230. https://doi.org/10.3102/00346543073002125
15.
BormanG. D.SlavinR. E.CheungA. C. K.ChamberlainA. M.MaddenN. A.ChambersB. (2007). Final reading outcomes of the national randomized field trial of success for all. American Educational Research Journal, 44(3), 701–731. https://doi.org/10.3102/0002831207306743
16.
BuzickH. M.Rhoad-DrogalisA.LaitusisC. C.KingT. C. (2019). Teachers’ views of their practices related to Common Core State Standards-aligned assessments. ETS Research Report Series, 2019(1), 1–18. https://doi.org/10.1002/ets2.12277
CharalambousC. Y.HillH. C.MitchellR. N. (2012). Two negatives don't always make a positive: Exploring how limitations in teacher knowledge and the curriculum contribute to instructional quality. Journal of Curriculum Studies, 44(4), 289–513. https://doi.org/10.1080/00220272.2012.716974
19.
CoburnC. E.HillH. C.SpillaneJ. P. (2016). Alignment and accountability in policy design and implementation: The Common Core State Standards and implementation research. Educational Researcher, 45(4), 243–251. https://doi.org/10.3102/0013189X16651080
20.
CohenJ. C.WyckoffJ.KatzV.BoguslavA.SadowskiK.WisemanE. A. (in press). Implementing targeted professional development at scale in the District of Columbia Public Schools. American Educational Research Journal.
ConleyD. T. (2014). The Common Core State Standards: Insight into their development and purpose. Council of Chief State School Officers.
23.
CookJ. B.MansfieldR. K. (2017). Task-specific experience and task-specific talent: Decomposing the productivity of high school teachers. Journal of Public Economics, 140, 51–72. https://doi.org/10.1016/j.jpubeco.2016.04.001
DeeT. S.JamesJ. K.WyckoffJ. (2021). Is effective teacher evaluation sustainable? Evidence from DCPS. Education Finance and Policy, 16(2), 313–346. https://doi.org/10.1162/edfp_a_00303
29.
DeeT. S.WyckoffJ. (2015). Incentives, selection, and teacher performance: Evidence from IMPACT. Journal of Policy Analysis and Management, 34(2), 267–297. https://doi.org/10.1002/pam.21818
30.
DesimoneL. M.StornaiuoloA.FloresN.PakK.EdgertonA. K.NicholsT. P.PlummerE.PorterA. (2019). Successes and challenges of the “new” college- and career- ready standards: Seven implementation trends. Educational Researcher, 48(3), 167–178. https://doi.org/10.3102/0013189X19837239
31.
District of Columbia Public Schools. (2014). 2014-15 IMPACT guidebook: Group 1: General education teachers with individual value-added student achievement data.
32.
District of Columbia Public Schools. (2015). 2015-16 IMPACT guidebook: Group 1: General education teachers with individual value-added student achievement data.
District of Columbia Public Schools. (2017). 2016-17 IMPACT guidebook: Group 1: General education teachers with individual value-added student achievement data.
EdgertonA. K. (2020). Learning from standards deviations: Three dimensions for building education policies that last. American Educational Research Journal, 57(4), 1525–1566. https://doi.org/10.3102/0002831219876566
37.
EdgertonA. K.DesimoneL. M. (2018). Teacher implementation of college- and career- readiness standards: Links among policy, instruction, challenges, and resources. AERA Open, 4(5), 1–22. https://doi.org/10.1177/2332858418806863
38.
FergusonR. F.DanielsonC. (2015). How framework for teaching and tripod 7Cs evidence distinguish key components of effective teaching. In KaneT. J.KerrK. A.PiantaR. C. (Eds.), Designing teacher evaluation systems: New guidance from the measures of effective teaching project (pp. 98–143). Jossey-Bass.
39.
FlodenR. E.RichmondG.AndrewsD. C. (2017). Responding to the challenge of new standards. Journal of Teacher Education, 68(3), 236–238. https://doi.org/10.1177/0022487117702380
40.
FullanM. (2001). Leading in a culture of change. Jossey-Bass.
41.
GarrettR.CitkowiczM.WilliamsR. (2019). How responsive is a teacher's classroom practice to intervention? A meta-analysis of randomized field studies. Review of Research in Education, 43(1), 106–137. https://doi.org/10.3102/0091732X19830634
42.
GillB.ShojiM.CoenT.PlaceK. (2016). The content, predictive power, and potential bias in five widely used teacher observation instruments (REL 2017-191). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Mid-Atlantic. https://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=4474
HafenC. A.HamreB. K.AllenJ. P.BellC. A.GitomerD. H.PiantaR. C. (2015). Teaching through interactions in secondary school classrooms: Revisiting the factor structure and practical application of the Classroom Assessment Scoring System–Secondary. Journal of Early Adolescence, 35(5–6), 651–680. https://doi.org/10.1177/0272431614537117
46.
HamiltonL. S.StecherB. M.MarshJ. A.McCombsJ. S.RobynA.RussellJ.NaftelS.BarneyH. (2007). Standards-based accountability under No Child Left Behind: Experiences of teachers and administrators in three states. RAND Corporation. https://www.rand.org/pubs/monographs/MG589.html
JamesJ. (2020). Measuring teacher quality: Does the test make a difference? [Working paper].
50.
JenningsJ. L.LauenD. L. (2016). Accountability, inequality, and achievement: The effects of the No Child Left Behind act on multiple measures of student learning. Russell Sage Foundation Journal, 2(5), 220–241. https://doi.org/10.7758/RSF.2016.2.5.11
KaneT. J.StaigerD. O. (2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains. Bill and Melinda Gates Foundation. https://files.eric.ed.gov/fulltext/ED540960.pdf
54.
KolluriS. (2018). Student perspectives on the Common Core: The challenge of college readiness at urban high schools. Urban Education. Advance online publication. https://doi.org/10.1177/0042085918772630
55.
KraftM. A.BlazarD.HoganD. (2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research, 88(4), 547–588. https://doi.org/10.3102/0034654318759268
56.
KraftM. A.GilmourA. (2017). RevisitingThe Widget Effect: Teacher evaluation reform and the distribution of teacher effectiveness. Educational Researcher, 46(5), 234–249. https://doi.org/10.3102/0013189X17718797
57.
KraftM. A.PapayJ. P. (2014). Can professional environments in schools promote teacher development? Explaining heterogeneity in returns to teaching experience. Educational Evaluation and Policy Analysis, 36(4), 476–500. https://doi.org/10.3102/0162373713519496
58.
KraftM. A.PapayJ. P.ChiO. L. (2020). Teacher skill development: Evidence from performance ratings by principals. Journal of Policy Analysis and Management, 39(2), 315–347. https://doi.org/10.1002/pam.22193
59.
LockwoodJ. R.SavitskyT. D.McCaffreyD. F. (2015). Inferring constructs of effective teaching from classroom observations: An application of Bayesian exploratory factor analysis without restrictions. Annals of Applied Statistics, 9(3), 1484–1509. https://doi.org/10.1214/15-AOAS833
McDonnellL. M.WeatherfordM. S. (2013). Evidence use and the Common Core State Standards movement: From problem adoption to policy adoption. American Journal of Education, 120(1), 1–25. https://doi.org/10.1086/673163
62.
McDuffieA. R.DrakeC.ChoppinJ.DavisJ. D.MagañaM. V.CarsonC. (2017). Middle school mathematics teachers’ perceptions of the Common Core State Standards for Mathematics and related assessment and teacher evaluation systems. Educational Policy, 31(2), 139–179. https://doi.org/10.1177/0895904815586850
63.
MeyerJ. P. (2016). Reliability of and validity evidence for Teaching Learning Framework scores for the District of Columbia public school system [Unpublished manuscript]. Curry School of Education, University of Virginia.
OortF. J.VisserM. R.SprangersM. A. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14(3), 599–609. https://doi.org/10.1007/s11136-004-0831-x
66.
OstB. (2014). How do teachers improve? The relative importance of specific and general human capital. American Economic Journal: Applied Economics, 6(2), 127–151. https://doi.org/10.1257/app.6.2.127
67.
PakK.DesimoneL. (2019). How do states implement college- and career-readiness standards? A distributed leadership analysis of standards-based reform. Educational Administration Quarterly, 55(3), 447–476. https://doi.org/10.1177/0013161X18799463
68.
PapayJ. P.KraftM. A. (2015). Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement. Journal of Public Economics, 130, 105–119. https://doi.org/10.1016/j.jpubeco.2015.02.008
69.
PapayJ. P.LaskiM. (2020). Understanding the dynamics of teacher productivity development: Evidence on teacher improvement in Tennessee [Working paper].
70.
PapayJ. P.TaylorE. S.TylerJ. H.LaskiM. E. (2020). Learning job skills from colleagues at work: Evidence from a field experiment using teacher performance data. American Economic Journal: Economic Policy, 12(1), 359–388. https://doi.org/10.1257/pol.20170709
PhippsA. R.WisemanE. A. (2021). Enacting the rubric: Teacher improvements in windows of high-stakes observation. Education Finance and Policy, 16(2), 283–312. https://doi.org/10.1162/edfp_a_00295
74.
PiantaR. C.LaParoK. M.HamreB. K. (2007). Classroom Assessment Scoring System (CLASS). Brookes.
75.
PlayerD.KatzV. (2016). Assessing school turnaround: Evidence from Ohio. Elementary School Journal, 116(4), 675–698. https://doi.org/10.1086/686467
76.
PolikoffM. S. (2012). Instructional alignment under No Child Left Behind. American Journal of Education, 118(3), 341–368. https://doi.org/10.1086/664773
77.
PolikoffM. S. (2015). How well aligned are textbooks to the Common Core Standards in mathematics?American Educational Research Journal, 52(6), 1185–1211. https://doi.org/10.3102/0002831215584435
RockoffJ. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94(2), 247–252. https://doi.org/10.1257/0002828041302244
80.
RonfeldtM.LoebS.WyckoffJ. (2013). How teacher turnover harms student achievement. American Educational Research Journal, 50(1), 4–36. https://doi.org/10.3102/0002831212463813
SchweigJ. D.KaufmanJ. H.OpferV. D. (2020). Day by day: Investigating variation in elementary mathematics instruction that supports the Common Core. Educational Researcher, 49(3), 167–187. https://doi.org/10.3102/0013189X20909812
83.
StosichE. L. (2016). Joint inquiry: Teachers’ collective learning about the Common Core in high-poverty urban schools. American Educational Research Journal, 53(6), 1698–1731. https://doi.org/10.3102/0002831216675403
SunM.PennerE.LoebS. (2017). Resource- and approach-driven multidimensional change: Three-year effects of school improvement grants. American Educational Research Journal, 54(4), 607–543. https://doi.org/10.3102/0002831217695790
TroiaG. A.GrahamS. (2016). Common Core writing and language standards and aligned state assessments: A national survey of teacher beliefs and attitudes. Reading and Writing, 29(9), 1719–1743. https://doi.org/10.1007/s11145-016-9650-z
WidamanK. F.FerrerE.CongerR. D. (2010). Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child Development Perspectives, 4(1), 10–18. https://doi.org/10.1111/j.1750-8606.2009.00110.x
91.
WigginsG. P.McTigheJ. (2005). Understanding by design (2nd ed.). ACSD.
92.
YuanK.LeV. (2012). Estimating the percentage of students who were tested on cognitively demanding items through the state achievement tests. RAND Corporation. https://www.rand.org/pubs/working_papers/WR967.html
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
