We investigate the predictive validity of teacher licensure tests using data from the Massachusetts Tests for Educator Licensure (MTEL). MTEL scores predict teachers’ in-service performance ratings and contributions to student test scores (i.e., value added). We then explore whether these relationships vary for teacher candidates of color. Although candidates of color have lower first-time pass rates and are less likely to retake licensure tests, we do not find consistent evidence that MTEL scores are less predictive of student achievement gains. Finally, we find that some evidence that MTEL scores are more predictive of teacher performance ratings for teachers of color than for White teachers.
AngristJ. D.GuryanJ. (2008). Does teacher testing raise teacher quality? Evidence from state certification requirements. Economics of Education Review, 27(5), 483–503.
2.
Bacher-HicksA.ChiO. L.OrellanaA. (2022). Two years later: How COVID-19 has shaped the teacher workforce. Annenberg Institute at Brown University. https://doi.org/10.26300/T5H7-Y366
3.
Bacher-HicksA.ChinM. J.KaneT. J.StaigerD. O. (2019). An experimental evaluation of three teacher quality measures: Value-added, classroom observations, and student surveys. Economics of Education Review, 73.
4.
Bacher-Hicks.A.GoodmanJ. (2021). The Covid-19 pandemic is a lousy natural experiment for studying the effects of online learning. Education Next, 21(4), 38–42.
5.
BackesB.CowanJ. (2019). Is the pen mightier than the keyboard? The effect of online testing on measured student achievement. Economics of Education Review, 68, 89–103. https://doi.org/10.1016/j.econedurev.2018.12.007
6.
BackesB.HansenM. (2018). The impact of Teach For America on non-test academic outcomes. Education Finance and Policy, 13(2), 168–193.
7.
BackesB.CowanJ.GoldhaberD.TheobaldR. (2023). How to measure a teacher: The influence of test and nontest value-added on long-run student outcomes. CALDER Working Paper No. 270-0423-2
8.
BakerS. (1995). Testing equality: The National Teacher Examination and the NAACP's legal campaign to equalize teachers’ salaries in the south, 1936-63. History of Education Quarterly, 35(1), 49–64.
9.
BallouD.PodgurskyM. J. (1998). Teacher recruitment and retention in public and private schools. Journal of Policy Analysis and Management, 17(3), 393–417.
10.
BartanenB.KwokA. (2021). Examining clinical teaching observation scores as a measure of preservice teacher quality. American Educational Research Journal, 58(5), 887–920.
11.
BennettC. I.McWhorterL. M.KuykendallJ. A. (2006). Will I ever teach? Latino and African American students’ perspectives on PRAXIS I. American Educational Research Journal, 43(3), 531–575. https://doi.org/10.3102/00028312043003531
12.
BonhommeS.ManresaE. (2015). Grouped patterns of heterogeneity in panel data. Econometrica, 83(3), 1147–1184.
Boston Public Schools. (2020). Recruitment, cultivation, and diversity initiatives. Boston Public Schools.
15.
BoydD.GoldhaberD.LankfordH.WyckoffJ. (2007). The effect of certification and preparation on teacher quality. The Future of Children, 17(1), 45–68.
16.
BoydD.LankfordH.LoebS.WyckoffJ. (2013). Analyzing the determinants of the matching of public school teachers to jobs: Disentangling the preferences of teachers and employers. Journal of Labor Economics, 31(1), 83–117.
17.
BrownR. P.JosephsR. A. (1999). A burden of proof: Stereotype relevance and gender differences in math performance. Journal of Personality and Social Psychology, 76(2), 246–257. https://doi.org/10.1037/0022-3514.76.2.246
18.
Center for Constitutional Rights. (2012). Federal court rules against NYC board of ed: Teacher exam discriminated. https://ccrjustice.org/node/1045
19.
ChenB.CowanJ.GoldhaberD.TheobaldR. (2023). Assessing the predictive validity of the Massachusetts candidate assessment of performance. Educational Policy. Advance online publication. https://doi.org/10.1177/08959048231174876
20.
ChettyR.FriedmanJ. N.RockoffJ. E. (2014). Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104(9), 2593–2632. https://doi.org/10.1257/aer.104.9.2593
21.
ChungB. W.ZouJ. (2021). Teacher licensing, teacher supply, and student achievement: Nationwide implementation of edTPA. Annenberg Institute at Brown University. https://www.edworkingpapers.com/ai21-440
22.
ClotfelterC. T.LaddH. F.VigdorJ. L. (2007). Teacher credentials and student achievement: Longitudinal analysis with student fixed effects. Economics of Education Review, 26(6), 673–682.
23.
ClotfelterC. T.LaddH. F.VigdorJ. L. (2010). Teacher credentials and student achievement in high school: A cross-subject analysis with student fixed effects. Journal of Human Resources, 45(3), 655–681. https://doi.org/10.3368/jhr.45.3.655
CohenJ.HuttE.BerlinR. L.MathewsH. M.McGrawJ. P.GottliebJ. (2020). Sense making and professional identity in the implementation of edTPA. Journal of Teacher Education, 71(1), 9–23. https://doi.org/10.1177/0022487118783183
CowanJ.GoldhaberD. (2016). National Board Certification and teacher effectiveness: Evidence from Washington State. Journal of Research on Educational Effectiveness, 9(3), 233–258. https://doi.org/10.1080/19345747.2015.1099768
30.
CowanJ.GoldhaberD.TheobaldR. (2018). Massachusetts teacher preparation and licensure: Performance Review Program for initial licensure study. American Institutes for Research.
31.
CowanJ.GoldhaberD.TheobaldR. (2022). Performance evaluations as a measure of teacher effectiveness when implementation differs: Accounting for variation across classrooms, schools, and districts. Journal of Research on Educational Effectiveness, 15(3), 510–531. https://doi.org/10.1080/19345747.2021.2018747
32.
CowanJ.GoldhaberD.HayesK.TheobaldR. (2017). Missing elements in the discussion of teacher shortages. Educational Researcher, 45(8), 460–462.
33.
De VotoC.OlsonJ. D.GottliebJ. J. (2021). Examining diverse perspectives of edTPA policy implementation across states: The good, the bad, and the ugly. Journal of Teacher Education, 72(1), 42–55. https://doi.org/10.1177/0022487120909390
34.
DeeT. S. (2004). Teachers, race, and student achievement in a randomized experiment. Review of Economics and Statistics, 86(1), 195–210.
35.
DeeT. S. (2005). A teacher like me: Does race, ethnicity, or gender matter?American Economic Review, 95(2), 158–165.
36.
DeeT. S.DomingueB. W. (2019). Did a “traumatic” test question create racial bias?Stanford Center for Education Policy Analysis.
37.
DrakeS.AulettoA.CowenJ. M. (2019). Grading teachers: Race and gender differences in low evaluation ratings and teacher employment outcomes. American Educational Research Journal, 56(5), 1800–1833.
EgaliteA. J.KisidaB.WintersM. A. (2015). Representation in the classroom: The effect of own-race teachers on student achievement. Economics of Education Review, 45, 44–52.
40.
FlippoR. F.RiccardsM. P. (2000). Initial teacher certification testing in Massachusetts: A case of the tail wagging the dog. Phi Delta Kappan, 82(1), 34–37. https://doi.org/10.1177/003172170008200111
41.
FordK. S. (2019). Observer-identification: A potential threat to the validity of self-identified race and ethnicity. Educational Researcher, 48(6), 378–381. https://doi.org/10.3102/0013189X19860803
FutrellM. H.RobinsonS. P. (1986). Testing teachers: An overview of NEA's position, policy, and involvement. The Journal of Negro Education, 55(3), 397–404. https://doi.org/10.2307/2295110
44.
GershensonS.HansenM. J.LindsayC. A. (2021). Teacher diversity and student success: Why racial representation matters in the classroom. Cambridge: Harvard Education Press.
45.
GershensonS.HartC. M. D.HymanJ.LindsayC.PapageorgeN. W. (2018). The long-run impacts of same-race teachers (Working Paper No. 25254). National Bureau of Economic Research. https://doi.org/10.3386/w25254
46.
GitomerD. H.BrownT. L.BonettJ. (2011). Useful signal or unnecessary obstacle? The role of basic skills tests in teacher preparation. Journal of Teacher Education, 62(5), 431–445. https://doi.org/10.1177/0022487111412785
47.
GitomerD. H.MartínezJ. F.BatteyD.HylandN. E. (2021). Assessing the assessment: Evidence of reliability and validity in the edTPA. American Educational Research Journal, 58(1), 3–31. https://doi.org/10.3102/0002831219890608
48.
GoldhaberD. (2004). Why do we license teachers? In HessF. M.RotherhamA. J.WalshK. (Eds.), A qualified teacher in every classroom?Cambridge: Harvard Education Press.
49.
GoldhaberD. (2007). Everyone’s doing it, but what does teacher testing tell us about teacher effectiveness?Journal of Human Resources, 42(4), 765–794.
50.
GoldhaberD. (2011). Licensure: Exploring the value of this gateway to the teacher workforce. In HanushekE. A.MachinS.WoessmannL. (Eds.), Handbook of the economics of education (Vol. 3, pp. 315–339). Elsevier. https://doi.org/10.1016/B978-0-444-53429-3.00006-5
51.
GoldhaberD.HansenM. (2010). Race, gender, and teacher testing: How informative a tool is teacher licensure testing?American Educational Research Journal, 47(1), 218–251. https://doi.org/10.3102/0002831209348970
52.
GoldhaberD.TheobaldR. (2023). Teacher turnover three years into the pandemic era: Evidence from Washington State. CALDER Policy Brief No. 32.
53.
GoldhaberD.CowanJ.TheobaldR. (2017a). Evaluating prospective teachers: Testing the predictive validity of the edTPA. Journal of Teacher Education, 68(4), 377–393.
54.
GoldhaberD.GratzT.TheobaldR. (2017b). What’s in a teacher test? Assessing the relationship between teacher licensure test scores and student STEM achievement and course-taking. Economics of Education Review, 61, 112–129. https://doi.org/10.1016/j.econedurev.2017.09.002
55.
GrissomJ. A.LoebS. (2017). Assessing principals’ assessments: Subjective evaluations of teacher effectiveness in low-and high-stakes environments. Education Finance and Policy, 12(3), 369–395.
56.
Gulino v. Board of Education, City of New York, 236 F. Supp. 2d 314 (S.D.N.Y. 2002)
57.
HaneyW.FowlerC.WheelockA.BebellD.MalecN. (1999). Less truth than error: Massachusetts teacher tests. Education Policy Analysis Archives, 7, 4. https://doi.org/10.14507/epaa.v7n4.1999
58.
HarrisD. N.SassT. R. (2011). Teacher training, teacher quality and student achievement. Journal of Public Economics, 95(7–8), 798–812. https://doi.org/10.1016/j.jpubeco.2010.11.009
HeckmanJ. J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.
61.
HendricksM. (2015). Public schools are hemorrhaging talented teachers: Can higher salaries function as a tourniquet? Available at SSRN 2564703.
62.
JacksonC. K. (2018). What do test scores miss? The importance of teacher effects on non–test score outcomes. Journal of Political Economy, 126(5), 2072–2107.
63.
JacobB. A.RockoffJ. E.TaylorE. S.LindyB.RosenR. (2018). Teacher applicant hiring and teacher performance: Evidence from DC public schools. Journal of Public Economics, 166, 81–97.
64.
JacobB.RothsteinJ. (2016). The measurement of student ability in modern assessment systems. The Journal of Economic Perspectives, 30(3), 85–108.
65.
JencksC. (1998). Racial bias in testing. In JencksC.PhillipsM. (Eds.), The Black-White test score gap (pp. 55–85). Brookings Institution Press.
66.
KaneT. J.StaigerD. O. (2008). Estimating teacher impacts on student achievement: An experimental evaluation (No. w14607). National Bureau of Economic Research.
67.
KaneT. J.McCaffreyD. F.MillerT.StaigerD. O. (2013). Have we identified effective teachers?Bill and Melinda Gates Foundation.
68.
KraftM. A. (2019). Teacher effects on complex cognitive skills and social-emotional competencies. Journal of Human Resources, 54(1), 1–36.
69.
KraftM. A.PapayJ. P.ChiO. L. (2020). Teacher skill development: Evidence from performance ratings by principals. Journal of Policy Analysis and Management, 39(2), 315–347.
LarsenB.JuZ.KaporA.YuC. (2020). The effect of occupational licensing stringency on the teacher quality distribution (No. w28158). National Bureau of Economic Research. https://doi.org/10.3386/w28158
73.
LudlowL. H.ShirleyD.RoscaC. (2002). Besieged institutions and the Massachusetts teacher tests. Education Policy Analysis Archives, 10, 50. https://doi.org/10.14507/epaa.v10n50.2002
74.
MelnickS. L.PullinD. (2000). Can you take dictation?: Prescribing teacher quality through testing. Journal of Teacher Education, 51(4), 262–275. https://doi.org/10.1177/0022487100051004002
75.
MitchellK. J.RobinsonD. Z.PlakeB. S.KnowlesK. T. (Eds.). (2001). Testing teacher candidates: The role of licensure tests in improving teacher quality. National Academy Press.
76.
MoreiraD.PérezS. (2021). Civil service reform and organizational practices: Evidence from the Pendleton Act (Working Paper No. 28665). National Bureau of Economic Research. https://doi.org/10.3386/w28665
77.
MotamediJ. G.LeongM.HansonH. (2018). Potential testing barriers for teacher candidates of color. REL Northwest.
78.
National Research Council. (2001). Testing teacher candidates: The role of licensure tests in improving teacher quality. National Academies Press. https://doi.org/10.17226/10090
79.
NettlesM. T.ScattonL. H.SteinbergJ. H.TylerL. L. (2011). Performance and passing rate differences of African American and White prospective teachers on PraxisTM examinations (ETS RR-11-08). NEA and ETS. https://www.ets.org/Media/Research/pdf/RR-11-08.pdf
80.
NiederleM.VesterlundL. (2010). Explaining the gender gap in math test scores: The role of competition. Journal of Economic Perspectives, 24(2), 129–144. https://doi.org/10.1257/jep.24.2.129
81.
Office of Management and Budget. (1997). Revisions to the standards for the classification of federal data on race and ethnicity. Federal Register, 62(210), 58782–58790.
PetchauerE.BoweA. G.WilsonJ. (2018). Winter is coming: Forecasting the impact of edTPA on Black teachers and teachers of color. The Urban Review, 50(2), 323–343. https://doi.org/10.1007/s11256-018-0453-1
86.
PutmanH.WalshK. (2021). Driven by data: Using licensure tests to build a strong, diverse teacher workforce. National Council on Teacher Quality.
RockoffJ. E.LockwoodB. B. (2010). Stuck in the middle: Impacts of grade configuration in public schools. Journal of Public Economics, 94(11–12), 1051–1061.
89.
RothsteinJ. (2009). Student sorting and bias in value-added estimation: Selection on observables and unobservables. Education Finance and Policy, 4(4), 537–571.
90.
RucinskiM.GoodmanJ. (2019) Racial diversity in the teacher pipeline: Evidence from Massachusetts. Harvard Kennedy School Policy Brief.
91.
RutledgeS. A.HarrisD. N.ThompsonC. T.IngleW. K. (2008). Certify, blink, hire: An examination of the process and tools of teacher screening and selection. Leadership and Policy in Schools, 7(3), 237–263. https://doi.org/10.1080/15700760701822132
92.
SassT. R. (2015). Licensure and worker quality: A comparison of alternative routes to teaching. The Journal of Law and Economics, 58(1), 1–35.
TaieS.GoldringR. (2020). Characteristics of public elementary and secondary school teachers in the United States: Results from the 2015–16 national teacher and principal survey first look (NCES 2017-072rev2). U.S. Department of Education. National Center for Education Statistics. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2017072rev2
TillmanL. C. (2004). (Un)intended consequences? The impact of the Brown v. Board of Education decision on the employment status of Black educators. Education and Urban Society, 36(3), 280–303.
99.
TylerL.WhitingB.FergusonS.EubanksS.SteinbergJ.ScattonL.BassettK. (2011). Toward increasing teacher diversity: Targeting support and intervention for teacher licensure candidates. NEA and ETS. https://www.ets.org/Media/Research/pdf/ETS-NEA-2011-01.pdf
100.
U.S. Department of Education. (2017). National Teacher and Principal Survey (NTPS) [“Public School Teacher Data File,” 2015-16]. National Center for Education Statistics, Schools and Staffing Survey.
101.
Van CleefV. (2022). Licensure tests as barriers to the profession. The State Education Standard, 22(3), 24–27.
VianoS.BakerD. J. (2020). How administrative data collection and analysis can better reflect racial and ethnic identities. Review of Research in Education, 44(1), 301–331. https://doi.org/10.3102/0091732X20903321
Wynter-HoyteK.LongS.McadooT. M.StricklandJ. D. (2020). “Losing one African American preservice teacher is one too many”: A critical race analysis of support for Praxis Core as African American students speak out. Teachers College Record, 122(11), 1–34. https://doi.org/10.1177/016146812012201107
