Despite recent shifts in research emphasizing the value of carefully designed experiments, the number of studies of teacher professional development with rigorous designs has lagged behind its student outcome counterparts. We outline a framework for the design of group randomized trials (GRTs) with teachers’ knowledge as the outcome and consider mathematics and reading knowledge outcomes designed to assess the types of content problems that teachers encounter in practice. To estimate design parameters, we draw on a national sample of teachers for mathematics and a state Reading First sample to estimate for reading. Our results suggest that there is substantial clustering of teachers’ knowledge within schools and professional development GRTs will likely need increased sample sizes to account for this clustering.
BallD. L.BassH. (2003). Toward a practice-based theory of mathematical knowledge for teaching. In DavisB.SimmtE. (Eds.), Proceedings of the 2002 annual meeting of the Canadian Mathematics Education Study Group (pp. 3-14). Edmonton, Alberta, Canada: CMESG.
2.
BallD. L.CohenD. K. (1999). Developing practice, developing practitioners: Toward a practice-based theory of professional education. In HammondL. D.SykesG. (Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 3-32). San Francisco: Jossey-Bass.
3.
BallD. L.ThamesM.PhelpsG. (2008). Content knowledge for teaching: What makes it special?Journal of Teacher Education, 59(5), 389-407.
4.
BirmanB.LeFlochK. C.KlekotkaA.LudwigM.TaylorJ.WaltersK.WayneA.YoonK. S. (2007). State and local implementation of the No Child Left Behind Act, volume II—Teacher quality under NCLB: Interim report. Washington, DC: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service.
5.
BloomH. (2005). Randomizing groups to evaluate place-based programs. In BloomH. S. (Ed.), Learning more from social experiments: Evolving analytic approaches (pp. 115-172). New York: Russell Sage Foundation.
6.
BloomH. S.Richburg-HayesL.BlackA. (2007). Using covariates to improve precision for studies that randomize schools to evaluate educational interventions. Educational Evaluation and Policy Analysis, 29, 30-59.
7.
BorkoH. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33(8), 3-15.
8.
BrownE. T.McGathaM.KarpK. (2006). Assessing teacher knowledge: Diagnostic assessments for elementary and middle school teachers. New England Journal of Mathematics, May, 37-50.
9.
CarlisleJ.KelceyB.RowanB.PhelpsG. (2011). Teachers’ knowledge about early reading: Effects on students’ gains in reading achievement. Journal for Research on Educational Effectiveness, 4, 289-321.
10.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
11.
CorrentiR. (2007). An empirical investigation of professional development effects on literacy instruction using daily logs. Educational Evaluation and Policy Analysis, 29(4), 262-295.
12.
CorrentiR.PhelpsG. (2010). Investigating the relationship between teachers’ knowledge, literacy practice and growth in student learning. Paper presented at the Annual Meeting of the American Educational Research Association, Denver, CO.
13.
CunninghamA. E.PerryK. E.StanovichK. E.StanovichP. J. (2004). Disciplinary knowledge of K-3 teachers and their knowledge calibration in the domain of early literacy. Annals of Dyslexia, 54(1), 139-167.
14.
Darling-HammondL.SykesG. (Eds.). (1999). Teaching as the learning profession: Handbook of policy and practice. San Francisco: Jossey-Bass.
15.
DesimoneL. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181-199.
16.
DesimoneL. M.PorterA. C.GaretM.YoonK. S.BirmanB. (2002). Does professional development change teachers’ instruction? Results from a three-year study. Educational Evaluation and Policy Analysis, 24(2), 81-112.
17.
DonnerA.KovalJ. J. (1982). Design considerations in the estimation of intraclass correlation. Annals of Human Genetics, 46, 271-277.
18.
GaretM. S.CronenS.EatonM.KurkiA.LudwigM.JonesW.…L.SilverbergM. (2008). The impact of two professional development interventions on early reading instruction and achievement (U.S. Department of Education Report NCEE 2008-4030). Washington, DC: U.S. Department of Education.
19.
GaretM. S.PorterA. C.DesimoneL. M.BirmanB.YoonK. S. (2001). What makes professional development effective? Analysis of a national sample of teachers. American Educational Research Journal, 38(3), 915-945.
20.
GaretM. S.WayneA. J.StancavageF.TaylorJ.EatonM.WaltersK.…DoolittleF. (2011). Middle school mathematics professional development impact study: Findings after the second year of implementation (U.S. Department of Education Report NCEE 2011-4024). Washington, DC: U.S. Department of Education.
21.
GoldschmidtP.PhelpsG. (2010). Does teacher professional development affect content and pedagogical knowledge: How much and for how long?Economics of Education Review, 29(3), 432-439.
22.
HedgesL.HedbergE. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60-87.
23.
HillC. H. (2009). Fixing teacher professional development. Phi Delta Kappan, 90(7), 470-476.
24.
HillH. C.BallD. L. (2004). Learning mathematics for teaching: Results from California’s Mathematics professional development institutes. Journal of Research in Mathematics Education, 35, 330-351.
25.
HillH. C.BlunkM. L.CharalambousY.LewisJ. M.PhelpsG.SleepL.BallD. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26, 430-511.
26.
HillH. C.RowanB.BallD. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42(2), 371-406.
27.
HillH. C.SchillingS.BallD. L. (2004). Developing measures of teachers’ mathematics knowledge for teaching. Elementary School Journal, 105(1), 11-30.
28.
HinkleyD. (1988). Bootstrap methods. Journal of the Royal Statistical Society B, 50, 321-337.
29.
HollandP. (1986). Statistics and causal inference. Journal of the American Statistical Associations, 81(396), 947.
30.
The Holmes Group. (1986). Tomorrow’s teachers: A report of the Holmes Group. East Lansing, MI: Author.
31.
IngersollR. (2001). Deprofessionalizing the teaching profession: The problem of out-of-field teaching. Educational Horizons, 80(1), 28-31.
32.
Institute of Education Sciences Education Research Grants. (2012). Institute of Education Sciences Education Research Grants Request for Applications for awards beginning in Fiscal Year 2013: CFDA Number 84.305A. U.S. Department of Education.
33.
JacobR.ZhuP.BloomH. (2010). New empirical evidence for the design of group randomized trials in education. Journal of Research on Educational Effectiveness, 3, 157-198.
34.
JeraldC. (2002). All talk, no action: Putting an end to out-of-field teaching, The Education Trust, 9, 1-14.
35.
KainJ.SingletonK. (1996). Equality of educational opportunity revisited. New England Economic Review, May/June, 109.
36.
KelceyB. (2011). Assessing the effects of teachers’ reading knowledge on students’ achievement using multilevel propensity score stratification. Educational Evaluation & Policy Analysis, 33, 458-482.
37.
KennedyM. M. (1998). Form and substance in in-service teacher education (Research Monograph No. 13). Arlington, VA: National Science Foundation.
38.
KreiM. S. (1998). Intensifying the barriers—The problem of inequitable teacher allocation in low-income urban schools. Urban Education, 33, 71-94.
39.
MoatsL. (1994). The missing foundation in teacher education: Knowledge of the structure of spoken and written language. Annals of Dyslexia, 44, 81-102.
40.
National Commission on Teaching and America’s Future. (1997). Doing what matters most: Investing in quality teaching. New York: Author.
41.
No Child Left Behind Act of 2001. (2002). Public Law #107-110, 115 Stat.1425.
42.
NyeB.KonstantopoulousS.HedgesL. V. (2004). How large are teacher effects?Education Evaluation and Policy Analysis, 26, 327-257.
43.
PhelpsG. (2009). Just knowing how to read isn’t enough! What teachers know about the content of reading. Educational Assessment, Evaluation, and Accountability, 21(2), 137-154.
44.
PhelpsG.SchillingS. (2004). Developing measures of content knowledge for teaching reading. Elementary School Journal, 105(1), 31-48.
45.
PutnamR.BorkoH. (1997). Teacher learning: Implications of new views of cognition. In BiddleB. J.GoodT. L.GoodsonI. F. (Eds.), The international handbook of teachers and teaching (pp. 1223-1296). Dordrecht, the Netherlands: Kluwer.
46.
RaudenbushS. W.BrykA. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
SmithP. S.BanilowerE. B. (2006, April). Measuring teachers’ knowledge for teaching force and motion concepts. Paper presented at the annual meeting of the National Association for Research in Science Teaching, San Francisco.
49.
Spear-SwerlingL.BruckerP. O. (2006). Teacher-education students’ reading abilities and their knowledge about word structure. Teacher Education and Special Education, 29(2), 116-126.
50.
SpybrookJ.RaudenbushS. W. (2009). An examination of the precision and technical accuracy of the first wave of group randomized trials funded by the Institute of Education Sciences. Educational Evaluation and Policy Analysis, 31(3), 298-318.
51.
WayneA. J.YoonK. S.ZhuP.CronenS.GaretM. S. (2008). Experimenting with teacher professional development: Motives and methods. Educational Researcher, 37(8), 469-479.
52.
WeiR. C.Darling-HammondL.AndreeA.RichardsonN.OrphanosS. (2009). Professional learning in the learning profession: A status report on teacher development in the United States and abroad. Dallas, TX: National Staff Development Council.
53.
WilsonS. M.BerneJ. (1999). Teacher learning and the acquisition of professional knowledge: An examination of research on contemporary professional development. Review of Research in Education, 24, 173-209.
54.
YoonK. S.DuncanT.LeeS. W.-Y.ScarlossB.ShapleyK. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, No. 033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest.
