States and districts are under increasing pressure to evaluate the effectiveness of their teachers and to ensure that all students receive high-quality instruction. This article describes some of the challenges associated with current effectiveness approaches, including paper-and-pencil tests of pedagogical content knowledge, classroom observation systems, and value-added models. It proposes development of a new teacher evaluation system using a virtual reality environment and describes how innovations in educational measurement and technology can be used to develop an improved teacher effectiveness measure.
AdairG. (1984). The Hawthorne effect: A reconsideration of the methodological artifact. Journal of Applied Psychology, 69, 334-345.
2.
American Recovery and Reinvestment Act of 2009: P. L. 111-5, as signed by the President on February 17, 2009: Law, explanation, and analysis. (2009). Chicago, IL: CCH.
3.
BakerE. L. (2008, April). Empirically determining the instructional sensitivity of an accountability test. Paper presented at the annual meeting of the American Educational Research Association, New York, NY.
4.
BallD. L.ThamesM. H.PhelpsG. (2008). Content knowledge for teaching: What makes it special?Journal of Teacher Education, 59, 389-407.
5.
BrophyJ. E.GoodT. L. (1986). Teacher behavior and student achievement. In WhitrockM. C. (Ed.), Handbook of research on teaching (pp. 328-375). New York, NY: Macmillan.
6.
CampbellD. T.KennyD. A. (1999). A primer of regression artifacts. New York, NY: Guilford.
7.
CampbellR. J.KyriakidesL.MujisR. D.RobinsonW. (2004). Differential teacher effectiveness: Towards a model for research and teacher appraisal. Oxford Review of Education, 29, 347-362.
8.
ChismN. V. N. (2007). Peer review of teaching: A sourcebook (2nd ed.). Bolton, MA: Anker.
9.
Clarke-MiduraJ.CodeJ.MayrathM.DedeC. (2011, April). Exploring inquiry processes in virtual performance assessments. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
10.
D’AgostinoJ. V.PowersS. J. (2009). Predicting teacher performance with test scores and grade point average: A meta-analysis. American Educational Research Journal, 46, 146-182.
11.
D’AgostinoJ. V.WelshM. E.CorsonN. M. (2007). Instructional validity of a state’s standards-based assessment. Educational Assessment, 12, 1-22.
12.
DanielsonC.McGrealT. L. (2000). Teacher evaluation to enhance professional practice. Princeton, NJ: Association for Supervision and Curriculum Development.
DomaleskiC.HillR. (2010). Considerations for using assessment data to inform determinations of teacher effectiveness. Dover, NH: National Center for Improving Educational Assessment. Retrieved from http://www.nciea.org/papers-UsingAssessmentData4-29-10.pdf
EricssonK. A.SimonH. A. (1993). Protocol analysis. Cambridge, MA: MIT Press.
17.
Federation of State Medical Boards of the United States & National Board of Medical Examiners. (2011). United States Medical Licensing Exam: 2011 Bulletin of information. Philadelphia, PA: Author. Retrieved from http://www.usmle.org/General_Information/bulletin/2011/2011%20BOI.pdf.
GoeL.BellC.LittleO. (2008). Approaches to evaluating teacher effectiveness: A research synthesis. Princeton, NJ: Educational Testing Service.
20.
GoeL.HoldheideL. (2011). Measuring teachers’ contributions to student learning growth for nontested grades and subjects. Washington, DC: National Comprehensive Center on Teacher Quality.
21.
GoldhaberD.AnthonyE. (2007). Can teacher quality be effectively assessed? National Board Certification as a signal of effective teaching. Review of Economics and Statistics, 89, 134-150.
22.
GoldhaberD.HansenM. (2010). Race, gender, and teacher testing: How objective a tool is teacher licensure testing?American Educational Research Journal, 47, 218-251.
23.
GordonR.KaneT. J.StaigerD. O. (2009). Identifying effective teachers using performance on the job. Washington, DC: Brookings Institution.
24.
HambletonR. H.JonesR. W. (1993). An NCME instructional module on comparison of classical test theory and item response theory and their application to test development. Educational Measurement: Issues and Practice, 12, 38-47.
25.
HenemanH. G.MilanowskiA.KimballS. M.OddenA. (2006). Standards-based teacher evaluation as a foundation for knowledge and skill-based pay (CPRE Research Report RB-45). Philadelphia: University of Pennsylvania, Consortium for Policy Research in Education. Retrieved from http://www.cpre.org/images/stories/cpre_pdfs/RB45.pdf
26.
HillH. C.BlunkM.CharalambousC.LewisJ.PhelpsG. C.SleepL.BallD. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26, 430-511.
27.
HillH. C.KapitulaL.UmlandK. (2011). A validity argument approach to evaluating teacher value-added scores. American Educational Research Journal, 48, 794-831.
28.
Johnson-GlenbergM. C.BirchfieldD.SavvidesP.Megowan-RomanowiczC. (2010). Semi-virtual embodied learning-real world STEM assessment. In AnnettaL.BronackS. (Eds.), Serious educational game assessment: Practical methods and models for educational games, simulations and virtual worlds (pp. 225-241). Rotterdam, Netherlands: Sense.
KromreyJ.RenfrowD. (1991, February). Using multiple choice examination items to measure teachers’ content-specific pedagogical knowledge. Paper presented at the annual meeting of the Eastern Educational Research Association, Boston, MA.
31.
LeightonJ. P. (2009, April). Two types of think aloud interviews for educational measurement: Protocol and verbal analysis. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego, CA.
32.
LockwoodJ. R.LouisT. A.McCaffreyD. F. (2002). Uncertainty in rank estimation: Implications for value-added modeling accountability systems. Journal of Educational and Behavioral Statistics, 27, 255-270.
33.
LohmanD. F.KorbK. A. (2006). Gifted today but not tomorrow? Longitudinal changes in ability and achievement during elementary school. Journal for the Education of the Gifted, 29, 451-484.
34.
McBeeM. (2010). Modeling outcomes with floor or ceiling effects: An introduction to the Tobit model. Gifted Child Quarterly, 54, 314-320.
35.
McCaffreyD. F.KoretzD. M.LockwoodJ. R.HamiltonL. S. (2003). Evaluating value-added models for teacher accountability. Santa Monica, CA: Rand.
36.
McCaffreyD. F.LockwoodJ. R.KoretzD. M.LouisT. A.HamiltonL. S. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29, 67-101.
37.
MeyerR. H. (2010, December). Value-added systems, accountability, and performance management in education: Promises and pitfalls. Paper presented at the 2010 Annual Conference of the Iowa Educational Research and Evaluation Association, Cedar Falls.
MislevyR. J.HaertelG. D. (2006). Implications of evidence-centered design for educational testing. Educational Measurement: Issues and Practice, 25, 6-20.
40.
MislevyR. J.RiconscenteM. M. (2006). Evidence-centered assessment design: Layers, concepts, and terminology. In DowningS.HaladynaT. (Eds.), Handbook of test development (pp. 61-90). Mahwah, NJ: Erlbaum.
41.
MujisD. (2006). Measuring teacher effectiveness: Some methodological reflections. Educational Research and Evaluation, 12, 53-74.
42.
No Child Left Behind Act of 2001, Pub. L. No. 107-110, § 115, Stat. 1425. (2002).
43.
PapayJ. P. (2011). Different tests, different answers: The stability of teacher value-added estimates across outcome measures. American Educational Research Journal, 48, 163-193.
44.
PetersS. J.GatesJ. C. (2010). The teacher observation form: Revisions and updates. Gifted Child Quarterly, 54, 179-188.
45.
PhelpsL.SchmitzC. D.BoatrightB. (1986). The effects of halo and leniency on cooperating teacher reports using Likert-type rating scales. Journal of Educational Research, 79, 151-154.
46.
PiantaR. C.HamreB. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38, 109-119.
47.
PolikoffM. S. (2010). Instructional sensitivity as a psychometric property of assessments. Educational Measurement: Issues and Practice, 29, 3-14.
48.
PophamJ. W. (2007). Instructional insensitivity of tests: Accountability’s dire drawback. Phi Delta Kappan, 89, 146-155.
49.
PrinceC. D.SchuermannP. J.GuthrieJ. W.WithamP. J.MilanowskiA. T.ThornC. A. (2009). The other 69 percent: Fairly rewarding the performance of teachers of nontested subjects and grades. Washington, DC: Center for Educator Compensation Reform. Retrieved from http://www.cecr.ed.gov/guides/other69Percent.pdf
50.
RamboK. E. (2011). How much do schools matter? Using summer growth patterns to assess the impact of schools on high achieving and gifted students (Unpublished doctoral dissertation). University of Connecticut, Storrs.
51.
ReisS. M.McCoachD. B. (2000). The underachievement of gifted students: What do we know and where do we go?Gifted Child Quarterly, 44, 152-170.
52.
ReisS. M.RenzulliJ. S. (2011). Challenging gifted and talented learners with a continuum of research-based intervention strategies. In KehleT. J.BrayM. A.NathanP. E. (Eds.), Oxford handbook of school psychology (pp. 456-482). Oxford, UK: Oxford University Press.
53.
RobinsonA.ShoreB.EnersonD. (2007). Best practices in gifted education: An evidence-based guide. Waco, TX: Prufrock.
54.
RowanB.SchillingS. G.BallD. L.MillerR. (2001). Measuring teachers’ pedagogical content knowledge in surveys: An exploratory study. Ann Arbor: University of Michigan.
55.
SandersW. L.WrightS. P.HornS. P. (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education, 11, 57-67.
56.
ShulmanL. S. (1987). Knowledge and teaching: Foundation of the new reform. Harvard Educational Review, 57, 1-22.
57.
StrongeJ. H.WardT. J.TuckerP. D.HindmanJ. L.McColskyW.HowardB. (2007). National Board certified teachers and non-National Board certified teachers: Is there a difference in teacher effectiveness and student achievement?Journal of Personnel Evaluation in Education, 20, 185-210.
58.
TomlinsonC. (2005). Quality curriculum and instruction for highly able students. Theory Into Practice, 44, 160-166.
59.
VanTassel-BaskaJ.AveryL.StruckJ.FengA.BrackenB. A.DrummondD.. . . QuekC. (2005). Classroom Observation Scale–Revised: User’s manual. Williamsburg, VA: The Center for Gifted Education, College of William and Mary.
60.
WestbergK. L.ArchambaultF. X.Jr.DobynsS. M.SalvinT. (1993). An observational study of instructional and curricular practices used with gifted and talented students in regular classrooms (Research Monograph 93104). Storrs: The National Research Center on the Gifted and Talented, University of Connecticut.
