This article presents the validity bias review feedback and outcomes of new performance-based assessments to evaluate candidates seeking principal licensure. Until now, there has been little empirical work on performance assessment for principal licensure. One state recently developed a multi-task performance assessment for leaders and has implemented it for statewide use in initial principal licensure decisions; this development process is described here, focusing on content validity and bias review, and incorporates candidate and program faculty validiation as well. The results demonstrate the content validity, relevance, and feasibility of this new performance assessment for leaders, and yield implications for leader assessment generally.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2.
AugustineC. H.GonzalezG.IkemotoG. S.RussellJ.ZellmanG. L.ConstantL.. . . DemboskyJ. W. (2009). Improving school leadership: The promise of cohesive leadership systems. Santa Monica, CA: Rand Corporation.
3.
BriggsK.CheneyG. R.DavisJ.MollK. (2013). Operating in the dark: What outdated state policies and data gaps mean for effective school leadership. Dallas, TX: George W. Bush Institute.
4.
BrykA. S.SebringP. B.AllensworthE.LuppescuS.EastonJ. Q. (2010). Organizing schools for improvement: Lessons from Chicago. Chicago, IL: University of Chicago Press.
5.
CliffordM.MenonR.GangiT.CondonC.HorungK. (2012). Measuring school climate for gauging principal performance: A review of the validity and reliability of publicly accessible measures. Washington, DC: American Institutes for Research.
6.
CondonC.CliffordM. (2010). Measuring principal performance: How rigorous are commonly used principal performance assessment instruments?Naperville, IL: Learning Points Associates.
7.
DavisS.EricksonD. E.KinseyG. W.Moore-StewardT.PadoverW.ThomasC.. . . WiseD. (2011). Reforming the California Public School Administrator Licensure System through the alignment of research, policy, and practice: Policy perspectives and recommendations from the California Association of Professors of Educational Administration (CAPEA). Education Leadership and Administration, 22, 66-82.
8.
DuckorB.CastellanoK. E.TelleK.WihardiniD.WilsonM. S. (2014). Examining the internal structure evidence for the Performance Assessemnt for California Teachers: A validation study of the elementary literacy teaching event for Tier I teacher license. Journal of teacher education, 65(5), 402-420.
9.
Educational Testing Service. (2009). Multi-state standard setting report: School Leaders Licensure Assessment (SLLA). Princeton, NJ: Author.
10.
Educational Testing Service. (n.d.). School leadership series: State requirements. Retrieved from https://www.ets.org/sls/states/
11.
GitomerD. H. (1993). Performance assessment and educational measurement. In WardW. C.BennettR. E. (Eds.), Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment (pp. 241-264). Hillsdale, NJ: Lawrence Erlbaum.
12.
GrissomJ. A.MitaniH.BlissettR. S. (2017). Principal licensure exams and future job performance: Evidence from the School Leaders Licensure Assessment. Educational Evaluation and Policy Analysis, 20(10), 1-33.
13.
HayesD.ChristieP.MillsM.LingardB. (2004). Productive leaders and productive leadership: Schools as learning organisations. Journal of Educational Administration, 42, 520-538.
14.
HoldenR. B. (2010). Face validity. In WeinerI. B.CraigheadW. E. (Eds.), The Corsini encyclopedia of psychology (4th ed., pp. 637-638). Hoboken, NJ: Wiley.
15.
KayeL. S. (2016). Requirements for certification of teachers, counselors, librarians, administrators for elementary and secondary schools, eighty-first edition, 2016-2017. Chicago, IL: University of Chicago Press.
16.
KhattriN.SweetD. (1996). Assessment reform: Promises and challenges. In KaneM. B.MitchellR. (Eds.), Implementing performance assessment: Promises, problems, and challenges (pp. 1-22). Mahwah, NJ: Lawrence Erlbaum Associates.
17.
KolbD. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.
18.
LaneS. (2010). Performance assessment: The state of the art. Stanford, CA:Stanford Center for Opportunity Policy in Education.
19.
LeithwoodK.JantziD. (2008). Linking leadership to student learning: The contributions of leader efficacy. Educational Administration Quarterly, 44, 496-528.
20.
LinnR. L.BakerE. L.DunbarS. B. (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20(8), 15-21.
21.
LitI.LotanR. (2013). A balancing act: Dilemmas of implementing a high-stakes performance assessment. The New Educator, 9, 54-76.
22.
LouisK. S.LeithwoodK.WahlstromK. L.AndersonS. E. (2010). Investigating the links to improved student learning: Final report of research findings. Minneapolis, MN: University of Minnesota.
23.
MartineauJ. (2004). Laying the groundwork: First steps in evaluating leadership development. Leadership in Action, 23(6), 3-8.
24.
MayH.SupovitzJ. A. (2011). The scope of principal efforts to improve instruction. Educational Administration Quarterly, 47, 332-352.
25.
MessickS. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.
26.
MessickS. (2005). Standards of validity and the validity of standards in performance assessment. Educational Measurement: Issues and Practice, 14(4), 5-9.
27.
MezirowJ. (2000). Learning as transformation: Critical perspectives on a theory in progress. San Francisco, CA: Jossey-Bass.
28.
PecheoneR. L.ChungR. R. (2006). Evidence in teacher education: The Performance Assessment for California Teachers. Journal of Teacher Education, 57, 22-36.
29.
PecheoneR. L.ShearB. W. A.Darling HammondL. (2013). 2013 edTPA Field Test: Summary report. Palo Alto, CA: Stanford Center for Assessment, Learning and Equity.
30.
PecheoneR. L.WeiR. C. (2007). Technical report of the Performance Assessment for California Teachers (PACT): Summary of validity and reliability studies for the 2003-2004 pilot year. Palo Alto, CA :Stanford Center for Assessment, Learning and Equity.
31.
PophamW. J. (2012). Assessment bias: How to banish it. Boston, MA: Pearson.
32.
RobinsonV. M. J.LloydC. A.RoweK. J. (2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44, 635-674.
33.
SebringP. B.AllensworthE.BrykA. S.EastonJ. Q.LuppescuS. (2006). The essential supports for school improvement. Chicago, IL: Consortium on Chicago School Research at the University of Chicago.
34.
SheltonS. V. (2011). Strong leaders strong schools: 2010 school leadership laws. Denver, CO: National Conference of State Legislatures.
35.
SheltonS. V. (2012). Preparing a pipeline of effective principals: A legislative approach. Denver, CO: National Conference of State Legislatures.
36.
StigginsR. (1987). Design and development of performance assessments. Educational Measurement: Issues and Practice, 6(3), 33-42.
37.
SunC. (2011). School leadership: Improving state systems for leader development. Arlington, VA: National Association of State Boards of Education.
38.
SupovitzJ. A.ChristmanJ. B. (2005). Small learning communities that actually learn: Lessons for school leaders. Phi Delta Kappan, 86, 649-651.
39.
TurnbullB. J.RileyD. L.ArcairaE. R.AndersonL. M.MacFarlaneJ. R. (2013). Six districts begin the principal pipeline initiative. Washington, DC: Policy Studies Associates.
40.
The Wallace Foundation. (2006). Leadership for learning: Making the connections among state, district, and school policies and practices. New York, NY: Author.
41.
U.S. Department of Education. (2009). Race to the top program executive summary. Washington, DC: Author.
42.
WeiR. C.PecheoneR. L. (2010). Performance-based assessments as high-stakes events and tools for learning. In KennedyM. M. (Ed.), Handbook of teacher assessment and teacher quality (pp. 69-132). San Francisco, CA: Jossey-Bass.
43.
WeissH. B.StephenN. (2009). From periphery to center: A new vision for family, school, and community partnerships. In ChristensonS.ReschleyA. (Eds.), Handbook of school-family partnerships (pp. 448-472). New York, NY: Routledge.
44.
WilsonF. R.PanW.SchumskyD. A. (2012). Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development, 45, 197-210.
45.
YoungM. E.CruessS. R.CruessR. L.SteinertY. (2014). The Professionalism Assessment of Clinical Teachers (PACT): The reliability and validity of a novel tool to evaluate professional and clinical teaching behaviors. Advances in Health Sciences Education, 19, 99-113.
