Developing a Psychometrically Sound Assessment of School Leadership: The VAL-ED as a Case Study

Abstract

Research has consistently shown that principal leadership matters for successful schools. Evaluating principals on the behaviors shown to improve student learning should be an important leverage point for raising leadership quality. Yet principals are often evaluated with the use of instruments with no theoretical background and little, if any, documented psychometric properties. To address this need, a team of researchers in principal leadership, assessment development, and psychometrics developed the Vanderbilt Assessment of Leadership in Education (VAL-ED). The purpose here is to report on iterative development work where the instrument was tested and revised across several cycles. Future work to investigate the instrument’s psychometric properties is identified. After an extensive item writing and instrument development phase, the authors embarked on a series of studies designed to guide improvements to the instrument. These studies include a sorting study, two rounds of cognitive interviews, a bias review, and two rounds of small- scale pilot tests. Results and implications from each study are discussed. The iterative development process helped improve the clarity of instructions and items while building a growing collection of preliminary validity and reliability evidence. At the end of the development process, the VAL-ED represents a promising instrument for assessing principal instructional leadership.The VAL- ED also represents a tool for possible use by principal leadership researchers in measuring the effectiveness of school principals.