We collected data from 691 teachers from 139 schools in Michigan to validate the instrument titled Data-informed Decision-making on High-impact Strategies, designed based on Marzano's (2003) 11 high-impact strategies. Results of confirmatory factor analysis strongly supported the 11-factor model as the most valid approach to measure teachers’ perceptions of their principals’ data-informed decision-making on high-impact strategies for the improvement of student achievement. According to Marzano's (2003) framework, the 11 factors fall into three categories of higher-order principal leadership concerning schools, teachers, and students. The higher-order-factor model came quite close to the 11-factor model in terms of model-data-fit so that the instrument could be used with satisfactory construct validity in the research of higher-order principal leadership. The scale property analysis showed favorable results for the 11-factor model and the higher-order-factor model. Implications of the results are discussed.
Alig-MielcarekJ. M., & HoyW. (2005). Instructional leadership: Its nature, meaning and influence. In MiskelC. G., & HoyW. K. (Eds.), Educational leadership and reform (pp. 29–52). Greenwich, CT: Information Age Publishing.
2.
AmsterdamC. E., JohnsonR. L., MonradD. M., & TonnsenS. L. (2003). A collaborative approach to the development and validation of a principal evaluation system: A case study.Journal of Personnel Evaluation in Education,17(3), 221–242.
3.
AndersonM. E. (1989). Evaluating principals: Strategies to assess their performance.Eugene: Oregon School Study Council.
4.
BernhardtV. (2005). Data tools for school improvement.Educational Leadership,62(5), 66–69.
5.
BernhardtV. (2009). Data use.Journal of Staff Development30(1), 24–27.
6.
BollenK. A. (1989). Structural equations with latent variables.New York, NY: John Wiley & Sons.
7.
Brown-SimsM. (2010). Evaluating school principals: Tips and tools.Washington, DC: National Comprehensive Center for Teacher Quality.
8.
CatanoN., & StrongeJ. H. (2006). What are principals expected to do? Congruence between principal evaluation and performance standards.NASSP Bulletin,90, 221–228.
9.
Council of Chief State School Officers (CCSSO) (2008). Educational leadership policy standards: ISLLC 2008.Washington, DC: Author.
10.
CelioM. B., & HarveyJ. (2005). Buried treasure: Developing a management guide from mountains of school data.New York, NY: Wallace Foundation.
11.
ChappuisS., & ChappuisJ. (2007). The best value in formative assessment.Educational. Leadership65(4), 14–19.
12.
ClaudetJ. (2006). Multimedia case simulations as professional learning and assessment tools for school leaders.Journal of Thought,41(1), 59–74.
13.
CoburnC. E., & TurnerE. O. (2012). The practice of data use: An introduction.American Journal of Education,118, 99–111.
14.
CurranP. J., BollenK. A., ChenF., PaxtonP., & KirbyJ. (2003). An empirical evaluation of the use of fixed cutoff points in RMSE A test statistic in structural equation models.Sociological, Methods Research,36, 462–494.
15.
Darling-HammondL. (1988). Policy and professionalism. In LiebermanA. (Ed.), Building a professional culture in organizations (pp. 55–77). New York, NY: Teachers College Press.
16.
DawR. W., & GageN. L. (1967). Effects of feedback from teachers to principals.Journal of Educational, Psychology,58, 181–188.
17.
DoudJ. L., & KellerE. P. (1998). The K-8 principal, in 1998: A ten-year study.Alexandria, VA: National Association of Elementary School Principals.
18.
GageN. L., RunkelP. J., & ChatterjeeB. B. (1960). Equilibrium, theory and behavior change: An experiment in feedback from pupils to teachers.Urbana: Bureau of Educational Research, College of Education, University of Illinois.
19.
GlassT. (2003). Superintendent, selection of secondary school, principals.Denver, CO: Education Commission of the States.
20.
GoldringE., PorterA., MurphyJ., ElliottS. N., & CravensX. (2009). Assessing learning-centered leadership: Connections to research, professional standards, and current practices.Leadership and Policy in Schools,8, 1–36.
21.
HallingerP. (1982, 1990). Principal instructional management rating scale.Sarasota, FL: Leading Development Associates.
22.
HalversonR., GriggJ., PritchettR., & ThomasC. (2007). The new instructional leadership: Creating data-driven instructional systems in school.Journal of School Leadership,17(2), 159–194.
23.
HeckR. H., & HallingerP. (2009). Assessing the contribution of distributed leadership to school improvement and growth in math achievement.American Educational Research Journal46, 626–658.
24.
HeckR. H., LarsenT. J., & MarcoulidesG. A. (1990). Instructional leadership and school achievement: validation of a causal model.Educational Administration Quarterly,26(2), 94–125.
25.
HessF. M., & FullertonJ. (2009). The numbers we need: Bringing balanced score-cards to education data.Phi Delta Kappan,90(9), 665–669.
26.
HonigM. I., & VenkateswaranN. V. (2012). School-Central Office Relationships in Evidence Use: Understanding Evidence Use as a Systems Problem.American Journal of Education,118, 199–222.
27.
HuL. T., & BentlerP. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification.Psychological, Methods,3, 424–453.
28.
HuL. T., & BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives.Structural. Equation Modeling,6, 1–55.
KeiserN. M., & ShenJ. (2000). Principals’ and teachers’ perceptions of teacher empowerment.Journal of Leadership & Organizational. Studies7(3), 115–121.
31.
KimballS. M., MilanowskiA. T., & McKinneyS. A. (2009). Assessing the promise of standards-based performance evaluation for principals: Results from a randomized trial.Leadership and Policy in Schools,8, 233–236.
32.
KnappM. S., SwinnertonJ. A., CoplandM. A., Monpas-HuberJ. (2006). Data-informed leadership in education.Seattle, WA: University of Washington.
33.
KnuthR. K., & BanksP. (2006). The essential leadership model.NASSP Bulletin90(1), 4–18.
34.
LaskyS., SchafferG., & HopkinsT. (2009). Learning to think and talk from evidence. In Professional learning conversations: Challenges in using evidence for improvement (pp. 95–107). Netherlands: Springer.
35.
LeithwoodK., LouisK. S., AndersonS., & WahlstromK. (2004). How leadership influences student learning.Minneapolis, MN: University of Minnesota
36.
LittleJ. W. (2012). Understanding data use practice among teachers: The contribution of micro-process studies.American Journal, of Education,118(2), 143–166.
37.
LoVetteO., WattsS., & WheelerJ. (2001, November). Teacher perceptions of principals: A new instrument. Paper presented at the annual meeting of the Mid-South Educational Research Association. Little Rock, AK.
MarzanoR. J. (2003). What works in schools.Alexandria, VA: ASCD.
40.
MarzanoR. J., WatersT., McNultyB. A. (2005). School leadership that works.Alexandria, VA: ASCD.
41.
MessickS. (1992). Validity of test interpretation and use. In AlkinM. (Ed), Encyclopedia of educational, research. (6th ed., pp. 1487–1495). New York, NY: Macmillan.
42.
Mid-continent Research for Education, and Learning. (2005). Balanced Leadership Profile [Online survey and Web site], Aurora, CO: Author.
43.
MilitelloM., BassL., JacksonK. T., & WangY. (2013). How data are used and misused in schools: Perceptions from teachers and principals.Education Sciences,3(2), 98–120.
44.
MossP. A. (2012). Exploring the Macro-Micro Dynamic in Data Use Practice.American Journal, of Education,118, 223–232.
45.
OwingsW. A., KaplanL. S., & NunneryJ. (2005). Principal quality, ISLLC standards, and student achievement: A Virginia study.Journal of school leadership,15, 99–119.
46.
PetzkoV. N. (2004). Findings and implications of the NASSP national study of leadership in middle level schools (volumes I and II): Teachers in middle level schools.NASSP Bulletin,88, 69–88.
47.
PorterA. C., PolikoffM. S., GoldringE., MurphyJ., ElliottS. N., & MayH. (2010). Developing a psychometrically sound assessment of school leadership: The VAL-ED as a case study.Educational Administration Quarterly,46(2), 135–173.
48.
ReevesD. (2009). Assessing educational leaders: Evaluating performance for improved individual and organizational results (2nd ed.). Thousand Oaks, CA: Corwin.
49.
RichardsJ. (2005, April). Principal behaviors that encourage teachers: Perceptions of teachers at three different stages. Paper presented at the annual meeting of the American Educational Research Association. Montreal, Canada.
50.
RowanB. (1990). Commitment and control: Alternative strategies for the organizational design. In CazdenC. B. (Ed.), Review of Research in Education, Vol. 16 (pp. 353–389). Washington, DC: American Educational Research Association.
51.
SengeP. (1990). The fifth discipline: The art and science of the learning organization.New York, NY: Doubleday.
52.
ShenJ. (2005). School principals.New York, NY: Peter Lang Publishing.
53.
ShenJ., & CooleyV. (2008). Critical issues in using data for decision-making.International Journal of Leadership in Education,11, 319–329.
54.
ShenJ., CooleyV., ReevesP., BurtW., RyanL., RaineyJ. M., & YuanW. (2010). Using data for decision-making: Perspectives from 16 principals in Michigan, USA.International Review of Education,56, 435–456.
55.
ShenJ., CooleyV. E., MaX., ReevesP. L., BurtW. L., RaineyJ. M., & YuanW. (2012). Data-informed decision making on high-impact strategies: Developing and validating an instrument for principals.Journal of Experimental Education,80, 1–25.
56.
ShenJ., & MaX. (2006). Does systemic change work? Curricular and instructional practice in the context of systemic change.Leadership and Policy in School,5, 231–256.
57.
SpillaneJ. P. (2012). Data in Practice: Conceptualizing the Data-Based Decision-Making Phenomena.American Journal of Education,118, 113–141.
58.
Students First. (2011). A challenge to states and districts: Policies that put students first, Sacramento, CA: Author.
ValentineJ. W. (1987, February). Performance/outcome based principal evaluation. Paper presented at the annual meeting of the American Association of School Administrators. New Orleans, LA.
61.
WahlstromK. L., LouiseK. S., LeithwoodK., & AndersonS. E. (2010). Investigating the links to improved student learning: Executive summary of research findings.Minneapolis, MN: University of Minnesota.
62.
WeldyG. R. (1961). Teachers evaluate their principal.NASSP Bulletin,45, 145–150.
63.
WeissK. (1989, March). Evaluation of elementary and secondary school principals. Paper presented at the annual meeting of the American Association of School Administrators. Orlando, FL.
64.
WestonC., McAlpineL., & BordonaroT. (1995). A model for understanding formative evaluation in instructional design.Educational. Technology Research and. Development,43, 9–48.
65.
ZimmermanS., & Deckert-PeltonM. (2003). Evaluating the evaluators: Teachers’ perceptions of the principal's role in professional evaluation.NASSP Bulletin,87, 28–37.
