Assessing the Effectiveness and Interactions of Instructional Coaches: Initial Psychometric Evidence for the Instructional Coaching Assessments–Teacher Forms
Restricted accessResearch articleFirst published online March, 2019
Assessing the Effectiveness and Interactions of Instructional Coaches: Initial Psychometric Evidence for the Instructional Coaching Assessments–Teacher Forms
The conceptual foundation and initial psychometric evidence are provided for the Instructional Coaching Rating Scales and Interaction Style Scales–Teacher Forms. These forms are part of a multicomponent online assessment system designed to evaluate the effectiveness of coaching skills and interactions that support the needs of teachers and students. Specifically, the article presents the theory, evidence, and measurement framework for the system. Findings indicate that the Rating Scales and Interaction Style Scales–Teacher Forms have very good internal structures based on multiple fit statistics for confirmatory factor analyses, high internal consistency, good item-to-scale total correlations, and freedom from item bias. Collectively, this promising statistical evidence is supportive of valid score inferences. Study limitations and directions for research are discussed.
American Institutes for Research. (2005). Conceptual overview: Coaching in the professional development impact study. Washington, DC: Author.
2.
American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
ArbuckleJ. L. (2015). IBM SPSS Amos 21 user’s guide (p. 635). Crawfordville, FL: Amos Development Corporation.
5.
BangorA.KortumP. T.MillerJ. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 6, 574–594.
6.
BentlerP. M. (1985). Theory and implementation of EQS: A structural equations program. Los Angeles, CA: BMDP Statistical Software.
7.
BerganJ. R.KratochwillT. (1990). Behavioral consultation and therapy. New York, NY: Plenum Press.
8.
BerganJ. R. (1977). Behavioral consultation. Columbus, OH: Charles E. Merrill.
9.
BrophyJ. E. (1981). Teacher praise: A functional analysis. Review of Educational Research, 51, 5–32.
10.
BrookeJ. (1996). SUS: A “quick and dirty” usability scale. In JordanP. W.ThomasB. A.WeerdmeesterMcClellandI. L. (Eds.), Usability evaluation in industry (pp. 189–194). London, England: Taylor & Francis.
11.
CicchettiD. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.
12.
ComreyA.LeeH. (1992). A first course in factor analysis. Hillsdale, NY: Lawrence Erlbaum.
13.
CrockerL.AlginaJ. (1986). Introduction to classical and modern test theory. New York, NY: Holt, Rinehart and Winston.
14.
DentonC. A.HasbrouckJ. (2009). A description of instructional coaching and its relationship to consultation. Journal of Educational and Psychological Consultation, 19, 150–175.
15.
DeussenT.CoskieT.RobinsonL.AutioE. (2007). “Coach” can mean many things: Five categories of literacy coaches in reading first (Issues & Answers, REL 2007-No. 005). Portland, OR: Regional Educational Laboratory Northwest.
16.
ErchulW. P. (2015). Put me in, coach: Observations on selected studies implementing supportive interventions to teachers. School Mental Health, 7, 74–79.
17.
ErchulW. P.SheridanS. (2014). Handbook of research in school consultation. New York, NY: Routledge.
18.
Every Student Succeeds Act of 2015 (ESSA), 20 U.S.C § 6311. et seq. (2015).
19.
FabianoG.ReddyL. A.DudekC. M. (2018). Use of teacher coaching and formative assessment for improving classroom practices. School Psychology Quarterly. Advance online publication. doi:10.1037/spq0000223
20.
FinstadK. (2006). The System Usability Scale and non-native English speakers. Journal of Usability Studies, 4, 185–188.
21.
GloverT. A. (2017). A data-driven coaching model used to promote students’ response to early reading intervention. Theory Into Practice, 56, 13–20.
22.
GloverT. A.ReddyL. A. (2017). Instructional coaching: Consideration of key competencies from multiple models. Theory Into Practice, 56, 1–2.
23.
GulamhusseinA. (2013). The core of professional development. American School Board Journal, 197, 22–26.
24.
Hagermoser SanettiL. M.LuiselliJ. K.HandlerM. W. (2007). Effects of verbal and graphic performance feedback on behavior support plan implementation in a public elementary school. Behavior Modification, 31, 454–465.
25.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.
26.
JacksonD. L.GillaspyJ. A.Jr.Purc-StephensonR. (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14, 6–23.
27.
JöreskogK. G.SörbomD. (1986). LISREL VI: Analysis of linear structural relationships by maximum likelihood, instrumental variables, and least squares methods. Uppsula, Sweden: Scientific Software.
KurzA.ReddyL. A.GloverT. (2017). A multidisciplinary framework of instructional coaching. Theory Into Practice, 56, 66–77.
30.
LaneM.RobbinsM.PriceD. (2013). Establishing content validity for a literacy coach performance appraisal instrument. Literacy Research and Instruction, 52, 230–253.
31.
MeadS. (2012). Recent state action on teacher effectiveness: What’s in state laws and regulation?New York, NY: Bellwether Education Partners.
32.
MeyersL. S.GamstG.GuarinoA. J. (2013). Applied multivariate research: Design and interpretation (2nd ed.). Thousand Oaks, CA: Sage Publications.
33.
MislevyR. J.AlmondR. G.LukasJ. F. (2003). A brief introduction to evidence-centered design. ETS Research Report Series, 2003(1), i–29.
34.
NoellG. H.WittJ. C.GilbertsonD. N.RanierD. D.FreelandJ. T. (1997). Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12, 77–88.
35.
ObermillerC.SpangenbergE. R. (1998). Development of a scale to measure consumer skepticism toward advertising. Journal of Consumer Psychology, 7, 159–186.
36.
OddenA. (2008). How to create world class teacher compensation. Saint Paul, MN: Freeload Press.
37.
ReddyL. A.Barboza-WhiteheadS.FilesT.RubelR. (2000). Clinical focus of consultation outcome research with children and adolescents. Special Services in the Schools, 16, 1–22. doi:10.1300/J008v16n01_01
38.
ReddyL. A.DudekC.LekwaA. J. (2017). Classroom strategies coaching model: Integration of formative assessment and instructional coaching. Theory Into Practice, 56, 46–55.
39.
ReddyL. A.DudekC.ShernoffE. (2016). Teacher formative assessment: The missing link in response to intervention. In JimersonS. R.BurnsM. K.VanDerHeydenA. M. (Eds.), Handbook of response to intervention: The science and practice of multi-tiered systems of support (2nd ed.; pp. 607-623). New York, NY: Springer Science.
40.
ReddyL.A.FabianoG.DudekC.HsuL. (2013). Development and construct validity of the Classroom Strategies Scale. Special Series: Assessment of general education teachers’ tier 1 classroom practices: Current science and practice. School Psychology Quarterly, 28, 317-341. doi:10.1037/spq0000043
41.
ReddyL. A.GloverT. A.ElliottS.KurzA. (2016, July). Measurement of instructional coaching effectiveness: Gaps in assessment practices and validity. Paper session: Assessment for Learning and Teachers’ Assessment Practices in the Classroom Contexts at the International Testing Commission, Vancouver, British Columbia, Canada.
42.
ReddyL. A.KettlerR. J.KurzA. (2015). School-wide educator evaluation for improving school capacity and student achievement in high-poverty schools: Year 1 of the school system improvement project. Journal of Educational and Psychological Consultation, 25, 90–108.
43.
ReddyL. A.ShernoffE. S.LekwaA. L. (2017, February). Examining the efficacy of the collaborative coaching model implemented in high poverty schools. In HoldawayA. (Chair), Coaching Teachers in Universal SEB Programming: Promising Research-Practice Models. Symposium submitted to the annual meeting of the National Association of School Psychologists, San Antonio, TX.
44.
ReynoldsC. R.CarsonD. (2005). Methods for assessing cultural bias in tests. In FrisbyC. L.ReynoldsC. R. (Eds.), Comprehensive handbook of multicultural school psychology (pp. 795–823). New York, NY: John Wiley.
45.
SawchukS. (2016, January). ESSA loosens reins on teacher evaluations, qualifications. Education Week, 35(15), 14–15.
46.
SchwartzF.WhiteK. (2000). Making sense of it all: Giving and getting online course feedback. In WhiteK. W.WeightB. H. (Eds.), The online teaching guide: A handbook of attitudes, strategies, and techniques for the virtual classroom (pp. 57–72). Boston, MA: Allyn & Bacon.
47.
SheridanS. M.BovairdJ. A.GloverT. A.GarbaczS. A.WitteA.KwonK. (2012). A randomized trial examining the effects of Conjoint Behavioral Consultation and the mediating role of the parent-teacher relationship. School Psychology Review, 41, 23–46.
48.
SheridanS. M.KratochwillT. R. (2008). Conjoint behavioral consultation: Promoting family-school connections and interventions. New York, NY: Springer.
49.
ShowersB.JoyceB.BennettB. (1987). Synthesis of research on staff development: A framework for future study and a state-of-the-art analysis. Educational Leadership, 45(3), 12–16.
50.
ShuteV. (2007). Research report: Focus on formative feedback. Princeton, NJ: Educational Testing Service.
