The purpose of this validity study was to improve measurement of athletes’ evaluations of their head coach’s coaching competency, an important multidimensional construct in models of coaching effectiveness. A revised version of the Coaching Competency Scale (CCS) was developed for athletes of high school teams (APCCS II-HST). Data were collected from athletes (N = 748) of seven relevant sports. Athlete observations were clustered within teams (G = 74). Multigroup confirmatory factor analyses of the asymptotic within-teams covariance matrix provided evidence for factorial invariance, except for one residual variance, by athlete gender (nmale = 427, nfemale = 321). An exploratory multilevel confirmatory factor analysis provided evidence for close fit of an oblique five-factor within-teams structure and a one-factor between-teams structure.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. ( 1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association .
2.
Asparouhov, T., & Muthén, B.O. (2006). Multilevel modeling of complex survey data. Proceedings of the Joint Statistical Meeting, USA. American Statistical Association Section on Survey Research Methods, 2718-2726.
3.
Asparouhov, T., & Muthén, B.O. (2007). Computationally efficient estimation of multilevel high-dimensional latent variable models. Proceedings of the Joint Statistical Meeting, USA, American Statistical Association Section on Biometrics, 2531-2535.
4.
Byrne, B.M., Shavelson, R.J., & Muthén, B.O. (1989). Testing for equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466.
5.
Cronbach, L.J. ( 1951). Coefficient alpha and the internal structure of tests . Psychometrika, 16, 297-334.
6.
Enders, C.K. ( 2006). Analyzing structural equation models with missing data . In R. C. Serlin (Series Ed.) & G. R. Hancock & R. O. Mueller (Vol. Eds.), Structural equation modeling: A second course (pp. 313-342). Greenwich, CT: Information Age.
7.
Feltz, D.L., Chase, M.A., Moritz, S.E., & Sullivan, P.J. (1999). A conceptual model of coaching efficacy: Preliminary investigation and instrument development. Journal of Educational Psychology, 91, 765-776.
8.
Feltz, D.L., Short, S.E., & Sullivan, P.J. (2008). Self-efficacy in sport . Champaign, IL: Human Kinetics .
9.
Grilli, L., & Rampichini, C. (2007). Multilevel factor models for ordinal variables. Structural Equation Modeling, 14, 1-25.
10.
Hancock, G.R., & Mueller, R.O. ( 2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. H. C. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Past and present. A festschrift in honor of Karl G. Jöreskog (pp. 195-261). Chicago: Scientific Software International.
11.
Henson, R.K., & Roberts, J.K. ( 2006). Use of exploratory factor analysis in published research . Educational and Psychological Measurement, 66, 393-416.
12.
Honaker, J., & King, G. ( 2007). What to do about missing values in time series cross-section data. Retrieved June 11, 2008, from http://gking.harvard.edu/amelia
13.
Honaker, J., King, G., & Blackwell, M. ( 2007). Amelia II: A program for missing data [Computer software] . Retrieved from http://gking.harvard.edu/amelia
14.
Horn, T.S. ( 2002). Coaching effectiveness in the sports domain. In T. S. Horn (Ed.), Advances in sport psychology (2nd ed., pp. 309-354). Champaign, IL: Human Kinetics.
15.
Hox, J.J., & Maas, C.J.M. ( 2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling, 8, 157-174.
16.
Hu, L., & Bentler, P.M. ( 1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
17.
Jöreskog, K.G. ( 1993). Testing structural equation models. In K. A. Bollen & J. S. Lang (Eds.), Testing structural equation models (pp. 294-316). Newbury Park, CA: Sage.
18.
Julian, M.W. ( 2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling, 8, 325-352.
19.
Lee, K.S., Malete, L., & Feltz, D.L. ( 2002). The strength of coaching efficacy between certified and non-certified Singapore coaches. International Journal of Applied Sport Sciences, 14, 55-67.
20.
Millsap, R.E., & Kwok, O.-M. ( 2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9, 93-115.
21.
Millsap, R.E., & Yun-Tein, J. ( 2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479-515.
22.
Muthén, B.O. ( 1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indictors. Psychometrika, 49, 115-132.
Muthén, L.K., & Muthén, B.O. (2007). Mplus user’s guide (5th ed.). Los Angeles : Muthén & Muthén.
25.
Muthén, B.O., du Toit, S.H.C., & Spisic, D. (in press). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Psychometrika.
26.
Myers, N.D., & Feltz, D.L. ( 2007). From self-efficacy to collective efficacy in sport: Transitional methodological issues. In G. Tenenbaum & R. C. Eklund (Eds.), The handbook of sport psychology (3rd ed., pp. 799-819). New York: Wiley.
27.
Myers, N.D., Feltz, D.L., Chase, M.A., Reckase, M.D., & Hancock, G.R. ( 2008). The Coaching Efficacy Scale II-High School Teams. Educational and Psychological Measurement, 68, 1059-1076.
28.
Myers, N.D., Feltz, D.L., Maier, K.S., Wolfe, E.W., & Reckase, M.D. ( 2006). Athletes’ evaluations of their head coach’s coaching competency. Research Quarterly for Exercise and Sport , 77, 111-121.
29.
Myers, N.D., Wolfe, E.W., & Feltz, D.L. ( 2005). An evaluation of the psychometric properties of the coaching efficacy scale forAmerican coaches. Measurement in Physical Education and Exercise Science, 9, 135-160.
30.
Myers, N.D., Wolfe, E.W., Feltz, D.L., & Penfield, R.D. (2006). Identifying differential item functioning of rating scale items with the Rasch model: An introduction and an application. Measurement in Physical Education and Exercise Science, 10, 215-240.
31.
Myers, N.D., Wolfe, E.W., Maier, K.S., Feltz, D.L., & Reckase, M.D. ( 2006). Extending validity evidence for multidimensional measures of coaching competency. Research Quarterly for Exercise and Sport, 77, 451-463.
32.
National Association for Sport and Physical Education. ( 2006). Quality coaches, quality sports: National standards for athletic coaches (2nd ed.). Reston, VA: Author.
33.
Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
34.
Penfield, R.D., Myers, N.D., & Wolfe, E.W. ( 2008). Methods for assessing item, step, and threshold invariance in polytomous items following the partial credit model. Educational and Psychological Measurement, 68, 717-733.
35.
Rubin, D.B. ( 1987). Multiple imputation for non-response in sample surveys . New York: Wiley.
36.
Schafer, J.L., & Olsen, M.K. ( 1998). Multiple imputation for multivariate missing data problems: A data analyst’s perspective. Multivariate Behavioral Research, 33, 545-571.
37.
Smoll, F.L., & Smith, R.E. ( 1989). Leadership behaviors in sport: A theoretical model and research paradigm. Journal of Applied Social Psychology, 19, 1522-1551.
38.
Sullivan, P.J., & Kent, A. ( 2003). Coaching efficacy as a predictor of leadership style in intercollegiate athletics. Journal of Applied Sport Psychology , 15, 1-11.
39.
Zumbo, B.D., Gadermann, A.M., & Zeisser, C. ( 2007). Ordinal version of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods , 6, 21-29.
