The purpose of this study was to develop a valid and reliable rubric to be used for the evaluation of large ensemble wind band performances. The guiding questions for this study were: (a) what are the psychometric qualities (i.e., reliability and validity) of the scale developed to assess wind band ensemble performance at the high school level? (b) how do the items fit the model and vary in difficulty? (c) how does the structure of the rating scale vary across individual items? and (d) how can the rating scale be transferred into an informative rubric? The primary data analysis tool used in this study was the Multifaceted Rasch Partial Credit Measurement Model. Music content experts (N = 20) were solicited to evaluate 40 wind band performances, each evaluator listening to four. A 4-point Likert-type rating scale (e.g., Strongly Agree, Agree, Disagree, and Strongly Disagree) was used to evaluate each recorded performance. Results indicated good model data fit and resulted in a final rubric containing 24 items ranging from two to four performance categories. Implications for classroom teaching and consequential validity are discussed.
AbelesH. F. (1971). An application of the facet-factorial approach to scale construction in the development of a rating scale for clarinet music performance (Unpublished doctoral dissertation). University of Maryland, College Park, MD.
2.
AbelesH. F.HofferC. R.KlottmanR. H. (1994). Foundations of music education (2nd ed.). New York, NY: Schirmer Books.
3.
AndrichD. A. (1996). Measurement criteria for choosing among models for graded responses. In von EyeA.CloggC. C. (Eds.), Analysis of categorical variables in developmental research (pp. 3–35). Orlando, FL: Academic Press.
4.
AsmusE. P. (1999). Music assessment concepts. Music Educators Journal, 2, 19.
5.
AustinJ. R. (1988). The effect of music contest format on self-concept, motivation, achievement, and attitude of elementary band students. Journal of Research in Music Education, 36(2), 95–107.
6.
BanisterS. (1992). Attitudes of high school band directors toward the value of marching band and concert band contests and selected aspects of the overall band program. Missouri Journal of Research in Music Education, 29, 49–57.
7.
BergeeM. J. (1987). An application of the facet-factorial approach to scale construction in the development of a rating scale for euphonium and tuba music performance (Unpublished doctoral dissertation). University of Kansas, Lawrence, KS.
8.
BerkR. A. (2010). The secret to the “best” ratings from any evaluation scale. Journal of Faculty Development, 24(1), 37–39.
9.
BondT. G.FoxC. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences. New York: Routledge.
ColwellR. (1970). The evaluation of music teaching and learning. Englewood Cliffs, NJ: Prentice-Hall.
12.
CookseyJ. M. (1977). A facet-factorial approach to rating high school choral music performance. Journal of Research in Music Education, 25(6), 100–114.
13.
DeCampC. B. (1980). An application of the facet-factorial approach to scale construction in the development of a rating scale for high school band performance. Dissertation Abstracts International, 41, 1462A.
14.
DeLucaC.BoldenB. (2014). Music performance assessment: Exploring three approaches for quality rubric construction. Music Educators Journal, 101(1), 70–76.
15.
EngelhardG. (1997). Constructing rater and task banks for performance assessments. Journal of Outcome Measurement, 1(1), 19–33.
16.
EngelhardG.Jr. (2009). Using item response theory and model-data fit to conceptualize differential item and person functioning for students with disabilities. Educational and Psychological Measurement, 69(4), 585–602.
17.
EngelhardG.Jr. (2013). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. New York, NY: Routledge.
18.
FranklinJ. O. (1979). Attitudes of school administrators, band directors, and band students towards selected activities of the public school band program (Unpublished doctoral dissertation). Northwestern State University of Louisiana, Natchitoches, LA.
19.
HowardK. K. (1994). A survey of Iowa high school band students’ self-perceptions and attitudes toward types of music contests (Unpublished doctoral dissertation). University of Iowa, Iowa City, IA.
20.
HowardR. L. (2002). Repertoire selection practices and the development of a core repertoire for the middle school concert band (Unpublished doctoral dissertation). University of Florida, Gainsville, FL.
21.
HurstC. W. (1994). A nationwide investigation of high school band directors’ reasons for participating in music competitions (Unpublished doctoral dissertation). The University of North Texas, Denton, TX.
22.
JonesH. (1986). An application of the facet-factorial approach to scale construction in the development of a rating scale for high school vocal solo performance (Unpublished doctoral dissertation). University of Oklahoma, Norman, OK.
23.
KanA.BulutO. (2014). Crossed random-effect modeling: Examining the effects of teacher experience and rubric use in performance assessments. Eurasian Journal of Educational Research, (57), 1–27.
24.
LinacreJ. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85–106.
25.
LinacreJ. M. (2014). Facets. Chicago, IL: MESA Press.
26.
MastersG. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: Macmillan.
30.
NicholsJ. P. (2005). A factor analysis approach to the development of a rating scale for snare drum performance. Dialogue in Instrumental Music Educaiton, 15(1), 11.
31.
O’NealC. (2012). Data-driven decision making: A handbook for school leaders. Eugene, OR: International Society for Technology in Education [ISTE].
32.
Pazitka-MunroeW. L. (2003). The development and validation of an audition instrument to measure vocal performance of college singers auditioning for choral ensembles (Unpublished doctoral dissertation). Indiana University, Bloomington, IN..
33.
PellegrinoK.ConwayC. M.RussellJ. A. (2015). Assessment in performance-based secondary music classes. Music Educators Journal, 102(1), 48–55.
34.
RussellB. E. (2010). The development of a guitar performance rating scale using a facet-factorial approach. Bulletin of the Council for Research in Music Education, 184, 21–34.
35.
ShermanC. (2006). A study of current strategies and practices in the assessment of individuals in high school bands. (Ed.D., Teachers College Columbia Univ., 2006). Dissertation Abstracts International Section A: Humanities & Social Sciences, 67(10), 3751.
36.
SmithB. P.BarnesG. V. (2007). Development and validation of an orchestra performance rating scale. Journal of Research in Music Education, 55(3), 268–280.
37.
SwanG.MazurJ. (2011). Examining data driven decision making via formative assessment: A confluence of technology, data interpretation heuristics and curricular policy. Contemporary Issues In Technology And Teacher Education (CITE Journal), 11(2), 205–222.
38.
SweeneyC. R. (1998). A description of student and band director attitudes toward concert band competition (Unpublished master’s thesis). University of Miami, Coral Gables, FL.
39.
U.S. Department of Education. (2009, November). Race to the Top program: Executive summary. Washington, DC: Author.
40.
VagiasW. M. (2006). Likert-type scale response anchors. Clemson, SC: Clemson International Institute for Tourism & Research Development.
41.
WaymanJ. C. (2005). Involving teachers in data-driven decision-making: Using computer data systems to support teacher inquiry and reflection. Journal of Education for Students Placed at Risk, 10(3), 295–308.
42.
WesolowskiB. C. (2012). Understanding and developing rubrics for music performance assessment. Music Educators Journal, 98(3), 36–42.
43.
WesolowskiB. C. (2014). Documenting student learning in music performance: A framework. Music Educators Journal, 101, 77–85.
44.
WesolowskiB. C.WindS. A.EngelhardG. (2015). Rater fairness in music performance assessment: Evaluating model-data fit and differential rater functioning. Musicae Scientiae, 29(2), 147–170.
45.
WesolowskiB. C.AmendR. M.BarnsteadT. S.EdwardsA. S.EverhartM.GoinsQ. R.GroganR. J.IIIHercegA. M.JenkinsS. I.JohnsP. M.McCarverC. J.SchapsR. E.SorrellG. W.WilliamsJ. D. (2017). The development of a secondary-level solo wind instrument performance rubric using the Multifaceted Rasch Partial Credit Measurement Model. Journal of Research in Music Education, 65(1), 95–119.
46.
WilsonM. (2005). Constructing measures: An item response modeling approach. New York, NY: Taylor & Francis.
47.
WindS. A.EngelhardG. E.WesolowskiB. C. (2016). Exploring th effects of rating designs and rater fit on achievement estimates within the context of music performance assessment. Educational Assessment, 21(4), 278–299.
48.
WrightB. D.LinacreJ. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370. Retrieved from https://www.rasch.org/rmt/rmt83b.htm
49.
ZdzinskiS. F.BarnesG. V. (2002). Development and validation of a string performance rating scale. Journal of Research in Music Education, 50(3), 245–255.
