The purpose of this study was to describe the development of a valid and reliable rubric to assess secondary-level solo instrumental music performance based on principles of invariant measurement. The research questions that guided this study included (1) What is the psychometric quality (i.e., validity, reliability, and precision) of a scale developed to assess secondary-level solo music performance? (2) Do the proposed items fit the measurement model, and if so, how do the items vary in difficulty? and (3) How does the structure of the rating scale vary across individual items? The psychometric considerations in this study included calibrations of items, persons, raters, school level, musical instrument, and rating scale structure using the Multifaceted Rasch Partial Credit Measurement Model. A 13-member cohort of music content experts participated as raters in this study. A total of 75 video performances of secondary-level solo and ensemble performances were evaluated. The result was the development of the Music Performance Rubric for Secondary-Level Instrumental Solos (MPR-2L-INSTSOLO), a 30-item rubric consisting of rating scale categories ranging from two to four performance criteria. Implications for consequential validity, rater training, standard setting, and benchmarking are discussed.
AbelesH. F. (1973). Development and validation of a clarinet performance adjudication scale. Journal of Research in Music Education, 21, 246–255.
2.
AndrichD. A. (1996). Measurement criteria for choosing among models for graded responses. In von EyeA.CloggC. C. (Eds.), Analysis of categorical variables in developmental research (pp. 3–35). Orlando. FL: Academic Press.
3.
BergeeM. J. (1987). An application of the facet-factorial approach to scale construction in the development of a rating scale for euphonium and tuba music performance. Dissertation Asbtracts International, 49(05), 1086A.
4.
BondT. G.FoxC. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences, 3rd edition. New York, NY: Routledge.
ChristensenK. B.EngelhardG.Jr.SalzbergerT. (2012). Ask the experts: Rasch vs. factor analysis. Rasch Measurement Transactions, 26, 1373–1378.
7.
CookseyJ. M. (1977). A facet-factorial approach to rating high school choral music performance. Journal of Research in Music Education, 25, 100–114.
8.
DCampC. B. (1980). An application of the facet-factorial approach to scale construction in the development of a rating scale for high school band performance. Iowa City: University of Iowa.
DumasJ. (1999). Usability testing methods: Subjective measures, part II - measuring attitudes and opinions. Washington, DC: American Institutes for Research.
11.
EngelhardG. (1997). Constructing rater and task banks for performance assessments. Journal of Outcome Measurement, 1, 19–33.
12.
EngelhardG. (2009). Using item response theory and model-data fit to conceptualize differential item and person functioning for students with disabilities. Educational and Psychological Measurement, 69, 585–602. doi:10.1177/0013164408323240
13.
EngelhardG. (2013). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. New York, NY: Routledge.
HorowitzR. A. (1994). The development of a rating scale for jazz guitar improvisation performance (Doctoral thesis) Columbia University, Teachers College, New York, NY.
16.
JonesH. (1986). An application of the facet-factorial appraoch to scale construction in the development of a rating scale for high school vocal solo performance. Dissertation Asbtracts International, 47(4), 1230.
17.
KreinerS.ChristensenK. B. (2004). Analysis of local dependence and multidimensionality in graphical loglinear Rasch models. Communications in Statistics—Theory and Methods, 33, 1239–1276.
18.
LinacreJ. M. (1989). Many facet Rasch measurement. Chicago, IL: MESA Press.
19.
LinacreJ. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3, 85–106.
20.
LinacreJ. M. (2014). Facets. Chicago, IL: MESA Press.
21.
MartonF. (1986). Phenomenography: A research approach to investigating different understandings of reality. Journal of Thought, 21, 29–49.
22.
MastersG. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: Macmillian.
25.
NicholsJ. P. (1991). A factor analysis approach to the development of a rating scale for snare drum performance. Dialogue in Instrumental Music Education, 15, 11–31.
26.
Pazitka-MunroeW. L. (2003). The development and validation of an audition instrument to measure vocal performance of college singers auditioning for choral ensembles. Dissertation Asbtracts International, 63(8), 2821.
27.
PfenningerR. C. (1990). The development and validation of three rating scales for the objective measurement of jazz improvisation achievement. Dissertation Asbtracts International, 51(8), 2674A.
28.
RussellB. E. (2010). The development of a guitar performance rating scale using facet-factorial approach. Bulletin of the Council for Research in Music Education, 184, 21–34.
29.
ShulerS. C.NorgaardM.BlakesleeM. J. (2014). The new national standards for music educators. Music Educators Journal, 101, 41–49. doi:10.1177/0027432114540120
30.
SmithB. P.BarnesG. V. (2007). Development and validation of an orchestra performance rating scale. Journal of Research in Music Education, 55, 268–280. doi:10.1177/002242940705500307
31.
SmithD. T. (2009). Development and validation of a rating scale for wind jazz improvisation performance. Journal of Research in Music Education, 57, 217–235. doi:10.1177/0022429409343549
WesolowskiB. C. (2014). Documenting student learning in music performance: A framework. Music Educators Journal, 101, 77–85. doi:10.1177/0027432114540475
38.
WesolowskiB. C. (2015). Tracking student achievement in music performance: Developing student learning objectives for growth model assessments. Music Educators Journal, 102, 39–47. doi:10.1177/0027432115589352
39.
WesolowskiB. C. (2016). Assessing jazz big band performance: The development, validation, and application of a facet-factorial rating scale. Psychology of Music, 44, 324–339. doi:10.1177/0305735614567700
40.
WesolowskiB. C.WindS. A.EngelhardG. (2015). Rater fairness in music performance assessment: Evaluating model-data fit and differential rater functioning. Musicae Scientiae, 19, 147–170. doi:10.1177/1029864915589014
41.
WesolowskiB. C.WindS. A.EngelhardG. (2016a). Examining rater precision in music performance assessment: An analysis of rating scale structure using the multifaceted Rasch partial credit model. Music Perception, 5, 662–678.
42.
WesolowskiB. C.WindS. A.EngelhardG. (2016b). Rater analyses in music performance assessment: Application of the Many Facet Rasch Model. In Connecting practice, measurement, and evaluation: Selected papers from the 5th International Symposium on Assessment in Music Education (pp. 335–356). Chicago, IL: Gia Publications.
43.
WilsonM. (2005). Constructing measures: An item response modeling approach. New York, NY: Taylor & Francis.
44.
WraughR. F.ChapmanE. S. (2005). An analysis of dimensionality using factor analysis (true-score theory) and Rasch measurement: What is the difference? Which method is better?Journal of Applied Measurement, 6, 80–99.
45.
WrightB. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14(2), 97–116.
46.
WrightB. D. (1996). Comparing Rasch measurement and factor analysis. Structural Equation Modeling, 3(1), 3–24.
47.
WrightB. D.DouglasG. A. (1986). The rating scale model for objective measurement. Retrieved from http://www.rasch.org/memo35.pdf
48.
WrightB. D.LinacreJ. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370.
49.
ZdzinskiS. F.BarnesG. V. (2002). Development and validation of a string performance rating scale. Journal of Research in Music Education, 50, 245–255.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
