This study investigated ways to improve the quality of music performance evaluation in an effort to address the accountability imperative in tertiary music education. An enhanced scientific methodology was employed incorporating ecological validity and using recognized qualitative methods involving grounded theory and quantitative methods involving confirmatory factor analyses. By distilling the disciplinary consensus, this approach enabled the specific definition of the constructs and standards used in university student classical music performance examinations, and provided their refinement with the empirical development of standards-based, criterion-specific rating scales for five instrument families. The study found that the examiners in each instrument family used between 15 to 17 constructs and associated descriptors, and at least two general dimensions referring to technical proficiency and another relating to musicality and interpretation. Analyses showed acceptable internal consistency and construct validity for the scales. Findings suggested that, although several construct and general dimension commonalities were found among the items across all scales, the presence of significant instrument-specific differences suggested that the use of generic rating scales may not provide sufficient content validity. Implications for the application of standards in music performance assessment were discussed.
AbelesH. F. (1973). Development and validation of a clarinet performance adjudication scale. Journal of Research in Music Education, 21, 246–255.
2.
AnnettJ. (2002). Subjective rating scales: Science or art?Ergonomics, 45(14), 966–987.
3.
ArbuckleJ. L. (2003). Amos 5.0 Update to the Amos User’s Guide. Chicago IL: SmallWaters Corporation.
4.
ArbuckleJ. L.WothkeW. (1999). Amos 4.0 User’s Guide. Chicago IL: SmallWaters.
5.
BergeeM. J. (1988). The use of an objectively constructed rating scale for the evaluation of brass juries: A criterion-related study. Missouri Journal of Research in Music Education, 5(5), 6–25.
6.
BergeeM. J. (1989). An objectively constructed rating scale for euphonium and tuba music performance. Dialogue in Instrumental Music Education, 13, 65–86.
7.
BergeeM. J. (1993). A comparison of faculty, peer and self-evaluation of applied brass jury performances. Journal of Research in Music Education, 41(1), 19–27.
8.
BergeeM. J. (1995). Primary and higher-order factors in a scale assessing concert band performance. Bulletin of the Council for Research in Music Education, 126, 1–14.
9.
BergeeM. J. (1997). Relationships among faculty, peer, and self-evaluations of applied performances. Journal of Research in Music Education, 45(4), 601–612.
10.
BergeeM. J. (2003). Faculty interjudge reliability of music performance evaluation. Journal of Research in Music Education, 51(2), 137–148.
11.
BiggsJ. (2003). Teaching for quality learning at university (2nd ed.). Berkshire, UK: Open University Press.
12.
BoyleJ. D. (1992). Evaluation of music ability. In ColwellR. (Ed.), Handbook of research on music teaching and learning (pp. 247–265). Basingstoke: Macmillan.
13.
BoyleJ. D.RadocyR. E. (1987). Measurement and evaluation of musical experiences. New York: Schirmer Books.
14.
BrownG. T. L. (2004). Teachers’ conceptions of assessment: Implications for policy and professional development. Assessment in Education, 11(3), 302–318.
15.
Cambridge International Dictionary of English. (1995). Cambridge: Cambridge University Press.
16.
ColwellR. (2002). Assessment’s potential in music education. In ColwellR.RichardsonC. (Eds.), The new handbook of research on music teaching and learning: A project of the Music Educators National Conference (pp. 1128–1158). Oxford: Oxford University Press.
17.
CookseyJ. M. (1977). A facet-factorial approach to rating high school choral music performance. Journal of Research in Music Education, 25, 100–114.
18.
CoteJ.SalmelaJ. H.AbderrahimB.RussellS. J. (1993). Organizing and interpreting unstructured qualitative data. The Sport Psychologist, 7, 127–137.
19.
DavidsonJ. W.CoimbraD. C. C. (2001). Investigating performance evaluation by assessors of singers in a music college setting. Musicae Scientiae, 5(1), 33–53.
20.
ElliottC. A. (1995). Race and gender as factors in judgements of musical performance. Bulletin of the Council for Research in Music Education (127), 50–55.
21.
FiskeH. E. (1977). Relationship of selected factors in trumpet performance adjudication reliability. Journal of Research in Music Education, 25, 256–263.
22.
FiskeH. E. (1983). Judging musical performances: Method or madness?Update, 1(3), 7–10.
23.
GabrielssonA. (2003). Music performance research at the millennium. Psychology of Music, 31(3), 221–272.
24.
GeringerJ. M.MadsenC. K. (1998). Musicians’ ratings of good versus bad vocal and string performances. Journal of Research in Music Education, 46, 522–534.
25.
HunterD.RussM. (1996). Peer assessment in performance studies. British Journal of Music Education, 13, 67–78.
26.
JohnsonP. (1997). Performance as experience: The problem of assessment criteria. British Journal of Music Education, 14(3), 271–282.
KlineR. B. (1998). Principles and practice of structural equation modeling. New York: The Guilford Press.
29.
KokotsakiD.DavidsonJ. W.CoimbraD. (2001). Investigating the assessment of singers in a music college setting: The students’ perspective. Research Studies in Music Education (16), 15–32.
30.
LeathwoodC.PhillipsD. (2000). Developing curriculum evaluation research in higher education: Process, politics and practicalities. Higher Education, 40, 313–330.
31.
MannsJ. W. (1998). Aesthetics. New York: M.E. Sharpe.
32.
MarginsonS.ConsidineM. (2000). The enterprise university. Cambridge: Cambridge University Press.
33.
McPhersonG. E.SchubertE. (2004). Measuring performance enhancement in music. In WilliamonA. (Ed.), Musical Excellence: Strategies and techniques to enhance performance (pp. 61–82). Oxford: Oxford University Press.
34.
MegillA. (1994). Introduction: Four senses of objectivity. In MegillA. (Ed.), Rethinking Objectivity (pp. 1–20). Durham, NC: Duke University Press.
35.
MilesM. B.HubermanA. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.
36.
MilesW. E. (1998). Improving the quality of performance assessments for high-stakes recital-based examinations in ensemble contexts. Victorian Journal of Music Education, 3–19.
37.
MillsJ. (1987). Assessment of solo musical performance: A preliminary study. Bulletin of the Council for Research in Music Education, 91, 119–125.
MintropH. (2004). High-stakes accountability, state oversight, and educaitonal equity. Teachers College Record, 106(11), 2128–2145.
40.
MucklerF. A.SevenS. A. (1992). Selecting performance measures: “Objective” versus “subjective” measurement. Human Factors, 34(4), 441–455.
41.
NicholsJ. P. (1991). A factor-analysis approach to the development of a rating scale for snare drum performance. Dialogue in Instrumental Music Education, 15, 11–31.
42.
PattonM. Q. (1990). Qualitative evaluation and research methods. Newbury Park, CA: Sage.
43.
PooleD. (2001). Moving towards professionalism: The strategic management of international education activities at Australian universities and their Faculties of Business. Higher Education, 42(4), 395–435.
44.
PriceM. (2005). Assessment standards: The role of communities of practice and the scholarship of assessment. Assessment & Evaluation in Higher Education, 30(3), 215–230.
RussellM.HigginsJ.RaczekA. (2004). Accountability, California style: Counting or accounting?Teachers College Record, 106(11), 2102–2127.
47.
RykielE. J. (2001). Scientific objectivity, value systems, and policymaking. Bioscience, 51(6), 433–436.
48.
SadlerD. R. (2005). Interpretations of criteria-based assessment and grading in higher education. Assessment & Evaluation in Higher Education, 30(2), 175–194.
49.
SagenD. P. (1983). The development and validation of a university band performance rating scale. Journal of Band Research, 18(2), 1–11.
50.
SaundersT. C.HolahanJ. M. (1997). Criteria-specific rating scales in the evaluation of high school instrumental performance. Journal of Research in Music Education, 45(2), 259–272.
51.
SheldonD. A. (1994). The effects of competitive versus noncompetitive performance goals on music students’ ratings of band performance. Bulletin of the Council for Research in Music Education, 121, 29–41.
52.
SmithB. P.BarnesG. V. (2007). Development and validation of an orchestral performance rating scale. Journal of Research in Music Education, 55(3), 268–280.
53.
SmithE. E. (1990). Categorization. In OshersonD. N.SmithE. E. (Eds.), An invitation to cognitive science: Thinking. Cambridge, MA: MIT Press.
54.
SmithJ. A. (1995). Semi-structured interviewing and qualitative analysis. In SmithJ. A.HarreR.LangenhoveL. V. (Eds.), Rethinking Methods in Psychology (pp. 9–27). London: Sage.
55.
StanleyM.BrookerR.GilbertR. (2002). Examiner perceptions of using criteria in music performance assessment. Research Studies in Music Education, 18, 43–52.
56.
SteigerJ. H. (1990). Structural model evaluation and modificaton: An interval estimation approach. Multivariate Behavioral Research, 25(2), 173–180.
57.
StraussA.CorbinJ. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: Sage.
58.
SwanickK. (1998). The perils and possibilities of assessment. Research Studies in Music Education (10), 1–11.
59.
ThompsonW. F.DiamondC. T. P.BalkwillL.-L. (1998). The adjudication of six performances of a Chopin Etude: A study of expert knowledge. Psychology of Music, 26, 154–174.
60.
ThompsonS.WilliamonA. (2003). Evaluating evaluation: Musical performance assessments as a research tool. Music Perception, 21, 21–41.
61.
WapnickJ.EkholmE. (1997). Expert consensus in solo voice performance evaluation. Journal of Voice, 11(4), 429–436.
62.
WapnickJ.FlowersP.AlegantM.JasinskasL. (1993). Consistency in piano performance evaluation. Journal of Research in Music Education, 41(4), 282–292.
63.
WapnickJ.RyanC.LacailleN.DarrowA. (2004). Effects of selected variables on musicians’ ratings of high-level piano performances. International Journal of Music Education, 22(1), 7–20.
64.
WebsterF.PepperD.JenkinsA. (2000). Assessing the undergraduate dissertation. Assessment & Evaluation in Higher Education, 25(1), 71–80.
65.
WinterN. (1993). Music performance assessment: A study of the effects of training and experience on the criteria used by music examiners. International Journal of Music Education, 22, 34–39.
66.
WrigleyW. J.EmmersonS. B. (2002). Staff and students’ attitudes to the performance assessment process at the Queensland Conservatorium Griffith University: Results and recommendations. Unpublished manuscript, Queensland Conservatorium Griffith University at Brisbane.
67.
ZdzinskiS. F.BarnesG. V. (2002). Development and validation of a string performance rating scale. Journal of Research in Music Education, 50(3), 245–255.
