Although a key concept in various writing textbooks, learning standards, and writing rubrics, voice remains a construct that is only loosely defined in the literature and impressionistically assessed in practice. Few attempts have been made to formally investigate whether and how the strength of an author’s voice in written texts can be reliably measured. Using a mixed-method approach, this study develops and validates an analytic rubric that measures voice strength in second language (L2) argumentative writing. Factor analysis of ratings from six raters on voice strength in a total of 400 TOEFL® iBT writing samples, together with qualitative analysis of four raters’ in-depth think-aloud and interview data, points to an alternative conceptualization of voice that sees authorial voice in written discourse as being realized primarily through the following dimensions: (1) the presence and clarity of ideas in the content; (2) the manner of the presentation of ideas; and (3) the writer and reader presence. Implications of such results for L2 writing instruction and assessment are discussed.
ArterJ. A.ChappuisJ. (2007). Creating and recognizing quality rubrics. Upper Saddle River, NJ: Pearson Education.
2.
BentlerP. M. (1985–2005). EQS for windows [Computer program]. Encino, CA: Multivariate Software.
3.
BowdenD. (1999). The mythology of voice. Portsmouth, NH: Heinemann.
4.
BrindleyG. (1998). Describing language development? Rating scales and second language acquisition. In BachmanL. F.CohenA. D. (Eds.), Interfaces between SLA and language testing research (pp. 112–114). Cambridge: Cambridge University Press.
5.
ByrneB. M. (2006). Structural equation modeling with EQS: Basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum.
6.
ComreyA. L.LeeH. B. (1992). A first course in factor analysis. Hillsdale, NJ: Lawrence Erlbaum.
7.
CummingA.KantorR.PowersD. E. (2002). Decision making while rating ESL/EFL writing tasks: A descriptive framework. The Modern Language Journal, 86, 67–96.
8.
DeRemerM. (1998). Writing assessment: Raters’ elaboration of the rating task. Assessing Writing, 5(1), 7–29.
9.
DiPardoA.StormsB. A.SellandM. (2011). Seeing voices: Assessing writerly stance in the NWP Analytic Writing Continuum. Assessing Writing, 16(3), 170–188.
10.
EdeL. (1992). Work in progress: A guide to writing and revising (2nd ed.). New York: St. Martin’s Press.
11.
ElbowP. (1968). A method for teaching writing. College English, 30(2), 115–125.
12.
ElbowP. (1981). Writing with power. New York: Oxford University Press.
13.
ElbowP. (1994). What do we mean when we talk about voice in texts? In YanceyK. B. (ed.), Voices on voice: Perspectives, definitions, inquiry (pp. 1–35). Urbana, IL: National Council of Teachers of English.
14.
ElbowP. (1999). Individualism and the teaching of writing: Response to Vai Ramanathan and Dwright Atkinson. Journal of Second Language Writing, 8(3), 327–338.
15.
FarmerF. (1995). Voice reprised: Three etudes for a dialogic understanding. Rhetoric Review, 13(2), 301–320.
16.
GravesD. H. (1983). Writing: Teachers & children at work. Portsmouth, NH: Heinemann.
17.
GravesD. H. (1994). A fresh look at writing. Portsmouth, NH: Heinemann.
18.
Helms-ParkR.StapletonP. (2003). Questioning the importance of individualized voice in undergraduate L2 argumentative writing: An empirical study with pedagogical implications. Journal of Second Language Writing, 12(3), 245–265.
19.
HinkelE. (2002). Second language writers’ text: Linguistic and rhetorical features. Mahwah, NJ: Lawrence Erlbaum.
20.
HinkelE. (2004). Teaching academic ESL writing: Practical techniques in vocabulary and grammar. Mahwah, NJ: Lawrence Erlbaum.
21.
HirvelaA.BelcherD. (2001). Coming back to voice: The multiple voices and identities of mature multilingual writers. Journal of Second Language Writing, 10(1–2), 83–106.
22.
HoldingM. (2005). Liberating the student’s voice: A teacher’s story of the college essay. English Journal, 94(4), 76–82.
23.
HylandK. (1994). Hedging in academic writing and EAP textbooks. English for Specific Purposes, 13(3), 239–256.
24.
HylandK. (1998). Hedging in scientific research articles. Amsterdam: John Benjamins.
25.
HylandK. (2000a). Hedges, boosters and lexical invisibility: Noticing modifiers in academic texts. Language Awareness, 9(4), 179–301.
26.
HylandK. (2000b). ‘It might be suggested that…’: Academic hedging and student writing. Australian Review of Applied Linguistics, 16, 83–97.
27.
HylandK. (2008). Disciplinary voices: Interactions in research writing. English Text Construction, 1(1), 5–22.
28.
IvaničR.CampsD. (2001). I am how I sound: Voice as self-representation in L2 writing. Journal of Second Language Writing, 10(1–2), 3–33. JefferyJ. (2007, November). Discourses of writing in high-stakes direct writing assessments. Paper presented at the National Reading Conference, Austin, TX.
29.
JefferyJ. (2009). Constructs of writing proficiency in US state and national writing assessments: Exploring variability. Assessing Writing, 14(1), 3–24.
30.
KaiserH. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.
31.
KnochU. (2011). Rating scales for diagnostic assessment of writing: What should they look like and where should the criteria come from?Assessing Writing, 16(2), 81–96.
32.
KumamotoChikako D. (2002). Bakhtin’s others and writing as bearing witness to the eloquent ‘I.’College Composition and Communication, 54(1), 66–87.
33.
LimG. S. (2011). Meeting multiple validation requirements in rating scale development. Paper presented at the 2011 Language Testing Research Colloquium, Ann Arbor, MI.
34.
LlosaL.BeckS. W.ZhaoC. G. (2011). An investigation of academic writing in secondary schools to inform the development of diagnostic classroom assessments. Assessing Writing, 16, 256–273.
35.
LumleyT. (2006). Assessing second language writing: The rater’s perspective. Frankfurt, Germany: Peter Lang.
36.
MatsudaP. K. (2001). Voice in Japanese written discourse: Implications for second language writing. Journal of Second Language Writing, 10(1–2), 35–53.
37.
MatsudaP. K.TardyC. M. (2007). Voice in academic writing: The rhetorical construction of author identity in blind manuscript review. English for Specific Purposes, 26, 235–249.
38.
MatsudaP. K.TardyC. M. (2008). Continuing the conversation on voice in academic writing. English for Specific Purposes, 27, 100–105.
39.
New York State Education Department (NYSED). (2007). Comprehensive examination in English: Scoring key and rating guide. Retrieved April15, 2007, from www.nysedregents.org/testing/engre/1rg-107.pdf.
O’LearyM. E. (1993). A voice of one’s own: Born, achieved, or thrust upon one?San Diego, CA: Retrieved October10, 2006, from ERIC database.
42.
PriorP. (2001). Voice in text, mind, and society: Sociohistoric accounts of discourse acquisition and use. Journal of Second Language Writing, 10(1–2), 55–81.
43.
RamanathanV.KaplanR. B. (1996). Audience and voice in current L1 composition texts: Some implications for ESL student writers. Journal of Second Language Writing, 5(1), 21–34.
44.
ReidJ. (2001). Advanced EAP writing and curriculum design: What do we need to know? In SilvaT.MatsudaP. K. (Eds.), On second language writing (pp. 143–160). Mahwah, NJ: Lawrence Erlbaum.
45.
SatorraA.,7BentlerP. M. (1988). Scaling corrections for chi-square statistics in covariance structure analysis. Proceedings of the American Statistical Association, 308–313.
46.
ShohamyE. (1996). Competence and performance in language testing. In BrownG.MalmkaerK.WilliamsJ. (Eds.), Performance and competence in second language acquisition (pp. 138–151). Cambridge: Cambridge University Press.
47.
StapletonP. (2002). Critiquing voice as a viable pedagogical tool in L2 writing: Returning spotlight to ideas. Journal of Second Language Writing, 11 (3), 177–190.
48.
StewartD. C. (1969). Prose with integrity: A primary objective. College Composition and Communication, 20(3), 223–227.
49.
StewartD. C. (1972). The authentic voice: A pre-writing approach to student writing. Dubuque, IA: Brown.
50.
StewartD. C. (1992). Cognitive psychologists, social constructionists, and three nineteenth- century advocates of authentic voice. Journal of Advanced Composition, 12(2), 279–290.
51.
TabachnickB. G.FidellL. S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon.
52.
TangR. (2006). Addressing self-representation in academic writing in a beginners’ EAP classroom. Journal of Language and Learning, 5(2), 76–85.
53.
TurnerC. E.UpshurJ. A. (1996). Developing rating scales for the assessment of second language performance. In WigglesworthG.ElderC. (Eds.), The language testing cycle: From inceptions to washback (Australian Review of Applied Linguistics Series S, No. 13, pp. 55–79). Melbourne: Australian Review of Applied Linguistics.
54.
UpshurJ. A.TurnerC. E. (1999). Systematic effects in the rating of second language speaking ability: Test method and learner discourse. Language Testing, 16(1), 82–111.
55.
WeigleS. C. (2002). Assessing writing. Cambridge: Cambridge University Press.
56.
WolfeE. W.BoltonS.FeltovichB.NidayD. M. (1996). The influence of student experience with word processors on the quality of essays written for a direct writing assessment. Assessing Writing, 3(2), 123–147.
57.
YanceyK. B. (1994). Introduction: Definition, intersection, and difference – mapping the landscape of voice. In YanceyK. B. (ed.), Voices on voice: Perspectives, definitions, inquiry (pp. vii–xxiv). Urbana, IL: National Council of Teachers of English.
58.
YehS. S. (1998). Validation of a scheme for assessing argumentative writing of middle school students. Assessing Writing, 5(1), 123–150.
59.
ZhaoC. G. (2010). The role of voice in high-stakes second language writing assessment. Unpublished doctoral dissertation, New York University.
60.
ZhaoC. G.LlosaL. (2008). Voice in high-stakes L1 academic writing assessment: Implications for L2 writing instruction. Assessing Writing, 13(3), 153–170.
