Through a field study using essays from the Analytical Writing Assessment (AWA)
of the Graduate Management Admission Test, we developed a set of analytical
tools to diagnose students' potential problems in MBA writing assignments. The
four tools—task, coherence, reasoning units, and error interference—can be used to
evaluate writing effectiveness based on success of the communicative interaction
between reader and writer. We tested the tools via blind and open scoring meth
ods and in direct writing consultations with students at two contrasting business
schools. Correlations between the AWA holistic scores reported by the Educational
Testing Service and our analytical tool scores suggest that the tools conceptualize
distinct traits in the AWA essays and can in turn be used to diagnose potential
deficiencies in MBA writing. In post-consultation surveys, students reported learn
ing about problems in their writing and about ways to address these needs, as
well as improved confidence as they transitioned into MBA writing. By accommo
dating reader and contextual differences, these analytical tools may bridge dis
courses where conventional checklists have failed.
Ackerman, J. (1991). Reading, writing, and knowing: The role of disciplinary knowledge in comprehension and composing. Research in the Teaching of English, 25, 133-178.
2.
Ackerman, J. (1993). The promise of writing to learn. Written Communication, 10, 334-370.
3.
Anson, C. M. (Ed.). (1989). Writing and response: Theory, practice, and, researchUrbana, IL: National Council of Teachers of English .
4.
Anson, C.M., & Forsberg, L.L. (1990). Moving beyond the academic community: Transitional stages in professional writing. Written Communication, 7, 200-31.
5.
Bakhtin, M. (1986). Speech genres and other late essays. (V. W. McGee, Trans.; C. Emerson & M. Holquist, Eds.). Austin: University of Texas Press. (Original work published in 1952-53)
6.
Bamberg, B. (1983). What makes a text coherent? College Composition and Communication, 34, 417-429.
7.
Barton, E.L. (1993). Evidentials, argumentation, and epistemological stance. College English, 55, 745-769.
8.
Barton, E.L. (1995). Contrastive and non-contrastive connectives. Written Communication, 12, 219-239.
9.
Barton, E.L., Halter, E., McGee, N., & McNeilley, L. (1998). The awkward problem of awkward sentences. Written Communication, 15, 69-98.
10.
Bazerman, C. (1988). Shaping written knowledge: The genre and activity of the experimental article in science. Madison: University of Wisconsin Press.
11.
Beach, R. (1989). Showing students how to assess: Demonstrating techniques for response in the writing conference. In C. M. Anson (Ed.), Writing and response: Theory, practice, and research (pp. 127-148). Urbana, IL: National Council of Teachers of English.
12.
Berkenkotter, C., & Huckin, T.N. (1995). Genre knowledge in disciplinary communication: Cognition/culture/power. Hillsdale, NJ: Lawrence Erlbaum.
13.
Bitzer, L.F. (1968). The rhetorical situation. Philosophy and Rhetoric, 1, 1-14.
14.
Bracewell, R.J. (1998). Commentary on "validation of a scheme for assessing argumentative writing of middle school students." Assessing Writing, 5, 151-157.
15.
Bridgeman, B., & Carlson, S.B. (1984). Survey of academic writing tasks. Written Communication, 1, 247-80.
16.
Broad, B. (1997). Reciprocal authorities in communal writing assessment: Constructing textual values within a "new politics of inquiry." Assessing Writing, 4, 133-167.
17.
Chambliss, M.J. (1995). Text cues and strategies successful readers use to construct the gist of lengthy written arguments. Reading Research Quarterly, 30, 778-807.
18.
Chambliss, M.J., & Garner, R. (1996). Do adults change their minds after reading persuasive text? Written Communication, 13, 291-313.
19.
Cheng, X., & Steffensen, M.S. (1996). Metadiscourse: A technique for improving student writing. Research in the Teaching of English, 30, 149-181.
20.
Cherry, R.D., & Meyer, P.R. (1993). Reliability issues in holistic assessment. In M. Williamson and B. Huot (Eds.), Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 109-141). Cresskill, NJ: Hampton Press.
21.
Cherry, R.D., & Witte, S.P. (1998). Direct assessments of writing: Substance and romance . Assessing Writing, 5, 71-87.
22.
Chin, E. (1994). Ethnographic interviews and writing research: A critical examination of the methodology. In P. Smagorinsky (Ed.), Speaking about writing: Reflections on research methodology. Sage Series in Written Communication, Vol. 8 (pp. 247-272). Thousand Oaks, CA: Sage.
23.
Clark, H.H., & Haviland, S. (1977). Comprehension and the given-new contract. In R. O. Freedle (Ed.), Discourse production and comprehension. Norwood, NJ: Ablex.
24.
Connor, U. (1990). Linguistic/rhetorical measures for international persuasive student writing. Research in the Teaching of English , 24, 67-87.
25.
Connor, U., & Johns. A. (1990). Coherence in writing. Alexandria, VA: Teachers of English to Speakers of Other Languages, Inc.
Connors, R., & Lunsford, A. (1988). Frequency of formal errors in current college writing. College Composition and Communication, 39, 395-409.
28.
Couture, B., & Rymer, J. (1993). Situational exigence: Composing processes on the job by writer's role and task value. In R. Spilka (Ed.), Writing in the workplace: New research perspectives (pp. 4-20). Carbondale, IL: Southern Illinois University Press .
29.
Crammond, J.G. (1998). The uses and complexity of argument structures in expert and student persuasive writing. Written Communication , 15, 230-68.
30.
Cross, G.A. (1994). Ethnographic research in business and technical writing. Journal of Business and Technical Communication, 8, 118-134.
31.
DeRemer, M.L. (1998). Writing assessment: Raters' elaboration of the rating task. Assessing Writing, 5, 7-29.
32.
Diederich, P., French, J., & Carlton, S. (1961). Schools of thought in judging excellence in English themes. Princeton, NJ: Educational Testing Service.
33.
Diederich, P.B. (1974). Measuring growth in English. Urbana, IL: National Council of Teachers of English .
34.
DiPardo, A. (1994). Stimulated recall in research on writing. In P. Smagorinsky (Ed.), Speaking about writing: Reflections on research methodology. Sage Series in Written Communication, Vol. 8 (pp. 163-181). Newbury Park, CA: Sage.
35.
Doheny-Farina, S. (1993). Research as rhetoric. In R. Spilka (Ed.), Writing in the workplace: New research perspectives (pp. 253-267). Carbondale, IL: Southern Illinois University Press.
36.
Driskill, L. (1989). Understanding the writing context in organizations . In M. Kogen (Ed.), Writing in the business professions. Urbana, IL: NCTE.
37.
Durst, R.K. (1987). Cognitive and linguistic demands of analytic writing . Research in the Teaching of English, 21, 347-376.
38.
Durst, R., Laine, C., Schultz, L.M., & Vilter, W. (1990). Appealing texts: The persuasive writing of high school students. Written Communication, 7, 232-255.
39.
Elbow, P. (1993). Ranking, evaluating, and liking: Sorting out three forms of judgment. College English, 55, 187-206.
40.
Faigley, L., Cherry, R.D., Jolliffe, D.A., & Skinner. A.M. (1985). Assessing writers' knowledge and processes of composing. Norwood, NJ: Ablex.
41.
Fairclough, N. (1993) Discourse and social change. Cambridge, UK: Polity Press.
42.
Farr, M. (1993). Essayist literacy and other verbal performances . Written Communication, 10, 4-38.
43.
Farr, M., & Nardini, G. (1996). Essayist literacy and sociolinguistic difference . In E. M. White, W. M. Lutz, & S. Kamusikiri (Eds.) Assessment of writing: Politics, policies, practices. New York: The Modern Language Association.
44.
Fleiss, J.L. (1981). Statistical methods for rates and proportions . New York: Wiley.
45.
Flower, L.S., & Hayes, J.R. (1980). The cognition of discovery: Defining a rhetorical problem. College Composition and Communication, 31, 21-32.
46.
Forman, J., & Rymer, J. (1999a). Defining the genre of the case write-up. The Journal of Business Communication, 36, 103-33.
47.
Forman, J., & Rymer, J. (1999b). The genre system of the Harvard case method. Journal of Business and Technical Communication, 13, 373-400.
48.
Freedman, A., Adam, C., & Smart, G. (1994). Wearing suits to class: Simulating genres and simulations as genre. Written Communication, 11, 193-226.
49.
Fulkerson, R. (1996a). Teaching the argument in writing. Urbana, IL: National Council of Teachers of English .
50.
Fulkerson, R. (1996b). The Toulmin model of argument and the teaching of composition. In B. Emmel, P. Resch, & D. Tenney (Eds.). Argument revisited; Argument redefined, (pp. 45-72). Thousand Oaks, CA: Sage.
51.
Geertz, C. (1983). Local knowledge: Further essays in interpretive anthropology. New York: Basic Books .
52.
GMAC (Graduate Management Admission Council) (n.d., distributed 1994, May). The GMAT analytical writing assessment: An introduction. Santa Monica, CA: Author.
Green, S., & Higgins, L. (1994). Once upon a time: The use of retrospective ccounts in building theory in composition. In P. Smagorinsky (Ed.), Speaking about writing: Reflections on research methodology. Sage Series in Written Communication, Vol. 8 (pp. 115-140). Newbury Park, CA: Sage.
Hairston, M. (1981). Not all errors are created equal: Nonacademic readers in the professions respond to lapses in usage. College English, 43, 794-806.
57.
Hairston, M. (1992). Successful writing (3rd Ed.) New York: Norton.
58.
Halliday, M., & Hasan, R. (1976). Cohesion in English. London : Longman.
59.
Harris, M. (1995). Talking in the middle: Why writers need writing tutors. College English, 57, 27-42.
60.
Harris, M., & Silva, T. (1993). Tutoring ESL students: Issues and options. College Composition and Communication, 44, 525-537.
61.
Haswell, R. (1989). Textual research and coherence: Findings, intuition, and application. College English, 51, 305-319.
62.
Haswell, R.H. (1998). Rubrics, prototypes, and exemplars: Categorization theory and systems of writing placement. Assessing Writing , 5, 231-268.
63.
Hayes, J.R., Hatch, J.A., & Silk, C.M. (2000). Does holistic assessment predict writing performance ? Written Communication, 17, 3-26.
64.
Hillocks, G., Jr. (1986). Research on written composition. Urbana, IL: National Conference on Research in English
65.
Hoetker, J. (1982). Essay examination topics and students' writing . College Composition and Communication, 33, 377-92.
66.
Horning, A.S. (1987). Teaching writing as a second language. Carbondale, IL: Southern Illinois University Press .
67.
Huckin, T.N. (1992). Context-sensitive text analysis. In G. Kirsch & P. A. Sullivan (Eds.), Methods and methodology in composition research (pp. 84-104). Carbondale, IL: Southern Illinois University Press .
68.
Huot, B.A. (1996). Toward a new theory of writing assessment. College Composition and Communication, 47, 549-566.
69.
Hynds, S., & Rubin, D. (Eds). (1990). Perspectives on talk and learning . Urbana: National Council of Teachers of English.
70.
Kent, T.L. (1984). Paragraph production and the given-new contract . The Journal of Business Communication, 2(4), 45-66.
71.
Kent, T.L. (1993). Paralogic rhetoric: A theory of communicative interaction. Lewisburg, PA: Bucknell.
72.
Knight, M. (1999). Management communication in US MBA programs: The state of the art. Business Communication Quarterly, 62(1), 9-32.
Kuriloff, P.C. (1996). What discourses have in common: Teaching the transaction between writer and reader. College Composition and Communication , 47, 485-501.
75.
Leki, I. (1992). Understanding ESL writers: A guide for teachers . Portsmouth, NH: Boynton/Cook.
76.
Leonard, D.J., & Gilsdorf, J.W. (1990). Language in change: Academics' and executives' perceptions of usage errors. The Journal of Business Communication, 27, 137-158.
77.
Lincoln, Y.S., & Guba, E.G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.
78.
Locker, K.O., & Keene, M.L. (1983). Using Toulmin logic in business and technical writing classes. In W. K. Sparrow & N. A. Pickett (Eds.), Technical and business communication in two-year programs (pp. 103-110). Urbana, IL: National Council of Teachers of English .
79.
Lorch, R.F., Jr., & O'Brien, E.J. (1995). Sources of coherence in reading. Hillsdale, NJ: Erlbaum .
80.
Lloyd-Jones, R. (1977). Primary trait scoring. In C. Cooper & L. Odell (Eds.), Evaluating writing: Describing, measuring, judging (pp. 33-69). Urbana, IL: National Council of Teachers of English.
81.
Markels, R.B. (1984). A new perspective on cohesion in expository paragraphs . Carbondale, IL: Southern Illinois University Press.
82.
Miller, C.R. (1984). Genre as social action. Quarterly Journal of Speech, 70, 151-167.
83.
Moss, P. (1994). Can there be validity without reliability? Educational Researcher, 23, 5-12.
84.
Noll, C.L., & Stowers, R.H. (1998). How MBA programs are using the GMAT's Analytical Writing Assessment. Business Communication Quarterly, 61(4), 66-71.
85.
Northey, M. (1990). The need for writing skill in accounting firms . Management Communication Quarterly, 3, 474-495.
86.
Nystrand, M. (1989). A social-interactive model of writing. Written Communication, 6, 66-85.
87.
Ochsner, R.S. (1990). Physical eloquence and the biology of writing . Albany: State University of New York Press.
88.
Odell, L., Goswami, D., & Herrington, A. (1983). The discourse-based interview: A procedure for exploring the tacit knowledge of writers in nonacademic settings . In P. Mosenthal, L. Tamor, & S. A. Walmsley (Eds.), Research on writing: Principles and methods (pp. 221-236). New York: Longman.
89.
Pajares, F., & Johnson, M.J. (1994). Confidence and competence in writing: The role of self-efficacy, outcome expectancy, and apprehension. Research in the Teaching of English, 28, 313-331.
90.
Penrose, A.M. (1992). To write or not to write: Effects of task and task interpretation on learning through writing. Written Communication , 9, 465-500.
91.
Pula, J.J., & Huot, B.A. (1993). A model of background influences on holistic raters . In M. M. Williamson & B. A. Huot (Eds.) Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 237-265). Cresskill, NJ: Hampton Press.
92.
Purves, A.C. (1992). Reflections on research and assessment in written composition. Research in the Teaching of English, 26, 108-122.
93.
Purves, A.C., Gorman, T.P., & Takala, S. (1988). The development of scoring scheme and scales. In T. Gorman, A. C. Purves, & R. E. Degenhart (Eds.), The IEA's study of written composition I: The international writing tasks and scoring scales, vol. 5 (pp. 41-58). New York: Pergamon.
94.
Raforth, B.A., & Rubin, D.L. (1984). The impact of content and mechanics on judgements of writing quality. Written Communication, 1, 446-58.
95.
Ray, R.E. (1993). The practice of theory: Teacher research in composition . Urbana, IL: National Council of Teachers of English.
Rogers, P.S. (1994). Analytic measures for evaluating managerial writing . Journal of Business and Technical Communication, 8, 380-407.
98.
Rogers, P.S. (2000). CEO presentations in conjunction with earnings announcements: Extending the construct of organizational genre through competing values profiling and user-need analysis. Management Communication Quarterly13, 426-485.
99.
Rogers, P.S., & Hildebrandt, H.W. (1993). Competing values instruments for analyzing written and spoken management messages. Human Resource Management, 32, 121-142.
100.
Rogers, P.S., & Rymer, J. (1995a). What is the relevance of the GMAT analytical writing assessment for management education? Management Communication Quarterly, 8, 347-367.
101.
Rogers, P.S., & Rymer, J. (1995b). What is the functional value of the GMAT analytical writing assessment for management education? Management Communication Quarterly, 8, 477-494.
102.
Rogers, P.S., & Rymer, J. (1996a). The GMAT analytical writing assessment: Opportunity or threat for management communication? Business Communication Quarterly, 60(2), 7-25.
103.
Rogers, P.S., & Rymer, J. (1996b). "What shall we do with these essays?" Using the analytical writing assessment for diagnostic purposes. Selections , 39 (Spring): 25-39.
104.
Rogers, P.S., & Rymer, J. (1996c). The Analytical Writing Assessment: Using the test results for diagnostic purposes (131 pp.). Prepared for the Graduate Management Admission Council, McLean, VA.
105.
Rogers, P.S., & Rymer, J. (1996d). The Analytical Writing Assessment diagnostic program (69 pp.). Prepared for the Graduate Management Admission Council, McLean, VA.
106.
Russell, D.R. (1991). Writing in the academic disciplines, 1870-1990 . Carbondale: Southern Illinois University Press.
107.
Ruth, L., & Murphy, S. (1988). Designing writing tasks for written assessment of writing. Norwood, NJ: Ablex.
108.
Rymer, J., Beard, J.D., & Williams, D.L. (1990, November). Correlating correctness and functionality in MBA writing assessments. Paper presented at the meeting of the Association for Business Communication , San Antonio.
109.
Schriver, K.A. (1989). Evaluating text quality: The continuum from text-focused to reader-focused methods. IEEE Transactions on Professional Communication, 32, 238-255.
110.
Schriver, K.A. (1992). Teaching writers to anticipate readers' needs: What can document designers learn from usability testing? In H. P. Maat & M. Steehouder (Eds.), Studies of functional text quality (pp. 141-157). Amsterdam: Rodopi.
111.
Shaughnessy, M. (1977). Errors and expectations: A guide for the teacher of basic writing. New York: Oxford University Press.
112.
Sloan, G. (1990). Frequency of errors in essays by college freshmen and by professional writers. College Composition and Communication , 41, 299-308.
113.
Smagorinsky, P. (1994). Speaking about writing: Reflections on research methodology. Sage Series in Written Communication, Vol. 8. Newbury Park, CA: Sage.
114.
Spivey, N.N. (1990). Transforming texts: Constructive processes in reading and writing. Written Communication, 7, 256-287.
115.
Straub, R. (1996). The concept of control in teacher response. College Composition and Communication, 47, 223-251.
116.
Suchan, J. (1998). The effect of high-impact writing on decision making within a public sector bureaucracy. The Journal of Business Communication, 35, 299-327.
117.
Swales, J. (1990). Genre analysis. Cambridge: Cambridge University Press.
118.
Swales, J.M. (1998). Other floors, other voices. Mahwah, NJ: Lawrence Erlbaum.
119.
Thomas, J.P. (1995, November). Coherence: Toward a logic for effective management documents. Paper presented at the meeting of the Association for Business Communication, Orlando .
120.
Toulmin, S.E. (1958). The uses of argument. Cambridge : Cambridge University Press.
121.
vanDijk, T.A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press.
122.
Walker, C., & Elias, D. (1987). Writing conference talk: Factors associated for high- and low-rated writing conferences. Research in the Teaching of English, 21, 226-85.
123.
Walker, R. (2001). Improving writing and analytic skills: The development of a student-centered system. Unpublished manuscript.
124.
Wall, S.V., & Hull, G.A. (1989). The semantics of error: What do teachers know ? In C. M. Anson (Ed.), Writing and response: Theory, practice, and research (pp. 261-292). Urbana, IL: National Council of Teachers of English .
125.
Walter, O.M. (1966). Speaking to inform and persuade. New York: Macmillan.
126.
Walvoord, B.E., & Anderson, V.J. (1998). Effective grading: A tool for learning and assessment. San Francisco: Jossey-Bass.
127.
Webster, L., & Ammon, P. (1994). Linking written language to cognitive development: Reading, writing, and concrete operations. Research in the Teaching of English, 28, 89-109.
128.
Weigle, S.C. (1999). Investigating rater/prompt interactions in writing assessment: Quantitative and qualitative approaches. Assessing Writing, 6, 154-178.
129.
White, E.M. (1994). Teaching and assessing writing. San Francisco: Jossey-Bass.
130.
Williams, J.M. (1981). Style: Ten lessons in clarity and grace. Glenview, IL: Scott, Foresman.
131.
Williams, J.M. (1982). The phenomenology of error. College Composition and Communication, 32, 152-168.
132.
Williams, J.M. (1991). Rhetoric and informal reasoning: Disentangling some confounding effects in good reasoning and good writing. In J. F. Voss, D. N. Perkins , & J. W. Segal (Eds.), Informal reasoning and education (pp. 225-246). Hillsdale, NJ: Lawrence Erlbaum.
133.
Williamson, M.M. (1993). An introduction to holistic scoring: The social, historical and theoretical context for writing assessment. In M. M. Williamson & B. A. Huot (Eds.), Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 1-43). Cresskill, NJ: Hampton Press.
134.
Winsor, D. (1996). Writing like an engineer: A rhetorical education . Mahwah, NJ: Lawrence Erlbaum.
135.
Witte, S.P., & Cherry, R.D. (1994). Think-aloud protocols, protocol analysis, and research design. In P. Smagorinsky (Ed.), Speaking about writing: Reflections on research methodology (pp. 69-85). Newbury Park, CA: Sage.
136.
Witte, S.P., & Flach, J. (1994). Notes toward an assessment of advanced ability to communicate. Assessing Writing, 1, 207-246.
137.
Wolcott, W., with Legg, S.M. (1998). An overview of writing assessment: Theory, research, and practice. Urbana, IL: National Council of Teachers of English.
138.
Writing Programs Fact Sheet, University of Michigan Business School (2001). Available: http://webuser.bus.umich.edu/Programs/BMC/WritingPrograms.html
139.
Yancey, K.B. (1999). Looking back as we look forward: Historicizing writing assessment. College Composition and Communication , 50, 483-503.
140.
Yates, J. & Orlikowski, W.J. (1992). Genres of organizational communication: A structurational approach to studying communication and media . Academy of Management Review, 17, 299-326.
141.
Yeh, S.S. (1998). Validation of a scheme for assessing argumentative writing of middle school students. Assessing Writing, 5, 123-150.
