A Decade of Research: Assessing Change in the Technical Communication Classroom Using Online Portfolios

Abstract

Over a period of 10 years, we have developed a sustainable process of online portfolio assessment that demonstrates both reliability and validity, using both qualitative and quantitative measures. The sustainable cycle is that, each semester, we assess a random sampling of the students' work that they have posted, as per our instructions, in an online portfolio. During the reading, the faculty score the documents for 11 variables, including writing, content, audience awareness, and document design. We achieved validity by a modified online Delphi that led to a redefinition of the construct of technical communication itself; we achieved reliability by adjudication resulting in adjacent scores. The results of our assessment meet the requirements of ABET and result in a continual cycle of improvement for our technical communication curriculum. Results from three semesters show an improving correlation between the course grade and the overall, holistic portfolio score.

Get full access to this article

View all access options for this article.

References

Huot

, (Re)Articulating Writing Assessment for Teaching and Learning, Utah State University Press, Logan, 2002.

For new paradigms in writing assessment, see the following: Huot

Brian

, (Re)Articulating Writing Assessment for Teaching and Learning (Utah State University Press, Logan, 2002); Broad

Bob

, What We Really Value: Beyond Rubrics in Teaching and Assessing Writing (Utah State University Press, Logan, 2003); Yancey

Kathleen Blake

, Postmodernism, Palimpset, and Portfolios: Theoretical Issues in the Representation of Student Work, College Composition and Communication, 55: 4, pp. 738–761, 2004, and Lynne

Patricia

, in Coming To Terms: A Theory of Writing Assessment, (Utah State University Press, Logan, 2004).

Criteria for ABET can be found on their website at: <http://www.abet.org/forms.shtml> August 19, 2006.

Several authors write about this topic. See Broad's

Bob

What We Really Value: Beyond Rubrics in Teaching and Assessing Writing (Utah State University Press, Logan, 2003) and Elliot's

Norbert

On a Scale: A Social History of Writing Assessment in America (Peter Lang, New York, 2005).

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, Standards for Educational and Psychological Testing, American Psychological Association, Washington, 1999.

Camp

, New Views of Measurement and New Models for Writing Assessment, in Assessment of Writing: Politics, Policies, and Practices, White

E. M.

Lutz

, and Kamusikiri

(eds.), MLA, New York, 1996.

Sternglass

M. S.

, Time to Know Them: A Longitudinal Study of Writing and Learning at the College Level, Lawrence Erlbaum, Mahwah, 1997.

Miller

C. M.

, Reuniting Wisdom and Eloquence within the Engineering Curriculum, in Liberal Education in Twenty-First Century Engineering: Responses to ABET/EC2000 Criteria, Ollis

D. F.

Nelly

K. A.

, and Lugenbiehl

H. C.

(eds.), Lang, New York, p. 49, 2004.

Johnson

and Elliot

, Portfolio Assessment in Program Evaluation, ASEE Mid-Atlantic Section 2004 Fall Conference, Engineering Workforce for the 21st Century, Digital Proceedings, October 22–23, 2004.

10.

Prados

Peterson

, and Lattuca

, Quality Assurance of Engineering Education through Accreditation: The Impact of Engineering Criteria 2000 and Its Global Influence, Journal of Engineering Education, 94: 1, pp. 165–184, 2005.

11.

Coakley

, Placement in English Composition Courses: When “One-Size-Doesn't-Fit-All,” Fourteenth International Conference on the First-Year Experience, 2001.

12.

Elliot

, On A Scale: A Social History of Writing Assessment in America, Peter Lang, New York, 2005.

13.

Elliot

Kilduff

, and Lynch

, The Assessment of Technical Writing: A Case Study, Journal of Technical Writing and Communication 24: 1, pp. 9–36, 1994.

14.

Coppola

, Setting the Discourse Community: Tasks and Assessment for the New Technical Communication Service Course, Technical Communication Quarterly, 8: 3, pp. 249–267, 1999.

15.

Yancey

K. B.

, Postmodernism, Palimpset, and Portfolios: Theoretical Issues in the Representation of Student Work, College Composition and Communication, 55: 4, pp. 738–761, 2004.

16.

Broad

, What We Really Value: Beyond Rubrics in Teaching and Assessing Writing, Utah State University Press, Logan, 2003.

17.

Coppola

and Elliot

, Assessment of Graduate Programs in Technical Communication: Constructing a Model to Engage Complexity, in Assessment in Professional and Technical Communication, Allen

and Hundleby

(eds.), SUNY Purchase, New York, forthcoming.

18.

Dayton

and Bernhardt

S. A.

, Results of a Survey of ATTW Members, 2003, Technical Communication Quarterly, 12: 1, pp. 13–43, 2004.

19.

Linstone

H. A.

and Turoff

, The Delphi Method: Techniques and Applications. Available at: <http://www.is.njit.edu/pubs/delphibook/> August 19, 2006.