The ouroboros of rubrics: A conundrum,a case,and a call

Abstract

Abstract

In this think piece, the author explores a conundrum and tension related to using rubrics to evaluate doctoral work. She ponders whether the use of rubrics provides beneficial ways for students to “crack the code” of academia, and/or whether the use of rubrics is perhaps a tool to engender conformity. With these competing ideas in mind, the author considers in what ways one might press for means to provide this on-ramp of access for students to the existing power structures, while at the same time seeking to change academia, to more equitably provide spaces for a range in ways of knowing, growing, expressing, framing, and presenting research. How might scholars support their students in engaging in what Rochelle Gutiérrez describes as simultaneously “playing the game” while also “changing the game?” Invoking challenges to existing power structures, such as those voiced in critical race theory and decolonizing epistemologies, this essay speaks to the tensions within the sacred spaces in academia which have been established by those with the greatest historical power.

The conundrum

How can we, as educators, simultaneously provide our students with unambiguous instructions for their assignments, while at the same time providing luxurious and loving opportunities for creative interpretations of the work we engage in? When it comes to assignments, how can we remove the expectation that students read our minds, while at the same time allow for unique and individual responses to prompts? How can we provide guideposts without limiting the paths students may walk? This is our grapple, our affliction, our bane. And, more specifically, how can we, as instructors, in recognition of our moral and ethical responsibilities as a part of our academic integrity, use a rigid rubric to evaluate the work of our doctoral students? Enter the ouroboros, the serpent devouring itself, speaking to the circular tensions in this conundrum. How can we be entirely transparent about expectations, while at the same time allowing for students to independently interpret what the work is intended to be?

The context

As a committed educator, I am the professor in a graduate program (at the Master’s and doctorate levels) with students who are curious and wise and eager, each hungry in different ways for the academic paths they are envisioning and constructing for themselves. All cluster their interests and passions beneath the umbrella of our department—Curriculum and Instruction—but with uniquely tailored points of focus for their research. A student may focus on the role of play in early childhood settings, the historical marginalization of teachers of color, or styles of teacher–family interactions that best serve the needs of students. Each voice adds a new layer, a new angle, a new nuance to the collective conversation, challenging, strengthening, and deepening our understandings on a kaleidoscope of topics, and speaking to the ideal Vygotsky (1966: 43) explained as “[b]ecom[ing] ourselves through others.”

As a professor, I do not hold unassailable expertise on the topics these students have selected. Rather, I have familiarity with the field of curriculum and instruction as a whole, and familiarity with the process of earning a doctorate, or what Shulman (1986: 9) would term “pedagogical content knowledge.” As professors and students working together, we live through these cycles of the ouroboros—again, the serpent eating its own tail—cycling into and out of knowing, wondering, wavering. I have experience in scholarly writing, and in research, and in what it means and what it takes to earn a doctorate. As such, we together form a community of practice (Lave and Wenger, 1991), learn in community, and labor in community with our students and with one another, spinning together, in Anzaldúa’s (1987) terms, tapestries (metaphorical, of course) of new truths and questions, threaded with both poetic and mundane ideas, fluidly moving between insights from the Buddha (on our syllabus) and from Kincheloe (in our textbooks). We press forward, metaphorical lamps in hand, squinting ahead into fresh territories and fixing our ears towards new tones, learning with and learning from one another through the process. This whirled richness, these voices of our students, is what enlivens and motivates and sustains us in the work.

The work

The work is here, in “the academy.” This work is both of the academy and in the academy, with each preposition acknowledging different facets of the same gem. In collaboration with my peers and students, we work within the confines of the structures, while at the same time occasionally jamming our elbows against the most rigid parts of the framework itself, mindful of the potential for life-changing consequences (such as failing to pass muster with institutional review boards, failing to successfully complete the dissertation, or injuring relationships with others).

The culminating projects

Each of the students in our graduate programs over the last few years has approached key academic milestones of various forms, sometimes as a comprehensive examination, or a dissertation proposal, or a final action research project. At our institution, as is the case in many institutes of higher education in the USA, these milestones often take the form of a lengthy paper, with an accompanying oral presentation—which, in our case, is bounded by a one-size-fits-all rubric, universally applied, whether the student is writing about accommodations in special education, policies of school suspension, or literacy strategies for emergent bilinguals. In our programs, we use one of several “universal” rubrics intended to evaluate work which may be focused on the role of faith identity in teachers, or the use of technology as a curricular focus, or the ways bias may cloud the views of school librarians.

One rubric for each type of assignment: the rubric for each type of assignment

The rubric: intended to both guide and evaluate. The rubric: a graciously constructed road map to guide the way or a clinically precise demand to conform? The rubric: a student’s savior or a student’s bane? This is not to suggest that such clear binaries—savior/bane—actually exist. Might a rubric be both, depending on the angle from which one views it?

Rubrics, defined by Popham (1997: 72) as “a scoring guide used to evaluate the quality of students’ constructed responses,” are seen as “writing assessment’s current sacred cow because they provide the appearance of objectivity and standardization” (Wilson, 2007: 66). While emphasizing this concept of objectivity and “fairness,” rubrics are also marketed to educators as a way to “save time” while grading. Critical of this self-serving rationale, Kohn’s (2006: 12) response to this notion of “saving time” was that “the best teachers would react to that selling point with skepticism, if not disdain.” As Zinn (as cited on goodreads.com) expressed:

I’m worried that students will take their obedient place in society and look to become successful cogs in the wheel—let the wheel spin them around as it wants without taking a look at what they’re doing. I’m concerned that students not become passive acceptors of the official doctrine that’s handed down to them from the White House, the media, textbooks, teachers and preachers. (Zinn 2016)

Like Zinn, I, too, fear that my students—not to mention myself—might take their obedient place in society, quietly accepting what has been handed down.

The case

The vividness of the tensions in the potential for bane with the use of rubrics has taken shape during the last few years of their use in my professional practice as a professor. As various students have spoken to the nuances of their chosen topics, it is clear that some students have, in many ways, been strongly guided, supported, and buoyed by the lift and loft provided by the rubric. Others, however, were lashed to the masts of the ship that was the rubric, working so hard to speak to each of the elements in that gridded “epistemological cage” (D’Ambrosio, 2015: 23) through the voice of their research passions.

Many students have been entirely at home as they open their presentations, just as the rubric outlines, by identifying their chosen “problem of practice” and recognizing its boundaries. In lockstep conformity with the boxes on the rubric, they describe the context, they validate that the problem exists, and they define their terms with clarity and deep insight. They speak with authority to the educational significance of the problem, invoking relevant theories to frame and articulate their work. The rubric often works well at this point, providing a tensile structure upon which students can unfurl their insights, their passions, their wonderings.

And then, often, there emerges some part of the rubric that requires students to focus on something, some element, not entirely (or not even tangentially) linked to their curiosity, their focus, their topic as a whole. The rubric demands something unanticipated by the author of the tool, something not connected to the reality lived by the student-author. The fit then moves from tightly mapped to awkwardly matched. As faculty, we are using a very specific organizational structure—one not chosen or designed by the students themselves—to evaluate the work the students themselves have decided upon. Narrative tensions arise in navigating how best to fit any and all topics into the structures defined by the rubric, a priori.

In considering the rubric itself, we, as scholars, are aware that the very structure of the rubric is a representation of a machine-like system, and that while using it to evaluate student work, we may be engaging in a highly impositional act, with an overt directive to conform. Suddenly, Popham’s (1997: 74) declaration that “the vast majority of rubrics are instructionally fraudulent” takes on new urgency. In what ways might we, as faculty, be participants in—or purveyors of—a kind of instructionally fraudulent act? Or rather, are we providing clear guidance and useful structures?

How, then, might we be colluding in a form of broad and bold acculturation? Some may argue that this is entirely the point of a graduate program—to elicit conformity to the master script, recasting our students in our own image. The creators of the rubrics themselves must surely see rubrics as gift-like, touting the benefits of rubrics for their transparency, while downplaying (or perhaps outright denying) any elements of conformity built into them. Points about transparency are entirely vivid and valid, but … what about the nagging problem with conformity? Does this troubling aspect somehow cancel out because the transparency is so useful?

In indignant support of rubrics, some have asked: “How, then, do you dismantle the master’s house?”—strongly suggesting that rubrics provide access to the master’s house, serving as “the tools” with which to dismantle said house, as if the work of scholarly research could possibly be so discrete, so bounded, so formulaic, or so technical.

Clearly seeking to reference the work of Audre Lorde, and in particular Lorde’s (1984) quote about “the master’s tools,” those posing this question (“How, then, do you dismantle the master’s house?”) have somehow reshaped the quote in their own thinking and, as a result, recast the meaning to align with their own genuinely benevolent beliefs and intentions about rubrics. What Lorde actually wrote was far more critical of the institutions of power:

Those of us who stand outside the circle of this society’s definition of acceptable women; those of us who have been forged in the crucibles of difference—those of us who are poor, who are lesbians, who are Black, who are older—know that survival is not an academic skill. It is learning how to take our differences and make them strengths. For the master’s tools will never dismantle the master’s house. They may allow us temporarily to beat him at his own game, but they will never enable us to bring about genuine change. And this fact is only threatening to those women who still define the master’s house as their only source of support. (Lorde, 1984: 112)

Many students at my institution (just like students across the USA) speak from the place Lorde described—“outside the circle.” At the same time, pro-rubric educators animate the idea that “many of the tenets of positivism are so embedded within Western culture, academia, and the world of education in particular that they are often invisible to researchers and those who consume their research” (Kincheloe and Tobin, 2009: 513). Similarly, just as Said (2003) explained: “Every empire, however, tells itself and the world that it is unlike all other empires, that its mission is not to plunder and control but to educate and liberate.” The pro-rubric community attempts to put forward the same idea—that the rubric itself is not a prison, but rather a gift to our students, serving to grant access.

So herein I return to my particular bane with regard to rubrics. As an institution, we want to provide access to the “keys to the kingdom,” while at the same time we want to disrupt and change the kingdom itself. Speaking to this tension, Kohn (2006: 13–14) points out that: “There’s a big difference between thinking about the content of a story you’re reading … and thinking about your own proficiency at reading.” How might we, as educators who seek to practice radical love, which West (2015: 3) describes as something which has “a radical humility and radical integrity,” simultaneously provide loving and structured support for our students, while at the same time knowing that access to the academy, in its current state, requires some degree of conformity and acculturation?

The conclusion (of sorts), and a call

So, where does this leave us, as scholars who may have already gained access to the academy, and perhaps not without loss, wound, or compromise, but with the deep desire to disrupt and change the oppressive and restrictive structures in place that serve to bar, to limit, and to deny others seeking access? Perhaps we are still animating the ouroboros, devouring our own tails through infinite cycles, using the rubric (because it is institutionally required) while simultaneously knowing that it is a limited and constrictive construct in so many, many ways. We have paid the ferryman—maybe Vasudeva or perhaps Charon—for safe passage across the river into the land of academe, but find ourselves wondering if now we have somehow become that very ferryman. As Kress wonders:

For scholars such as myself who have been immersed in Western culture and Western notions of research and science, colonialism and positivism have been normalized to the point where they feel commonsensical and are often hidden from me even as I might embody and enact these norms. (Kress, 2011: 270)

Do we compromise standards and quality and rigor, as defined by some communal (but external) experts, and permit our students to create works that somehow fall outside of what is considered to be “scholarly” by these measures? Do we cede to the need for ethical inter-rater reliability to ensure that all students have equitable chances of passing their high-stakes comprehensive examinations? Or do we support our students in growing along their own trajectories, trusting that they will succeed in this process, and allowing them to carve new paths that we, ourselves, may have not yet come to recognize or understand?