Abstract
One path to improving adolescents’ literacy skills is to integrate reading and writing into the content areas in which such work occurs. Although argumentative writing has been found to help students understand historical content and transform information, scholars do not know the influence of specific task structures on students’ writing or historical reasoning. To learn more, the authors administered four document-based writing tasks on the origins of the Cold War to 101 students from 10th or 11th grade. Using multiple regression, the authors found that writing tasks explained 31% of the variance in the quality of students’ overall historical reasoning after accounting for differences in students’ background. A closer analysis of different aspects of historical reasoning using a different rubric (and as analyzed using MANOVA) indicated that students’ skill in recognizing and reconciling historical perspectives significantly improved with writing tasks that asked them to engage in sourcing, corroboration, and causal analysis. The task that asked students to imagine themselves as historical agents and write in the first person was significantly different and resulted in the lowest mean essay scores.
Academic literacy is critical to success in secondary schools and professional life. As adolescents prepare for the demands of college and beyond, they must learn to read and write increasingly complex and specialized forms of text. When writing, in particular, students must go beyond telling what they know with text, to engaging in knowledge construction, reasoning, and discourse with text (Bereiter & Scardamalia, 1987). In history, this written work requires disciplinary thinking such as taking an investigative stance toward the past and an understanding of the norms of knowledge construction and communication in the discipline (cf. Bain, 2006). As students make the transition from basic to academic literacy, adolescent writers must adapt to a variety of tasks, rhetorical structures, and standards that vary from one discipline to the next (Ackerman, 1991; Beaufort, 2004; Coffin, 2004; Geisler, 1994; Moje, 2008; Shanahan & Shanahan, 2008; R. Stevens, Wineburg, Herrenkohl, & Bell, 2005). Unfortunately, data from the National Assessment of Educational Progress (NAEP) suggest that most adolescent writers are not prepared to make this transition. A recent NAEP report found that fewer than 35% of eighth graders wrote essays at or above the proficient level—a standard defined as “solid academic work for the grade level” (Salahu-Din, Persky, & Miller, 2008). The data suggest that there is a large population of students who struggle with the demands of academic literacy in writing. In response, recent reports such as Writing Next (Graham & Perin, 2007) and the National Commission on Writing (Magrath & Ackerman, 2003) and reviews such as those found in this journal (Faggella-Luby, Ware, & Cappozoli, 2009) have called for increased attention to adolescent writing instruction that is embedded in content courses.
Disciplinary literacy has emerged as a pathway to advance students’ literacy skills and necessitates the integration of literacy and subject matter (cf. Moje, 2008). Such an emphasis highlights the ways of thinking and knowing in a discipline as key to learning how to reason, read, write, and discuss. As Moje and her colleagues (2004) argue, an integral part of learning a discipline involves learning the oral and written language of the discipline. By adolescence, students confront subject-specific texts and tasks that require specialized forms of knowledge. Ways of thinking and reasoning associated with a particular discipline are embedded in subject-specific texts and tasks and must be attended to if we are to help adolescents become proficient readers and writers (Coffin, 2004; Moje, 2008; Monte-Sano, 2010; Shanahan & Shanahan, 2008). Any focus on writing, therefore, implicates ways of thinking in the content area in which the writing occurs. We know from prior work that writing essays in history can improve students’ mastery and understanding of the content (Bangert-Drowns, Hurley, & Wilkinson, 2004; De La Paz, 2005; Smith & Niemi, 2001), enhance their ability to integrate content from sources with their own thinking (Wiley & Voss, 1999; Young & Leinhardt, 1998), and promote historical thinking (Monte-Sano, 2010). Although this research on writing correlates general tasks with disciplinary outcomes, it does not address the potential impact of the structure of the task. Do all argumentative writing tasks provide students with the same opportunity to exhibit or develop their historical thinking or writing? Or might some ways of framing questions and/or combinations of materials with questions promote historical thinking and writing better than others? In this study, we set out to understand whether the structure of a writing task—in particular, the structure of an argumentative writing task—could influence students’ historical thinking and writing.
Reform-oriented history instruction emphasizes reading and writing from historical documents and places high demands on struggling writers. Students must analyze first- and secondhand accounts of events in history and then write an essay that either advances an interpretation of events or advocates for a position based on information available to decision makers at the time. To date, much of the research on history writing has focused on how students draw on multiple sources in constructing argumentative essays (De La Paz, 2005; De La Paz & Felton, 2010; Monte-Sano, 2010; Young & Leinhardt, 1998). Some attention has been granted to how students represent and construct historical arguments (Britt, Rouet, Georgi, & Perfetti, 1994; Stahl, Hynd, Britton, McNish, & Bosquet, 1996; Wiley & Voss, 1999) in comparison with other genres of writing such as narratives, summaries, and explanations. But within the genre of argument, the structure of particular tasks and their influence on students’ performance remain unexamined. Although research has examined what students do when asked to write history essays, little research has been conducted on what we ask students to write and how that affects their disciplinary thinking and writing.
Conceptual Framework
Embedding adolescent writing instruction in history courses requires connecting disciplinary reasoning with writing. To do so is not a stretch, for making the case for a particular interpretation in writing is the keystone of history qua discipline (Mink, 1987). Toulmin’s (1958) argumentation framework lays out such key aspects of writing as claim, data, warrant, and counterargument, components that can apply to most disciplines, including history. Historical writing shares an argumentation stance with other forms of writing, but the nature of the data and warrants (the evidence and connection between evidence and claim) seem to be discipline specific (Bruner, 1960; Hexter, 1971; Monte-Sano, 2010; Schwab, 1978). In constructing historical arguments, writing is often inextricable from a disciplinary way of thinking and working with evidence. According to history experts, the use and framing of evidence in historical writing indicate key aspects of disciplinary reasoning including recognizing biases in sources, comparing evidence, situating evidence in its context, and taking into account different perspectives and multiple causes (Carr, 1961; Coffin, 2004; Collingwood, 1943; Hexter, 1971; Mink, 1987).
Historical interpretations rely on the public display of evidence to substantiate claims: that is, a claim cannot stand without evidence to support it (Collingwood, 1943; Hexter, 1971). The inclusion of examples, details, footnotes, and quotations from primary and secondary sources exemplifies this aspect of reasoning. Stating where evidence comes from (i.e., sources of quotations and information) allows others to understand and evaluate the basis for one’s claim. Furthermore, historical interpretations must account for the available evidence (Hexter, 1971). This may involve altering interpretations to accommodate contradictory evidence. Comparing different and contrasting documents is a visible form of this type of reasoning.
Because the goal of historical interpretation is to understand the past, historical reasoning involves reading and writing about sources from the perspectives of those who created it and placing sources into their historical context. Such contextualization is central to history, in that historians may only interrogate artifacts from the past. The events under study cannot be repeated: Historians have usually not witnessed the events about which they write, and the authors of documents used to analyze the past are inaccessible (cf. Hexter, 1971). To write an interpretation that argues why something happened in the past, or what compelled someone to write a particular text, historians must situate authors and events in the context of contemporary events, peers, and ideas; such writing highlights the relationships between contiguous events (cf. Mink, 1987). Historians do not look for generalizable rules that can be applied to future situations but, rather, seek specialized understanding of the particular circumstances of past events. Absent context, historical understanding is at risk.
Key aspects of historical writing cited by philosophers of history and historians are consistent with research identifying the nature of disciplinary thinking in history. Wineburg’s (1991) seminal study identified three heuristics that historians use when approaching texts. He found that historians interrogated historical documents by looking at authors and their biases (sourcing), situating documents in the time and place of their creation (contextualization), and comparing documents (corroboration) to find points of agreement and contradiction. In defining expert approaches to historical texts, Wineburg identified discipline-specific ways of reading and thinking. For these historians, primary documents were regarded as excerpts of social interactions that had to be reconstructed to render the documents comprehensible. Recent research confirms that these ways of thinking are apparent in the argumentative, document-based history essays of students whose teachers emphasize historical thinking and argumentative writing (Monte-Sano, 2010). In particular, researchers have identified attention to evidence, historical context, perspective, author point of view, accuracy, comparison of sources, and causality in students’ writing (Coffin, 2004; De La Paz & Felton, 2010; Monte-Sano, 2010; Young & Leinhardt, 1998). These forms of reasoning can theoretically be expressed in different forms of writing, including narrative, argumentative, or descriptive writing. However, one study of novice teachers showed that summary writing can coincide with an emphasis on reporting information in history classes, rather than disciplinary thinking (Monte-Sano & Cochran, 2009). In the research reported report here, we focus on argumentative writing and whether framing the writing task can promote such historical thinking.
Writing arguments has been compared with other writing genres (e.g., narration and description) and found to promote greater audience awareness and syntactic complexity in high school students (Crowhurst & Piche, 1979). Writing arguments, in particular, may actually help students integrate historical content into their essays because students must interpret and organize information from historical documents in a new way (Newmann, 1990). Results from one study with high school students provide support for this idea. Stahl and his colleagues (1996) found that writing arguments helped 10th grade students become better at historical reasoning about controversial topics. Asking students to write opinions resulted in their ability to produce more global statements integrating more than one source as compared to their descriptions, in which they tended to stay closer to the readings, with most of their statements coming from a single source. Similarly, Wiley and Voss (1999) found that, more so than with other genres, writing arguments with multiple documents improved students’ content understanding and skill in transforming information into an integrated essay.
However, writing argumentative essays, in itself, does not always promote disciplinary thinking (Grant, Gradwell, & Cimbricz, 2004; Young & Leinhardt, 1998). Greene (1994) found that historians consistently approach writing as argument, whereas students may perceive the same tasks as demanding summary of information. In other words, students do not understand writing tasks in the same way as experts—students’ task representations for writing history do not tend to include argument or interpretation. If students do recognize that a task demands argument, they may read the prompt, figure out what they want to say, and then use the documents to support their argument (Young & Leinhardt, 1998). From a disciplinary standpoint, this is problematic since the historical evidence is never fully considered in forming the argument. The role for instruction, then, is to help students understand historical writing as a form of argument that involves constructing an interpretation that reflects the available evidence.
Monte-Sano’s (2011) work suggests that providing writing prompts that require close reading of texts or consideration of authors’ perspectives in constructing an argument support historical thinking and understanding of content. However, no study has tested this theory. Brooks (2008) found that prompts asking students to write in the first person as though they were living at the time they write about can both improve and weaken students’ historical empathy as compared to prompts that direct students to write in the third person about the perspectives of historical agents. Although small-scale studies give hints about the impact of different writing tasks, it remains an open question whether a focus on the nature of a writing prompt can help teachers consistently elicit their students’ disciplinary thinking.
Method
Given this background, we ask the following questions: Do the structure and focus of the writing prompt affect the quality of students’ historical reasoning? If so, how? With this purpose in mind, we designed assessment tasks to determine how the type of writing prompt affects the quality of students’ written arguments. We constructed four reading and writing tasks using the same historical documents and randomly assigned one task to each student within 11 classes. We spent 3 days in students’ social studies classes—one day to administer a subtest from a published, standardized writing test; a second day to observe classroom sessions focused on learning background knowledge and analyzing historical documents in preparation for writing; and a third day to observe teachers’ administration and assignment of the writing tasks. More detail about each component of our study follows.
Writing Tasks
We randomly assigned students within each class to complete one of the four assessment tasks (Table 1 shows the outcome of this assignment). Each assessment task presented the same background information adapted from students’ social studies textbook:
The United States and the Soviet Union emerged from World War II as the two strongest powers in the world. Each had different political and economic systems—totalitarian government and communism in the Soviet Union; democracy and capitalism in the United States. Both countries wanted their own political and economic systems to spread to other countries. The different goals of the U.S. and the Soviet Union led to the Cold War, a conflict in which neither country directly confronted the other on a battlefield. The following documents focus on the concerns of the US and its ally, Great Britain, regarding the Soviet Union at the outset of the Cold War.
Summary of Student Characteristics by Condition
Note: Grades converted to 4.0. WIAT = Written Expression subtest of the Wechsler Individual Achievement Test, 2nd edition; Quiz = percentage correct on 10-item multiple-choice understanding of background information related to the Cold War topic.
Grades were available for 99 of 101 students.
Quizzes were completed by 91 of 101 students.
An assessment task prompt was given to students after they read the background information (see Table 2). Each writing prompt focused on understanding Western views about the origins of the Cold War conflict and why Western leaders might speak out against the Soviet Union. However, we worded each prompt differently to frame the issue of Cold War causes from a variety of historical angles. The first prompt (situated) encouraged students to imagine they heard these speeches and write as though they were living in 1947. We developed this prompt based on Brooks’s (2008) finding that first person prompts that solicited a first person stance as though students were a historical agent led to more inferential thinking for middle school students. The intent of the second prompt (sourcing) was to encourage students to focus on the motivations of each author in making their respective speeches. We designed this prompt in part from a belief that encouraging students to attend to the author’s perspective would elicit more sophisticated historical thinking in students’ writing. This was from our study of Monte-Sano’s (2011) work in which students’ historical thinking improved when faced with repeated prompts that focused students on the perspectives of the authors of historical documents. Prompt 3 (document analysis) encouraged students to identify the similarities and differences in the two documents. Analysis of text is a common form of writing in history (e.g., Benjamin, 2004; Marius & Page, 2005). This prompt is similar to those found on the International Baccalaureate history exam (Paper 1) in which students analyze historical documents in writing. 1 It also builds off of Monte-Sano’s (2011) work in the same classroom in which students were directed to read carefully and corroborate sources as they wrote. Writers in this classroom showed marked improvement in historical thinking over time. The final prompt (causal) asked students why Churchill and Truman spoke out against the Soviet Union and communism directly. This prompt is typical in historical writing in its focus on how and why something happened as well as the significance of events (e.g., Coffin, 2004; Marius & Page, 2005; Rampolla, 2010). Each prompt uses a different method of historical analysis—imagining the setting, sourcing, document analysis, and causal reasoning—to understand a major event in history (e.g., Holt, 1990; Wineburg, 1991).
Cold War Writing Tasks for 10th to 12th Grade
At the end of each prompt, we asked students to write two “M.E.A.L. paragraphs” in their response. M.E.A.L. stands for main idea, evidence, analysis, and link to thesis. The teachers at the participating high school had taught students this mnemonic in an effort to improve their argumentative writing. The teachers believed this mnemonic would help students understand what they were being asked to do; in addition, it was regularly provided on nearly all extended writing assignments.
Participants and Setting
Martin Luther King, Jr. High School is a public charter school in the inner city of a major urban area in the mid-Atlantic region. In the 2007-2008 academic year, the school reported that of the 427 students, 73% were African American and 26% were Latino. Of the students, 71% were eligible for free or reduced-price meals, and 15% of the student body qualified for special education services. In addition, 6% were English language learners. In the academic year in which this study took place, 41% of 10th grade students scored below grade level on the district-administered high-stakes reading assessment (other grade levels do no participate in this assessment). The school had struggled to meet adequate yearly progress for both reading and math over the past 3 years.
Two U.S. and two Modern World History teachers and their students participated in the study (see Table 1 for descriptive information). The Modern World History course was primarily intended for 10th graders, and the U.S. History course was primarily intended for 11th graders. As shown in Table 1, older students occasionally participated in these classes because they had yet to pass them. A total of 101 student participants completed the pretest, experimental task, and assented to our review of academic records. These students were included in the analyses of student performance on the different writing tasks. We had additional data on the majority of the participants: 91 of these students also completed a quiz to determine their general understanding of the topic and 99 students had previous English and social studies grades recorded. The 101 students participated in one of 11 class sections (68 students from 8 Modern World History classes and 33 students from 3 U.S. History classes). Three of the four teachers reported regular use of primary sources and an emphasis on historical thinking in their classrooms. All identified improving students’ writing as a central goal for the year. The teachers identified the Cold War as a topic that could be used for both courses and as a topic that students in both grade levels would learn about, so we used this topic to include students from more than one grade. Of the 101 participants, 25% spoke English as their second language and 10% had previously received special education services or participated in a remedial reading program in middle school.
Students’ scores on the Written Expression subtest of the revised Wechsler Individual Achievement Test (WIAT-2), administered before the study began, were compared to determine whether students who completed the four assessment prompts differed significantly in initial writing ability. A one-way analysis of variance test evaluated the relationship between the four groups and performance. Students performed at virtually the same levels, F(3, 93) = 0.512, MSE = 176.93, p = .73. See Table 1 for descriptive information regarding students.
In addition, grades from the previous semester were available for 99 students in English and 99 students in social studies. There were significant differences for grades earned in English, F(3, 98) = 3.40, MSE = 1.37, p = .021, and grades earned in social studies, F(3, 98) = 4.89, MSE = 1.12, p = .003. Bonferroni tests were conducted to determine the source of these differences. Students’ English grades were significantly different between individuals who wrote on the sourcing prompt and those who wrote on the document analysis prompt. Students’ social studies grades were significantly different between individuals who wrote on the document analysis prompt and those who wrote on the causal and the sourcing prompts. See Table 1 for descriptive information regarding students.
General Procedures
All classes had covered the Cold War before we administered the writing task. As part of that exposure, all students had read the same background reading in their course textbook. All of our raters also read these pages of the textbook so that they knew what students had been exposed to before participating in any of the research tasks.
As an indicator of students’ general understanding of the topic, they completed a 10-question multiple-choice quiz on the Cold War that had been jointly constructed by teachers from both the Modern and U.S. History courses. This quiz included five questions from each course. A sample item is as follows:
The Iron Curtain was:
a. A physical barrier between the USSR and the rest of the world
b. A symbolic barrier separating capitalism and communism in Europe
c. A wall in China
d. The border between the US and Mexico
The day before completing the essay, teachers gave students an overview of the topic and introduced students to two primary sources (see Appendix A). Students then read and analyzed the primary sources using a worksheet designed to help students both comprehend and think historically about the documents. Students had used this worksheet before, titled “SOAPSTone” (see Appendix B). Using this mnemonic and two guiding questions per letter, students considered the speaker (e.g., Who made this?), occasion (e.g., What else was going on?), audience (e.g., Who was this document made for?), purpose (e.g., What did the author want?), subject (e.g., What is it about?), and tone (e.g., How does the author feel?). Students completed one worksheet per document and included the title and date of each document. Students were allowed to work individually or in small groups, and we asked teachers to debrief students’ responses at the end of the class.
We selected two primary sources that emphasized different viewpoints on the Cold War by Western leaders: Winston Churchill’s Iron Curtain Speech from March 1946 and Harry Truman’s Speech to Congress from March 1947. Together these documents focus on causes and origins of conflict with the Soviet Union, given the views of leaders in the United States and Great Britain. We made several adaptations to these documents to facilitate reading comprehension and minimize the influence of weak reading skills on the writing task. Adapting sources is a common approach in history education designed to give all students access to complex, subject-specific texts (cf. Wineburg & Martin, 2009). We shortened documents to one page (Churchill’s speech was originally three pages and Truman’s was seven pages). We substituted simpler vocabulary terms for more complicated words. For example, in the first paragraph of Churchill’s adapted speech we substituted peak for pinnacle, serious for solemn, blame for reproaches, and responsibility for accountability. Finally, we eliminated punctuation and markers of these adaptations (e.g., ellipses or brackets around a substituted word) that can be distracting for struggling readers. We took care to preserve the intended meaning and author’s perspective represented in the texts and consulted with participating teachers throughout this process to ensure readability for their students.
The day after students analyzed the Churchill and Truman sources, they wrote in response to one of four writing tasks using those historical documents and their document analysis worksheets. In administering the assessment tasks, teachers were directed to give students their historical documents and SOAPSTone worksheets, to go over the background information, and to distribute the writing tasks in the previously determined random order. Teachers were told not to read each task aloud so that students could focus only on the task given to them. Teachers were allowed to answer students’ questions on an individual basis to clarify the tasks. Students were then given 40 minutes (a full class period) to write. Teachers gave students the quiz on the background information as students turned in their essays; some students finished these quizzes during the same class period, whereas other students completed them the next day.
Data Analysis
Students’ topic knowledge and work with primary sources
Students earned 10 points for each question they answered correctly on the Cold War knowledge quiz. We used scores from this quiz to infer what students had learned from reading their textbook and participating in classroom activities on the topic of the Cold War. Students’ scores on this measure were compared to determine whether the four groups differed significantly in general understanding of the topic. A one-way ANOVA evaluated the relationship between the four groups and performance for the 91 students who completed this task. Students performed at virtually the same levels, F(3, 93) = 0.747, MSE = 338.87, p = .73. See Table 1 for descriptive information regarding students.
Students received two scores for their work with each document on the SOAPSTone worksheet. One score, Source Use, focused on whether students had annotated the document or generated any written notes in the margins of the text (1 point for each for a total of 2 points). The second score, SOAPSTone, focused on whether students answered both questions for each letter of the mnemonic correctly (1 point per question for a total of 12 points). After initial training, two raters gave two scores to the document analysis worksheet for each document (r = .88 for the Source Use score; r = .89 for the SOAPSTone score).
Students’ Cold War essays
Two variables were developed to analyze students’ writing. The first was a holistic, global measure of students’ writing (overall quality) and was measured with a 0 to 6 rubric (see Table 3). We adapted this rubric from Monte-Sano’s (2008) prior work on students’ historical writing, De La Paz and Felton’s (2010) prior work with argumentative writing in history, and qualitative analyses of patterns in the strengths and weaknesses of this particular set of essays. As a whole, this rubric measured the extent to which students synthesized ideas from documents into an argument, or interpretation, provided supporting evidence, contextualized their argument, and recognized the perspectives of historical actors in their written work (see Table 4 for excerpts from essays with different scores).
Holistic Scoring Rubric of Overall Historical Quality
Excerpts From Students’ Essays Using the Holistic Historical Quality Rubric
We trained two raters in two sessions lasting 2 hours each. During this time, we reviewed the historical background (textbook excerpts) and primary source documents, shared benchmark essays for each level on the rubric, and explained distinctions between each level on the rubric. Then, the raters learned to use the scoring rubric with 19 sample papers from students who had initially been included in the study but were subsequently determined to be nonparticipants because they had not completed all tasks. Once they achieved an acceptable reliability rate, they scored all 101 essays in the data set (Pearson r = .82).
The second dependent variable, historical reasoning, served as a measure of specific aspects of historical thinking that allowed us to analyze multiple dimensions of historical reasoning so that we could detect the influence of writing task on such reasoning with greater sensitivity. Scoring this dependent variable relied on an analytic trait rubric that focused on three specific aspects of historical reasoning—substantiation, perspective recognition, and contextualization—and resulted in a separate score for each (see Table 5). Substantiation emphasized the extent to which students provided evidence and explanation in support of a claim. Perspective recognition focused on students’ skills in presenting the texts as authors’ viewpoints that could be evaluated rather than as authoritative words to be accepted. Contextualization highlighted the extent to which students identified and situated their argument in the appropriate time, place, and setting, thus linking related events (see Table 6 for excerpts from essays with different scores). These are all aspects of historical reasoning that Monte-Sano (2010) has found in adolescents’ history essays.
Pretest–Posttest Analytic Rubric of Specific Aspects of Historical Reasoning
Students’ Essay Excerpts Using the Historical Reasoning Analytic Trait Rubric
We trained two different raters to use the analytic trait rubric in a manner similar to the holistic rubric training. The only difference was that we focused on one trait at a time when we looked at benchmarks, talked through distinctions in scores, and so on. Raters scored the complete set of 101 essays and achieved satisfactory reliability for each analytic trait (for substantiation Pearson r = .98, for perspective r = .99, for contextualization r = .95).
Results
Overall Quality
We performed a block-entry hierarchical regression to examine the effect of demographic information about students, information about their reading and background knowledge about the topic, and the four assessment prompts on predictions of overall writing quality. Theoretical predictions led us to expect that demographic and background information would be useful but not as important as the type of writing prompt in determining students’ overall writing quality. Therefore, blocks were entered in the following order:
Demographic information (including the students’ first language, prior grades in English and social studies, prior special education services, score on the WIAT-2 Written Expression subtest, and the student’s current social studies teacher)
Background information on the topic (score on the 10-item quiz, his or her annotations and overall scores on the SOAPSTone worksheets), which we used as a proxy for reading comprehension
The prompt or actual writing task
The results indicate that at Step 1, the demographic predictors included in the regression equation, R2 = .14, F(6, 73), p = .09, indicated the students’ prior learning profiles and writing ability did not reliably affect the overall historical quality of their essays. Moreover, after Step 2, the background predictors included in the regression equation, R2 = .23, F(5, 68), p = .17, indicated the students’ knowledge of the topic and reading comprehension of the primary sources did not reliably affect the overall historical quality of their essays. In Step 3, the writing prompt was added to the regression equation, and the R2 increased to .31, F(1, 67), p = .006, indicating that the assessment task that students responded to was predictive of a fair degree of the students’ overall historical quality in their writing. In other words, when considering students’ knowledge of the topic, their reading comprehension of the primary sources, and the different types of writing prompts, only the writing prompt significantly influenced student performance.
Historical Reasoning
Table 7 presents the means and standard deviations for the elements of historical reasoning (substantiation, perspective, and context) by writing prompt. We computed a 4 × 3 multivariate analysis of variance (MANOVA) to examine the effects of assessment prompt on the above-mentioned measures. We tested the assumptions of univariate (Levene’s test) and multivariate (Box’s M) homogeneity, and the latter was violated. Results of MANOVA are robust in the face of this violation when group sizes are at least 20 and fairly equal (see J. P. Stevens, 2002), as was the case in this study. Nevertheless, to be conservative, we used Pillai’s trace to test the multivariate significance of the independent variables because Monte Carlo studies demonstrate that it is robust to violations of the multivariate assumptions of homogeneous variance–covariance matrices (see Tabachnick & Fidell, 2007). Using Pillai’s trace, we found that the MANOVA revealed significant multivariate effects for assessment prompt, F(9, 291) = 4.68, p = .000, partial η2 = .13.
Means and Standard Deviations for Historical Reasoning Element by Writing Prompt
We then computed univariate ANOVAs to examine the effects of writing prompt on historical analysis using Bonferroni corrections in follow-up tests. Univariate analyses indicated the perspective variable was significantly different with different writing prompts only, F(3, 100) = 10.352, p = .000, partial η2 = .24. Substantiation, F(3, 100) = 1.655, p = .182, partial η2 = .049, and contextualization were not significantly different given different writing prompts, F(3, 100) = .929, p = .43, partial η2 = .028. Bonferroni tests indicated that with respect to the dependent variable of perspective, the situated writing prompt was significantly different from the three other writing prompts (sourcing, document analysis, and causal). This result was highly significant (p = .000). Students in the situated writing prompt condition demonstrated significantly weaker attention to or reconciliation of historical perspectives in their essays. In contrast, students in the sourcing, document analysis, and causal conditions demonstrated significantly stronger attention to or reconciliation of historical perspectives in their essays.
In sum, the regression analyses indicated that 31% of the variance in the overall historical quality of students’ writing can be explained by the writing task, in combination with other background factors. In looking more closely at particular aspects of historical reasoning through the use of a second rubric, the MANOVA analyses indicate that Tasks 2, 3, and 4 (the sourcing, document analysis, and causal prompts) are associated with significantly higher student scores for perspective recognition. Task 1 (the situated prompt) was significantly different than the other tasks for the perspective score and had the lowest mean score of all the tasks.
Discussion
These results indicate that writing prompts focused on sourcing, corroboration of documents, and causation are more likely to elicit adolescents’ attention to historical perspectives than prompts that ask students to imagine themselves as historical agents. When sourcing (Wineburg, 1991), students consider the authors of documents, their motivations and intentions, and their reliability. When corroborating (Wineburg, 1991), students compare and contrast documents, noting where documents agree and disagree with one another. In analyzing causation, students ideally consider the context of historical events and the relationships between them as well as the influence of historical agents (cf. Lee, 2005). Inherent in each of these forms of historical reasoning is some attention to the viewpoints of historical actors in their role as authors of historical documents.
Recognizing authorship of historical documents is a key step in advancing students’ historical reasoning. For when students identify and reconcile multiple perspectives, they can begin to see history as an interpretive enterprise based on the deliberation of varying accounts of the past (cf. Monte-Sano, 2011). Sourcing and corroboration can help students see the subtext inherent in any historical document (Wineburg, 1991). Each document is shaped by the author who produced it and his or her circumstances. Likewise, comparing documents brings differences of opinion into relief and can highlight the factors that shaped the differing accounts. The particular causal prompt in this study emphasizes why these authors (Churchill and Truman) spoke out against the Soviet Union, highlighting the circumstances and views of the authors.
In contrast, the situated task in this study encouraged students to imagine themselves as historical actors and authors in devising a written response to the words of other historical actors. Initially, we hoped this prompt might help students make inferences, as Brooks (2008) found. However, our data indicate that students appeared to have responded to this task by using their own personal views and transposing them on the particular historical situation in the prompt. There is less in this prompt that directs students to consider the historical documents in constructing their answer than in the other prompts. We suspect that this prompt promoted students’ tendency toward “presentism” (Seixas, 1993)—that is, to see the past through their own lens rather than from the point of view of historical actors. Although we had hoped this kind of first person task might encourage students to empathize with people in the past, it appears to have led students to consider their own views first and foremost, rather than to carefully consider the evidence presented and recognize the historical perspectives present in that evidence.
Because so much of history relies on evidence-based thinking to construct interpretations of past events, prompts that focus students more directly on sources of evidence (e.g., documents) may be more likely to promote historical reasoning. Historical reasoning begins with questioning records of the past (cf. Collingwood, 1943). Any question put to evidence is directed toward trying to understand the meaning of the evidence as it relates to the historical inquiry. The iterative process of moving between questions and evidence eventually leads historians to make a case for a particular interpretation of the past. Historical reasoning includes analyzing evidence, understanding the meaning of evidence, and using evidence to construct and explain historically plausible accounts of the past. Historians typically express these accounts as written arguments. Not only does work with evidence lead to conclusions, but evidence is also publicly conveyed in writing to justify and substantiate these conclusions. Therefore, prompts that direct students’ attention to historical evidence would be more likely to produce essays with grounded interpretations—essays that recognize and reconcile the perspectives present in the evidence. This is consistent with Monte-Sano’s (2011) findings that a teacher who focused on careful, close analysis of text as well as authors’ perspectives helped his students’ historical writing improve.
Although consideration and citation of evidence is paramount in history, there are multiple ways of thinking associated with evidence analysis. As work in Britain and Canada highlights, historical reasoning is not monolithic (Lee, 2005; Lee & Ashby, 2000; Seixas, 2006; Shemilt, 1983). This study is consistent with this conception of historical reasoning in that the rubric that broke historical reasoning into three traits produced more nuanced results. Indeed, there are multiple facets to historical reasoning, and each facet does not necessarily grow in concert with others or in a linear fashion (Lee & Ashby, 2000). As the work of history education scholars points out, there are additional aspects of historical reasoning that go beyond the three traits we looked for in our results. Based on the work of the Schools Council History Project in the United Kingdom, Lee (2005) identifies second-order concepts that apply to any field of history, including time, change, empathy, cause, evidence, and accounts. The current Historical Thinking Project in Canada defines six concepts that are essential to understanding history: establish historical significance, use primary document evidence, identify continuity and change, analyze cause and consequence, take historical perspectives, and understand moral dimensions of history (Seixas, 2006). These concepts define major components of historical thinking, identify what adolescents should learn in history class, and provide the basis for learning tasks and assessments. The field would be well served by developing rubrics that define hierarchies of thinking for different aspects of historical reasoning and then identifying the kinds of tasks that promote each kind of reasoning—historical reasoning as a whole is likely advanced by multiple kinds of tasks.
But the results of this study suggest that the writing process itself is a key factor in facilitating students’ reasoning, conceptual change, and content area learning. Langer and Applebee (1987) frame writing as a learning activity that allows students to recognize their knowledge of the topic and organize the ideas they will write about. Mason and Boscolo’s (2000) work supports this conception of writing to learn. In their study, students whose learning of photosynthesis included regular writing opportunities demonstrated a better understanding of the topic than their peers who had no writing opportunities. In addition, a review of writing-to-learn interventions by Bangert-Drowns et al. (2004) indicated that writing interventions had positive effects on school achievement and content learning, although the effects were relatively small. One factor limiting our results may have been the use of conventional academic measures for content learning as opposed to more nuanced instruments that measure reasoning or concept development. It is likely that the experience of writing facilitated the reasoning we detected in this study.
Rivard (1994) has pointed out not only that writing has an impact but also that the type of task influences students’ conceptual frameworks differently. History education research that precedes this study supports this idea. De La Paz and Felton (2010) found that both poor and advanced writers developed conceptual understandings and demonstrated disciplinary reasoning through a historical document-based writing task at the 8th and 11th grades. Wiley and Voss (1999) found that college students who wrote historical arguments (as opposed to narratives, summaries, or explanations) demonstrated a better understanding of the content and essays with more transformed content. Young and Leinhardt (1998) also found that high school students who wrote historical-document-based questions learned to transform the content into their own argument over time. Argumentative writing certainly appears to have a positive impact on students’ understanding of history and conception of the topic. In this study, all four writing tasks called for argument, and we sought nuance within the genre. Given our results, the sourcing, document analysis, and causal writing tasks may have not only provided students with a better opportunity to express their reasoning but also altered and enhanced their reasoning altogether. Likewise, these tasks may have given students a better opportunity to develop their understanding of the Cold War.
Limitations
The findings herein are certainly modest but indicate that there is some promise in investigating the nuances of what we ask students to write when we analyze the results of their writing. The scope of these findings is limited by the instructional context in which the study took place. The M.E.A.L. and SOAPSTone frameworks may have resulted in less differentiation among students and among tasks since these frameworks directed students to focus on certain aspects of historical reasoning that we measured. Offering writing prompts and readings with fewer guidelines to a wider range of students (e.g., students from different schools who have learned how to approach writing in different ways) might have resulted in greater distinctions in students’ essays. Our data indicated students followed these guidelines, so the fact that evidence and analysis were parts of each writing prompt may have confounded our attempt to study four assessment prompts. A follow-up study in contexts where students have not learned the M.E.A.L. and SOAPSTone approaches to historical writing and reading would address this limitation.
Although we believe there are important links between students’ reasoning and writing, we do not know the extent to which the process of writing facilitated students’ reasoning or content-area learning. Certainly the writing prompts directed students to consider evidence and construct an argument, in addition to the specific foci of each prompt. But did the process of writing help students attend to these details or enhance their reasoning? We gained some insight into the products of students’ work, but we did not investigate students’ thought processes as they completed the writing tasks or how the process of writing influenced and represented their reasoning. Including interviews with students as they complete these tasks would lend insight into how students interact with and make sense of the various tasks and whether students are able to express the full extent of their reasoning in writing. Another approach would be to measure students’ content learning, conceptual development, and reasoning about the topic before and after writing tasks. Comparing outcomes on these measures to students in classes without writing (similar to Mason & Boscolo, 2000) would help identify the impact of writing as a feature in and expression of their reasoning processes. This is a promising area for further study that would extend the work in history begun by Greene (1994) and help us identify more closely the intersections and distinctions between disciplinary reasoning and writing.
Conclusion
Of 12th grade teachers, 41% state that their students “write reports” in social studies classes a minimum of one to two times per month (National Center for Education Statistics, 2002). A more recent survey lends some insight into the nature of writing in many classes. Kiuhara, Graham, and Hawken (2009) found that when teachers assign writing in history classrooms, the focus typically involves summary of information. Yet a focus on summary writing in history inhibits students’ historical reasoning because history is interpretive and relies on the reconciliation of multiple sources of evidence. This study uses argumentative writing as a foundation for developing students’ disciplinary thinking and writing and tests the relative impact of four different prompts on students’ performance. Prompts that emphasize corroboration, sourcing, and causal analysis hold promise for eliciting adolescents’ historical reasoning—particularly their ability to recognize and reconcile historical perspectives—in their writing.
Cultivating students’ historical reasoning and writing skills in the classroom is difficult work. Writing demands that students read and analyze texts, organize their thoughts, and compose essays, keeping their reading and analysis in mind. Examining not only the nature of instruction but also the nature of the tasks we ask students to complete can give educators a greater range of tools from which to draw as we work to develop adolescents’ disciplinary literacy.
Footnotes
Appendix A
Appendix B
Acknowledgements
The authors would like to thank the College of Education at the University of Maryland for the SPARC grant that supported this project. The authors would also like to thank the incredible teachers at Martin Luther King, Jr., who worked with them on this project and made it possible. The opinions herein are solely those of the authors.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: The College of Education at the University of Maryland provided the SPARC grant.
Notes
Bios
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
