The Influence of Sources in the Reading of Mathematical Text

Abstract

In a recent article published in this journal, Shanahan, Shanahan, and Misischia investigated the differences in how chemists, historians, and mathematicians read text specific to their disciplines. Unlike the chemists and historians, the pair of mathematicians in this study did not consider sources when reading and evaluating their text. In this response, we contend that most mathematicians regularly do consider sources when reading mathematical arguments and papers. We summarize the results of four empirical studies, which suggest that this is the case.

In a recent article, Shanahan, Shanahan, and Misischia (2011) studied expert reading in three disciplines: history, mathematics, and chemistry. As mathematics educators who study students’ and mathematicians’ reading of mathematical arguments, we applaud this topic of research. Mathematics majors and secondary mathematics teachers often cannot determine if the mathematical arguments that they read are correct (e.g., Knuth, 2002; Selden & Selden, 2003; Weber, 2010), and mathematics majors typically learn little from the mathematical arguments that they read (e.g., Conradie & Frith, 2000; Cowen, 1991). Hence, research into how experts read mathematical arguments and how this could inform instruction is urgently needed.

We found many of the findings from Shanahan et al.’s (2011) study to be interesting and insightful. However, we also felt some of the claims, particularly those about the role of sourcing when reading mathematical text, to be misleading and refuted by recent empirical studies. As Shanahan et al. claim to have conducted “the first expert performance study in mathematics” reading (p. 397), we are concerned that their article may lead to misconceptions in the literacy community about how mathematicians read text and the nature of mathematics in general. The purpose of this article is to present data showing that, contrary to Shanahan et al.’s findings, sourcing has a significant effect on how mathematicians read text.

At a broad level, we are concerned that Shanahan et al. (2011) are propagating a pernicious and widespread myth about mathematics. This myth holds that because mathematics is a discipline based solely on logic, mathematical knowledge and truth are independent of time and culture. For instance, even though philosophers and historians argue that the standards and styles of mathematical argumentation have evolved over time (Kleiner, 1991), even among contemporary branches of mathematics (Rav, 1999), the mathematicians interviewed by Shanahan et al. were the only experts in their study who believed that the utility and interpretation of their domain-specific text was not time dependent. Consistent with this myth is the belief that when mathematicians read arguments, they do not need to consider the source of the arguments, only the contents of the argument itself. Based on their interviews with two mathematicians, Shanahan et al. advanced this hypothesis in their article, appropriately qualifying their findings because of the small sample size in their study. The purpose of this reply is to argue that this hypothesis is false. In particular, we disagree with the following claims advanced in Shanahan et al.:

• Claim 1: Shanahan et al. (2011) have conducted “the first expert performance study” on the reading of mathematical text (p. 397).

• Claim 2: The author of a mathematical work is inconsequential to an expert reader in interpreting that work. “The quality of ideas” in a mathematical paper “could only be evidenced in the work itself” (p. 408).

• Claim 3: Mathematicians “make an active effort to not use source as an interpretive consideration” (p. 406).

Claim 2 and Claim 3 are clearly similar, but one can plausibly accept Claim 3 while rejecting Claim 2 or vice versa. For instance, mathematicians might try their best not to be prejudiced by the reputation of the author of a paper, but sometimes not be successful. Alternatively, mathematicians might claim to consider sourcing when evaluating a paper but not actually do so. Nonetheless we contend that we have evidence that both Claim 2 and Claim 3 are not true.

Shanahan, Shanahan, and Misischia’s Claims About Sourcing

In Shanahan et al.’s (2011) study, pairs of professional historians, chemists, and mathematicians were observed as they read and discussed texts specific to their respective domains. Shanahan et al. observed that the two historians considered sources, particularly the author of the text, as an important factor in interpreting the text they were reading. The two chemists also considered who wrote a research article, but not as a means of interpreting the meaning of what they read. Rather, the chemists used the reputation of the author of an article to judge its credibility and decide what papers were worth reading. For instance, Shanahan et al. quoted a chemist as claiming that the author of an article and their affiliation gives their articles “more or less credibility,” noting that articles from dubious sources would be viewed more suspiciously: “If [the article is] from a third world country, it may have a little less [credibility], I’d read it with a little more suspicion than if it came from a highly ranked university” (p. 409). The two mathematicians differed from both groups in that they claimed not to consider the author of a text when reading it. Both mathematicians claimed that the identity of the author was interesting but ultimately inconsequential; what mattered was the text on the page (pp. 408-409). In particular, they “made an active effort not to use source as an interpretive consideration” (p. 406).

We believe Shanahan et al. are likely correct in that mathematicians are not concerned about the bias of the author and that the source of the article does not affect their interpretation of the meaning of statements in the text: The claim that, say, “f(x) is a uniformly continuous function” would have the same meaning regardless of who wrote the claim or where the claim appeared. In this respect, we agree that the reading of mathematicians is different from that of historians and similar to that of chemists. However, we have conducted a series of studies that indicate that mathematicians do consider the source of an argument when evaluating its accuracy and persuasiveness. In this sense, we argue that most mathematicians seem to use sourcing similarly to the chemists that Shanahan et al. describe. ¹

Our Response to Shanahan, Shanahan, and Misischia’s Claims

• Claim 1: Shanahan et al. (2011) have conducted the first expert study on the reading of mathematical text.

A theoretical mathematics paper is usually centered on the demonstration that a particular significant claim is true (or that a small number of significant claims are true). Such a paper typically begins with a brief exposition about why the mathematical claim is important and how it relates to open questions and previously established claims in the literature. Most of the paper comprises definitions, theorems concerning these definitions, and proofs that demonstrate that these theorems are correct. These culminate with the presentation of the central theorem of the paper and a proof that it is correct. The paper then usually concludes with open questions and conjectures that were not established in the paper. From a mathematician’s perspective, the most important parts of a mathematical paper are its claims and proofs (although occasionally a conjecture posed at the end of the paper will be very influential). Our account is consistent with Fulda’s claim (as cited in Shanahan et al.) that in mathematics discourse, “in general mathematics is concerned with axioms, definitions, theorems, and problems.” We agree, adding the caveat that arguments and proofs are also absolutely essential components of mathematical discourse.

As a result, when mathematicians and mathematics educators discuss reading mathematics, what they usually mean is the reading of definitions, theorems, arguments, and proofs. Shanahan et al. (2011) are correct that studies in this area are sparse, but they do exist. Weber (2008), Wilkerson-Jerde and Wilensky (2011), and Inglis and Alcock (2012) have conducted studies on the processes that mathematicians use when reading mathematical text (although the Inglis and Alcock study was published after Shanahan et al.’s article). Inglis and Mejia-Ramos (2009a, 2009b) conducted psychological experiments examining what factors mathematicians consider when evaluating written mathematical arguments. Weber and Mejia-Ramos (2011) and Müller-Hill (2010) have interviewed mathematicians on how they read mathematical arguments. Also of interest is Thurston’s (1994) introspective account of an eminent mathematician’s view on the communicative function of proof, including how he reads mathematical arguments, and Geist, Löwe, and Van Kerkhove’s (2010) exposition on the reviewing process in mathematics.

We also note that Shanahan et al. (2011) contended that with the exception of the work of Fulda, studies of mathematical discourse focus entirely on story problems in mathematics pedagogy (p. 400). Readers may also be interested in the work of Konior (1993) and Burton and Morgan (2000), who investigate the writing of published mathematicians. In summary, we agree with Shanahan et al. (2011) that research on the reading and nature of mathematical text is sparse, but some research in these areas does exist, and this can inform the plausibility of hypotheses about mathematicians’ reading.

• Claim 2: The author of a mathematical argument does not influence how mathematicians read it.

In the first of a series of experiments, Inglis and Mejia-Ramos (2009a) asked 190 research-active mathematicians to read a mathematical argument and rate how persuasive they found it on a scale of 1 through 100. Half the mathematicians were randomly assigned to the “anonymous group” and given the argument, but not told who wrote it. The other half of the mathematicians were assigned to the “named group” and given the same argument, except in this group participants were informed that the argument had been taken from a textbook by J. E. Littlewood, a famous mathematician. The named group’s rating of the persuasiveness of the argument was more than 17 points higher than that of the anonymous group, a statistically reliable difference. Inglis and Mejia-Ramos replicated this result using Littlewood’s argument and a separate argument written by another eminent mathematician, Timothy Gowers: In each case, mathematicians rated the argument as significantly more persuasive if they were told the name of the famous mathematician who wrote it. Although these results were obtained only within one genre of mathematical argumentation, they demonstrate that there are cases in which the identity of the author of an argument significantly influences how persuasive mathematicians find it.

• Claim 3: Mathematicians make an active effort not to consider the source of an argument when interpreting it.

In a study in which nine prominent mathematicians were interviewed on how and why they read the published proofs of their colleagues (Weber & Mejia-Ramos, 2011), several discussed how sourcing influenced the way in which they read proofs. For instance, some participants would not check proofs for correctness if they appeared in reputable journals, as we illustrate in the excerpt below:

Interviewer:

One of the things you didn’t say was you would read it to be sure the theorem was true. Is that because it was too obvious to say or is that not why you would read the proof?

Mathematician:

Well, I mean, it depends. If it’s something in the published literature then. . . . I’ve certainly encountered mistakes in the published literature, but it’s not high in my mind. So in other words I am open to the possibility that there’s a mistake in the proof, but I . . . it’s not. . . . [pause]

Interviewer:

But you act on the assumption that it’s probably correct?

Mathematician:

Yeah, that’s right. That’s right. (Weber & Mejia-Ramos, 2011, p. 344)

In another study, eight mathematicians were observed reading and evaluating eight mathematical arguments (Weber, 2008). Six of the eight participants described how the author of the proof would influence how they read an argument, mostly in the context of comparing student- and mathematician-generated written arguments. A representative comment from one mathematician in this study is provided below:

Mathematician: There’s something about trusting the source [italics original]. My assumption would be when reading a proof generated by a mathematician friend and they were pretty sure it was true, my assumption would be that the steps are correct and that I need to work hard to make sure that I can understand and justify every step. In a student proof, I’m less inclined to work through things that strike me as odd. (Weber, 2008, p. 448)

In this excerpt, the mathematician indicates that when he reads an unusual statement in a proof, he would be inclined to simply dismiss the statement as wrong. However, if it was written by a mathematician, he would assume the statement was likely correct and would invest the time to understand it. Although comments of this type were usually made within the context of comparing arguments produced by students and mathematicians, qualitative evidence from other studies supports the notion that expert readers let the credibility of the author increase their confidence in a claim in an argument. For instance, Müller-Hill (2010) reported one mathematician who said,

Let’s say a famous mathematician comes up with a paper and I have to referee it. Then I am preoccupied with the fact that he is a famous mathematician and so that it will probably be correct. And then you say, “Yes, this really seems plausible, but I’m not really sure if it is true” and then you end up with the question, “Is this because I don’t have enough knowledge?” (Müller-Hill, 2010, as cited and translated by Geist et al., 2010, p. 162)

These data illustrate how the source of an argument influences mathematicians’ purposes for reading an argument (i.e., some mathematicians will not check a proof published in a journal for correctness because it underwent a peer-review process) and how the trustworthiness of the author of the proof influences how carefully it is read (i.e., some mathematicians will be slower to reject dubious arguments within a proof if it was written by an authoritative source).

To test the hypotheses that we generated from the qualitative studies in Weber and Mejia-Ramos (2011) and Weber (2008), we collected survey data from 118 research-active mathematicians. In this survey, mathematicians were asked to indicate whether they agreed with, disagreed with, or were neutral toward the following statements:

• C1: It is not uncommon for me to believe that a proof is correct because it is published in an academic journal.

• C2: When I read a proof in a respected journal, it is not uncommon for the quality of the journal to increase my confidence that the proof is correct.

• C3: When I read a proof in a respected journal, it is not uncommon for me to be very confident that the proof is correct because it was written by an authoritative source that I trust.

We found that 72% of the participants agreed with C1 (12% disagreed), 67% agreed with C2 (17% disagreed), and 83% agreed with C3 (7% disagreed). This provides evidence, by mathematicians’ own self-report, that the source of a mathematical argument influences their judgment of it (Mejia-Ramos & Weber, in press).

Summary

We have argued that expert studies on mathematicians were written before the Shanahan, Shanahan, and Misischia’s (2011) article and that some of these articles are relevant to the hypotheses advanced by Shanahan et al. We have provided evidence that mathematicians do consider who wrote a mathematical argument in deciding how persuasive they found it (Inglis & Mejia-Ramos, 2009a). Furthermore, the majority are consciously aware that they consider the source of the argument and are willing to admit that they do this (Mejia-Ramos & Weber, in press), and some mathematicians are able to articulate the different ways that sourcing influences how they read mathematical arguments (Weber, 2008; Weber & Mejia-Ramos, 2011). Hence, contrary to the claims made by the mathematicians in Shanahan et al.’s (2011) study, mathematicians seem to make an active effort to consider sources.

On the Different Status of Mathematical Arguments

We conclude by critically examining why many believe that mathematicians do not consider sources when reading and evaluating arguments. Mathematics has been referred to as an “epistemic exception,” with its knowledge being more secure than knowledge in the sciences and the humanities (Geist et al., 2010). As opposed to historical texts in which distinguishing between fact and interpretation has proven extremely difficult, ² or scientific texts that rely on empirical evidence that readers do not have direct access to, mathematical arguments rely on pure reason. In history, the stance and bias of the author are essential in interpreting and evaluating his or her arguments. In science, a reader must estimate the reliability of the empirical evidence in a paper, in part by considering the technical expertise of the associated research team (and perhaps the quality of their equipment, their reputation for producing replicable results, and so on). Mathematical arguments ostensibly are different in that they are free from the vagaries of biased human interpretation, the randomness of empirical sampling, and the fallibility of human perception.

In principle, one can check and understand a mathematical argument based on its merits alone without considering its source. In practice, there are three factors that may prevent a mathematician from doing so. The first is technical expertise. Mathematics is a fragmented discipline, and it is frequently the case that mathematicians in one area of mathematics would be unable to follow all the arguments in another area (e.g., Thurston, 1994). In these cases, Auslander (2008) contended that many mathematicians are willing to accept that aspects of these arguments are correct when they appear in a journal because experts in this area have verified them. A second factor is motivation. When reading proofs in journals, mathematicians’ primary aim is not to check the proof for correctness, but to obtain some other source of insight, such as techniques they can use to tackle problems they are currently studying (Rav, 1999; Weber & Mejia-Ramos, 2011). Indeed, 74% of the 118 mathematicians surveyed in Mejia-Ramos and Weber (2012) claimed they sometimes would not check proofs published in journals for correctness. Thus, even if a mathematician had the background knowledge to carefully check and understand every part of the argument, it might not be worth his or her effort to do so. A third factor is time. Checking every step in a complex mathematical argument is a lengthy process that could take months to complete. Indeed, one mathematician in Shanahan et al.’s study said, “It sometimes took years to work through a theorem until he clearly understood it” (p. 420). Mathematicians sometimes therefore do not do this, even when they referee (Geist et al., 2010).

Based on their research with mathematicians’ and undergraduates’ proof reading, Inglis and Mejia-Ramos (2009a) put forth a model that describes situations for when the source of a mathematical argument influences readers’ level of persuasion in that argument. If readers have the resources to definitively determine whether an aspect of the argument is correct or incorrect, they will actively ignore the source of that argument. For instance, we believe all mathematicians would accept the claim “2 + 2 = 4” regardless of its source but reject the claim “2 + 2 = 5,” even if it came from an eminent mathematician. However, if the readers lacked the time, motivation, or technical expertise to determine with a high degree of certainty if some aspect of the argument is correct, they may weigh external factors in making their judgment. These include the reputation of the author of the argument as well as whether the argument underwent a rigorous reviewing process. As Geist et al. (2010) argued, these situations are surprisingly common in the practice of professional mathematicians. In this sense, technical details of some mathematical arguments are analogous to empirical data presented in scientific papers. For various reasons, the mathematicians and scientists reading these papers are not able to verify these parts of the papers and sometimes use sources as a factor in considering their credibility. In this sense, we believe mathematicians are actually quite similar to the chemists discussed in Shanahan et al.’s study.