The Consequences of Teaching Critical Sociology on Course Evaluations

Abstract

While there has been a great deal of literature addressing students’ perceptions of bias in the classroom, there has been little in the way of examining the relationship between the student evaluations and the pedagogy used to examine sociological issues in class. We conducted a case study that reviewed and coded the written comments on Professor Prew’s Introduction to Sociology course evaluations in the 2008-2009 academic year. We assessed whether there was a correlation between the quantitative feedback on evaluations and the open-ended comments regarding the professor’s critical sociological perspective. Using means, ANOVA, and correlations, we specifically tested to see if students who left qualitative comments that contain overtly negative assessments of Professor Prew’s critical sociological perspective evaluated the course more negatively than those who are neutral or complimentary regarding his perspective. The findings show a positive correlation between negative attitudes regarding critical sociology and lower evaluation scores.

Introduction

We examined the effect of teaching introductory sociology using a critical sociological perspective and how Professor Prew’s college students evaluated the course based on their perception, reception, or rejection of the critical sociological perspective. What effect does teaching critical sociology have on the more mundane aspects of our professional assessment, specifically course evaluations? Our research asks, “Is there a discernable difference between the evaluations of students who express disagreement with a professor’s perspective and those who do not?”

We critically evaluate the literature on students’ perception of bias and balance in the classroom. While there tends to be some support for the idea that the perspective of the professor does have an effect on professors’ evaluations (Dixon & McCabe, 2006; Kelly-Woessner & Woessner, 2006; Tollini, 2009), we focus more specifically on critical sociology. We describe the concept “critical sociology” (Buechler, 2008b) and tie it to the approach Professor Prew takes in the classroom. For this article, we code qualitative responses on evaluation forms for perceived bias of the professor. After coding these qualitative responses, we test to see if there are significant correlations between students who perceive bias and how they evaluate Professor Prew’s Introduction to Sociology course. Because we focus on the critical sociological perspective, this research will take an exploratory case study approach. We would like to first identify a relationship between students’ negative evaluations and the critical sociological perspective before moving on to more elaborate comparative studies. If we extended the analysis to other professors, identifying the presence (or absence) of critical sociology would necessitate an observational analysis of faculty’s pedagogy. While this may be an approach for the future, this study is an attempt to identify a relationship between a critical sociological pedagogy and student evaluations before conducting more labor-intensive comparative work. As a result, our study is limited to 473 student evaluations of one professor’s course.

Review of the Literature

Most prior research of bias in the classroom (Dixon & McCabe, 2006; Tollini, 2009) tends to focus on bias related to professors providing balance and alternative viewpoints, but at least one study (Kelly-Woessner & Woessner, 2006) investigates how professors’ and students’ perspectives affect evaluation outcomes. In one study of bias, Tollini (2009) attempts to differentiate what behaviors students would classify as political bias. Tollini’s study identified a number of behaviors that the majority of students considered bias. When the professor did not present, criticize, or allow viewpoints in the classroom other than the professor’s own views, the majority of students were likely to consider these behaviors biased. In contrast, behaviors that few students found biased included discussing political or controversial topics in class. Despite the fact that fewer students felt discussing political or controversial issues was an example of bias, between 13% and 27% did. Tollini’s study tends to suggest that even though some behaviors are not considered bias by the majority of students, some students feel that the mere mention of political or controversial issues is bias. These findings are particularly interesting in that a small but noticeable number of students (up to 27%) can perceive bias in the discourse, even if the information was presented within the context of the discipline.

Another study focused on the issue of viewpoints provided by the professor. According to Dixon and McCabe’s (2006) study of student evaluations, the classes tended to perceive their instructors as balanced, especially in terms of discussing points of view other than their own and inviting criticism of their ideas. When professors discussed viewpoints other than their own, their classes tended to give them higher evaluations. Dixon and McCabe also found some correlations between students’ expected grades and how well the students evaluated the course and instructor effectiveness. Dixon and McCabe concluded that balance is not the most important element in the classroom, but indicated that it is a significant aspect of the classroom experience.

Another research study in political science by Kelly-Woessner and Woessner (2006) targets the political perspective of the student and the perspective of the professor. Students are asked to provide an assessment of the political orientation of the professor and also their own political ideology. These assessments were then used to see if they had an impact on evaluation criteria. The authors concluded that the more disparate the perceived difference between the professor’s views and those of the student, the lower the evaluation scores. This suggests that students’ estimations of balance or bias are also influenced by their own biases and ideological views.

Similar to Kelly-Woessner and Woessner (2006), some studies do address the issue of students’ reactions to course material. Research in disciplines other than sociology documents some of the reactions faced by professors attempting to teach critically (Bérubé, 2006; Seibel Trainor, 2002). Students react negatively to the course content despite the efforts of the professor to provide an inclusive environment and a variety of views. When these views clash with the students’, they lash out, sometimes in highly inappropriate and disruptive ways. Some students may react negatively to material that clashes with their worldview and then refuse to meet the course material on its own terms. The students may bring their own perspectives into the academic discussion of the material, derailing what is an intellectual exercise and transforming it into a polemical debate.

Michael Bérubé (2006) begins his discussion of the purported liberal bias in higher education by outlining a particularly troublesome student that reacted negatively and antagonistically to the content of his postmodernism and American fiction course. The student staunchly took an assimilationist stance regarding African Americans and unilaterally defended the Japanese internment camps in the United States during World War II (Bérubé, 2006). The student consistently dominated the course in an effort to be the “only countervailing conservative voice in a classroom full of liberal-left think-alikes,” as Bérubé (2006, p. 5) perceived it.

Another author, Seibel Trainor (2002), outlines her own experiences with a difficult student. When Professor Seibel Trainor showed pictures of police officers beating migrant workers during a strike, a student “thought the photos were ‘biased’ and [argued] that if ‘you want to learn about history, you have to have objective sources’” (Seibel Trainor, 2002, p. 642). This kind of attitude is difficult to address within the classroom because it is extremely challenging to make the act of police brutality or other social ills “objective.” There are certain historical truths that are difficult for some students to accept, but does their very inclusion in a course mean that the professor and the course are hopelessly biased? Some students who disagree with these facts would argue in the affirmative, and Tollini’s (2009) research would tend to confirm that some students will perceive bias regardless of the objective nature of the material presented. However, general education goals are aimed directly at introducing these issues into the classroom and exposing students to issues of civil liberties, discrimination, and oppression (see below).

As the quantitative research above (Dixon & McCabe, 2006; Kelly-Woessner & Woessner, 2006) tends to suggest, some students will react negatively to the mere mention of issues that differ from their own views. In addition, Tollini’s (2009) research tends to suggest that the mere mention of a politicized issue will inflame a minority of students to decry the content as biased. If professors are to honestly incorporate general education goals, or by extension critical sociology, into their classroom, they will inevitably offend this minority of students. Because negative reactions are limited to a small group of students and the vast majority of students do not react to professor’s perspective, we are attempting to investigate the effect of this small group of students. We do not wish to negatively characterize this small group of students or students generally, but we are interested in discerning if there is a noticeable effect on a professor’s evaluations from students who disagree with the professor’s perspective.

Our research deviates from previous research in that it specifically targets critical sociology. Prior research tends to address issues central to the critical thinking model (discussed below): discussing contrasting viewpoints. For example, previous studies focused on the students’ perceptions of whether the professor “discusses points of view other than his/her own” (Dixon & McCabe, 2006, p. 116) provides an “objective presentation” (Kelly-Woessner & Woessner, 2006, p. 497), or presents a “position,” “view,” or “side” of a political or social issue (Tollini, 2009, p. 383). A critical thinking approach may increase the student’s perceptions that the class is balanced by creating the impression that they are exposed to a variety of viewpoints regardless of how “critical” these views may be. The prior research does not distinguish between professors who specifically take a critical sociological (or other disciplinary) perspective from those that encourage critical thinking or do not take a critical approach at all. In fact, the research of Kelly-Woessner and Woessner (2006) appears to suggest that bias is in the eye of the beholder. Whether the professor is viewed as objective is directly correlated with how similar the student is ideologically to the professor.

Critical Sociology Versus Critical Thinking

To understand the specific issue in this paper, it is necessary to outline critical sociology and how it is integrated into Professor Prew’s Introduction to Sociology class. His approach to teaching enjoys an indefatigable lean toward critical sociology as defined by Buechler (2008a, 2008b). With respect to critical approaches in the classroom, there has been a great deal of discussion of radical approaches in sociology (Ballard, 1998; Gaianguest, 1998; Gimenez, 1998; Long, 1998; Sweet, 1998a, 1998b), and critical pedagogy (Callero & Braa, 2006; Fobes & Kaufman, 2008; Jay & Graff, 1995; Kaufman, 2006; Monchinski, 2008; Seibel Trainor, 2002). In this paper, we focus specifically on critical sociology, in large part, because Professor Prew’s Introduction to Sociology class regularly enrolls between 110 and 160 students in each section. While attempting more interactive approaches, the class does not approach the dialogue method mentioned by critical pedagogy writers (Callero & Braa, 2006; Fobes & Kaufman, 2008; Freire, 1996; Sweet, 1998a). For this reason, we will focus on a critical sociological approach.

Before we discuss critical sociology, we have to draw a clear distinction between critical sociology and critical thinking. While the notion of critical thinking is becoming increasingly popular, it is essentially depoliticized. Critical thinking is inherently uncritical in that it does not necessitate an understanding of why social phenomena occur, nor does it challenge status quo explanations. Critical thinking is merely the development of analytical ability, but it does not demand we demystify the social forces governing our lives. Critical thinking may provide the skills to aid in the understanding of our social world, but it alone cannot tell us why social phenomena occur. Critical sociology is distinguished from critical thinking in that critical thinking is a “decontextualized, generic skill applicable to virtually any issue” (Buechler, 2008b, p. 318) while critical sociology is a deeper critical analysis specifically addressing issues of inequality and power. Steven Buechler (2008b) outlines the critical sociological perspective that closely mirrors Professor Prew’s pedagogical approach in the classroom. In general, sociology is critical in that it “is critical to our ability to define, analyze and respond to pressing social issues,” scrutinizes “familiar truths and established facts,” and “is explicitly based on the values of freedom, equality, and justice” (Buechler, 2008b, p. 319). While mainstream sociology may contain an inherent “debunking” quality that incessantly questions the status quo, critical sociology goes beyond this cynicism to focus specifically on issues of inequality and power. For critical sociology, the notion of objectivity in the physical sciences is not necessarily transferable to social sciences. When applied to the social sciences, objectivity is inherently tied to the acceptance of the status quo relationships of inequality in society. Acknowledging that dispassionate objectivity is problematic, the critical sociological perspective demands that commonly accepted social practices supporting inequality be critically analyzed and understood. “Critical sociologists deliberately focus on relations of domination, oppression, or exploitation because they so obviously violate personal freedom and social equality” (Buechler, 2008b, p. 324).

Using an example by Mollborn and Hoekstra (2010), we can highlight the difference between critical sociology and critical thinking. Mollborn and Hoekstra outlined a critical thinking exercise dealing with gender roles in an article in Teaching Sociology. We acknowledge that Mollborn and Hoekstra were not attempting to surpass critical thinking, but were focused on the difficult task of engaging students in large lecture formats. Their exercise revolves around students’ ideas about who does more housework in their family and contrasting their ideas with a critical examination of the human capital explanation. The students are asked to “identify shortcomings in the human capital explanation” (Mollborn & Hoekstra, 2010, p. 22). Instead of requiring the students to challenge the contemporary gender relations, “students are . . . prompted to think critically about the extent to which sociological theories and concepts fit the experiences of ‘people like me’” (Mollborn & Hoekstra, 2010, p. 23, italics added). Students are expected to identify the “fit” of the theory and not why contemporary gender arrangements are inequitable. As Buechler (2008b) argues, this technique tends to fall into the category of critical thinking because it fails to contextualize the broader power relationships and historical development of gender inequality over time. Mollborn and Hoekstra’s example is reduced to an intellectual exercise of evaluating a perspective, which may increase the perception among students that the professors are providing more than their own views to the class, but it does not fundamentally address the issues of inequality and power. It does not demystify the contemporary arrangements to allow the student to challenge the contemporary inequitable gender arrangements.

Compared with critical thinking, critical sociology may appear radical to some, but it is essentially the pedagogical realization of university goals to be more inclusive, forward-thinking institutions. The core tenets of critical sociology can be found in many university mission statements. Minnesota State University−Mankato is no exception in this regard. In fact, the focus of the critical sociologist is incorporated explicitly in the general education goals (see Appendix A) at Minnesota State University–Mankato (2009). Based on the general education goals, Professor Prew’s course must challenge students to, “develop and communicate alternative explanations or solutions for contemporary social issues,” “analyze specific international problems illustrating cultural, economic, . . . social, and political differences which affect their solution,” and “acquire a substantive knowledge base to identify the impact of oppression for individuals from diverse populations” (Minnesota State University–Mankato, 2009). The issues of equality and justice in critical sociology are explicitly outlined in the general education requirements, “Purple courses allow students to explore basic concepts such as oppression, prejudice, discrimination, racism and ethnocentrism and responses to each” (Minnesota State University–Mankato, 2009). Professor Prew uses critical sociology to promote the general education goals of the university.

Critical Sociological Pedagogy in the Classroom

To provide context to the reasons why students may react negatively to the perspective in the classroom, it is necessary to describe the critical sociological perspective that Professor Prew uses in his classroom. While not part of the formal data collection, describing the practices in Professor Prew’s classroom brings the concept of critical sociology to life. In his Introduction to Sociology course, Professor Prew focuses on issues of inequality and demystifying the students’ contemporary social world. Early in the course, Professor Prew challenges the students to develop an understanding of the various barriers to accurate information by examining the role of media as a vital agent of socialization. The students do not merely survey the research and findings with respect to the media. They use theory to identify why there is a difference between media accounts and actual events. The class begins the exercise by discussing a homework assignment asking the students to summarize a news article from CNN (Wallace, 2003). After discussing the content, the class watches the actual events unfold as documented in the film, Death in Gaza (Miller, 2004). After discussing the egregious difference between the CNN account and the actual events, Professor Prew then turns to the theory of the “filters of propaganda” by Herman and Chomsky (1988) to explain how the two accounts could differ so widely. Students are then challenged to apply the filters to issues presented in the film, Outfoxed (Greenwald, 2004). Students take the five filters of propaganda and are asked to apply them to issues and events in the film to understand why the information they receive from the media is profoundly limited and distorted. The class discusses how the different filters fit the actions and structure of the Fox News Channel. Because the class uses theory to understand the media, students are encouraged to apply the filters to other media outlets, not only Fox News, once they leave the classroom.

To encourage the greatest possibility for learning, Professor Prew weaves together these exercises, lectures, and film. He creates interest through the first exercise because of the vast gulf between what was reported and what actually occurred. Then he explains, theoretically through the filters of propaganda, how the media operates to present a limited and distorted view of reality. Finally, he walks the students through their attempts to apply these theoretical concepts. In this way, Professor Prew maximizes comprehension and retention through examples, theory, and application by the students with the intent of encouraging the students to leave the classroom more informed and critical citizens.

In a more active and participatory exercise using the technique of “role taking” mentioned by Buechler (2008b, p. 328), Professor Prew begins the issues of race/ethnicity by conducting a small-group in-class exercise. In the exercise, the students are asked to contemplate how they would feel, and what they would do, if their grandparents were subject to a zoning law that devalued their property dramatically while increasing the property values of neighboring residents (see Appendix B). After dividing the class into small groups, Professor Prew gives figures for property values as well as wages based on the inequitable education systems in the two communities. Students are asked to discuss their feelings about the effects of the inequalities created by the zoning law. After returning from the small groups, Professor Prew leads a discussion with the entire class and collected responses using a “clicker” personal response system. The questions ask how they feel about the situation and what they feel would be necessary to remedy the problem (see Appendix C). After collecting their responses, the class watches the third part of the film, Race the Power of an Illusion (Smith, 2003), dealing with redlining and block busting after World War II. The class then discusses the implications of how society defines race over time, and the effects of structural inequality like segregation.

Using the clicker devices again, Professor Prew asks the students a new set of questions dealing specifically with housing inequality in the United States (see Appendix C). The results are interesting and are used to demonstrate how students learn from the exercise and how some students will view the same issue differently when it involves race/ethnicity. In the results, there is a tendency for more students to agree that people should be compensated for lost property values and they should make the school districts equal. From the perspective of critical sociology, students have become more sensitive to the institutional constraints placed on people of color as a result of the history of structural inequality. Consistent with the goals of critical sociology, some students have changed their attitudes to be more consistent with the sociological research findings, and many are better equipped to understand the causes of inequality in the world around them. Unfortunately, there may be slightly more students who tend to say that only removing redlining and block busting is enough to solve the problem of inequality compared with removing the zoning law in the previous exercise.

To place the increase in responses favoring less intervention in context, Professor Prew expresses to the students that this is part of the issue facing people of color. While most students are in favor of addressing housing inequality in general in the first exercise, there are a few people in this group who change their minds when it becomes intertwined with race/ethnicity and now feel that simply banning overtly discriminatory practices is enough. This shift in thinking is characteristic of racial/ethnic discrimination. Some of the students’ attitudes are less favorable of equality when the issue is specifically linked to race/ethnicity. Professor Prew links this issue to a later discussion of inequitable treatment of people of color in the health care profession to demonstrate the continued influence of discrimination. Health professionals tend to treat patients differently based on racial/ethnic markers, and the health outcomes are worse for people of color as a result (Smedley, Stith, & Nelson, 2003). In both situations—desegregation and discrimination in health care—differential treatment continues to occur solely based on racial/ethnic markers.

Returning to the example of critical thinking by Mollborn and Hoekstra (2010), their critical thinking approach differs in significant ways from Professor Prew’s critical sociological approach. The example from Professor Prew’s course forces the students to contemplate their own feelings about racial/ethnic inequality by placing them in the shoes of the exploited, and then demonstrates how the same process happened to others based on their racial/ethnic background. Mollborn and Hoekstra are asking the students to examine the issue of household labor externally by evaluating the fit of a theoretical perspective. In Professor Prew’s course, the institutional forces that gave rise to contemporary racial/ethnic inequality are described historically and theoretically through the discussion of institutional racism. For the students to dismiss institutional discrimination, they must dismiss their own feelings of injustice that they felt in the first exercise. Contrasted to Mollborn and Hoekstra where students are not asked to develop feelings of injustice, Professor Prew’s class requires that the students experience their own feelings of inequality before learning about the inequality of others. While a few do not accept the evidence regarding institutional discrimination, others are able to grow from the exercise and develop a new understanding of racial/ethnic inequality as well as a better appreciation for the experiences of others.

Describing in detail how Professor Prew’s course follows a critical sociological perspective is important to understanding why professors following a critical sociological perspective may be more likely to receive negative comments. Students may perceive critical thinking exercises favorably by creating the impression that professors are providing balance in perspectives and views other than their own. Notions of providing alternative views are central to some research studies of bias (Dixon & McCabe, 2006; Tollini, 2009). Comparing and contrasting a variety of theoretical perspectives while not challenging the “social arrangements that create conflicting interests between people” (Buechler, 2008b, p. 324) is less likely to provoke students to charges of bias than a critical sociological perspective that clearly identifies the source of these inequalities.

Even though some literature (Dixon & McCabe, 2006) demonstrates that the presentation of diverse perspectives is associated with higher evaluations, other literature (Kelly-Woessner & Woessner, 2006) shows that the difference in perspective between professor and student is an important predictor of evaluation scores. Thus, having diverse perspectives in the classroom may be necessary, but not sufficient, for higher evaluations if students’ perspectives differ from their professors’. Students with rigid worldviews will tend to cling to their own perspective (Nyhan & Reifler, 2010) and will refuse to recognize the diversity of perspectives offered in the course, leading to lower evaluations of the course.

While there are limitations to a case study approach including only Professor Prew’s class, the perspective of the class is known and can serve as a starting point for future comparative research. We hope to promote research that begins to address pedagogical differences as the source of student perceptions of bias. This paper is an attempt to begin research into the pedagogical influences on students’ perceptions of bias.

Method

The data for our research are taken from evaluation forms collected in Professor Prew’s Introduction to Sociology courses in Fall 2008 and Spring 2009. Of the 662 students enrolled in the classes, 473 completed evaluations for a response rate of more than 70%. The evaluations included quantitative and qualitative sections (see Appendix D). The quantitative portion of the evaluations included questions that addressed the course in general and other questions that were more specific to the professor’s pedagogy. The students could choose a number between 1 and 5, 1 being low and 5 being high. On the back of the evaluations, students were asked to provide open-ended, qualitative comments on the course pedagogy.

We coded the written comments for inclusion of remarks addressing the professor’s critical sociological perspective. These codes are merely to identify how the student perceived the course in terms of perspective. We did not code for a positive or negative evaluation of the course in general, but focused specifically on how they viewed the perspective in the course. Did they mention positive or negative comments that specifically addressed the perspective Professor Prew used in the classroom? In the process of coding the evaluations, we developed codes for critical thinking, positive comments, mixed comments, and negative comments regarding how the student viewed the critical sociological perspective of Professor Prew. As students do not possess the lexicon to describe critical sociology, they tended to use the terms and ideas associated with critical thinking. A critical thinking comment would include statements like the following, “Helps critical thinking, [sic] the different Sociological perspectives helped with that greatly.” They tend to focus on seeing things from different perspectives and often use the term, critical thinking. Positive comments regarding the perspective would resemble the following: “Receiving the clear cut, unbiased truth [sic] . . . I loved this course. Presentation of current events & social issues [sic].” Mixed comments blurred positive and negative comments regarding the perspective in the classroom. They tended to identify bias, but still saw the approach in the class as beneficial. For instance, the “instructor is bias [sic] to what he teaches, but everyone likes to show their views on topics. I think that understanding these social problems and learning from a different perspective than my own was extremely beneficial.” An example of a negative comment would be, “He didn’t teach sociology! He taught what he thought was soc! He also only showed one side of the stories!” This comment includes two components that we looked for in negative comments on the perspective. First, we looked for comments that argued the class was biased or “showed one side.” Second, we looked for comments that argued that the class was another discipline or not sociology. We included these ideas in the coding because some students seem to have a confused idea of what sociology really is. For example, a student exclaimed on an evaluation, “I thought I was in ethnic studies/political science class. NOT sociology” [sic]. For some students, critical sociology is not only biased; it does not even qualify as sociology because it does not fit their preconception of the range of sociological topics. We will refer to this variable as Student Written Comments. Student Written Comments is a nominal-level variable designed to divide the students into separate groups. While there is an intuitive order to these identifications (negative comments regarding perspective, mixed comments, positive comments regarding perspective), we are treating the variable as nominal.

Our main hypothesis is that students who provide negative comments regarding the professor’s critical sociological perspective will tend to negatively evaluate the course as a whole. Our dependent variable is a measure on the evaluation form that asked the students to rank “the course as a whole” (Course as a Whole). We looked at other independent variables that may influence Course as a Whole, such as grade expected in the class, the student’s class standing (freshman, sophomore, etc.), and if the class was in their major. We included the grade expected in the class because previous research (Dixon & McCabe, 2006) found a link between grades and course evaluations. The year in school may influence evaluations if students who are further along in their education take it more seriously than new students. Students in the major may evaluate the course more favorably because they are more familiar with the material and interested in the subject.

To assess the relationship between the Student Written Comments and Course as a Whole, we ran means for Course as a Whole based on the various categories in Student Written Comments. We conducted ANOVA analysis to test for significant difference between the means of the various groups in Student Written Comments and our independent variables. Included in the ANOVA analysis were Instructor’s Contribution, Instructor’s Knowledge, Course as a Whole, Explanations, Amount Learned, Answers to Questions, and Grade Expected in Class.

To test the strength of the relationships, we also ran correlations, but it was necessary to recode our variable Student Written Comments. Because Student Written Comments can be considered a nominal-level variable, we dichotomized Student Written Comments into negative and positive comments regarding the perspective of Professor Prew. The negative comments consisted of those coded as negative regarding the perspective. Positive comments included mixed, positive, and critical thinking comments regarding the perspective in the class. The dichotomized variable is called Negative and Positive Comments. We ran correlations between Negative and Positive Comments and a number of variables from the evaluations, including Course as a Whole, “The instructors contribution to the course,” “Student confidence in instructor’s knowledge,” “What grade do you expect to receive?” “Class,” and “Is this course: in your major, not in your major but required for a program, an elective, other.” To test the impact of dichotomizing Student Written Comments, we compared the correlations between Student Written Comments and the dichotomized variable, Negative and Positive Comments. Dichotomizing the variable, Student Written Comments, did not change any of the significant relationships, but it did slightly increase the strength of the correlations compared with the nondichotomized variable.

Results and Discussion

The results tend to support our hypothesis that students who comment negatively on the critical sociology perspective in Professor Prew’s classroom have a negative effect on the course evaluations. To begin, the means of the Student Written Comments contrasted with the variable, Course as a Whole, show a pattern consistent with lower evaluations scores associated with negative comments (see Table 1). While the total mean for Course as a Whole is 3.74, the mean for students who leave negative comments regarding Professor Prew’s perspective is 2.82.

Table 1.

Means: Course as a Whole by Student Written Comments.

Student written comments	M	n	SD
Negative comments	2.8214	56	1.09722
No comments	3.6897	87	0.81141
No positive or negative comments	3.8846	312	0.91054
Mixed comments	4.0000	6	0.89443
Positive comments	4.1429	7	1.06904
Critical thinking	4.8000	5	0.44721
Total	3.7378	473	0.98020

Out of a 5-point scale, this is nearly a full point lower. However, students who comment that they appreciate the critical thinking in the class have a mean of 4.8 (although the N is very low at five cases). For a graphic representation of the difference in means, see Figure 1. Students who leave negative comments regarding the professor’s perspective dip sharply below the overall mean for the class and the university. As the graph progresses toward positive comments and critical thinking, the graph line moves above the class and university mean. Some of the difference between Professor Prew’s average evaluation scores and the university in general may be explained by the fact that his course is arguably more difficult than other large lecture courses. For example, the number of students expecting A’s in Professor Prew’s class is 23% of those filling out evaluations compared with 42% in the university large lecture courses and 54% in other large lecture Sociology courses for the 2008-2009 academic year.

Figure 1.

Means: Course as a Whole by student written comments.

The ANOVA analysis supports the idea that students who comment negatively on the perspective in the class have significantly different evaluations of the course than other groups, including those who do not comment or comment positively on the perspective. The results of the ANOVA analysis were significant at the .001 level for all variables tested except Grade Expected in Class. Thus, Instructor’s Contribution, Instructor’s Knowledge, Course as a Whole, Explanations, Amount Learned, and Answers to Questions were all significant with respect to Student Written Comments (see Appendix E for ANOVA results). The ANOVA results support the idea that students who comment negatively on the course perspective do, indeed, evaluate it differently from other groups of students. The lower means for students who comment negatively on the course perspective is significantly lower than the means for other groups of students. Thus, if students perceive the professor’s perspective to contrast with their own, they will rate the class lower. In this case, Professor Prew’s critical sociological approach elicits negative responses from some students that can be statistically identified. Prior research (Bérubé, 2006; Dixon & McCabe, 2006; Kelly-Woessner & Woessner, 2006; Seibel Trainor, 2002; Tollini, 2009) is consistent with our findings. Students who disagree with the perspective in the classroom tend to react negatively to the content and will evaluate the course lower than those who do not perceive bias or sympathize with the perspective in the classroom.

The correlations tend to demonstrate a significant relationship between the students’ ranking of Course as a Whole and the comments regarding Professor Prew’s perspective in the classroom (see Table 2 above). After dichotomizing Student Written Comments into Negative and Positive Comments regarding the perspective of the class, we ran correlations with our other variables. Of all of the factors listed on the evaluation form (see Appendix D), Negative and Positive Comments were most closely correlated with “The course as a whole” (.516) followed by “Amount you learned in the course” (.479). Next, “The instructor’s contribution to the course” (.473) was followed by “Would you recommend the course?” (.460; not shown in table). Other correlations of note (not shown in table) include “Explanations by instructor” (.457), “Student confidence in instructor’s knowledge” (.352), and “Answers to student questions” (.343).

Table 2.

Correlations.

	Comment	Course	Contribution	Learned	Grade	Class	Major
Negative and positive comments		.516**	.473**	.479**	.141	.227	−.060
Course as a whole	.516**		.680**	.754**	.222**	−.030	.104
Instructor’s contribution	.473**	.680**		.649**	.065	.018	.102
Amount learned in course	.479**	.754**	.649**		.229**	−.047	.078
Grade expected	.141	.222**	.065	.229**		.048	.009
Class/year in school	.227	−.030	.018	−.047	.048		.017
In major, required, elective	−.060	.104	.102	.078	.009	.017

**

Correlation is significant at the .01 level (two-tailed).

The close tie of these survey items to Negative and Positive Comments can be related to how students view the course. First, there is a close tie to how the student evaluates the content provided by the professor in the form of confidence in the instructor’s knowledge, contribution to the course, explanations, and answers to questions. If a student were reacting negatively to the critical sociological perspective, it would be expected that the student would not agree with the theories and research findings presented in the class. The student may also find the explanations and answers to students’ questions unsatisfactory because they do not agree with the student’s worldview. Consistent with the correlations found here, evidence in prior research tends to suggest that students who hold dogmatic ideological viewpoints are not only less likely to accept factual information, but they are also more likely to believe more strongly their own misperceptions (Nyhan & Reifler, 2010).

Second, the Negative and Positive Comments are closely tied to evaluations of the course in terms of how they feel about the course as a whole, what the student learned, and whether they would recommend the course. Students who perceive bias in the class would disapprove of the course as a whole and state that they did not learn in the class. Because it differs from their own preconceived worldview, the students will reject the course material, leading them to the impression that they did not learn and the course was not worthwhile. For example, one student who self-identified as being in the major stated,

The strengths were [sic] that he was confident in what he was saying[,] but his weakness was that the only opinion that he ever portrayed throughout the course was his own[,] and he was not open to others. He let his liberal views run the course rather than teaching true curriculum. I felt as if I learned nothing & that I learned more in sociology in high school by a teacher assistant[,] who was 18, then [sic] him. [K]eep the professor’s bias, [sic] liberal agenda out of the curriculum.

This attitude about the class may be explained by the difference between critical sociology and noncritical approaches. In contrast, another student recently expressed gratitude to Professor Prew for his critical sociological approach because the student’s high school course taught what happens when “someone farts in an elevator.” Certain types of sociology, especially taught in the restrictive environments of secondary education, may focus on issues like the elevator example above. The reliance on breaching experiments and humorous insights is termed “whoopee cushion sociology” by Professor Prew. If the students are having fun, they perceive that they are learning, but if they are made to feel vulnerable, then they feel that the discomfort they experience is not conducive to learning. Students may get a distorted impression of the wide range of sociological topics if instructors do not stray too far from safe topics that make students giggle. When they do take a class that teaches sociology using a critical sociological perspective, students may embrace or reject the change in approach depending on their own preexisting biases, consistent with Kelly-Woessner and Woessner (2006).

The correlations also reveal some interesting nonsignificant results. First, a major charge of “liberal” higher education is that left-leaning professors will make it more difficult for nonliberal students to earn higher grades (Giroux, 2010), or even that the professor will lower grades for students who disagree with their perspective (Bérubé, 2006). Negative and Positive Comments were not significantly correlated with “What grade do you expect to receive?” (see Table 2) indicating that even though some students felt Professor Prew’s class was “liberal” or “not sociology,” their perceptions of the course perspective did not correlate with their grades. There is also no correlation between Negative and Positive Comments and whether the class was in their major, or an elective. While it may be assumed that sociology majors would tend to adopt a critical sociological perspective and be less likely to view the class as biased, crosstabs revealed that two students who identified as majors did express negative comments about the perspective in the class and two others expressed mixed comments.

He seems to have done a lot of research on the subject matters[,] and he seems to really care about said subject as well, but he is far too biased and opinionated to fully trust as an informational source. I almost expected the instructor to burst out into a rant about the man. Ironically, I agree with most of what he says, but he seems to [sic] close-minded. [B]e more open-minded to others’ viewpoints. The world is not black and white.

Another student in the major stated, “Some [sic] of the major strengths of the professor is his knowledge[,] but one of his weaknesses is that he isn’t [sic] bias [sic] about certain issues.” In many cases, like these comments, it is difficult to understand the students’ concerns as the course emphasizes the complexity of social life and eschews simplistic, often ethnocentric, explanations.

Based on our results, we tend to find significant relationships between the negative comments regarding the critical sociological approach and the outcome of the course evaluations. Prior literature suggests a connection between perceived imbalance in the classroom and course evaluations, but is it possible that professors who provide a variety of perspectives in the classroom are still perceived as biased simply because they use a critical sociological perspective? Bérubé’s (2006) and Seibel Trainor’s (2002) research would suggest that professors can strive to be inclusive and provide a variety of perspectives, but still be perceived as biased. Professor Prew included guest speakers as well as covered a variety of topics outside of his area of core interest. The list of perspectives covered in Professor Prew’s classroom is long, and he does not agree, nor identify, with all of them. To introduce students to the various theoretical perspectives in sociology early in the course, he applies functionalism, interactionism, and conflict theory to the same social issue. Using the example of genocide in Rwanda, Professor Prew demonstrates (without denigrating those perspectives with which he disagrees) how all three perspectives would contribute to the understanding of the issue. Later in another portion of the course, Professor Prew outlines the goals, intentions, and actions of the World Trade Organization (WTO) from its perspective based on its own documents. Students are then asked to contemplate the effects of those goals in the nations where it operates. In this way, students are exposed to the claims to benefits by the WTO as well as a critical sociological analysis of the effects of WTO policies. Despite the fact that Professor Prew provides the WTO perspective and the critical sociological critique, some students may look past the different perspectives offered and perceive bias anyway. Again, it must be stressed that, as with prior research (Tollini, 2009), only a minority of students perceived bias or commented negatively on the perspective. The majority of students did not comment on the perspective in any form, but the small proportion of students who did had a discernable effect on the evaluations.

We found a strong correlation in our research between Negative and Positive Comments and “Student confidence in instructor’s knowledge.” This correlation is very interesting, especially given that seven students specifically acknowledged the breadth of Professor Prew’s knowledge, but argued his class is too biased. “He knows topic well but[,] he is biased & very opinionated.” Thus, some students agree that Professor Prew knows the correct information, but still argue that he must provide an alternative to this correct information. For these students, they obsess over the idea of portraying different sides of the issue to the exclusion of what is truly accurate information. Balance is lauded over social scientific findings.

Not only is the concept of “balance” vague, it is an impossible ideal to achieve; it revives discredited positive ideologies that professors can adopt a neutral (rather than always-already biased) stance and always give equal time and consideration to opposing viewpoints. The concept of “balance” and “equal” time are absurd if one is required to teach the KKK along with Martin Luther King Jr., flat-earth approaches in addition to contemporary astronomy. (Best, Nocella, & McLaren, 2010, p. 68)

Given that many students who complain about the critical sociological perspective concede Professor Prew’s knowledge base, criticisms of one-sidedness are not really about balancing the perspectives within the course because accurate information has already been communicated in the classroom; these criticisms are really about changing, fundamentally, the content of the course.

Conclusion

We consider the results of this study to be illuminating and disturbing. Only a small portion (about 12%) of students actively voiced negative attitudes toward the perspective in the class. The general education goals at Minnesota State University−Mankato are well suited to a critical sociological perspective, but some students reject the attempt to seriously engage those goals. Because of this small group who can affect the outcome of evaluations, sociology professors, like many other faculty from other disciplines, may not seriously engage the general education goals. Thus, many faculty may water down the critical sociological analysis, putting them in a more advantageous position when it comes to evaluations and other institutional rewards. Given the laundry list of social problems facing the world today, it is necessary to challenge current students to understand their world more deeply. Without the ability to “develop alternative explanations or solutions for contemporary social issues” (Minnesota State University−Mankato, 2009), students will be ill equipped to address the problems they face.

The presence of students in our classrooms who contest the perspective of the professor, or critical sociology specifically, intentionally, or unintentionally presents unique challenges. In the classroom, students cannot avoid or opt out of their professors’ critical sociological approach. Accepting course content that addresses inequality, oppression, and civil liberties may be difficult for some students because it contrasts with their understanding of why such issues persist. Their understanding tends to be rooted in the popular notions of individual explanations and personal motivations. When students are exposed to the critical sociological perspective, they might experience cognitive dissonance (Kelly-Woessner & Woessner, 2006). One solution is to withdraw from controversial issues in the classroom and hide the fact that, as a scientist, the professor has an obligation to present the most accurate and contemporaneous research possible. Due to the pressures of institutional rewards, faculty may succumb to simply presenting various perspectives, as opposed to challenging the students’ understanding of the causes of social problems. When professors do not go beyond merely presenting various perspectives, this leaves the impression that one perspective is just as good as another, thus not challenging students and their understanding. Just as the reality of climate change has remained in a limbo of “debate” over its existence, students may leave the classroom assuming that the research regarding wealth inequality as a result of segregation is also debatable and believing that racism, inequality, and so on is the result of an individual’s characteristics based on the flawed notion that our society is an open and equal system based on merits only.

The critical sociological approach asserts that social science evidence, regardless of its status as controversial, should be communicated. As a result, the critical sociological perspective will inevitably prompt ire in those whose belief systems disagree with contemporary social science evidence. One student clearly expressed his discomfort with the critical sociological approach, especially the discussions of repression and agent provocateurs.

While the instructor brings passion to his teaching, he fails in teaching important class concepts and instead opts for stressing his own political beliefs. He uses “a new and controversial learning style that may make you uncomfortable” as his reason. The professor should stick to teaching important class concepts & not his personal viewpoint. He had many negative and dangerous lectures about his negative views on law enforcement. This was not conducive to a learning environment.

What students like these are asserting is that ethnocentrism, alienation, filters of propaganda, repression, gender inequality, institutional racism, climate change, and so on are the professor’s personal views and not “important class concepts.”

Simply because some students emphatically deny social scientific evidence, professors should not be goaded into watering down their courses by sticking to “important class concepts” to make everyone feel comfortable. As Howard Zinn (2002) so famously asserts, “you can’t be neutral on a moving train.” As a result of viewing a critical sociology or other disciplines as politically charged, professors may argue that they are not trying to “effect a change in the political opinions of its students” (Jay & Graff, 1995, p. 206). To us, this is an odd stance to take in that critical sociology is no more (or less) an expressly political project than other disciplines like mechanical engineering, anthropology, physics, or economics. In opposition to Kelly-Woessner and Woessner’s (2006, p. 500) admonitions to “strive for political balance,” we suggest that focus should move away from the issue of the political orientation of the professor to a better understanding of how the students’ own perspectives may influence faculty evaluations.

Research in this area needs to provide more specificity regarding what causes students to react negatively to the course content. Administrators and supervisors should be aware of the effects of the students’ prejudices on the evaluations of professors who take a critical sociological approach. While Bérubé (2006) and others (Nocella, Best, & McLaren, 2010) look at the influence of the radical right to influence what is taught on college campuses, research could also focus more specifically on how this media assault on academic freedom has created intolerance among some students, especially those required to take general education courses.

Our research is a case study and only provides a limited insight; however, it does provide guidance for future research. A more ambitious project could be developed to analyze the content of Introduction to Sociology courses to determine the level of critical sociology in the course. Differences in the application of critical sociology could be contrasted with the degree to which students criticize the perspective in the classroom and the evaluation scores of the professor. Further studies could be expanded to other disciplines. Issues of race/ethnicity, gender, sexuality, and class can be incorporated into new research in this area. Despite a critical sociological approach, is there White privilege that produces higher evaluation scores for White, heterosexual males compared with other groups? While previous studies tended to focus on a balance of views in the classroom to accommodate students as opposed to challenging their individualistic perspective, new research could be reoriented toward attempting to understand how much the student’s own bias influences the evaluation outcomes. Despite its limitations, our study does demonstrate a direct connection between students’ negative evaluations of the professor’s critical sociological perspective and lower evaluations of the course as a whole. These results have profound implications for faculty’s pedagogical choices and how they affect the institutional system of rewards within the academy, such as tenure, promotion, grants, course releases, and sabbaticals.