Decoding the Disciplines: An Approach to Scientific Thinking

Abstract

The Decoding the Disciplines methodology aims to teach students to think like experts in discipline-specific tasks. The central aspect of the methodology is to identify a bottleneck in the course content: a particular topic that a substantial number of students struggle to master. The current study compared the efficacy of standard lecture and readings (Control) to the Decoding the Disciplines methodology in teaching Introductory Psychology students about the scientific process. Relative to the Control group (N = 45), students taught using the Decoding the Disciplines methodology (N = 46) created better hypotheses and operational definitions and correctly identified more variables. These results suggest that using the Decoding the Discipline methodology may facilitate student learning about scientific inquiry.

Introduction

A critical understanding and appreciation for the scientific method is a competency recommended by the American Psychological Association (APA) (American Psychological Association, 2007) for undergraduate psychology programs. This outcome has been adopted by many psychology programs and psychology faculty will often cite the development of scientific thought as a crucial skill for their students. In support of this goal, over 50% of Introductory Psychology courses have learning objectives related to the science and application of psychology (American Psychological Association, 2014). Students, however, often struggle to reconcile their ideas about psychology with the fundamental role of the scientific method in psychology. When discussing empirical evidence in a class, students commonly dispute or reject the research because they know a contradictory anecdotal example. This highlights students’ struggle to conceptualize psychology in a scientific manner. This struggle provides a foundational challenge to students’ mastery of the scientific understanding of psychology. It may be that the early unit in introduction to psychology that focuses on the scientific method is a bottleneck for student understanding. Consequently, students are unable to apply scientific thought and processes to later chapters about sub-discipline within psychology because they do not understand the scientific method.

This inability seems in stark contrast to the common outcome of scientific competency recommended by the APA. Indeed, even before the APA curriculum guidelines, one study noted that most undergraduate psychology classes place central emphasis on empirical research (Friedrich, 1996). Whereas instructors focus on teaching empirical studies in their classes, they often bemoan their students’ inability to grasp common ideas in research and scientific methodology. A common complaint among nearly all psychology instructors is their students’ inability to differentiate between independent and dependent variables. Friedrich suggested encouraging students to think about psychology as a scientific pursuit. More recent research, however, has shown that even psychology majors with extended coursework in the field do not view psychology as a science, though their knowledge of the scientific method increases (Holmes & Beins, 2009).

There is a tension between what psychologists want their students to know and what the students know. This tension is most keenly felt in Introductory Psychology courses (Hailstorks, 2007). Using a laboratory course in Introductory courses has been posited as one way to increase students’ knowledge and appreciation of the scientific method and empirical research (e.g., Berthold, Hakala, & Goff, 2003; Thieman, Clary, Olson, Dauner, & Ring, 2009). Unfortunately, with limited faculty lines and large teaching loads, this is not always a feasible approach for psychology programs.

Thus, teaching students about psychology as a science is critical for student learning, but students often struggle with this conceptualization of psychology. Consequently, getting students to think scientifically about psychology is a bottleneck in student learning. Bottlenecks occur in student learning when there is an aspect of the course that is challenging and disruptive to students’ mastery of the content. These bottlenecks can take a multitude of forms (Diaz, Midendorf, Pace, & Shopkow, 2008; Middendorf & Pace, 2004). An emotional bottleneck may be problematic for students who have an emotional reaction to the content that impedes their ability to learn. For example, students who have experience with mental illness may be too distressed by their personal connection to that content to be able to learn about the topic in an academic manner. Other students may become stuck at an epistemological bottleneck where students lack an understanding of how knowledge is constructed. An example of this type of bottleneck may be when students attempt to use anecdotal stories to argue against empirical evidence (such as arguing that they were spanked as children but experienced no deleterious effects, so the research on spanking is nullified). Finally, a procedural bottleneck occurs when students encounter a multi-step process but they fail to master some steps within the process. Consequently, students are unable to successfully complete the task. The current research centers on a procedural bottleneck: students’ ability to use the scientific method to answer psychological questions.

Many Introductory Psychology students experience a bottleneck in their learning around the application of the scientific method to psychology. This bottleneck impedes student learning related to one of the major disciplinary goals of psychology (critical understanding and appreciation for the scientific method). To help students move through this bottleneck, a supplementary activity using the Decoding the Disciplines methodology (Middendorf & Pace, 2004) attempted to improve student mastery of the scientific method. The Decoding the Disciplines methodology is an attempt to help students learn to think in disciplinary-specific ways by modeling how experts approach the topic. Central to the methodology is defining a “bottleneck” in the course. Once the barrier bottleneck has been defined, the instructor can use disciplinary knowledge and expertise to break down the bottleneck into smaller conceptual units and processes to aid student learning.

The goal of the current study is to facilitate students’ mastery of the scientific process and its application to psychological questions.

Method

Participants

Participants were 91 students enrolled in two Introductory Psychology courses at the University of Wisconsin–Superior. All students received partial course credit in exchange for their participation.

Materials

First, I constructed research questions relating to topics covered in Introductory Psychology. All research questions can be found in the Appendix.

At the completion of the lesson participants were given a worksheet containing a novel research question (e.g., Are people happier after exercising or before exercising?). The worksheet included prompts to generate hypotheses, brief methodologies, independent, and dependent variables and operational definitions.

Procedure

Two classes were selected to take part in this experiment; the classes were assigned to the Decoding the Disciplines group or Control group by a coin flip. The Control group (N = 45) received approximately a class period and a half (approximately 100–115 minutes) of a traditional lecture about the topic of research methodologies in psychology (the lecture covered the scientific method, operational definitions, hypotheses, research questions, and different methods commonly used in psychology). Additionally, the students read a textbook chapter on the topic. The Decoding the Discipline group (N = 46) took part in a class period and a half (approximately 100–115 minutes) of the activities that are outlined in steps three through six below; they were also instructed to read the relevant textbook chapter before class (same chapter and textbook as the Control group). Following the in-class learning experience student learning was assessed (Step 6) with a worksheet that included prompts to generate hypotheses, brief methodologies, independent, and dependent variables and operational definitions.

Steps 1 and 2 were completed prior to the class by the instructor.

Step 1: Identify the Bottleneck. As previously discussed, the bottleneck of interest is students’ ability to make a psychological question into a testable and measurable scientific experiment. This bottleneck was chosen because assessment in my Introduction to Psychology courses found that students showed low levels of competency in scientific thinking. Typically, questions regarding operational definitions and variables (independent versus dependent) had the lowest accuracy rates throughout the whole course. As a result, students who failed to master these concepts in the early sections of the course struggled to apply them in later sections. Further, poor performance on those exam questions, in my course, predicted lower overall grades in the course. This analysis of student learning (particularly as it relates to course and program outcomes) led to the choice of bottleneck.

Step 2: Identify discipline specific ways of thinking about the bottleneck. There are multiple aspects of scientific thinking that are second nature to professionals in psychology, namely, the ability to turn abstract questions about the mind and behavior into concrete measurable questions. In doing these tasks, psychologists must master the following concepts: operational definitions; manipulation; hypotheses; and variables. These concepts are central to our students’ ability to approach psychology scientifically. (Detailed discussion of how to deconstruct disciplinary thinking can be found in either Middendorf and Pace (2004) or Pace and Middendorf (2004).)

Step 3: Model disciplinary thinking. The class period began by introducing the concept of empiricism to the students. The instructor framed empiricism as something that the students already do by providing real world examples. (For example, when buying a car, a person will often test drive multiple options. The prospective buyer can then use their experience to make their choice.) Then the instructor transitioned into a discussion of how psychologists can be confident that things are true (centering on observation and experimentation).

Next, the instructor introduced a six-step scientific method.

A question is identified—what exactly does the scientist want to know?

Derive a hypothesis from previous knowledge (the instructor provided a “hypothesis”).

Identify terms within the hypothesis that need to be measured or quantified (introduce concept of operational definitions). Create an operational definition of terms within hypothesis.

Take the operational definitions and construct a methodology.

Identify the independent and dependent variables.

Conduct the experiment, analyze the data and revisit the original hypothesis.

Once this process had been outlined, the instructor wrote out a research question that will be revisited later in the course content (see Appendix for samples). Using the research question as an anchor, the instructor modeled the preceding steps. For the question, the instructor and class identified a hypothesis, operational definition, experimental methodology and labeled the independent and dependent variables. For example, for the research question: “How does vision influence balance?,” the researcher asked the students to identify the independent variable (vision) and the dependent variable (balance). Next the instructor guided the students to develop operational definitions for vision (i.e., eyes are open and corrected to normal vision versus eyes are closed) and balance (i.e., length of time, in seconds, a person can stand on one foot). Next the class discussed these variables and developed a hypothesis (i.e., vision improves balance such that when eyes are open people will be able to stand on one leg longer than when eyes are closed).

The modeling process was repeated for three questions; with each example, the instructor offered less input into the process, allowing for more student input. This process took approximately 45 minutes.

Step 4: Practice and feedback. Next, students worked in pairs to complete a hypothesis, method, operational definition and variable identification for two questions. Following completion, students turned in their assignment. Students were given 25 minutes to complete the practice sessions.

The instructor wrote (ungraded) feedback on each pairs’ assignment after the class period ended. This feedback included praise and corrections. In the next class period, the assignments were handed back. The entire class then discussed common mistakes on the assignments and ways to correct those errors.

Step 5: Motivate students. Prior to assessment, the class worked through one more research question example. Students designed and conducted an experiment to answer the research question. The research question in this step should be something that is easily completed in class and engaging (e.g., does vision influence balance or does coffee drinking dull the sense of taste?). Step 5 took approximately five minutes.

Step 6: Assess student learning of task. Students completed the previously described worksheet assessing their ability to generate hypotheses, create methodologies and operational definitions and identify independent and dependent variables for each of 6 example questions. Step 6 took approximately 35 minutes.

For each question, three responses were scored: the generated hypothesis; operational definitions for the variables; and identifying the independent and dependent variables. Each response was coded as either a 0 (incorrect) or 1 (correct). A correct hypothesis that was directly related to the research question would earn a score of “1.” For example, the hypothesis “Stress makes people remember fewer items from a list” would be counted as correct for the research question “Does stress help or hurt memory?” For the operational definition and independent variable questions, there were two possible points (correct identification of and operational definition for the independent and dependent variables). From the previous example, an operational definition of memory such as “number of items correctly recalled from a list” would be scored as correct while “items kept in the brain for a period of time” would be scored as incorrect.

Results

Responses for all six questions were scored by two independent coders who were naïve to the purpose of the experiment. Inter-rater reliability between the two coders was 0.877. Mean scores for the three dependent variable tasks (hypothesis generation, operational definition creation, and variable identification) can be found in Table 1.

Table 1.

Points possible, mean correct score (M), and standard deviation (SD) presented for the three dependent variable tasks as a function of the Decoding the Disciplines (DtD) group and the Control group.

	DtD Group		Control Group
Task	M	SD	M	SD	Points Possible
Hypothesis Generation	0.945	0.206	0.678	0.468	1
Operational Definitions	1.57	1.017	0.852	0.405	2
Variable Identification	1.368	0.807	0.545	0.787	2

Three independent samples t-tests compared performance on each of the assessment measures (hypothesis generation, identifying variables, and creating operational definitions). Students who were taught using the Decoding the Disciplines technique were more likely to generate complete hypotheses that looked at relationships between two variables, t(89) = 3.54, standard deviation (SD) = 0.08, p < 0.01; d = 0.738. Similarly, students in the Decoding the Disciplines condition outperformed students in the Control condition in writing operational definitions for the variables, t(89) = 10.838, SD = 0.16, p < 0.01; d = 0.928. Finally, students in the Decoding the Disciplines group more accurately identified the independent and dependent variables than students in the Control group, t(89) = 4.767, SD = 0.18, p < 0.01; d = 1.033.

General Discussion

The Decoding the Disciplines method aided student learning about how to derive scientific questions about everyday psychological phenomena. Part of the Decoding the Disciplines methodology’s success is encouraging psychologists to look at their course content from a student perspective, not as an expert. Identifying student bottlenecks offers insight into why students may show deficits in some areas of their learning while they excel in others. By breaking down the bottleneck task into small component parts, the expert easily models scientific thinking for students. The Decoding the Disciplines method makes the scientific method more transparent for students by encouraging experts to engage in metacognitive processes about their discipline.

One weakness of the study is the extent of difference in the classroom experience between the two groups. One group received a class period and a half of active learning; one group received an equal amount of time in a relatively passive learning environment. An ideal comparison would be between the Decoding the Discipline condition and a class that received an equal amount of time engaged in active learning. Despite this concern, it is clear that the Decoding the Disciplines methodology offers an effective form of active learning. The Decoding the Disciplines method also offers potential for other bottlenecks for students within the field of psychology such as the basis of neural communication, the logic of p values, or the Opponent-Process Theory.

One potential implication of this research is that using the scientific method to answer questions about psychology is a difficult task for students and one that requires feedback and explicit instruction. Although most psychology courses focus on empirical evidence from the field, it is possible that faculty do not model the steps involved in the creation of knowledge for students. In this instance, our entrenched disciplinary ways of thinking provide a roadblock for novice psychologists. The Decoding the Disciplines methodology is one way to address this issue and teach students to think like psychologists. Additionally, this methodology may help to align a goal of psychology faculty, to increase understanding and appreciation for psychological science, with student outcomes.