Abstract
Although students make some epistemological progress during college, most graduate without developing meaning-making strategies that reflect an understanding that knowledge is socially constructed. Using a pre-test–post-test design and a within-subjects 2 × 2 mixed-design ANOVA, this study reports on empirical findings which support the Socratic method of teaching as effective in challenging and changing psychology capstone students’ levels of epistemological maturity as measured by the Learning Environment Preferences survey and Perry’s model of intellectual maturity.
Keywords
Introduction
Many students leave academia without developing meaning-making strategies that reflect an understanding that knowledge is socially constructed (Pizzolato, 2006). In particular, as Cuseo (2007) reported, students need to develop critical thinking skills that can be applied outside of the classroom, in real-life situations. This is of international interest (Buehl, 2008) and particular concern for the discipline of psychology because the American Psychology Association’s (APA) guidelines for the teaching of psychology (APA, 2013) highlight the importance of psychological literacy, while addressing specific goals required of a psychology major – including critical and creative thinking and the ability to apply psychological principles to personal, social, and organizational issues. Furthermore, the goals include the need for students to learn how to tolerate ambiguity and respect international diversity and behavior (Zagoria, 2014). However, Dunn et al. (2010) noted that institutions in the USA had largely ignored past attempts to implement guidelines in psychology teaching, specifically the inclusion of a capstone course during the final year. Instead, Halpern et al. (2010) reported the growing trend of psychology departments to deliver more specialized courses (e.g., cognitive psychology, social psychology) as opposed to providing a solid undergraduate course that delivers key experiences and standards within the psychology discipline.
The promotion of culminating projects, or capstone courses, is presented extensively by the Association of American Colleges and Universities (Carey, 2013). Capstone course goals are aligned with APA objectives and typically include opportunities for students to integrate, create/synthesize and apply knowledge (Bangasser et al., 2016). Encouraging students to question their understanding of the nature of their knowledge in a rigorous, intellectually authentic manner can be transformative (Ashworth, 2004; Formenti, 2015; Goodman, Murphy, & D’Andrea, 2014; Illeris, 2015). Different instructional approaches may achieve equally valid outcomes, but empirical work associated with specific approaches remains limited (Buehl & Alexander, 2001; ten Dam & Volman, 2004). The use of a discussion-based format (Nunn, 1996), especially one based on the Socratic method, can be highly effective (Bagshaw, 2014; Pihlgren, 2008).
Meaning Making Through a Capstone Course
Bloom’s original taxonomy (1956) of learning objectives was revised by Anderson and Krathwohl (2001) resulting in three main categories: analysis, evaluation and creating (also called synthesis). Meaning making is creating, the outcome of which is successfully combining elements to form a coherent or functional whole. It has two fundamental aspects: intellectual maturity; and situations that encourage autonomous problem solving (Huber, 2005; Pizzolato, 2006). Clegg (2015) identified five key knowledge practices or forms of meaning making: description, personal reflection and reflexivity, explicitly theoretical, and data-driven. Capstone courses vary in the proportional use of these, but intellectual maturity requires personal reflection and reflexivity – of the instructor and the students (Avalos, 2016; Bangasser et al., 2016; Bendixen, 2002).
Perry’s full learning model (1970) is based on a stage model of intellectual development and encompasses nine stages or positions. Starting with basic duality (in which knowledge is basically right or wrong), it culminates in a sense of personal identity and commitment despite uncertainty or conflicting sources (Thomas, 2008). As Remy (2015) noted, Bloom’s taxonomy, used in tandem, provides a hierarchical framework of learning objectives and Perry’s model offers a degree of insight into how students’ epistemological maturity may interact with tasks aimed at particular learning outcomes. Myers (2010, p. 124), summarized this nicely: ‘… dualist students want to receive information, multiplist students want to learn how to think and contextual relativist students want to exercise their ability to think’. An awareness of these differences allows one to anticipate and shape course discussion by using questions of increasing ambiguity while moving subtly between the stages identified by Perry (or related theoretical models). Rapaport (2002, p. 48), by way of a student comment, provides a simple example: I really enjoyed this course. I had lots of trouble till about 2/3 into the course, ’cause I was looking for answers. Once I realized there were no answers and you had to figure things out for yourself, it became easier.
Metacognition refers to the knowledge of one’s cognitive processes and how these can affect one’s learning performance (Delahaij & van Dam, 2016; Muis & Franco, 2010). Epistemic metacognition seeks to integrate metacognition and developmental epistemology. Current work applies much of Perry’s developmental model (and those of a number of other theorists; see also Maggioni, VanSledright, & Alexander, 2009) by highlighting a useful distinction between the cognitive and affective processes associated with changing one’s beliefs and the new [beliefs] themselves (Bendixen, 2002; Hofer, 2002, 2016; Hofer & Sinatra, 2010; Sinatra, 2016).
Inquiry-based teaching and capstone experiences help students ‘integrate, apply, and evaluate … diverse findings as well as the psychological perspectives’ (Wade, 1997, p. 151). The capstone course is intended to allow an instructor to assess the student’s overall collegiate learning experience and, by its very nature, is a method of summative evaluation. Capstone course expectations include a display of a mastery of learning and the ability to apply it to new, unusual and integrated project requirements. A primary goal of teaching and outcomes assessment in the capstone course involves presenting the students with authentic challenges connected with knowledge mastery, reasoning proficiency, and professional expectations (Dunlap, 2005; Stiggins, 1997). Reflective questions can make students aware of what they know and what they don’t know. These types of questions are key to the Socratic method. Thoughtfully challenging a capstone student’s assumptions based on metacognitive principles is one way of prompting autonomy and mastery by way of meaning making (Ford, Smith, Weissbein, & Gully, 1998).
Using the Socratic Method in a Capstone Course
The Socratic method uses enquiry in varying degrees (Furedy & Furedy, 1982; Pang, 2008). Introducing dissonance by way of follow-up questioning, an instructor can effectively, and respectfully, prompt consideration of changes in Perry’s epistemological positions by way of accommodation (Bendixen, 2002; Proulx & Heine, 2010). Processing various cognitive and affective components of oneself and others, and learning to receive and provide feedback, are integral components of the course (Berman, 2007; Biesta, 2013; Paivi, 1998; Richardson, 2013).
According to Jonassen (2002), in this learning environment the teacher acts as a participant–facilitator and is thus able to guide and scaffold the learning process by providing related experiences about which students may not have first-hand knowledge. This includes modeling reasoning and generalization of similar situations. For example, in the ‘loss and grief’ section of the course many students initially respond that they have not ‘lost’ anyone, but when the discussion shifts to relationship break-ups, and then circles back to the empirical work (Wortman & Silver, 1989; Wortman & Boerner, 2011) students are able to connect affectively with the topic.
Students May Often Know What They Think, But Not Why
The weekly selection of primary readings purposely incorporates a diverse range of alternative perspectives and competing world views (e.g., course syllabi here); for example, Cain (2006) on the consequences of parental suicide, Huss and Cwikel (2008) on the use of art as a technique for socially acceptable expressions of distress in Bedouin women, or Hansbury (2004) and his narrative characterizing the cognitive, affective, and behavioral experiences of undergoing sexual reassignment from female to male. This is not a ‘controversial issues’ approach but, rather, a ‘novel viewpoints, never considered before’ approach. Thus the teacher, acting as facilitator, then uses the questions submitted by students to extend into discussions that use to advantage the students’ differing perspectives, interpretations, and alternative ways of resolving the divergent vantage points. Wrestling cooperatively with different theoretical and empirical standpoints on a weekly basis introduces a sustained degree of ambiguity and access to Perry’s (1970) culminating stage of personal identity (Moore, 2009). In a safe discussion-based setting this type of enquiry and ensuing conversation is intended to prompt an educated commitment to the students’ beliefs based on why they reach a particular conclusion.
Course processes are modeled after the Socratic method described by Paul (1995): This consists in teachers wondering aloud about the meaning and truth of students’ responses to questions. The Socratic teacher models a reflective, analytic listener. One that actively pursues clarity of expression. One that actively looks for evidence and reasons. One that actively considers alternative points of view. One that actively tries to reconcile differences of viewpoint. One that actively tries to find out not just what people think but whether they think it is actually so. (Paul, 1995, p. 297)
Some Questions Are Better Than Others
Developing the skillful exchange of questions that respectfully challenge thinking and to promote open discussion (Toledo, 2015) is a key aspect of the Socratic method and an inquiry-based approach to teaching. Discussion in the classroom prompts creating/synthesis and promotes the use of higher order aspects of knowledge use as identified by Bloom (1956) and others. As an example, based on the original work by Andrews (1980), explicit fact-based questions (e.g., ‘What was the name of that institution?’) yield the fewest subsequent associated number of student statements (NSS), 1.45 (it has only one answer) compared with a focal, invitation to explore a topic (e.g., ‘Can we make any generalizations about the play as a whole, from the nature of the opening lines?’), which yields an average number of 5.08 subsequent related student statements. By formatting questions properly the facilitator can model the process of learning itself, and using the Socratic method authentically can result in epistemological shifts in students’ perceptions of themselves and the world (Asterhan & Schwarz, 2016; Tredway, 1995). This is precisely the goal of evaluation and creation (synthesis).
While it is the case that various instructional methods may be aligned with these and other APA Guidelines (APA, 2013), the current study seeks to examine whether Perry’s (1970) framework of cognitive development can assess particular outcomes associated with the Socratic method effectively. Does this instructional approach prompt meaningful epistemological shifts in student’s knowledge structures compared to traditional lecture-only based courses within an academic semester? To examine this question difference scores were calculated (Time 2 score minus Time 1 score). The hypothesis is that by using a pre-post design and Moore’s (1989) Learning Environment Preference survey (LEP) to assess Perry’s model (1970) of epistemological maturity within sections of a psychology capstone course based on Socratic principles, demonstrable epistemological shifts should be evident in the students’ perspectives. Similarly, following the same pre-post design in a comparison group of lecture-only participants, no differences in epistemological perspective are expected.
Method
The capstone course described in this report is predicated on the constructivist learning environment model outlined by Jonassen (2002; see also ten Dam & Volman, 2004). The goal of this model is to foster a classroom atmosphere where students are actively involved in dealing with messy, ill-structured real-life topics. Using a graduate-style seminar approach, weekly readings are selected to allow students not only to use what they have learned but also to challenge that knowledge together with unexplored assumptions acquired along each student’s particular path through the curriculum as a whole. The course is fully discussion-based and the focus of learning involves questions which the students are required to submit. These are shared with the class in a fully anonymous manner. Student’s questions are thus discussed openly in class, but in an entirely safe way.
Participants
Lecture-only (comparison) group
Participants comprised approximately 90% psychology undergraduate students, 116 (87 women, 15 male, and 14 gender not specified) enrolled in two different upper-level lecture-only courses (two separate class sections of each) at a large Midwestern comprehensive university. Upper-level psychology courses often involve the study of theories and methods, and developing an understanding of the applications and limitations of those theories. As a result, students generally take upper-division courses in their junior and senior years. A semester is 15 weeks in duration. The age of students ranged from 19 to 41 years (M = 22.2, SD = 3.1). While course material differed for the two courses, they were taught by the same instructor and offered a very similar structure (course grades were based on four multiple-choice exams and course writing exercises formed roughly the same proportion of students’ grades between the two courses). Both were upper-level core classes (Psy-303 Psychopathology and Psy-420 Theories of Personality). Both classes involve conceptual work and are composed of predominantly junior/senior psychology undergraduate students. These courses aim past simple content and attempt to encourage students to avoid rote memorization and progress toward a deeper understanding of the material. Informed consent regarding completion of the LEP survey was obtained, and participation was entirely voluntary.
Socratic method group
There were 135 participants (116 women, 17 male, and 2 gender not specified), undergraduate students enrolled across six (separate) capstone classes (taught by the same instructor) at a large Midwestern comprehensive university. Each class had an average size of about 22 students. A semester is 15 weeks in duration. The age of students ranged from 20 to 55 years (M = 23.1, SD = 5.6). All students enrolled in the capstone course except one were seniors. Informed consent regarding completion of the LEP survey was obtained and participation was entirely voluntary. Of an initial 150 students, 16 students did not complete the study (ten dropped the course and six failed to complete the Time 2 portion); differences between completers and non-completers on the measure at Time 1 were not significant, F(1,165) = 1.28, p = 0.73. No other demographic data were collected.
Procedure
Data collection followed the same procedure across all semesters. Students completed the survey twice, initially during the first week of the semester (Time 1) and then a second time in the final week of the semester (Time 2) using a within-subjects pre-test–post-test design. An ANOVA found no significant differences between the students by semester based on total credits completed or cumulative GPA, thus data from all three semesters were combined.
Measures
The LEP survey (Moore, 1989; LEP) is designed to be used with student populations, primarily in colleges and universities. It can be used either to measure patterns of longitudinal intellectual development across various subgroups of students or for pre-post evaluations of specific courses or groups of courses. It consists of 65 items subdivided into five domains (course content, the roles of instructors and peers, classroom atmosphere and activities, and course evaluation) related to approaches to learning. Adequate validity and reliability has been reported (Moore, 1989; 2000). The coefficient alpha ranges from 0.72 to 0.84 for the four levels examined (Moore, 2000). The measure assesses differences in intellectual and cognitive development in college students based on Perry’s (1970) model, specifically Positions 2 through 5. 1 Respondents rate each item on a four-point Likert-style self-report scale according to how significant they find that item in relation to their perception of the ideal learning environment. Respondents then rank the three most significant items to them personally from each of the five domains. Items range from simpler to more complex across each of the domains. For example, ‘Teach me all the facts and information I am supposed to learn’, ‘Include straightforward, not “tricky”, tests, covering only what has been taught and nothing else’, ‘Challenge students to present their own ideas, argue with positions taken, and demand evidence for their beliefs’, or ‘Allow students to demonstrate that they can think on their own and make connections not made in class’. A summative value, the Cognitive Complexity Index (CCI), is the primary score of the LEP and represents a respondent’s epistemological position. The CCI scale of intellectual development has a range of 200 (Perry Position 2) to 500 (Perry Position 5). The Cognitive Complexity Index (CCI) showed a test–retest correlation of 0.89 (Moore, 2000).
Results
Descriptive Statistics
Note. CCI = Cognitive Complexity Index.
Lecture-only (comparison) group N = 116; Socratic method group N = 135.
Comparison Group
In an effort to ascertain if the two comparison courses could be combined, a two-sample independent t-test was conducted to examine if a difference was present in the mean difference CCI score between Time 1 and Time 2 between student participants in the two lecture-only courses (four sections total; two sections of each course). The mean difference in CCI scores from Time 1 to Time 2 between the comparison sections were not significant, t(114) = –0.129, p = 0.898. This indicates that there is insufficient evidence to conclude that a significant difference exists in the mean difference in CCI scores from Time 1 to Time 2. Given the lack of differences by cumulative credits, grade point average, or changes in the CCI scores between the students in the comparison sections, and that the preponderance of students in the classes were psychology students, the course sections were combined to create a single comparison group.
Main Study
2 × 2 Mixed ANOVA (CCI Between Groups and Time Within-Subjects)
Notes: CCI = Cognitive Complexity Index. Time 1 to Time 2 = 15 weeks.
Figure 1 displays the mean CCI plot by group and time. Hypothesis tests on the contrasts were used to examine if a difference was present in the mean CCI score before and after the semester for both the students in the Socratic method group and the lecture-only group. The difference in mean CCI scores was not significant for the lecture-only group (F(1,247) = 1.72, p = 0.19); in contrast, the difference in mean CCI scores was significant for the Socratic method group (F(1,249) = 30.26, p = < 0.0001). This indicates that only the Socratic method group demonstrates sufficient evidence to conclude that a significant difference exists in the mean CCI scores before and after the semester. A 95% confidence interval for the mean difference in CCI scores between Time 1 and Time 2 for the Socratic method group is (10.18, 28.26), and a 95% confidence interval for the mean difference in CCI scores for the lecture-only group is (–4.77, 14.56). In conclusion, examination of the confidence intervals for the mean differences reveals that the Socratic method group reported increased cognitive complexity and the lecture-only group failed to demonstrate improvement (because the confidence interval did not include 0).
Socratic method versus lecture-only groups.
Discussion
The results of this study demonstrated a significant shift in students’ epistemological beliefs; specifically, beliefs that would normally be resistant to change (Schommer-Aikins, 2004; Swann, 1997; Sweeny & Miller, 2012). By using a capstone course based entirely upon the Socratic teaching method, psychology students were able to demonstrate the ability to tolerate ambiguity within novel meaning-making processes. This included concomitant challenges to their preconceived beliefs, albeit with ample scaffolding and thorough discussion.
The LEP provides a theoretically useful way to assess cognitive development in college and university students. In each semester capstone student CCI scores started, roughly, at the transition between Perry positions 3 and 4, and at the conclusion of the course, 15 weeks later, had shifted to the initial stages of position 4 (Moore, 1989; Hassaskhah, Sepahi, & Azarnava, 2015). Shifting from the dualistic view of position 2 (with a belief that there should be only one right answer), to a position 3 view that there may be multiple right answers is a big step. Between position 3 and 4 students have typically accepted the likelihood of multiple answers as correct, but efforts to encourage consideration of some answers as a better fit for the facts than others can be met with frustration. This is where conversations in class are very beneficial. Students can often ‘hear’ other students, even when views differ, if someone with a bit more insight explains why they have come to a particular conclusion. When the instructor is able to pose Socratic questions that encourage students to reflect in this way and then share their thoughts, scaffolding as such can thus be subtle and helpful. As a result, this type of capstone course can motivate peer-to-peer modeling, allaying anxiety and encouraging students to consider new perspectives (Asterhan & Schwartz, 2016; Bendixen, 2002). Current findings suggest that for teachers of psychology courses whose goal is to have students change the way they understand the nature of their knowledge, the LEP scale captures some of these changes with sufficient sensitivity.
The finding that mean CCI scores did improve in the Socratic method group demonstrates that a single 15-week semester is an adequate interval of time for students in a capstone course to provide evidence of significant improvement. The constructivist model posits one type of learning as a change in identity. Scaffolding incorporates the premise of Vygotsky’s zone of proximal development. This serves as an effective way to provide an appropriate degree of challenge and personal risk-taking within a context of sufficient perceived personal safety when faced with increasing ambiguity (Asterhan & Schwartz, 2016). These principles are central to the effective use of the Socratic method and account for much of the pedagogical value embedded in the dialectic approach.
Limitations and Future Research
There are limitations to this study that should inform future research. For example, the data for this study lacked external comparison, because all sections were taught by the same instructor. It may be the case that the instructor is more effective in discussion-based versus lecture-only courses. While the comparison courses were not capstone courses, they were upper-level core courses for the psychology major, including predominantly junior and senior level psychology students. As such, one of the goals in the upper-level courses is, per Elby (2001, p. S54), ‘helping students understand the importance of consistency and coherence, and the difference between rote memorization and deeper understanding’. By the same token, questions on the psychology subject test of the Graduate Record Examination are derived from material associated with these core courses. Full coverage of text material is therefore expected and these courses may default to content and focusing on key words, thus limiting the use of metacognitive strategies, meaning making, and epistemological development. Future work using different comparison groups would be useful.
In addition, the study samples contained a larger proportion of women. Psychology undergraduate courses tend to have a predominance of female applicants and so future research should provide a comparison of this method across psychology courses and, if possible, examine whether gender differences are influenced by this style of teaching. Future work examining the degree of change associated with the Socratic method in other types of discussion-based classes and other fields of study might prove beneficial. The present findings, in conjunction with recent work in the field, suggest further investigation of the role of epistemic doubt and its resolution will be important. A better understanding of underlying metacognitive processes associated with Socratic methods would prove useful (Bagshaw, 2014; Dinsmore & Alexander, 2016). In addition, the researchers have interest in extending the duration of this study to determine whether the improvement in epistemological perspective is retained over time.
Conclusions
Despite the limitations summarized above, the current findings suggest that the Socratic teaching method can enable students to challenge and reconcile different points of view (Paul, 1995) and at the same time allow the teacher to reliably assess valid student learning outcomes across the curriculum. Future research aimed at specific capstone assignments that might track bigger changes in specific domain scores on the LEP as well as the CCI composite scores could lend itself to a rich discussion on how best to assess particular course components, not only in capstone courses but also in courses in general.
Assessment in the capstone offers a unique vantage point on the science of psychology as an evolving discipline best understood from an array of differing perspectives (Mentkowski et al., 2016). This study proposes using the Learning Environment Preferences (LEP) scale as one approach to assessing course goals as identified by the APA and other professional bodies (e.g., in Europe and Australia). If debates about the best way to teach could be based on reliable and valid assessments of desired outcomes, it might do a lot to improve teaching. The thoughtful examination of passionate but empirically unverified claims about the efficacy of different teaching strategies has steadily increased in importance. The Socratic method can be meaningfully assessed, and is aligned with APA (2013) goals – consistently challenging faculty and students to reflect deeply, and to strive to ask ever more insightful questions about cognition, behavior, and experience (McGovern et al., 1991; Kazanjian & Choi, 2016).
Footnotes
Declaration of Conflicting Interest
The author(s) declare that they have no conflict of interest with respect to the authorship, research, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
