Abstract
There is considerable debate both in Australia and internationally about the best way for schools to deliver the thinking skills required for university study and professional work life – skills that are often referred to by the term “critical thinking”. This article reviews the literature on the effective teaching of critical thinking and looks at one example of a secondary school subject aimed at developing critical thinking skills in its graduates, namely the Theory of Knowledge (ToK) subject in the International Baccalaureate Diploma Program (DP). ToK, a core element of the DP, has no set curriculum. Instead, ToK requires students to think deeply about knowledge problems and to formulate coherent views on the nature of knowledge. This article reports on an Australian study into the teaching and learning of ToK and presents associated outcomes in cohorts of both secondary and tertiary students.
Keywords
Introduction: Teaching critical thinking
The study reported in this article examined the relationship between the International Baccalaureate (IB) Diploma Program (DP) subject “Theory of Knowledge” (ToK) – a compulsory (100-hour), pre-tertiary, epistemological subject – and the practice of critical thinking (CT), which has been defined as: … [t]he ability to engage in purposeful, self-regulatory judgement based on rigorous intellectual concepts and principles … and which is widely recognised as an essential academic skill (Facione, 1990, p. 2).
To contribute to this debate, the research reported here focused on the ToK subject which is currently the only stand-alone course in Australia that intends explicitly to deliver critical thinking skills in the pre-tertiary curriculum. According to its “Subject Brief”, ToK is … … [A] course about critical thinking and inquiring into the process of knowing, rather than about learning a specific body of knowledge. It plays a special role in the DP by providing an opportunity for students to reflect on the nature of knowledge, to make connections between areas of knowledge and to become aware of their own perspectives and those of the various groups whose knowledge they share (International Baccalaureate Organisation (IBO), 2014, p. 1).
This study presents some preliminary data on the effectiveness of ToK delivery in Australian schools through 1) a comparison of critical thinking skills in ToK students across year cohorts (years 11 and 12), as cross-sectional proxy measure of their skill development and 2) a comparison of critical thinking skills in DP graduates and non-DP graduates within a cohort of first year university students.
Given the benefit of a mixed-methods approach for triangulation of data (Creswell & Plano-Clark, 2011), this study employed ToK teacher interviews, classroom observations and artefact analyses in four case study schools in the Australian state of New South Wales (NSW) in order to understand better the complexities of students’ critical thinking skill development within the ToK course.
In Australia, the skill of critical thinking is, by and large, developed within key learning areas across the curriculum, including science, mathematics, English and history, and is understood as evidence of higher-order thinking or metacognition (Flavell, 1977) within the subject areas (McGuire, 2012; Pintrich, 2002). While metacognition is a crucial aspect of critical thinking (see Bruer, 2000), if schools frame critical thinking as thinking about knowledge in discrete, domain-specific ways (e.g. thinking about knowledge in science or thinking about knowledge in English), it may have limited use as a general thinking skill, since students may not receive explicit instruction in its transfer or use beyond the domain in which it is presented. Although some attempts have been made to conceptualise the teaching of critical thinking as a cross-curricular capacity (Australian Curriculum, Assessment and Reporting Authority, 2012), and to provide cross-curricular support for the development of higher-order thinking skills, cross-curricular innovations (such as the Tasmanian ‘Essential Learnings’; see Watt, 2006) have not been maintained. Proponents of the domain-specificity approach argue that it is easier to learn to think critically within a given subject area than it is to learn to think critically in a generic sense (Willingham, 2007), contending that domain-specific knowledge is necessary for critical thinking since what constitutes valid evidence, arguments and standards varies across domains (Bailin, 2002). However, traditional, content-based curricula dominate in Australia, which can push the specific teaching and learning of critical thinking to the periphery, potentially resulting in minimal explicit instruction of critical thinking skills. In other words, if critical thinking is an implicit expectation within discrete subject areas across the curriculum, it is more likely to be dropped from a given subject as curricular demands increase, particularly if it is not positioned as a measurable outcome.
This implicit, ‘infusion’ approach to developing students’ critical thinking skills is markedly at odds with available research on what works in the purposeful teaching and learning of critical thinking. In their meta-analysis of 117 studies with 20,698 participants, Abrami et al. (2008) concluded that critical thinking skills were developed best through explicit teaching methods rather than implicit teaching, which positioned critical thinking as a by-product of the subject being learned. In their comprehensive evaluation of empirical studies of critical thinking development in students, students’ greatest critical thinking skill gains occurred where critical thinking was explicitly taught in a stand-alone way (Abrami et al., 2008). Echoing findings from the above meta-analysis, Marin and Halpern (2011) reported on results from their two studies by comparing direct/explicit and indirect/implicit instruction in critical thinking within a population of high school students from disadvantaged, low-performing schools in the United States. Across both afterschool/supplementary classes and instruction during school hours, students in classrooms with explicit critical thinking instruction demonstrated much larger, statistically significant, gains in critical thinking skills. These gains were larger both in terms of the differences between pre- and post-instructional intervention measures and compared with outcomes of students who received implicit critical thinking skills instruction. These findings are critical to the field, as Marin and Halpern’s (2011) research remains the only randomised controlled trial undertaken in high schools to compare both approaches to teaching critical thinking. The results provided by these studies support the explicit instruction of critical thinking in a stand-alone subject such as ToK, as opposed to the implicit instruction of critical thinking across the curriculum. Furthermore, these results reinforce the need for further investigation into the role of explicit critical thinking instruction and, in particular, the impact of such instruction on students’ long-term employment of critical thinking skills and related measures of academic success across primary, secondary and tertiary education.
In addition to emphasising the importance of how the instruction of critical thinking is framed, research has shown that both teacher training and pedagogy have a significant impact on students' critical thinking outcomes. In their aforementioned meta-analysis, Abrami et al. (2008) found that students’ critical thinking skills acquisition was highest in groups where their teachers had received specific pedagogical training in the teaching of critical thinking skills and where their teaching in this area was assessed and monitored. In contrast, student outcomes were lowest when there were no efforts at professional development or elaboration of course design and implementation by and for the teacher. The authors concluded that better outcomes for critical thinking instruction can be achieved through active, purposeful training and teacher support. Unfortunately, studies of critical thinking in schools have shown that teachers’ critical thinking pedagogic knowledge and skill can be lacking (Aliakbari & Sadeghdaghighi, 2013; Thomas, 1999), which has implications for teacher education, in-service teacher training, and in the development of pedagogic strategies for the teaching and learning of critical thinking over time.
Two large-scale studies by Gardiner (1995) and Paul, Elder, and Bartell (1997) reported that while teachers say that they place primary importance on teaching critical thinking (90% and 97% respectively), authors concluded that “only 19% [of interviewees] gave a clear explanation of what critical thinking is … [and] only 9% of the respondents were clearly teaching for critical thinking on a typical day in class” (Paul et al., 1997, p. 18). Of greater concern is evidence which seems to suggest that many teachers see critical thinking as relevant only to high achieving students, despite evidence to the contrary (Dweck, 2002; Halpern, 2003; Kuhn, 2000; Zohar & Dori, 2003). Studies in the USA have demonstrated a difference in the critical thinking instruction given to high and low achieving high school students, with students in honours and high achieving classes having greater access to exemplary teachers who demonstrably encourage critical thinking through their teaching (see Thomas, 1999 for a California case study of this phenomenon). Furthermore, Stanton-Salazar (1997) points out that many disadvantaged students do not have access to the benefits of critical thinking in the curriculum and are likely to fall behind in the skills of problem solving, decision making, reasoning through arguments, and identifying how to avoid subtle coercion. As a consequence, these students can be deprived of the skills that are required to make positive long-term decisions and goals for their futures. Such conditions are likely to contribute to the cycle of underachievement, failure and exclusion from participation in productive and meaningful occupations that disadvantaged students face.
The benefits of explicit critical thinking instruction have been shown to have long-term positive impact across a variety of student populations, including those from disadvantaged backgrounds. Coca, Johnson, and Kelley-Kemple (2011), for example, provide longitudinal evidence of the impact of the IB DP on students of low socio-economic status (SES) in schools across the Chicago-metro area. As discussed, while the DP program does imbed critical thinking across multiple domains, the ToK subject is positioned as a central, cross-curricular and independent means to raise the academic performance of students through the specific teaching and learning of critical thinking skills. This study which compared the performance of DP graduates and non-DP graduates across a group of low-SES schools in the Chicago area revealed that: 1) DP graduates were 40 per cent more likely to complete a four-year college course and 50 per cent more likely to attend a more selective college and 2) DP graduates in four-year colleges were significantly more likely to persist in their tertiary education for two years or more (Coca et al., 2011). The two demographic groups with the strongest gains in academic performance as a result of their inclusion in the DP stream were African Americans and Latina/Latinos, who were prepared for the DP starting in year 9. Over the past three decades the connection between students’ epistemological approach and their intellectual development has been bolstered by research across age ranges, academic subjects, and in different cultures (see Gow & Kember, 1993; Samuelowicz & Bain, 1992 for excellent reviews of the literature on this topic).
At the time of writing, only Australian schools which offer the IB DP deliver the subject of ToK; contrariwise, the current state of critical thinking teaching and learning across the curriculum in non-IB schools across Australia is largely piecemeal and uncoordinated (Cole et al., 2014). Two states will be used here as examples, namely NSW and Victoria. In NSW, critical thinking is mentioned principally as one of a repertoire of skills that is developed through the selection by examination of students in terms of ‘opportunity classes’ (e.g. invitation-only classes in government schools for top-performing students in years 5 and 6 of primary school), ‘selective schools’ (e.g. invitation-only government secondary schools for top-performing students) and ‘selective streams’ in mainstream government secondary schools (e.g. invitation-only academic streams for top-performing students). This strategy has been justified by the NSW Department of Education and Community (DEC) with respect to the literature on ‘gifted and talented’ students, which has shown that these students need specific extension activities to keep them engaged in the learning process (New South Wales, Department of Education, 1991, 2001). As a result, the NSW strategy can have the unwanted consequence of reserving the provision of critical thinking education for students who have been categorised (via standardised testing) as being high achievers, contrary to evidence demonstrating the benefits of critical thinking for students from a range of academic levels (Coca et al., 2011). In Victoria, critical thinking has been bundled with creativity and meta-cognition as skills and dispositions that underpin the curriculum as an inter-disciplinary ‘thinking’ capacity – Critical and Creative Thinking (CCT) (Victorian Curriculum and Assessment Authority, 2013). One could argue that the lack of both a specific, well-designed and properly resourced critical thinking subject and targeted teacher training that aids the instruction of this course could make the provision of critical thinking in Victorian schools reliant on the inclinations of individual teachers and school leaders. This curricular structure persists despite the Victorian Department of Education and Training (DET) making online CCT-focused resources and strategies available to teachers as well as psychometrically testing 4,900 Victorian schoolchildren in 2014 for evidence of their creative and critical thinking skills and dispositions. Concurrently, CCT is being promoted as a cross-disciplinary focus to be embedded across the new Australian curriculum (Australian Curriculum, Assessment and Reporting Authority, 2012) as evidence of Australia’s ‘general capacity’ approach to critical thinking. However, teachers may well be confused about exactly how to implement this general capacity, particularly given a recent government report into the implementation of the Australian curriculum (Donnelly & Wiltshire, 2014) which sided with the domain-specific ‘side’ of the critical thinking debate as discussed above, effectively writing the CCT general capacity out of the Australian national curriculum.
While empirical evidence appears to support the short- and long-term benefits of an explicit, stand-alone approach to teaching critical thinking skills, no comparative data have been collected in the Australian context. Accordingly, the study reported in this article sought to investigate the influence of the IB DP ToK subject and its stand-alone approach to the delivery of critical thinking skills.
Furthermore, this study explored one of the key claims in a previous empirical investigation of the DP in the Australian and New Zealand contexts (Coates, Rosicka, & MacMahon-Ball, 2007) – namely that the DP, inclusive of its specific provision of critical thinking skills via the ToK course, provides an optimal preparation in the academic skills needed for university. Critical thinking skills appear to constitute the difference between the rote learning of content knowledge and being able to purposefully manipulate any new knowledge to a specified end (Coates et al., 2007; Paul et al., 1997).
Method
This study followed a three stage, convergent, mixed-methods approach (Creswell, 2012), with data collection occurring concurrently for each stage. This approach allowed for the triangulation of the information in order to understand better the link between students’ completion of the ToK subject and their critical thinking skill development. The quantitative component of the full study involved data collection across both ToK student and ToK teacher cohorts and explored a larger number of related variables (see Cole et al., 2014 for the full report). In this article, a snapshot of 1) ToK student data relevant to critical thinking skills acquisition and 2) comparative outcomes for a population of DP and non-DP first year university students is reported.
In the first stage of the study, the IBO sent all Australian DP school leadership staff an email invitation to participate in the national study, inclusive of text with the online survey URL to distribute to their population of ToK students in years 11 and 12. As students were aged 16 and older, parental consent was waived for the purposes of this study and survey completion was taken as evidence of informed consent, as participant information was located at the start of the online survey. The survey assessed critical thinking outcomes using both an original measure, developed for the purposes of this study, namely the Critical Thinking Strategies Scale (CTSS), as well as the five-item “Critical Thinking” subscale of the Motivated Strategies for Learning Questionnaire (MSLQ) developed by Pintrich, Smith, Garcia, and McKeachie (1993). The latter was used to enable additional comparison of critical thinking skills with a well-established, validated measure of high internal validity (Cronbach’s alpha = .80).
The 14-item CTSS was developed using evidence of critical thinking skills as outlined in the “Delphi Report” (Facione, 1990), widely regarded as the leading synthesis of expert consensus on measureable critical thinking skills. Students were asked “How likely is it that you will …” and exposed to a series of actions associated with critical thinking ability (e.g. “How likely is it that you will work to clarify meaning and define terms with which you are unfamiliar?”), with responses measured on a 7-point Likert scale ranging from “extremely likely” (7) to “not very likely” (1). Exploratory factor analysis (EFA) was used to reduce the number of items in the scale. This resulted in a 14-item measure which showed excellent internal reliability (Cronbach’s alpha = .96).
In stage two of the study, a convenience sample of four NSW schools offering the IB DP curriculum were chosen as sites for in-depth case study. Semi-structured interviews (see Appendix A for questions) were conducted with teachers, students (focus groups) and administrators, who were chosen for interview if available and at the discretion of the schools. The interviews took place in the schools, and asked how respondents viewed the subject of ToK, and where the subject of ToK was located in their learning programmes, both in terms of the value of ToK, and the how their programming was organised. In addition, ToK lesson observations (see Appendix A for observation focuses) were undertaken through which ToK lesson notes were taken, and document analysis of any documents produced by the schools with reference to ToK, for example, school reports, curriculum documents, student help guides. Data were synthesised in terms of agreed upon qualitative themes, that came about after rigorous discussion between the researchers involved in the project, and attempted to frame and develop ways in which ToK currently operated in the four schools. Researchers returned regularly to the schools over a four-month period, which provided an opportunity for follow up interviews, extra observational work and clarification with respect to the ways in which ToK provided a platform for CT skill development or otherwise (some researchers visited schools a maximum of 4 times).
Stage three involved data collection from a convenience sample of Australian first year university students, including those who had previously completed the ToK subject. DP graduates from the year prior to data collection were recruited through the Australian IBO Alumni contact list and approached via email by the IBO with the online survey URL. Non-DP graduates were solicited via targeted email/online announcements on their various units’ online content management systems (e.g. Blackboard). While a number of universities were targeted based on completion data for DP students in efforts to generate a strong comparison sample, only courses at the University of Sydney and the Queensland University of Technology assisted with recruitment efforts.
During stage three, DP and non-DP graduates in their first year of university students were compared using the 75-item California Critical Thinking Disposition Inventory (CCTDI; Insight Assessment, 2000), a well-regarded commercial measure of the students’ dispositional tendencies to deploy critical thinking as a ‘natural thinking habit’. This measure has been shown to have a very high internal consistency (Cronbach’s alpha = .90) with reliabilities for the seven subscales ranging from 0.67 to 0.82 (see Appendix B, for a complete list).
In addition, given the aforementioned links between critical thinking skill development and university success (Coates et al., 2007; Gardiner, 1995), as well as the ways in which enhanced critical thinking skills ease the cognitive load of myriad subject areas via a more discerning set of thinking skills and improved strategies for learning over time (Higgins et al., 2004), university students’ general academic outcomes were also measured in the study. To this end, the 10-item Academic Self-concept (General) Scale (Marsh, 1992) was used as a reliable, internally consistent (Cronbach’s alpha = 0.92) proxy measure of students’ actual academic outcomes (Byrne & Worth-Gavin, 1996). This proxy measure was deemed preferable to students’ reports of their actual academic achievement, given the challenges of standardising marks across multiple schooling institutions. Students were compared on two further measures. First, students were asked to report their Australian Tertiary Admissions Rank (ATAR) which is a standardised measure of their year 12 results. Second, a six-item measure of their anticipated university outcomes, specifically designed for this study was used to assess students’ self-beliefs regarding course completion, anticipated marks and anticipated unit failures (sample item: “I will finish my course without having failed a single unit”; Cronbach’s alpha = 0.78).
Sample
Number of schools in Australia which offer the IBO Diploma Programme (ToK) by state/territory, together with number of student questionnaires received.
In stage two, members of the research team performed: 1) 22 teacher interviews; 2) eight focus groups with current ToK students; 3) five interviews with administration or other school personnel; 4) a document analysis of relevant ToK literature produced by the schools, and 5) 10 observations of ToK lessons. Interviews with each teacher lasted for approximately 40 minutes while focus groups took approximately half an hour each and consisted of between four and seven students. The final sample of stage three participants included 389 first year students, of whom 99 (=25% of the total sample) were DP graduates.
Results and discussion
Mean differences in CTSS individual item and full scale scores, by ToK year cohort.
NS: non-significant.
Note: Missing cases eliminated listwise (n = 1262 for items; n = 1242 for total); *Indicates statistical significance. Items measured on a 7-point scale, with higher scores indicating greater use.
These differences were further confirmed by students’ outcomes on the five-item Critical Thinking subscale of the MSLQ. The mean scale score for the Year 12 cohort (M = 4.69, SD = 1.15) was higher than for their Year 11 counterparts (M = 4.48, SD = 1.22), with students’ year level shown to have a statistically significant effect (t(1240) = 3.04, p < 0.01). While both cohorts of students’ scores fell between the anchor point of “Moderately true of me” (4) and the high end of the scale (“Very true of me” [7]), the Year 12 cohort mean was moderately closer to the higher end of the measurement scale.
Taken together, these two quantitative outcomes point to greater awareness and use of critical thinking skills as students move from their first to their second year of the ToK subject. Analysis of the four case study schools aids in the interpretation of these findings.
The case studies focused on how the different schools conceptualised and operationalised their critical thinking skills improvement programmes through the ToK subject. While each of the four schools framed students’ critical thinking skill development as central to their school’s ‘mission(s)’, analysis of the case study data highlighted differences in the schools’ approaches to meeting this goal.
Summary of the case study results for critical thinking (CT) skills.
Mean differences on the CCTDI for DP and non-DP graduates.
Note: Former ToK-DP students (n = 42); non-ToK-DP students (n = 126); this drop in participants was likely due to the secondary online location of the commercial CCTDI, access to which required login details, which may have resulted in participant fatigue/disinterest. Additional details are available in the full report (Cole et al., 2014).
Mean differences in ATAR, ASC & perceived university outcomes, by cohort.
Indicates statistical significance.
These findings point to the tangible impact of completion of the DP curriculum, inclusive of the ToK subject, on students’ university readiness and access to competitive courses, as well as their own personal perceptions of their academic ability and their future university successes. Results suggest a link between the DP focus on students’ development of critical thinking skills and increases in both habits of critical thinking and representative measures of academic success.
Limitations and implications of the study
ToK requires DP students to think deeply about the knowledge and skills that they are learning across the curriculum and asks that they formulate answers to questions with respect to the nature of knowledge. Given the body of empirical research supporting the benefits of explicit, stand-alone critical thinking education (Abrami et al., 2008; Marin & Halpern, 2011), this study sought to explore these benefits in an Australian context via the IBs’ ToK subject.
While the findings reported in this article do appear to support explicit delivery of critical thinking education via ToK, it should be noted that students who have completed the ToK course have also completed the full DP course, of which ToK is only one component. Given our cross-sectional research design, it was not possible to separate the effects of ToK completion from the effects of completing the DP curriculum as a whole.
Still, these findings provide an interesting and extended picture of not only differential critical thinking outcomes for DP graduates, of which ToK is a defining feature, but also improved academic outcomes which are potentially linked to the development of critical thinking skills.
Results, therefore, highlight the need for longitudinal research with DP students before, during and post-ToK completion in efforts to parcel out the predictive impact of ToK on students’ critical thinking, short- and longer-term academic outcomes. Furthermore, small tertiary student sample sizes and a convenience, rather than a representative, statistically matched, sample of first-year university students mean that the tertiary student findings provide tentative, rather than unequivocal, evidence of impact.
Case study findings point to the need for greater clarification regarding the precise school climate and framing of ToK to allow for enhanced critical thinking skills, particularly around the ‘translation’ from epistemology as an overarching concept to the exploration and practical application of new and varied ways of thinking and knowing. There is much to learn about the best pedagogical methods for ToK teachers to employ in order to enable this translation and the optimal school conditions to support this work. To this end, case study findings highlight differences between schools in terms of ToK subject interpretation and operationalisation. For some schools, these interpretations were not always helpful, nor did they result in purposeful and explicit teaching of critical thinking. This was reflected in School 2, where student focus groups, teachers’ reports and lesson observations revealed an over-reliance on whole-class discussion work. Additional case study explorations could help to elucidate the mechanisms by which teachers of ToK interpret the IB’s official ToK subject outcomes in order to design and resource their lessons.
The present curricula position of ToK at the inter-disciplinary centre of the DP makes it an invaluable arena for critical thought and debate, and a possible ‘spark’ for students to go further in their own thinking, research and inquiry. Case study findings revealed that while students may struggle initially in their ToK classes (which we called ‘ToK-shock’), this is a subject that many will come back to, and will positively appreciate in the future, as they learn to think critically in an adult and mature manner, to question assumptions, and to more fully articulate their own knowledge and thinking beliefs. Further, findings from the case study element of this research supported the idea that key to ToK’s success (e.g. enhancement of students’ critical thinking skills) lies in the fact that the ToK subject is not subsidiary to any other subject, as it does not have learnt curriculum knowledge tied to its functioning (see, Cole & Throssell, 2008); instead, ToK allows students to fully use and explore their knowledge through structured thinking exercises and cross-curricula activities. Indeed, one of the case study schools had copied (but rebadged for propriety reasons) the ToK course for its non-IB students, and runs it as a compulsory aspect of its pre-tertiary education for all students. The local enthusiasm for exploring knowledge claims fully and the explicit modes of questioning that ToK can generate in communities are vital aspects of the classroom experience of ToK, and do contribute to the continued smooth functioning of ToK in DP schools as a ‘thinking mechanism’ (see, Cole, 2005) and a form of critical thinking practice with wide-ranging benefit.
Findings presented here add to the evidence and rationale for teaching critical thinking as an explicit subject. However, the implementation of a similar subject across the Australian national curriculum would require wholesale education reform and present great challenges for an already crowded curriculum landscape. Coupled with the previous empirical data on the benefits of such curricular inclusions, the current findings suggest benefits if schools in Australia considered to initiate pre-tertiary, ‘stand-alone’ critical thinking courses to enhance students’ academic success rather than relying on the implicit teaching of critical thinking through the key learning areas of the curriculum. It is acknowledged, however, that this is not a simple proposition, but rather one with significant implications for a number of areas, including pre-service teacher training, school staffing, curriculum organization and content as well as timetabling.
Footnotes
Authors’ note
The full report can be found at the IBO research web site: http://ibo.org/globalassets/publications/ib-research/dp/tokfinalreport_000.pdf. A summary of the report can be found here: 
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: the International Baccalaureate Organization (IBO).