Critical thinking skills in the International Baccalaureate’s “Theory of Knowledge” subject: Findings from an Australian study

Abstract

There is considerable debate both in Australia and internationally about the best way for schools to deliver the thinking skills required for university study and professional work life – skills that are often referred to by the term “critical thinking”. This article reviews the literature on the effective teaching of critical thinking and looks at one example of a secondary school subject aimed at developing critical thinking skills in its graduates, namely the Theory of Knowledge (ToK) subject in the International Baccalaureate Diploma Program (DP). ToK, a core element of the DP, has no set curriculum. Instead, ToK requires students to think deeply about knowledge problems and to formulate coherent views on the nature of knowledge. This article reports on an Australian study into the teaching and learning of ToK and presents associated outcomes in cohorts of both secondary and tertiary students.

Keywords

Theory of Knowledge (ToK)critical thinking mixed-method research International Baccalaureate (IB) in Australia academic self-concept

Introduction: Teaching critical thinking

The study reported in this article examined the relationship between the International Baccalaureate (IB) Diploma Program (DP) subject “Theory of Knowledge” (ToK) – a compulsory (100-hour), pre-tertiary, epistemological subject – and the practice of critical thinking (CT), which has been defined as:

… [t]he ability to engage in purposeful, self-regulatory judgement based on rigorous intellectual concepts and principles … and which is widely recognised as an essential academic skill (Facione, 1990, p. 2).

Despite the agreed significance of critical thinking for academic and professional success (Abrami et al., 2008; Paul, 1992), a global debate continues as to the best pedagogical approach for students’ development of critical thinking skills, abilities, and understandings (Hatcher, 2006). At the heart of this debate is whether it is best to teach critical thinking explicitly and systematically in an epistemological course, as is the approach of the IB’s DP course through their ToK subject, or whether the learning of critical thinking should develop implicitly as a bi-product of the inculcation of students into specific academic disciplines (see, Aizikovitsh & Amit, 2010 and Swartz, 1992 for their summative perspectives on this debate).

To contribute to this debate, the research reported here focused on the ToK subject which is currently the only stand-alone course in Australia that intends explicitly to deliver critical thinking skills in the pre-tertiary curriculum. According to its “Subject Brief”, ToK is …

… [A] course about critical thinking and inquiring into the process of knowing, rather than about learning a specific body of knowledge. It plays a special role in the DP by providing an opportunity for students to reflect on the nature of knowledge, to make connections between areas of knowledge and to become aware of their own perspectives and those of the various groups whose knowledge they share (International Baccalaureate Organisation (IBO), 2014, p. 1).

ToK gives students in years 11 and 12 the opportunity to understand and develop their critical thinking skills in an environment without the reproduction of learnt curriculum knowledge (Cole, Gannon, Ullman, & Rooney, 2014; Cole & Hager, 2010). Furthermore, while ToK does stand alone as a discrete subject, teachers across the DP, which services students in years 11 and 12 in schools following the IB curriculum, are encouraged to develop their students’ critical thinking skills and facilitate transfer of their students’ learning and insights from ToK through to their other DP subjects.

This study presents some preliminary data on the effectiveness of ToK delivery in Australian schools through 1) a comparison of critical thinking skills in ToK students across year cohorts (years 11 and 12), as cross-sectional proxy measure of their skill development and 2) a comparison of critical thinking skills in DP graduates and non-DP graduates within a cohort of first year university students.

Given the benefit of a mixed-methods approach for triangulation of data (Creswell & Plano-Clark, 2011), this study employed ToK teacher interviews, classroom observations and artefact analyses in four case study schools in the Australian state of New South Wales (NSW) in order to understand better the complexities of students’ critical thinking skill development within the ToK course.

In Australia, the skill of critical thinking is, by and large, developed within key learning areas across the curriculum, including science, mathematics, English and history, and is understood as evidence of higher-order thinking or metacognition (Flavell, 1977) within the subject areas (McGuire, 2012; Pintrich, 2002). While metacognition is a crucial aspect of critical thinking (see Bruer, 2000), if schools frame critical thinking as thinking about knowledge in discrete, domain-specific ways (e.g. thinking about knowledge in science or thinking about knowledge in English), it may have limited use as a general thinking skill, since students may not receive explicit instruction in its transfer or use beyond the domain in which it is presented. Although some attempts have been made to conceptualise the teaching of critical thinking as a cross-curricular capacity (Australian Curriculum, Assessment and Reporting Authority, 2012), and to provide cross-curricular support for the development of higher-order thinking skills, cross-curricular innovations (such as the Tasmanian ‘Essential Learnings’; see Watt, 2006) have not been maintained. Proponents of the domain-specificity approach argue that it is easier to learn to think critically within a given subject area than it is to learn to think critically in a generic sense (Willingham, 2007), contending that domain-specific knowledge is necessary for critical thinking since what constitutes valid evidence, arguments and standards varies across domains (Bailin, 2002). However, traditional, content-based curricula dominate in Australia, which can push the specific teaching and learning of critical thinking to the periphery, potentially resulting in minimal explicit instruction of critical thinking skills. In other words, if critical thinking is an implicit expectation within discrete subject areas across the curriculum, it is more likely to be dropped from a given subject as curricular demands increase, particularly if it is not positioned as a measurable outcome.

This implicit, ‘infusion’ approach to developing students’ critical thinking skills is markedly at odds with available research on what works in the purposeful teaching and learning of critical thinking. In their meta-analysis of 117 studies with 20,698 participants, Abrami et al. (2008) concluded that critical thinking skills were developed best through explicit teaching methods rather than implicit teaching, which positioned critical thinking as a by-product of the subject being learned. In their comprehensive evaluation of empirical studies of critical thinking development in students, students’ greatest critical thinking skill gains occurred where critical thinking was explicitly taught in a stand-alone way (Abrami et al., 2008). Echoing findings from the above meta-analysis, Marin and Halpern (2011) reported on results from their two studies by comparing direct/explicit and indirect/implicit instruction in critical thinking within a population of high school students from disadvantaged, low-performing schools in the United States. Across both afterschool/supplementary classes and instruction during school hours, students in classrooms with explicit critical thinking instruction demonstrated much larger, statistically significant, gains in critical thinking skills. These gains were larger both in terms of the differences between pre- and post-instructional intervention measures and compared with outcomes of students who received implicit critical thinking skills instruction. These findings are critical to the field, as Marin and Halpern’s (2011) research remains the only randomised controlled trial undertaken in high schools to compare both approaches to teaching critical thinking. The results provided by these studies support the explicit instruction of critical thinking in a stand-alone subject such as ToK, as opposed to the implicit instruction of critical thinking across the curriculum. Furthermore, these results reinforce the need for further investigation into the role of explicit critical thinking instruction and, in particular, the impact of such instruction on students’ long-term employment of critical thinking skills and related measures of academic success across primary, secondary and tertiary education.

In addition to emphasising the importance of how the instruction of critical thinking is framed, research has shown that both teacher training and pedagogy have a significant impact on students' critical thinking outcomes. In their aforementioned meta-analysis, Abrami et al. (2008) found that students’ critical thinking skills acquisition was highest in groups where their teachers had received specific pedagogical training in the teaching of critical thinking skills and where their teaching in this area was assessed and monitored. In contrast, student outcomes were lowest when there were no efforts at professional development or elaboration of course design and implementation by and for the teacher. The authors concluded that better outcomes for critical thinking instruction can be achieved through active, purposeful training and teacher support. Unfortunately, studies of critical thinking in schools have shown that teachers’ critical thinking pedagogic knowledge and skill can be lacking (Aliakbari & Sadeghdaghighi, 2013; Thomas, 1999), which has implications for teacher education, in-service teacher training, and in the development of pedagogic strategies for the teaching and learning of critical thinking over time.

Two large-scale studies by Gardiner (1995) and Paul, Elder, and Bartell (1997) reported that while teachers say that they place primary importance on teaching critical thinking (90% and 97% respectively), authors concluded that “only 19% [of interviewees] gave a clear explanation of what critical thinking is … [and] only 9% of the respondents were clearly teaching for critical thinking on a typical day in class” (Paul et al., 1997, p. 18). Of greater concern is evidence which seems to suggest that many teachers see critical thinking as relevant only to high achieving students, despite evidence to the contrary (Dweck, 2002; Halpern, 2003; Kuhn, 2000; Zohar & Dori, 2003). Studies in the USA have demonstrated a difference in the critical thinking instruction given to high and low achieving high school students, with students in honours and high achieving classes having greater access to exemplary teachers who demonstrably encourage critical thinking through their teaching (see Thomas, 1999 for a California case study of this phenomenon). Furthermore, Stanton-Salazar (1997) points out that many disadvantaged students do not have access to the benefits of critical thinking in the curriculum and are likely to fall behind in the skills of problem solving, decision making, reasoning through arguments, and identifying how to avoid subtle coercion. As a consequence, these students can be deprived of the skills that are required to make positive long-term decisions and goals for their futures. Such conditions are likely to contribute to the cycle of underachievement, failure and exclusion from participation in productive and meaningful occupations that disadvantaged students face.

The benefits of explicit critical thinking instruction have been shown to have long-term positive impact across a variety of student populations, including those from disadvantaged backgrounds. Coca, Johnson, and Kelley-Kemple (2011), for example, provide longitudinal evidence of the impact of the IB DP on students of low socio-economic status (SES) in schools across the Chicago-metro area. As discussed, while the DP program does imbed critical thinking across multiple domains, the ToK subject is positioned as a central, cross-curricular and independent means to raise the academic performance of students through the specific teaching and learning of critical thinking skills. This study which compared the performance of DP graduates and non-DP graduates across a group of low-SES schools in the Chicago area revealed that: 1) DP graduates were 40 per cent more likely to complete a four-year college course and 50 per cent more likely to attend a more selective college and 2) DP graduates in four-year colleges were significantly more likely to persist in their tertiary education for two years or more (Coca et al., 2011). The two demographic groups with the strongest gains in academic performance as a result of their inclusion in the DP stream were African Americans and Latina/Latinos, who were prepared for the DP starting in year 9. Over the past three decades the connection between students’ epistemological approach and their intellectual development has been bolstered by research across age ranges, academic subjects, and in different cultures (see Gow & Kember, 1993; Samuelowicz & Bain, 1992 for excellent reviews of the literature on this topic).

At the time of writing, only Australian schools which offer the IB DP deliver the subject of ToK; contrariwise, the current state of critical thinking teaching and learning across the curriculum in non-IB schools across Australia is largely piecemeal and uncoordinated (Cole et al., 2014). Two states will be used here as examples, namely NSW and Victoria. In NSW, critical thinking is mentioned principally as one of a repertoire of skills that is developed through the selection by examination of students in terms of ‘opportunity classes’ (e.g. invitation-only classes in government schools for top-performing students in years 5 and 6 of primary school), ‘selective schools’ (e.g. invitation-only government secondary schools for top-performing students) and ‘selective streams’ in mainstream government secondary schools (e.g. invitation-only academic streams for top-performing students). This strategy has been justified by the NSW Department of Education and Community (DEC) with respect to the literature on ‘gifted and talented’ students, which has shown that these students need specific extension activities to keep them engaged in the learning process (New South Wales, Department of Education, 1991, 2001). As a result, the NSW strategy can have the unwanted consequence of reserving the provision of critical thinking education for students who have been categorised (via standardised testing) as being high achievers, contrary to evidence demonstrating the benefits of critical thinking for students from a range of academic levels (Coca et al., 2011). In Victoria, critical thinking has been bundled with creativity and meta-cognition as skills and dispositions that underpin the curriculum as an inter-disciplinary ‘thinking’ capacity – Critical and Creative Thinking (CCT) (Victorian Curriculum and Assessment Authority, 2013). One could argue that the lack of both a specific, well-designed and properly resourced critical thinking subject and targeted teacher training that aids the instruction of this course could make the provision of critical thinking in Victorian schools reliant on the inclinations of individual teachers and school leaders. This curricular structure persists despite the Victorian Department of Education and Training (DET) making online CCT-focused resources and strategies available to teachers as well as psychometrically testing 4,900 Victorian schoolchildren in 2014 for evidence of their creative and critical thinking skills and dispositions. Concurrently, CCT is being promoted as a cross-disciplinary focus to be embedded across the new Australian curriculum (Australian Curriculum, Assessment and Reporting Authority, 2012) as evidence of Australia’s ‘general capacity’ approach to critical thinking. However, teachers may well be confused about exactly how to implement this general capacity, particularly given a recent government report into the implementation of the Australian curriculum (Donnelly & Wiltshire, 2014) which sided with the domain-specific ‘side’ of the critical thinking debate as discussed above, effectively writing the CCT general capacity out of the Australian national curriculum.

While empirical evidence appears to support the short- and long-term benefits of an explicit, stand-alone approach to teaching critical thinking skills, no comparative data have been collected in the Australian context. Accordingly, the study reported in this article sought to investigate the influence of the IB DP ToK subject and its stand-alone approach to the delivery of critical thinking skills.

Furthermore, this study explored one of the key claims in a previous empirical investigation of the DP in the Australian and New Zealand contexts (Coates, Rosicka, & MacMahon-Ball, 2007) – namely that the DP, inclusive of its specific provision of critical thinking skills via the ToK course, provides an optimal preparation in the academic skills needed for university. Critical thinking skills appear to constitute the difference between the rote learning of content knowledge and being able to purposefully manipulate any new knowledge to a specified end (Coates et al., 2007; Paul et al., 1997).

Method

This study followed a three stage, convergent, mixed-methods approach (Creswell, 2012), with data collection occurring concurrently for each stage. This approach allowed for the triangulation of the information in order to understand better the link between students’ completion of the ToK subject and their critical thinking skill development. The quantitative component of the full study involved data collection across both ToK student and ToK teacher cohorts and explored a larger number of related variables (see Cole et al., 2014 for the full report). In this article, a snapshot of 1) ToK student data relevant to critical thinking skills acquisition and 2) comparative outcomes for a population of DP and non-DP first year university students is reported.

In the first stage of the study, the IBO sent all Australian DP school leadership staff an email invitation to participate in the national study, inclusive of text with the online survey URL to distribute to their population of ToK students in years 11 and 12. As students were aged 16 and older, parental consent was waived for the purposes of this study and survey completion was taken as evidence of informed consent, as participant information was located at the start of the online survey. The survey assessed critical thinking outcomes using both an original measure, developed for the purposes of this study, namely the Critical Thinking Strategies Scale (CTSS), as well as the five-item “Critical Thinking” subscale of the Motivated Strategies for Learning Questionnaire (MSLQ) developed by Pintrich, Smith, Garcia, and McKeachie (1993). The latter was used to enable additional comparison of critical thinking skills with a well-established, validated measure of high internal validity (Cronbach’s alpha = .80).

The 14-item CTSS was developed using evidence of critical thinking skills as outlined in the “Delphi Report” (Facione, 1990), widely regarded as the leading synthesis of expert consensus on measureable critical thinking skills. Students were asked “How likely is it that you will …” and exposed to a series of actions associated with critical thinking ability (e.g. “How likely is it that you will work to clarify meaning and define terms with which you are unfamiliar?”), with responses measured on a 7-point Likert scale ranging from “extremely likely” (7) to “not very likely” (1). Exploratory factor analysis (EFA) was used to reduce the number of items in the scale. This resulted in a 14-item measure which showed excellent internal reliability (Cronbach’s alpha = .96).

In stage two of the study, a convenience sample of four NSW schools offering the IB DP curriculum were chosen as sites for in-depth case study. Semi-structured interviews (see Appendix A for questions) were conducted with teachers, students (focus groups) and administrators, who were chosen for interview if available and at the discretion of the schools. The interviews took place in the schools, and asked how respondents viewed the subject of ToK, and where the subject of ToK was located in their learning programmes, both in terms of the value of ToK, and the how their programming was organised. In addition, ToK lesson observations (see Appendix A for observation focuses) were undertaken through which ToK lesson notes were taken, and document analysis of any documents produced by the schools with reference to ToK, for example, school reports, curriculum documents, student help guides. Data were synthesised in terms of agreed upon qualitative themes, that came about after rigorous discussion between the researchers involved in the project, and attempted to frame and develop ways in which ToK currently operated in the four schools. Researchers returned regularly to the schools over a four-month period, which provided an opportunity for follow up interviews, extra observational work and clarification with respect to the ways in which ToK provided a platform for CT skill development or otherwise (some researchers visited schools a maximum of 4 times).

Stage three involved data collection from a convenience sample of Australian first year university students, including those who had previously completed the ToK subject. DP graduates from the year prior to data collection were recruited through the Australian IBO Alumni contact list and approached via email by the IBO with the online survey URL. Non-DP graduates were solicited via targeted email/online announcements on their various units’ online content management systems (e.g. Blackboard). While a number of universities were targeted based on completion data for DP students in efforts to generate a strong comparison sample, only courses at the University of Sydney and the Queensland University of Technology assisted with recruitment efforts.

During stage three, DP and non-DP graduates in their first year of university students were compared using the 75-item California Critical Thinking Disposition Inventory (CCTDI; Insight Assessment, 2000), a well-regarded commercial measure of the students’ dispositional tendencies to deploy critical thinking as a ‘natural thinking habit’. This measure has been shown to have a very high internal consistency (Cronbach’s alpha = .90) with reliabilities for the seven subscales ranging from 0.67 to 0.82 (see Appendix B, for a complete list).

In addition, given the aforementioned links between critical thinking skill development and university success (Coates et al., 2007; Gardiner, 1995), as well as the ways in which enhanced critical thinking skills ease the cognitive load of myriad subject areas via a more discerning set of thinking skills and improved strategies for learning over time (Higgins et al., 2004), university students’ general academic outcomes were also measured in the study. To this end, the 10-item Academic Self-concept (General) Scale (Marsh, 1992) was used as a reliable, internally consistent (Cronbach’s alpha = 0.92) proxy measure of students’ actual academic outcomes (Byrne & Worth-Gavin, 1996). This proxy measure was deemed preferable to students’ reports of their actual academic achievement, given the challenges of standardising marks across multiple schooling institutions. Students were compared on two further measures. First, students were asked to report their Australian Tertiary Admissions Rank (ATAR) which is a standardised measure of their year 12 results. Second, a six-item measure of their anticipated university outcomes, specifically designed for this study was used to assess students’ self-beliefs regarding course completion, anticipated marks and anticipated unit failures (sample item: “I will finish my course without having failed a single unit”; Cronbach’s alpha = 0.78).

Sample

Table 1 shows the national sample of ToK student participants from stage one of the data collection alongside the total number of Australian schools offering the DP in each State and Territory. With the exception of the Northern Territory, all States and Territories of Australia were represented, representing approximately 37 per cent of the total number of Year 11 and 12 ToK students in Australia at the time of data collection, according to estimates provided by the IBO.

Table 1.

Number of schools in Australia which offer the IBO Diploma Programme (ToK) by state/territory, together with number of student questionnaires received.

	# DP schools	ToK students in sample (Yrs. 11 & 12)
	Frequency	Frequency	%	# of schools represented
QLD	10	423	31.6	8
NSW	16	363	27.1	14
VIC	14	263	19.7	8
SA	9	208	15.5	7
WA	5	36	2.7	4
ACT	5	34	2.5	5
TAS	1	11	.8	1
NT	1	0	0.0	0
Total	N = 61	N = 1,338	100.0	N = 47

In stage two, members of the research team performed: 1) 22 teacher interviews; 2) eight focus groups with current ToK students; 3) five interviews with administration or other school personnel; 4) a document analysis of relevant ToK literature produced by the schools, and 5) 10 observations of ToK lessons. Interviews with each teacher lasted for approximately 40 minutes while focus groups took approximately half an hour each and consisted of between four and seven students. The final sample of stage three participants included 389 first year students, of whom 99 (=25% of the total sample) were DP graduates.

Results and discussion

Stage one was designed to allow for a cross-sectional comparison of group means between the cohort of Year 11 and Year 12 students of ToK as a proxy measure of growth in critical thinking skills across the two years of study. An examination of student means on the CTSS measure showed that students in year 12 reported a statistically significantly higher mean score for both the full scale measure, as well as for 11 of the 14 individual items (see Table 2), indicating a higher reported likelihood of employing these critical thinking skills.

Table 2.

Mean differences in CTSS individual item and full scale scores, by ToK year cohort.

CTSS measure: Commenced with prompt, “How likely is it that you will …?”	Year of school	N	Mean (SD)	Comparison of means (Yr. 11–12)
Work to clarify meaning and define terms with which you are unfamiliar? (skill 1)	Year 11	630	4.72 (1.45)	t(1260) = 3.38*
	Year 12	632	4.99 (1.39)	p < 0.01
Work to understand the significance of the new material to see how it fits with your prior knowledge or experience? (skill 2)	Year 11	630	4.60 (1.36)	t(1260) = 2.05*
	Year 12	632	4.76 (1.35)	p < 0.05
Express the new question in several ways to clarify its meaning and scope? (skill 3)	Year 11	630	4.23 (1.41)	t(1260) = 3.68*
	Year 12	632	4.53 (1.46)	p < 0.001
Break the question into sub-questions? (skill 4)	Year 11	630	4.36 (1.49)	t(1260) = 2.44*
Break the question into sub-questions? (skill 4)	Year 12	632	4.57 (1.52)	p < 0.05
Seek to identify arguments being made? (skill 5)	Year 11	630	4.67 (1.39)	t(1260) = 3.94*
Seek to identify arguments being made? (skill 5)	Year 12	632	4.97 (1.37)	p < 0.001
Work to analyse arguments being made? (skill 6)	Year 11	630	4.72 (1.39)	t(1260) = 4.04*
Work to analyse arguments being made? (skill 6)	Year 12	632	5.03 (1.36)	p < 0.001
Seek to assess claims being made (rather than passively accepting those claims)? (skill 7)	Year 11	630	4.73 (1.48)	t(1260) = 4.70*
	Year 12	632	5.12 (1.48)	p < 0.001
Imagine alternative ways to solve the same problem? (skill 8)	Year 11	630	4.74 (1.46)	t(1260) = 3.57*
	Year 12	632	5.03 (1.39)	p < 0.001
Draw conclusions about a problem based on the evidence at hand? (skill 9)	Year 11	630	4.90 (1.41)	t(1260) = 1.61
	Year 12	632	5.00 (1.46)	p = 0.25 (NS)
Restrict your claims to those supported by the evidence you have? (skill 10)	Year 11	630	4.29 (1.44)	t(1260) = 0.07
	Year 12	632	4.28 (1.48)	p = 0.95 (NS)
Search for information that opposes your position as well as information that supports it? (skill 11)	Year 11	630	4.66 (1.50)	t(1260) = 2.37*
	Year 12	632	4.86 (1.49)	p < 0.05
Consider how your assumptions are shaping your point of view? (skill 12)	Year 11	630	4.67 (1.45)	t(1260) = 2.43*
	Year 12	632	4.87 (1.48)	p < 0.05
Be able to justify the strategies that you used to solve a problem or create an argument? (skill 13)	Year 11	630	4.61 (1.41)	t(1260) = 2.04*
	Year 12	632	4.78 (1.43)	p < 0.05
Be able to present your argument to others in a way that they will understand? (skill 14)	Year 11	630	4.84 (1.46)	t(1260) = 1.84
	Year 12	632	4.99 (1.42)	p = 0.07 (NS)
Total score on the 14-item CTSS	Year 11	620	4.62 (1.08)	t(1240) = 3.04*
Total score on the 14-item CTSS	Year 12	622	4.85 (1.05)	p < 0.001

NS: non-significant.

Note: Missing cases eliminated listwise (n = 1262 for items; n = 1242 for total); *Indicates statistical significance. Items measured on a 7-point scale, with higher scores indicating greater use.

These differences were further confirmed by students’ outcomes on the five-item Critical Thinking subscale of the MSLQ. The mean scale score for the Year 12 cohort (M = 4.69, SD = 1.15) was higher than for their Year 11 counterparts (M = 4.48, SD = 1.22), with students’ year level shown to have a statistically significant effect (t(1240) = 3.04, p < 0.01). While both cohorts of students’ scores fell between the anchor point of “Moderately true of me” (4) and the high end of the scale (“Very true of me” [7]), the Year 12 cohort mean was moderately closer to the higher end of the measurement scale.

Taken together, these two quantitative outcomes point to greater awareness and use of critical thinking skills as students move from their first to their second year of the ToK subject. Analysis of the four case study schools aids in the interpretation of these findings.

The case studies focused on how the different schools conceptualised and operationalised their critical thinking skills improvement programmes through the ToK subject. While each of the four schools framed students’ critical thinking skill development as central to their school’s ‘mission(s)’, analysis of the case study data highlighted differences in the schools’ approaches to meeting this goal.

While such data do not allow for generalisations, the cases here highlighted links between the selected schools’ culture and openness (or otherwise) to critical thinking, its definition and framing of critical thinking as well as delivery of explicit critical thinking instruction (Table 3). The case studies illustrated how the history and makeup of a school, including the experience and mix of the teacher community, school policy and ethos, and the intellectual heritage that the school calls upon, can all influence the specific teaching and learning of critical thinking in the classroom. Table 3 shows how ‘School 1’ has taken ToK as the centrepiece of the DP’s critical thinking learning programme and has, therefore, integrated the learning strategies used in ToK across the whole school; in contrast, ‘School 2’ has taken the domain-specific approach to critical thinking improvement. ‘Schools 3’ & ‘4’ have evolved their own approaches to critical thinking instruction: ‘School 3’ based on Socratic dialogue and logic, ‘School 4’ on life-world skills (e.g. everyday questions of logic and problem solving) and issue discussion.

Table 3.

Summary of the case study results for critical thinking (CT) skills.

School	Means to deliver critical thinking skills	Links between ToK & critical thinking	Extent to which ToK delivers critical thinking skills
1	ToK links; inter-disciplinary work; inquiry-based learning; questioning of knowledge assumptions; setting up challenges to conventional knowledge; research-based activities; resources that spark inquiry around knowledge issues; ways of knowing & areas of knowledge	Based around published ToK curriculum and IBO documentation, e.g. analysis of the areas of knowledge and ways of knowing.	The practice of ToK has been aligned with CT in school 1. This is helped by a whole school approach to IBO concepts and curriculum, which are shared, for example, in devotional time.
2	Discussion work; open debates around knowledge issues (for and against); thematic work around knowledge issues; ToK essay writing practice; knowledge as a cultural practice; scientific examination of knowledge	Other specialist subject areas heavily influenced the teaching of CT through ToK, e.g. the examination of evidence in history or the study of rhetoric in English.	CT skills are mainly delivered through the connection between ToK and other subjects such as history, English and science.
3	Logic; philosophical debate; detailed specialist knowledge about CT; analysis of philosophical arguments (i.e. premise/conclusions); teaching of philosophical methods, e.g. Socratic dialogue; ToK journal writing	CT linked primarily to philosophy and logical argument. ToK therefore is conceived as a means to improve CT through close examination of how to argue logically.	The focus on CT skills as a traditional aspect of philosophy in school 3 means that the subject of ToK has been moulded in this light. The subject of ToK is a subset of philosophical studies.
4	‘Lifeworld’ discussions; ethics; specific linguistic preparation for ToK assessments; publication of ToK work; introduction to university-style CT skills; links to Community–Action–Service (CAS) issues/debates; teaching thinking routines	ToK has been embedded within a whole-school approach to CT. ToK sessions are therefore an opportunity to focus and improve CT skills learnt elsewhere and to link to subjects such as psychology.	The delivery of CT skills was a central part of the school’s mission to encourage critical thinking & independent girls. The ToK course was an aspect of this bigger picture school ethos.

Results from phrase three of the data collection allowed for an exploration of the longer-term differences in students who had completed the ToK curriculum and those who had not. A comparison of mean results on the CCTDI revealed higher scores¹ for the former ToK students across all seven categories of the inventory (Table 4).

Table 4.

Mean differences on the CCTDI for DP and non-DP graduates.

	Former ToK-DP student?	Mean (SD)
Truth-seeking	Yes	37.02 (5.88)
Truth-seeking	No	35.00 (6.33)
Open-mindedness	Yes	43.48 (5.41)
Open-mindedness	No	42.85 (5.42)
Inquisitiveness	Yes	48.17 (6.45)
Inquisitiveness	No	46.96 (6.35)
Analyticity	Yes	44.95 (5.39)
Analyticity	No	44.75 (5.02)
Systematicity	Yes	39.74 (6.59)
Systematicity	No	38.91 (6.78)
Confidence in reasoning	Yes	45.93 (5.96)
Confidence in reasoning	No	43.75 (6.12)
Maturity of judgement	Yes	41.98 (7.06)
Maturity of judgement	No	41.90 (6.35)
CCTDI total score	Yes	300.95 (30.05)
CCTDI total score	No	293.98 (27.75)

Note: Former ToK-DP students (n = 42); non-ToK-DP students (n = 126); this drop in participants was likely due to the secondary online location of the commercial CCTDI, access to which required login details, which may have resulted in participant fatigue/disinterest. Additional details are available in the full report (Cole et al., 2014).

Table 5 shows the mean differences between DP and non-DP graduates’ academic outcome measures, including their reported ATAR scores, their academic self-concept and their perceived university outcomes. DP graduates had statistically significantly higher mean scores across all three of these predictors of postsecondary school success.

Table 5.

Mean differences in ATAR, ASC & perceived university outcomes, by cohort.

Measure	Secondary school	N	Mean (SD)	Comparison of means
ATAR score	DP graduate	94	94.13 (6.79)	t(290) = 4.76*
ATAR score	Non-DP graduate	267	89.29 (12.07)	p < 0.001
General academic self-concept (10-item scale; Marsh, 1992)	DP graduate	99	6.25 (0.96)	t(380) = 4.24*
General academic self-concept (10-item scale; Marsh, 1992)	Non-DP graduate	283	5.75 (1.02)	p < 0.001
Anticipated university outcomes (6-item scale)	DP graduate	99	3.43 (0.43)	t(380) = 2.25*
Anticipated university outcomes (6-item scale)	Non-DP graduate	283	3.32 (0.47)	p < 0.001

Indicates statistical significance.

These findings point to the tangible impact of completion of the DP curriculum, inclusive of the ToK subject, on students’ university readiness and access to competitive courses, as well as their own personal perceptions of their academic ability and their future university successes. Results suggest a link between the DP focus on students’ development of critical thinking skills and increases in both habits of critical thinking and representative measures of academic success.

Limitations and implications of the study

ToK requires DP students to think deeply about the knowledge and skills that they are learning across the curriculum and asks that they formulate answers to questions with respect to the nature of knowledge. Given the body of empirical research supporting the benefits of explicit, stand-alone critical thinking education (Abrami et al., 2008; Marin & Halpern, 2011), this study sought to explore these benefits in an Australian context via the IBs’ ToK subject.

While the findings reported in this article do appear to support explicit delivery of critical thinking education via ToK, it should be noted that students who have completed the ToK course have also completed the full DP course, of which ToK is only one component. Given our cross-sectional research design, it was not possible to separate the effects of ToK completion from the effects of completing the DP curriculum as a whole.

Still, these findings provide an interesting and extended picture of not only differential critical thinking outcomes for DP graduates, of which ToK is a defining feature, but also improved academic outcomes which are potentially linked to the development of critical thinking skills.

Results, therefore, highlight the need for longitudinal research with DP students before, during and post-ToK completion in efforts to parcel out the predictive impact of ToK on students’ critical thinking, short- and longer-term academic outcomes. Furthermore, small tertiary student sample sizes and a convenience, rather than a representative, statistically matched, sample of first-year university students mean that the tertiary student findings provide tentative, rather than unequivocal, evidence of impact.

Case study findings point to the need for greater clarification regarding the precise school climate and framing of ToK to allow for enhanced critical thinking skills, particularly around the ‘translation’ from epistemology as an overarching concept to the exploration and practical application of new and varied ways of thinking and knowing. There is much to learn about the best pedagogical methods for ToK teachers to employ in order to enable this translation and the optimal school conditions to support this work. To this end, case study findings highlight differences between schools in terms of ToK subject interpretation and operationalisation. For some schools, these interpretations were not always helpful, nor did they result in purposeful and explicit teaching of critical thinking. This was reflected in School 2, where student focus groups, teachers’ reports and lesson observations revealed an over-reliance on whole-class discussion work. Additional case study explorations could help to elucidate the mechanisms by which teachers of ToK interpret the IB’s official ToK subject outcomes in order to design and resource their lessons.

The present curricula position of ToK at the inter-disciplinary centre of the DP makes it an invaluable arena for critical thought and debate, and a possible ‘spark’ for students to go further in their own thinking, research and inquiry. Case study findings revealed that while students may struggle initially in their ToK classes (which we called ‘ToK-shock’), this is a subject that many will come back to, and will positively appreciate in the future, as they learn to think critically in an adult and mature manner, to question assumptions, and to more fully articulate their own knowledge and thinking beliefs. Further, findings from the case study element of this research supported the idea that key to ToK’s success (e.g. enhancement of students’ critical thinking skills) lies in the fact that the ToK subject is not subsidiary to any other subject, as it does not have learnt curriculum knowledge tied to its functioning (see, Cole & Throssell, 2008); instead, ToK allows students to fully use and explore their knowledge through structured thinking exercises and cross-curricula activities. Indeed, one of the case study schools had copied (but rebadged for propriety reasons) the ToK course for its non-IB students, and runs it as a compulsory aspect of its pre-tertiary education for all students. The local enthusiasm for exploring knowledge claims fully and the explicit modes of questioning that ToK can generate in communities are vital aspects of the classroom experience of ToK, and do contribute to the continued smooth functioning of ToK in DP schools as a ‘thinking mechanism’ (see, Cole, 2005) and a form of critical thinking practice with wide-ranging benefit.

Findings presented here add to the evidence and rationale for teaching critical thinking as an explicit subject. However, the implementation of a similar subject across the Australian national curriculum would require wholesale education reform and present great challenges for an already crowded curriculum landscape. Coupled with the previous empirical data on the benefits of such curricular inclusions, the current findings suggest benefits if schools in Australia considered to initiate pre-tertiary, ‘stand-alone’ critical thinking courses to enhance students’ academic success rather than relying on the implicit teaching of critical thinking through the key learning areas of the curriculum. It is acknowledged, however, that this is not a simple proposition, but rather one with significant implications for a number of areas, including pre-service teacher training, school staffing, curriculum organization and content as well as timetabling.

Footnotes

Authors’ note

The full report can be found at the IBO research web site: http://ibo.org/globalassets/publications/ib-research/dp/tokfinalreport_000.pdf. A summary of the report can be found here:

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: the International Baccalaureate Organization (IBO).

Notes

Appendix A

Appendix B

References

Abrami

P. C.

Bernard

R. M.

Borokhovski

Wade

Surkes

M. A.

Tamim

Zhang

(2008) Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research 78(4): 1102–1134.

Aizikovitsh

Amit

(2010) Evaluating an infusion approach to the teaching of critical thinking skills through mathematics. Procedia – Social and Behavioral Sciences 2(2): 3818–3822.

Aliakbari

Sadeghdaghighi

(2013) Teachers’ perception of the barriers to critical thinking. Procedia – Social and Behavioral Sciences 70(0): 1–5.

Australian Curriculum, Assessment and Reporting Authority (2012) General capabilities in the Australian curriculum. Retrieved from: http://www.acara.edu.au (Critical and creative thinking): pp. 53–63.

Bailin

(2002) Critical thinking and science education. Science & Education 11(4): 361–375.

Bruer

J. T.

(2000) Schools for thought: A science of learning in the classroom, Cambridge, MA: MIT Press.

Byrne

Worth-Gavin

(1996) The Shavelson model revisited: Testing for the structure of academic self-concept across pre-, early, and late adolescents. Journal of Educational Psychology 88(2): 215–228.

Coates

Rosicka

MacMahon-Ball

(2007) Perceptions of the International Baccalaureate Diploma Programme among Australian and New Zealand Universities, Canberra: Australian Council for Educational Research.

Coca

Johnson

Kelley-Kemple

(2011) Working to my potential: The postsecondary experiences of CPS students in the International Baccalaureate Diploma Programme, Chicago: University of Chicago, Consortium on Chicago School Research.

10.

Cole

D. R.

(2005) An examination of Hegelian and Spinozian Philosophy and their relationships with the International Baccalaureate subject, Theory of Knowledge. Journal of Research in International Education 4(2): 211–226.

11.

Cole

D. R.

Gannon

Ullman

Rooney

(2014) Report for IBO: IB Programme: Theory of Knowledge (TOK): Exploring learning outcomes, benefits, and perceptions, Washington: International Baccalaureate Organization. Singapore, pp. 1–115.

12.

Cole

D. R.

Hager

(2010) Learning-practice. The ghost in the education machine. Education Inquiry 1(1): 21–40.

13.

Cole

D. R.

Throssell

(2008) Epiphanies in action: Teaching and learning in synchronous harmony. The International Journal of Learning 15(7): 175–184.

14.

Creswell

J. W.

(2012) Educational research: Planning, conducting, and evaluating quantitative and qualitative research, (4th ed). Upper Saddle River, NJ: Pearson Education.

15.

Creswell

J. W.

Plano-Clark

V. L.

(2011) Designing and conducting mixed methods research, (2nd ed). Thousand Oaks, CA: Sage Publications.

16.

Donnelly

Wiltshire

(2014) Review of the Australian curriculum, Canberra: Australian Government Department of Education.

17.

Dweck

C. S.

(2002) Beliefs that make smart people dumb. In: Sternberg

R. J.

(ed.) Why smart people can be so stupid, New Haven, CT: Yale University Press.

18.

Facione

P. A.

(1990) The Delphi Report: Executive summary, Fullerton: California University Press.

19.

Flavell

J. H.

(1977) Cognitive development, Englewood Cliffs, NJ: Prentice-Hall.

20.

Gardiner

(1995) Redesigning higher education: producing dramatic gains in student learning. ASHE-ERIC Higher Education Report, Vol. 23, No. 7. The George Washington University, Graduate School of Education and Human Development, Washington, DC.

21.

Gow

Kember

(1993) Conceptions of teaching and their relation to student learning. British Journal of Educational Psychology 31: 93–97.

22.

Halpern

D. F.

(2003) Thought and knowledge: An introduction to critical thinking (4th ed.), Mahwah, NJ: Lawrence Erlbaum Associates.

23.

Hatcher

(2006) Stand-alone vs. integrated critical thinking courses. The Journal of General Education 55(3 & 4): 247–272.

24.

Higgins

Baumfield

Lin

Moseley

Butterworth

Downey

Thacker

(2004) Thinking skills approaches to effective teaching and learning: What is the evidence for impact on learners? In: Research evidence in education library, London: EPPI-Centre, Social Science Research Unit, Institute of Education.

25.

Insight Assessment. (2000). California critical thinking dispositional inventory web site. Retrieved from: http://www.insightassessment.com/Products/Critical-Thinking-Attributes-Tests/California-Critical-Thinking-Disposition-Inventory-CCTDI.

26.

International Baccalaureate Organization (2014) International Baccalaureate Diploma Programme Subject Brief Diploma Programme core: Theory of Knowledge First assessments 2015 – Last assessments 2021, Cardiff, UK: International Baccalaureate Organization.

27.

Kuhn

(2000) Metacognitive development. Current Directions in Psychological Science 9: 178–181.

28.

McGuire

(2012) Teach students how to learn: Metacognition is the key, Louisiana State University, Center for Academic SuccessRetrieved from: https://www.alabamacca.org/pdfs/2-ACCA-Birmingham-2012-Metacognition.pdf.

29.

Marin

L. M.

Halpern

D. F.

(2011) Pedagogy for developing critical thinking in adolescents: Explicit instruction produces greatest gains. Thinking Skills and Creativity 6(1): 1–13. DOI: http://dx.doi.org/10.1016/j.tsc.2010.08.002.

30.

Marsh

H. W.

(1992) Self-Description Questionnaire (SDQ) III: A theoretical and empirical basis for the measurement of multiple dimensions of late adolescent self-concept. An interim test manual and research monograph, Macarthur, Australia: University of Western Sydney.

31.

New South Wales, Department of Education (NSW DOE) (1991) Implementation plan for the education of gifted and talented students, Sydney: NSW Government.

32.

New South Wales, Department of Education (NSW DOE) (2001) Gifted and talented education programs and initiatives of the NSW Department of Education and Training, Sydney: NSW Government.

33.

Paul

R. W.

(1992) Critical thinking: What, why, and how? New Directions for Community Colleges 77: 3–24.

34.

Paul

Elder

Bartell

(1997) California teacher preparation for instruction in critical thinking: Research findings and policy recommendations, State of California: California Commission on Teacher Credentialing.

35.

Pintrich

(2002) The role of metacognitive knowledge in learning, teaching and assessing. Theory into Practice 41(4): 219–225.

36.

Pintrich

Smith

Garcia

Mckeachie

(1993) Reliability and predictive ability of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement 53: 801–813.

37.

Samuelowicz

Bain

(1992) Conceptions of teaching held by academic teachers. Higher Education 24: 93–112.

38.

Stanton-Salazar

R. D.

(1997) A social capital framework for understanding the socialization of racial minority children and youths. Harvard Educational Review 67(1): 1–39.

39.

Swartz

(1992) Critical thinking, the curriculum, and the problem of transfer. In: Perkins

Bishop

Lochhead

(eds) Thinking: The second international conference, Hillsdale, NJ: Erlbaum, pp. 261–284.

40.

Thomas

(1999) Critical thinking instruction in selected greater Los Angeles High Schools. Unpublished dissertation, Azusa Pacific University.

41.

Victorian Curriculum and Assessment Authority (VCAA) (2013) The AusVels curriculum, Victoria, Australia: Victoria Department of Education.

42.

Watt, M. G. (2006). Looking back at curriculum change in Tasmania: Is the essential learnings framework promoting successful curriculum reform? In M. G. Watt (ed.), Proceedings of the 2005 National Biennial Conference of the Australian Curriculum Studies Association. ACSA Biennial Conference, Sunshine Coast.

43.

Willingham

D. T.

(2007) Critical thinking: Why is it so hard to teach? American Educator. 8–19.

44.

Zohar

Dori

(2003) Higher order thinking skills and low-achieving students: Are they mutually exclusive? Journal of the Learning Sciences 12(2): 145–182.