Sage Journals: Discover world-class research

Abstract

Civics courses strive to promote students’ political competencies, which according to the model of Detjen et al. incorporate content knowledge, abilities to make political judgements and take political action, as well as motivational skills and attitudes. For achieving these goals, high hopes are placed on active learning tools such as political simulations. Looking at positive expectations generally placed on simulation games, they seem very suitable for addressing key problems of teaching about European Union identified earlier. However, simulation games have also been regarded as demanding and time-consuming, with the ‘fun-factor’ outweighing the ‘learning factor’. There is a profound lack of systematically won empirical evidence for such positive and negative expectations. This study addresses this research deficit, focusing on the effects of short European Union simulation games of the decision-making of the European Parliament on pupils and their political competence controlling for different background variables (e.g. gender, cultural capital, pre-knowledge, political interest). Data were collected from 2013 to 2016 in secondary schools of Lower Saxony and North Rhine-Westphalia (N = 308; average age = 16.75 years) using partly standardised questionnaires accompanied by interviews. The intervention study (pre–post design) focuses on the competency dimensions content knowledge, motivation (political interest, self-efficacy), volition (willingness to participate politically) and attitudes (e.g., perceived responsiveness of European Union and perceived relevance of European Union for everyday life).

Keywords

civic education simulation game political competence European Union intervention study

Theoretical background

Simulation Games have become an established teaching method in civic education. Their use is not only part of seminars or visits to parliament in extracurricular citizenship education but has also found its way into school civics lessons in Germany. Surveys show that while they are not employed universally, they are certainly no longer seen as an exotic novelty.¹ In civic education, such games usually simulate a political decision-making process and involve clearly opposed political interests as well as a certain time pressure, reducing reality to a simplified model so that participants may experiment in a safe environment (cf. Klippert, 1984; Massing, 1997).

This action-, experience- and process-oriented didactic method entails several positive expectations (cf. Guasti et al., 2015; Hartmann and Weber, 2013; Klippert, 1984; Massing, 1997; Raiser et al., 2015; Rappenglück, 2004): simulation games are intended to help reduce the complexity of politics – especially regarding political processes – in order to render them easier to understand and to provide insights into the difficulties of reaching political compromises. The aim is to link the political to learners’ everyday reality and to disseminate knowledge in an experience-based and hence sustainable fashion. Ultimately, it is assumed that, thanks to the particular self-dynamism and the self-activity of the players, simulation games motivate learners to engage with the subject matter and awaken respectively (resp.) deepen their interest in politics. These (anticipated) advantages of the method are particularly relevant regarding the specific problems of teaching on Europe in civics lessons (see below).

Today, the European Union (EU) is a compulsory element of secondary-level civics lessons in general schools in all of Germany’s federal states (see Von Geyr et al., 2007). This is to be welcomed in view of the relevance of EU-related competencies for (aspiring) citizens: if at least 30% of all laws made at the federal level are instigated by a ‘European impulse’ (Töller, 2008), then the EU level of the European multi-level system must be taken into consideration by any attempt to analyse, evaluate or influence politics in Germany (cf. Oberle, 2012). Current social problems such as migration and climate change highlight the significance of political decision-making by the EU.

The central aim of civic education in schools is to foster pupils’ political competence. According to the model developed by Detjen et al. (2012), this entails subject knowledge, the ability to make political judgements, the ability to take political action, as well as motivations and attitudes regarding politics. This model of political competence is based on a domain-specific but relatively broad pedagogical-psychological concept of competence, embracing not only cognitive but also affective, motivational and volitional dispositions (although Detjen et al. give scant consideration to the volitional aspect).²

Motivational orientations include, above all, political interest and perceived political competence resp. self-efficacy. One can distinguish between internal and external efficacy (Vetter, 1997; see also Oberle, 2012: 65–67): While internal political efficacy refers to perceptions of one’s own politics-related skills, external political efficacy is one’s subjective perception of the system’s responsiveness to citizens’ interests, that is, one’s assessment of whether ‘the political system is open to and reacts to the influence of its citizens’ (Vetter and Maier, 2005: 57). Perceptions of responsiveness are closely related to trust in the political system or diffuse political support for it (cf. Massing, 2012: 28). As an aspect of self-efficacy, perceptions of political responsiveness can be categorised, on one hand, as political motivations; on the other hand, they can also be seen as political attitudes in so far as they are an element of political support.

Of course, as elements of political competence, perceptions of responsiveness and diffuse political support depend to a large extent on the trustworthiness of a political system, its institutions and actors. At the same time, it must be emphasised that without a modicum of citizens’ trust in their political institutions, representative democracy cannot survive (cf. Fuchs et al., 2002). It is unclear, however, just how much trust is required to maintain a democratic political system (Schöne, 2017). Wilhelm Knelangen (2015) discusses this question in the context of the EU on the basis of data provided by the Eurobarometer: writing before the British referendum voted in favour of Brexit, he identifies a crisis of trust among EU citizens that could pose a serious threat to the existence of the EU.

Peter Massing (2012) claims that pupils’ perception of responsiveness and trust in their political system are not directly fostered by civics lessons, rather they are influenced indirectly: they are ‘at best a by-product of school as a social institution and the design of civics lessons’ (p. 29). Promoting attitudes in civics lessons is problematic due to the widely recognised Beutelsbach Consensus (see Oberle, 2013, 2016; Wehling, 1977), which prohibits overwhelming the pupil and requires that controversial topics are treated as controversial in civic education. However, here, it must be taken into consideration that certain values are intrinsic to these principles. Conveying ‘democratic virtues’ and values corresponding to free democracies remains a balancing act, and the influence of empathic ethics of conviction is often quite rightly criticised. Students must also become familiar with critique of the European unification process as well as alternative development scenarios and institutional designs. However, the openness towards Europe set out in the Basic Law (Art. 23 of the Basic Law, see above) justifies a fundamentally positive evaluation of the European integration efforts in civic education. Following Easton (1965), it makes sense to differentiate between general and performance-based attitudes, that is, between a ‘hard’ and a ‘soft’ Euroscepticism (see Knelangen, 2011; Weßels, 2009). While a ‘fundamental’ rejection of the EU is not part of the goal of today’s school education in Germany, a ‘constructive’ scepticism towards the EU certainly corresponds to the goals of EU civic education (cf. Oberle, 2015).

Drawing on studies by Patzelt (1998, 2003) among others, Helmar Schöne (2016) assumes that certain misconceptions contribute to a loss of citizens’ trust in their political institutions, especially party-state institutions (parliament and government). Using group discussions, he demonstrates that it is quite common even for students in teacher training specialising in politics to be fed up with political processes (prozessverdrossen; cf. Schöne, 2011). He identifies a lack of ‘appreciation for the interplay between political dispute and the search for compromise’ (Schöne, 2017). In order to promote understanding of how politics is done, Schöne draws on micro-politics in recommending that civic education should focus more on political actors’ perspectives as well as processes of political conflict and consensus. For this, he considers simulation games and field trips to be promising didactic approaches.

Willingness to participate in politics (volition, cf. Weinert, 2001: 27–28) can also be seen as a component of political competence, while this is controversial in the scientific discourse (cf., for example, the debate on different Bürgerleitbilder (= citizen models as a goal of civic education); Detjen, 2013: 215 ff.; Massing, 2004). Obviously, pupils’ willingness to engage politically and their political attitudes cannot be graded by schools. However, they can certainly be one of the aims of civic education (also in schools) and can be defined as a desirable outcome, at least in a form that is legal and fundamentally supports the basic free democratic order or in the form of civil disobedience as an expression of defensive democracy. It thus makes sense to consider learners’ political volition when examining the effects of civic education.

Ultimately, knowledge represents a central component of competence. According to Klieme (2004), competence entails ‘knowledge and skills’ (p. 13). Franz E. Weinert (1999) introduces various concepts of competence, his preferred concept being one that explicitly entails knowledge alongside skills, a ‘knowledge-centered concept of competence’ (p. 28). Weinert (1999) argues that

key competencies, however, are no substitute for content-specific knowledge, especially when solving more difficult problems. Even the hope that one could use a few key competencies to quickly acquire content-specific knowledge through electronic media is a purely utopian view. Only those with enough initial knowledge are able to use new knowledge effectively.

Here too, Weinert emphasises the significance of motivational orientations: ‘all measures of competence should also include measures of competence-specific motivational aspects’ (p. 29). Subject-specific competencies can be neither acquired nor used without content knowledge (cf. also Weinert, 2000). The systematic dissemination of basic conceptual knowledge, along with the continual promotion of other competence facets, thus does not undermine the paradigm of competence orientation, but is inherent to it (cf. Oberle and Tatje, 2014: 63).

Clearly, establishing the EU as part of the core curriculum does not necessarily guarantee its discussion in class, let alone effective promotion of competence. Indeed, civic education on the EU faces particular challenges. As early as 2004, Detjen identified key problems of teaching about the EU, foremost among them the complexity of the multi-level political system. In addition to the complexity and dynamics of European integration, further challenges are presented by a perceived distance between the EU and its citizens as well as insufficient relevance to their everyday life, a lack of interest in the EU, a lack of prior knowledge and prejudices on the part of learners. A survey of 179 teachers confirms this theory (Oberle and Forstmann, 2015b: 201): the EU’s complexity is perceived as the greatest barrier to teaching on it. This is a serious issue, even if some might say that local politics is also highly complex and is rarely dealt with in its entirety in the classroom. Yet, the issue of local politics clearly seems far less cognitively taxing for (aspiring) citizens than the EU, which the teachers themselves perceive to be hyper-complex.

Considering their aforementioned advantages, simulation games seem to be particularly well-suited to addressing these problems of promoting pupils’ EU-related skills in civics lessons (see also Usherwood, 2014). Their approach to teaching and learning also corresponds well to the negotiation system of the EU (cf. Brunazzo and Settembri, 2012). Admittedly, there are plenty of teachers (and it is not just teachers) who have their reservations about implementing political simulation games, particularly in the classroom context, above all due to the large amount of time they take up and a perceived lack of compatibility with the schedule of regular school lessons. There are also fears that such games do not allow a serious enough examination of the subject matter and hence lead to a low learning effect, and that they distort reality too strongly resp. create an illusion of reality. Furthermore, simulation games are considered to require a great deal of preparation. Additionally, the self-dynamics of the game situation can put some teachers off since there is a limit to the extent the envisaged learning processes can be directed.

There are, however, surprisingly few empirical findings arrived at by systematic research regarding the perceived potential and risks of the simulation game method (cf. Gosen and Washbush, 2004). As Schnurr et al. (2014) state, ‘much of the existing evidence assessing the learning impacts of simulations is anecdotal and, by and large, unsystematic’ (p. 402). Occasionally, studies have conducted qualitative interviews or reflective essays seeking the opinions of pupils, students and teachers (see, for example, Dierßen and Rappenglück, 2015; Jozwiak, 2013; Zeff, 2003). Quantitative studies on the effects and conditions of political simulation games (especially, but not only, in schools) supporting generalisations are rare (on the development of interests, albeit for the natural sciences, cf. Knogler and Lewalter, 2014; regarding University students, cf. Jones and Bursens, 2015; Rünz, 2015; for conditions influencing the substantive negotiation outcomes, see Fink, 2015; several interesting quantitative studies on simulation games’ effects on students’ political knowledge or attitudes use very small-n analyses, for example, Krain and Lantis, 2006). This is even more the case for teaching on the EU in school civics lessons. The extent to which relatively short simulation games that can be used in the classroom are able to foster pupils’ EU-related competencies remains unclear and forms the focus of this study.

Research aims and questions

The intervention study seeks to systematically examine the effects of short EU simulation games on learners’ EU-related skills. The basis is formed by a broad definition of competence that includes not only cognitive but also affective, evaluative and motivational dispositions (see above). The study’s focus lies on subject knowledge and political attitudes, motivations and volitions (cf. the model of political competence in Detjen et al., 2012). It analyses, on one hand, the transformations in the constructs captured both before and after the simulation games were conducted and, on the other hand, pupils’ subjective assessment of its quality and impact. Data were collected both before and after the games concerning learners’ political knowledge of the EU, various aspects of attitudes towards the EU and interest in the EU, EU-related internal political efficacy and pupils’ willingness to participate in EU-related politics. This includes analysis of the influence of socio-demographic background variables (gender, age, type of school) and general political interest on the impact of simulation games (a) as measured in pre-post comparison and (b) as perceived by the pupils. The following questions can be posed: (how) does participation in a short simulation game affect the pupils’ knowledge about the EU, attitudes towards the EU, interest in the EU, EU-related internal political efficacy and willingness to participate in politics? How do the pupils themselves rate the simulation games and their effects? Do the effects of simulation games that have been ascertained display systematic differences for different groups of pupils (e.g. in terms of age, prior knowledge, political interest)? The study’s findings are intended to contribute to a more appropriate assessment of the didactic value of these games. The potential of the research methodology is also examined in order to further develop the use of (EU) simulation games in the classroom on the basis of empirical evidence.

Study design

The short EU simulation game

The study examined short EU simulation games developed by planpolitik³ and conducted by their staff at participating schools. The games simulate decision-making by the European Parliament as part of the EU’s co-decision procedure on (a) asylum policy, (b) data protection or (c) CO₂ regulations (in relation to passenger cars). It is assumed that the relevance of these issues for pupils’ lives is relatively accessible for today’s adolescents. In this simulation game, pupils do not take on the roles of certain personalities but are prescribed those of political actors (e.g. member of a committee allied to a political fraction represented in parliament and of a particular nationality). Each game consists of different phases such as parliamentary meetings, negotiations in specialist committees and plenary sessions with a final vote. The games last approximately 3 hours, including an introduction regarding content and organisation and debriefing. The simulations are kept short in recognition of the fact that in regular school lessons there is rarely time for more extensive simulation games.

Sample and data collection

Data were collected from 2013 to 2016 in pre–post design using partly standardised questionnaires. The survey examined 13 short EU simulation games in a total of 10 schools. The sample comprises 308 pupils (51.2% girls) who participated in one of the simulation games and both the pre- and post-collection surveys at the secondary level at grammar schools (57.8%), comprehensive schools (11.7%) and vocational schools/vocational colleges (30.5%) in Lower Saxony and North Rhine-Westphalia. Pupils’ average age was 16.75 years (standard deviation (SD) = 1.98). The written survey was conducted either a few days before or directly before and a few days after the simulation game, care being taken to always ensure that there was no other teaching on the EU except the simulation game unit between pre- and post-collection. In addition to the written questionnaires, guided interviews with pupils were conducted after several games.

Survey instruments

The questionnaire consisted of closed, semi-open and open questions. The closed questions on pupils’ political dispositions and their assessments of the simulation games generally consisted of four-point Likert scales allowing agreement or disagreement with statements (to aid interpretation in this article, scores have been reversed in a coherent manner: 1 = disagree entirely, to 4 = agree entirely). In order to assess the changes in EU-related dispositions brought about by the simulation game, scales validated by earlier studies were collected both before and after the simulation game (items according to, inter alia, Deutsche Shell, 2010; Gille et al., 2006; Kerr et al., 2010; Oberle, 2012; Oberle and Forstmann, 2015a; Vetter, 2013; Westle, 2006). A five-factor construct was modelled for EU-related political attitudes, consisting of general attitudes on the EU, perceptions of the EU’s responsiveness, attitudes on the EU’s performance (focusing on democratic legitimacy), the significance of European elections and perceptions of the EU’s relevance to everyday life. Internal political efficacy (with regard to EU) was estimated in a two-factor model, with the two dimensions (a) subjective knowledge and (b) discourse-related self-efficacy.

A two-factor model was also created for willingness to participate in EU-related politics, whereby an empirical and theory-led distinction could be made between basic and advanced willingness to participate. These two sub-facets were captured using (four-point) Likert scale items: (a) basic engagement, involving relatively little effort (e.g. voting in European elections) and (b) advanced, more strenuous engagement (e.g. working in a party or other political group on EU issues, supporting the election campaign of a candidate for the European Parliament and standing for election oneself). Here, we might introduce the differentiation category of ‘effort’ (particularly investing time or giving up free time, ‘information costs’, cf. Pickel, 2012). It would appear that the classic categories of conventionality and institutionalisation, on the other hand, no longer serve as explanations for the distinction between empirically evident types of participation (Pickel, 2012). The sub-dimensions of basic and advanced willingness to participate in politics have already been established by the Göttingen study ‘Pupils’ knowledge and attitudes regarding the European Union’ (WEUS-II; see Oberle and Forstmann, 2015a; in WEUS-II, other than in this study, ‘willingness to participate illegally’ was integrated as a third dimension) and also correspond to the International Civic and Citizenship Education Study (ICCS; cf. Kerr et al., 2010). Learners’ interest in the EU was determined using a four-level scaled single item (‘Are you interested in the European Union? Please tick the answer that most closely reflects you: very interested – not interested at all’).

Furthermore, the pre-test collected data on interest in politics (five-item scale adapted from Köller et al., 2000) and the socio-demographic background variables of gender, age, type of school and cultural capital of the parental home (proxy indicator: books at home; scale: 1 = no/low cultural capital; 6 = high cultural capital; for example, ICCS 2009, see Kerr et al., 2010). The three-factor construct concerning pupils’ satisfaction with the simulation game or their assessment of the simulation game’s effects – (a) general satisfaction with the simulation game, (b) experienced learning effect and (c) perceived increase in interest in EU resp. politics and motivation to engage politically, collected of course only in the post-test – was newly developed in this study and also provides a good data fit. Both measurement models and advanced calculations were modelled using Mplus 7.4, treating Likert-scale items as categorical variables and always taking into consideration the hierarchical structure of the data (Mplus command = type is COMPLEX). For the measurement models of the constructs and sample items, see Table 1.

Table 1.

Measurement models and item examples (for pre-test/post-test).

	Number of Items	α	χ2	CFI/TLI	RMSEA	Item example
Attitudes towards EU (5-factorial)
I = in general	I = 4	I = .78/.78				I = ‘I am glad that Germany is a member of the European Union’.
II = responsiveness	II = 3	II = .67/.59				II = ‘The politicians of the European Union do not really attend to what people like me are thinking’. (Recoded)
III = (democratic) performance	III = 3	III = .71/.68	196.03 (109)***/	.96/.94//	.05/.05	III = ‘How satisfied are you with the extent of people’s participation regarding political decisions of the EU?’IV = ‘It matters to me which candidate gains a seat and becomes a member of the European Parliament’.
IV = relevance of EP-elections	IV = 2	IV = .78/.77	204.32 (109)***	.95/.93
V = relevance to everyday life	V = 5	V = .79/.78				V = ‘The policy decisions of the EU have an impact on my life’.
EU-related internal efficacy (2-factorial)
I = subjective knowledge	I = 5	I = .82/.74	45.56 (19)**/	.99/.99//	.07/.04	I = ‘Altogether, I am familiar with the European Union’.
II = discourse-related efficacy	II = 3	II = .82/.77	26.72 (19); ns	.99/.99		II = ‘When the European Union is being discussed, I usually have something to say about it’.
Willingness to participate in EU (2-factorial)						‘Today, there are various opportunities to participate in the EU. In which of the following could you see yourself taking part?’
I = basic	I = 2	I = .68/.72	15.64 (13); ns/	1.00/.99//	.03/.06	I = ‘… voting in European elections’.
II = advanced	II = 5	II = .80/.82	26.53 (13)*	.99/.98		II = ‘…getting in touch with a member of the EP’.
Interest in politics	5	.90	6.18 (5); ns	1.00/.100	.03	‘Learning about politics is very important to me – regardless of school and other people’.
Assessment of simulation (3-factorial)			320.12 (186)***	.96/.95	.05
I = in general	I = 11	I = .86				I = ‘Altogether, how satisfied are you with the simulation game?’
II = learning effect	II = 6	II = .82				II = ‘Through the simulation game … I all in all better understand how the EU works’.
III = motivation reg. EU and politics	III = 4	III = .89				III = ‘The simulation game … has motivated me to further occupy myself with the EU’.

CFI: Comparative Fit Index; RMSEA: Root Mean Square Error of Approximation; EU: European Union; TLI: Tucker–Lewis index; EP: European Parliament.*p<0.05, **p<0.01, ***p<0.001.

Objective knowledge about the EU was captured by 24 multiple choice items with mainly three distractors and one correct answer (developed building on Oberle, 2012; Oberle and Forstmann, 2015b). Content-wise, the focus was on EU institutions and law-making processes (especially co-decision procedure as ordinary legislative procedure), particularly the role of the European Parliament, as well as general knowledge (e.g. the EU’s goals, its number of member states, its dynamics). Using ConQuest, a Rasch model was applied showing a good fit to the data (Weighted Likelihood Estimate (WLE)/Expected A Posteriori Estimate (EAP) = .73/.74, Variance = .66, Discrimination = .20–.48; for map of EU knowledge displaying item difficulty for pre- and post-test, see Figure 1).

Figure 1.

EU knowledge item difficulties for pre- and post-test.

Results

There is a high level of subjective pupil satisfaction with the simulation game and its effects. In all, 94.8% of participants were satisfied (35.3% ‘very satisfied’) with the game and 77.1% stated they would like to participate in such a simulation again; 90.6% would recommend the game they experienced to others. The participants were asked to associate the game they played with a selection of adjectives. In summary, from the pupils’ perspective, the simulation games conducted may be described as an interesting, varied, instructive method with an appropriate level of difficulty and a duration that appears to be suitable in the school context (cf. Oberle and Leunig, in press-b).

Manifest mean scales were created in order to analyse the mean values and mean differences (see Table 2). On the whole, the simulation game was rated positively (M = 3.08; SD = .31) and pupils generally considered it to have large learning effects (M = 3.09; SD = .47). When it comes to the questions of whether the game had increased their interest in politics and the EU and whether it had motivated them to further engage with the topic of EU and to participate in politics, there was a medium level of agreement (M = 2.49) which varied more strongly across interviewees (SD = .66).

Table 2

Mean values (M) and standard deviations (SD) in the pre- and post-test; effect size of changes (Cohen’s d) for constructs captured before and after the simulation game.

		Pre-test		Post-test		Cohen’s d
		M	SD	M	SD	Cohen’s d
Attitudes towards EU	In general	3.25	.53	3.31	.50	.12
	Responsiveness	2.37	.53	2.58	.47	.42
	Performance	2.78	.55	2.90	.46	.24
	Relevance of EP-elections	2.45	.76	2.63	.74	.24
	Relevance to everyday life	2.82	.55	2.96	.53	.26
Willingness to politically participate (EU)	Basic	3.10	.71	3.13	.74	.04
Willingness to politically participate (EU)	Advanced	2.03	.57	2.13	.66	.16
EU-related internal efficacy	discourse-related efficacy	2.33	.72	2.51	.64	.26
EU-related internal efficacy	subjective knowledge	2.75	.54	3.02	.41	.56
Objective knowledge (EU)		14.33	4.40	16.01	3.50	.42
Interest in EU		2.60	.65	2.69	.60	.14
Interest in politics		2.59	.70
Participants’ evaluation of simulation game	general contentment			3.08	.31
	learning effects			3.09	.47
	motivation (regarding EU, politics)			2.49	.66

EU: European Union; EP: European Parliament.

1 = strongly disagree, 2 = somewhat disagree, 3 = somewhat agree, 4 = strongly agree; objective knowledge: 0–24.

Classification according to Cohen’s d: d ≥ .20 = weak effect; d ≥ .50 = medium effect; d ≥ .80 = strong effect.

The pupils express a clear increase in their understanding of political processes and the way the EU functions (see Figure 2). They state that gaining an insight into how difficult it is to arrive at common decisions, compromises or ‘solutions’ made a profound impression. This learning effect seems particularly relevant given that reservations about the EU (and politics in general) are often related to the length of decision-making processes or long-winded and conflict-laden political discourses. The pupils also confirm the positive effect of the simulation game on their perception of the EU’s relevance to everyday life (also see Table 2).

Figure 2.

Effects of simulation game from the pupils’ perspective (percentages).

Multiple regressions and structural equations (cf. Figure 3) show that pupils who initially were not interested or only slightly interested in politics rate the simulation game on the whole less positively than their politically more interested peers and that pupils who had no interest in politics initially were significantly less motivated by the game to continue to engage with the EU. In order to test whether their assessment of the game is only relatively lower or actually negative, three interest types were generated using a latent class analysis (entropy value = .88; average latent class probabilities between .93 and .98). Comparisons of assessments of the simulation game made by the three groups (cf.Oberle and Leunig, 2016,in press-b) show that the pupils with a strong interest (M = 3.24; SD = .35) do indeed give it the highest rating (M = 3.11; SD = 32), but they also show that on the whole the markedly uninterested learners (M = 1.33; SD = 24) also rate it positively (M = 2.98; SD = .29). Pupils uninterested in politics also state that they experienced an increase in learning (M = 2.96; SD = .48). However, this group does not tend to demonstrate an increase in motivation to continue to concern oneself with the EU or to engage in politics (M = 2.01; SD = 59). After participating in EU simulation games, including the short subject-specific introduction by the staff of planpolitik, on the whole pupils demonstrate significantly greater knowledge about the EU. Mean value comparisons (Table 2) reveal a medium-size effect (Cohen’s d = .42). Multiple regressions and structural equations (cf. Figure. 3) do not display direct differences in relation to gender, age, cultural capital or type of school. However, above-average gains in knowledge were attained by learners with a low level of prior knowledge about the EU. This may be due to a ceiling effect (items were too easy for some pupils, especially in post-test, see Figure 1), but also indicates that learners without much prior knowledge were not uncoupled. The fact that there is only a vague correlation between objective growth in knowledge and perceived learning effect can be interpreted as indicating that from the pupils’ perspective a different kind of political learning was promoted by the game relating – we assume on the basis of the accompanying interviews – to the general processes of politics and the vividness of their EU knowledge.⁴ Further studies are required for more precise analysis.

Figure 3.

Latent structural equation of pupils’ assessment of the simulation game.

Moreover, the pre–post mean value comparisons (Table 2) reveal participation in the simulation game had slight to medium effects on pupils’ attitudes towards the EU, these being more positive in the post-test, as well as on the learners’ internal political efficacy, which was more noticeable after participation in the game. What is striking is the medium-sized effects for subjective knowledge about the EU (Cohen’s d = .56) and perceptions of the EU’s responsiveness (Cohen’s d = .42) – here, significantly higher values can be observed after participation in the game. The general attitudes concerning the EU (M = 3.25; SD = .53) are an exception – these were already relatively positive before the game and did not undergo significant changes. Nor were the effects of participation particularly significant for the total sample regarding interest in the EU, which was captured by a single item or willingness to participate in EU-related politics.

The comparatively significant change in the pupil’s perception of the EU’s responsiveness is surprising since they only experienced a simulation of the political process without getting to know real actors personally. This change could be considered problematic, given that the aim of civics lessons and simulation games is not to create unreflective euphoria concerning the EU or an illusion of reality. Hence, the development of the perception of responsiveness in the sample must be investigated more closely. Before participation in the simulation game, learners’ perception of EU responsiveness was estimated to be slightly negative on the whole (M = 2.37); however, SD = .53 shows variation in the sample. The point of departure for our analysis is to ask which groups of pupils display a change in their perception of responsiveness, resp. whether the changes observed are rather due to reinforcement of positive attitudes or owe to a weakening of prior negative attitudes.

In order to ascertain how pupils with different perceptions of responsiveness develop between the pre- and the post-test, ‘responsiveness types’ were generated: on the basis of the pre-test data, latent class analyses (LCA) were conducted in Mplus 7.4, which suggest a two-class solution (see Table 3). The two types differ in terms of their level of perceived EU responsiveness. Type 1 (n = 151 pupils) perceives the EU as rather unresponsive (M = 1.96; SD = .36), while type 2 (n = 150 pupils) tends towards a slightly positive estimation of its responsiveness (M = 2.77; SD = .52). The group of pupils belonging to type 1, with a negative perception of responsiveness, contains a higher number of vocational school pupils and has less cultural capital than the group with a more positive perception of responsiveness.

Table 3.

Pupil types based on pre-test responsiveness perception: composition and attitudinal changes.

	Responsiveness types^a		Cohen’s d^b
	Type 1 (n = 151)Lower Perceived Responsiveness	Type 2 (n = 150)Higher Perceived Responsiveness	Cohen’s d^b
Gender female^c	52.0%	48.0%
Gender male^c	47.9%	52.1%
Grammar school^c	48.9%	51.1%
Comprehensive school^c	36.4%	63.6%
Vocational school^c	57.4%	42.6%
Cultural capital of the parental home	4.63 (1.27)	4.91 (1.17)	.23
Interest in politics	2.52 (.76)	2.66 (.64)	.20
Responsiveness pre-test	1.96 (.36)	2.77 (.52)	1.81
Responsiveness post-test	2.52 (.47)	2.63 (.48)	.23
Cohen’s d^b within the type	1.34	−.28

Two-type solution with relatively lowest Bayesian information criterion (BIC), entropy value = .84, average latent class probabilities = .95–.97.

Classification according to Cohen’s d: d ⩾ .20 = weak effect; d ⩾ .50 = medium effect; d ⩾ .80 = strong effect.

All percentage values in a row add up to 100%.

The changes brought about by the simulation game regarding the pupils’ perception of EU responsiveness distinguished by the two ‘responsiveness types’ are displayed in Table 3. A significant change in perception of responsiveness can only be seen in type 1, this being extraordinarily strong (Cohen’s d = 1.34) and tending towards a medium value (M = 2.52; SD = .47). Both groups demonstrate a clear growth of objective knowledge about the EU (type 1: Cohen’s d = .44; type 2: Cohen’s d = .38). What is striking is that there is no difference in level of knowledge across both groups either before or after the simulation game. However, significant differences in the perceived relevance of EU for everyday life and subjective EU knowledge could be observed before the game, with type 1 displaying more negative attitudes and less subjective knowledge; after having participated in the simulation, these differences are no longer existent, due to a change towards higher values in group 1 (for more details, see Oberle and Leunig, in press-c).

The analyses of the responsiveness types demonstrate, then, that the simulation game did not contribute to an increase in prior positive perceptions of responsiveness or indeed to euphoric feelings about the EU, but led to a less negative attitude in pupils with a prior sceptical, negative assessment of responsiveness. It can be assumed that the simulation game illustrated relevance of EU for pupils’ daily lives and that adopting the roles of parliamentarians increased identification with politicians in the EU. In this way, increased understanding of the difficulties of finding solutions to political problems and reaching political comprises may have caused pupils to question their general assumptions regarding the incompetence or aloofness of political actors.⁵ Further empirical studies should investigate possible effect mechanisms.

Outlook

The results of this empirical study underline the potential of simulation games to help pupils ‘understand parliaments from the inside’ (Schöne, 2017). They confirm the thesis that simulation games can contribute to overcoming problems peculiar to teaching about the EU, including a perceived lack of relevance to everyday life and perceived hyper-complexity. Pupils’ satisfaction with the simulation is largely independent of the socio-demographic background variables captured, and the growth in knowledge about the EU does not vary systematically when prior knowledge is controlled for. Along with knowledge on the EU, important motivational orientations such as internal political efficacy are fostered by participation in the game. Learners’ EU attitudes tend to become more positive, with changes in the perception of the EU’s relevance to everyday life and the EU’s responsiveness in particular. Perceptions of responsiveness change mainly in pupils who had a low prior opinion; in these cases, the change is extraordinarily pronounced. Presumably the simulation game reduces these pupils’ prejudices, while their adopting a role increased identification with members of the European Parliament. The game did not cause pupils to be euphoric about the EU. Also in view of the scant time resources available in civics lessons, it does make sense to implement short simulation games in schools. Cooperation with extracurricular partners specialising in the subject and methodology seems to offer great potential in that regard since teachers might lack experience, competencies and perhaps also the courage to implement simulation games successfully in class.

The survey method selected for this pilot study seems to be very promising, both in terms of the multi-perspective collection of pupils’ subjective assessments of the simulation game and measuring changes in relevant latent constructs. Further research is required: on one hand, the sample size should be increased or replication studies should be undertaken; on the other hand, follow-up data need to be collected (with at least a third point of measurement) in order to assess the long-term effects of simulation games. It would definitely be useful to have a control group without intervention (see Oberle and Leunig, in press-b), and it would also be highly interesting to have a third group receiving only brief content input by planpolitik but not participating in the game itself. This could be complemented by interviews with pupils or teachers in order to re-examine the results of the questionnaire. There would also appear to be much potential in observing how the simulation game is carried out (for instance, by using video). Furthermore, new survey instruments are required for political judgment and (especially communicative) action competencies in order to investigate their development, too. Ultimately, the competencies of the teachers should be examined as a prerequisite of simulation games’ successful implementation.

Finally, it must be stressed that the aim of this article is not to hold up simulation games as an educational silver bullet; successful civics lessons certainly require the interplay of a variety of approaches and methods (see Oberle and Tatje, 2014; on political excursions in the context of teaching about the EU, see, for example, Weber, 2015; on teaching about the EU in a cognitively activating way with specially developed text-based material, see Weißeno and Landwehr, 2015; on EU-related WebQuests, see Manzel, 2007). Furthermore, tackling the subject of the EU is a process involving many steps – as indeed demonstrated by university seminars; sustained development of understanding requires several attempts and repeated engagement with the object of study. This implies that it is not ideal to wait until the more cognitively advanced sixth form level to examine the complex topic of the EU; rather, it should be introduced early on, ideally already at primary school, while paying appropriate attention to the process dimension of politics. The results of a recent study as part of Göttingen’s Jean Monnet Project PEP⁶ (Oberle and Leunig, in press-a) demonstrate that, contrary to widespread doubts, simulation games can also be an appropriate approach to teaching about (European) politics at primary school.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

Notes

References

Brunazzo

Settembri

(2012) Experiencing the European Union: Learning How EU Negotiations Work Through Simulation Games. Soveria Mannelli: Rubbettino.

Detjen

(2004) Europäische Unübersichtlichkeiten. In: Weißeno

(ed.) Europa verstehen lernen. Eine Aufgabe des Politikunterrichts. Schwalbach am Taunus: Wochenschau, pp.126–143.

Detjen

(2013) Politische Bildung: Geschichte und Gegenwart in Deutschland, 2nd edn. München: Oldenbourg.

Detjen

Massing

Richter

. (2012) Politikkompetenz – ein Modell. Wiesbaden: Springer VS.

Deutsche

Shell

(ed.) (2010) 16 Shell Jugendstudie. Jugend 2010. Eine pragmatische Generation behauptet sich. Frankfurt: Fischer.

Dierßen

Rappenglück

(2015) Europabezogene Planspiele und ihre Wirkungen. In: Oberle

(ed.) Die Europäische Union erfolgreich vermitteln. Perspektiven der politischen EU-Bildung heute. Wiesbaden: Springer VS, pp.223–234.

Easton

(1965) A System Analysis of Political Life, 2nd edn. New York: John Wiley & Sons.

Fink

(2015) 36 different chocolate directives: How does the setting influence negotiation outcomes in an EU simulation? European Political Science 14: 241–253.

Fuchs

Gabriel

Völkl

(2002) Vertrauen in politische Institutionen und politische Unterstützung. Österreichische Zeitschrift für Politikwissenschaft 31(4): 427–450.

10.

Gille

Sardei-Biermann

Gaiser

. (2006) Jugendliche und junge Erwachsene in Deutschland. Lebensverhältnisse, Werte und gesellschaftliche Beteiligung 12- bis 29-Jähriger. Wiesbaden: Springer VS.

11.

Gosen

Washbush

(2004) A review of scholarship on assessing experiential learning effectiveness. Simulation & Gaming 35(2): 270–293.

12.

Guasti

Muno

Nieman

(2015) Introduction – EU simulations as a multi-dimensional resource: From teaching and learning tool to research instrument. European Political Science 14(3): 205–217.

13.

Hartmann

Weber

(2013) Planspiel EU-Emissionshandel – zur Praxis außerschulischer politischer Bildungsprojekte an Schulen. In: Juchler

(ed.) Projekte in der politischen Bildung. Bonn: Bundeszentrale für politische Bildung, pp.136–151.

14.

Jones

Bursens

(2015) The effects of active learning environments: How simulations trigger affective learning. European Political Science 14: 254–265.

15.

Jozwiak

(2013) ‘Vegelate’ and Greece: Teaching the EU through simulations. European Political Science 12(2): 215–230.

16.

Kerr

Sturman

Schulz

. (2010) ICCS 2009 European Report: Civic Knowledge, Attitudes, and Engagement Among Lower-Secondary Students in 24 European Countries. Amsterdam: International Association for the Evaluation of Educational Achievement.

17.

Klieme

(2004) Was sind Kompetenzen und wie lassen sie sich messen? Pädagogik 56(6): 10–13.

18.

Klippert

(1984) Wirtschaft und Politik erleben. Planspiele für Schule und Lehrerbildung. Weinheim: Beltz.

19.

Knelangen

(2011) Vertrauenskrise – die EU und das schwierige Verhältnis zu den Bürgerinnen und Bürgern. In: Varwick

(ed.) Die Europäische Union. Krise, Neuorientierung, Zukunftsperspektiven. Schwalbach am Taunus: Wochenschau, pp.74–88.

20.

Knelangen

(2015) Die Europäische Union und die Bürgerinnen und Bürger: Stimmungsschwankungen oder handfeste Vertrauenskrise? In: Oberle

(ed.) Die Europäische Union erfolgreich vermitteln. Perspektiven der politischen EU-Bildung heute. Wiesbaden: Springer VS, pp.13–25.

21.

Knogler

Lewalter

(2014) What makes simulation games motivating? Design-based research on learners’ motivation in simulation gaming. In: Kriz

(ed.) The Shift from Teaching to Learning: Individual, Collective and Organizational Learning through Gaming Simulation. Bielefeld: Bertelsmann, pp.301–312.

22.

Köller

Schnabel

Baumert

(2000) Der Einfluss der Leistungsstärke von Schulen auf das fachspezifische Selbstkonzept der Begabung und das Interesse. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie 32(2): 70–80.

23.

Krain

Lantis

(2006) Building knowledge? Evaluating the effectiveness of the global problems summit simulation. International Studies Perspectives 7(4): 395–407.

24.

Manzel

(2007) Kompetenzzuwachs im Politikunterricht: Ergebnisse einer Interventionsstudie zum Kernkonzept Europa. Münster: Waxmann.

25.

Massing

(1997) Handlungsorientierter Politikunterricht – Ausgewählte Methoden. Schwalbach am Taunus: Wochenschau.

26.

Massing

(2004) Bürgerleitbilder – Anknüpfungspunkte für eine europazentrierte Didaktik des Politikunterrichts. In: Weißeno

(ed.) Europa verstehen lernen. Eine Aufgabe des Politikunterrichts. Schwalbach am Taunus: Wochenschau, pp.144–157.

27.

Massing

(2012) Die vier Dimensionen der Politikkompetenz. Aus Politik und Zeitgeschichte 46–47: 23–29.

28.

Oberle

(2012) Politisches Wissen über die Europäische Union. Subjektive und objektive Politikkenntnisse von Jugendlichen. Wiesbaden: Springer VS.

29.

Oberle

(2013) Der Beutelsbacher Konsens – Richtschnur oder Hemmschuh politischer Bildung? politische bildung 46(1): 156–161.

30.

Oberle

(2015) Learning about the European Union at school – Theoretical considerations and empirical evidence. Australian & New Zealand Journal of European Studies 7(2): 26–44.

31.

Oberle

(2016) Der Beutelsbacher Konsens – eine kritische Würdigung. In: Widmaier

Zorn

(eds) Brauchen wir den Beutelsbacher Konsens? Bonn: Bundeszentrale für politische Bildung, pp.251–259.

32.

Oberle

Forstmann

(2015a) Förderung EU-bezogener Kompetenzen bei Schüler/innen – zum Einfluss des politischen Fachunterrichts. In: Oberle

(ed.) Die Europäische Union erfolgreich vermitteln. Perspektiven der politischen EU-Bildung heute. Wiesbaden: Springer VS, pp.81–98.

33.

Oberle

Forstmann

(2015b) Lehrerfortbildungen zur politischen EU-Bildung – eine empirische Begleitstudie In: Oberle

(ed.) Die Europäische Union erfolgreich vermitteln. Perspektiven der politischen EU-Bildung heute. Wiesbaden: Springer VS, pp.193–209.

34.

Oberle

Leunig

(2016) Planspiele im Politikunterricht – nur etwas für politische interessierte Jugendliche? In: Goll

Oberle

Rappenglück

(eds) Herausforderung Migration. Perspektiven der politischen Bildung. Schwalbach am Taunus: Wochenschau, pp.125–133.

35.

Oberle

Leunig

(in press-a) Politische Planspiele und EU-Vermittlung in der Grundschule – ein doppeltes No-Go? In: Engartner

(ed.) Politische und ökonomische Bildung – Integration oder Separation? Schwalbach am Taunus: Wochenschau.

36.

Oberle

Leunig

(in press-b) Political motivations as prerequisite and effect of EU simulation games with pupils: Results of an empirical study. In: Bursens

Gijbels

Donche

. (eds) Simulations of Decision-Making in Political Science. Dordrecht: Springer.

37.

Oberle

Leunig

(in press-c) Wirkungen politischer Planspiele auf Einstellungen, Motivationen und Kenntnisse von Schüler/innen zur Europäischen Union. In: Ziegler

(ed.) Politische Bildung in der Demokratie. Zürich; Chur: Rüegger.

38.

Oberle

Tatje

(2014) Die Europäische Union vermitteln – Vorschläge für eine kompetenzorientierte EU-Didaktik. In: Manzel

(ed.) Politisch mündig werden. Politikkompetenz in der Schule aufbauen und diagnostizieren. Opladen: Barbara Budrich, pp.63–77.

39.

Oberle

Tatje

(in press) Schulbuchnutzung im Politikunterricht – eine empirische Studie. In: Manzel

Schelle

(eds) Empirische Forschung zur Politischen Bildung. Wiesbaden: Springer VS.

40.

Patzelt

(1998) Ein latenter Verfassungskonflikt? Die Deutschen und ihr parlamentarisches Regierungssystem. Politische Vierteljahresschrift 38: 725–757.

41.

Patzelt

(2003) Die Deutschen und ihre politischen Missverständnisse. politische bildung 36(3): 58–74.

42.

Pickel

(2012) Das politische Handeln der Bürgerinnen und Bürger – ein Blick auf die Empirie. In: Weißeno

Buchstein

(eds) Politisch Handeln. Modelle, Möglichkeiten, Kompetenzen. Opladen: Barbara Budrich, pp.39–57.

43.

Raiser

Schneider

Warkalla

(2015) Simulating Europe: Choosing the right learning objectives for simulation games. European Political Science 14: 228–240.

44.

Rappenglück

(2004) Europäische Komplexität vermitteln – Chancen und Grenzen europabezogener Simulationen zur kognitiven und habituellen Kompetenzförderung junger Erwachsener. Schwalbach am Taunus: Wochenschau.

45.

Rünz

(2015) Beyond teaching: Measuring the effect of EU simulations on European identity and support of the EU. European Political Science 14: 266–278.

46.

Schnurr

De Santo

Green

(2014) What do students learn from a role-play simulation of an international negotiation? Journal of Geography in Higher Education 38(3): 401–414.

47.

Schöne

(2011) Institutionenperzeption und politische Partizipation – Wie die Bürger über politische Institutionen denken und welche Schlussfolgerungen daraus für die Förderung politischer Beteiligung zu ziehen sind. Politische Psychologie 1(1): 54–66.

48.

Schöne

(2017) Politikwissenschaftliche Mikroanalyse und Politische Bildung. In: Oberle

Weißeno

(eds) Politikwissenschaft und Politikdidaktik. Wiesbaden: Springer VS, pp.87–101.

49.

Töller

(2008) Mythen und Methoden. Zur Messung der Europäisierung der Gesetzgebung des Deutschen Bundestages jenseits des 80-Prozent-Mythos. Zeitschrift für Parlamentsfragen 39(1): 3–18.

50.

Usherwood

(2014) Constructing effective simulations of the European Union for teaching: Realising the potential. European Political Science 13: 53–60.

51.

Vetter

(1997) Political Efficacy – Reliabilität und Validität, alte und neue Messmodelle im Vergleich. Wiesbaden: Deutscher Universitäts-Verlag.

52.

Vetter

(2013) Zufriedenheit der Bürger mit politischen Institutionen und der Demokratie. In: Van Deth

Tausendpfund

(eds) Politik im Kontext: Ist alle Politik lokale Politik? Individuelle und kontextuelle Determinanten politischer Orientierungen. Wiesbaden: VS Verlag, pp.359–380.

53.

Vetter

Maier

(2005) Mittendrin statt nur dabei? Politisches Wissen, politisches Interesse und politisches Kompetenzgefühl in Deutschland, 1994–2002. In: Gabriel

(ed.) Wächst zusammen, was zusammengehört? Stabilität und Wandel politischer Einstellungen im wiedervereinigten Deutschland. Baden-Baden: Nomos, pp.51–90.

54.

Von Geyr

Hornung

Noack

. (2007) Die Europäische Dimension in den Lehrplänen der deutschen Bundesländer: Vergleichende Studie im Auftrag der Europäischen Kommission. Available at: www.jugendpolitikineuropa.de/downloads/4-20-2352/eab_studie%5B1%5D.pdf (accessed 15 July 2016).

55.

Weßels

(2009) Spielarten des Euroskeptizismus. In: Decker

Höreth

(eds) Die Verfassung Europas: Perspektiven des Integrationsprojekts. Wiesbaden: Springer VS, pp.50–68.

56.

Weber

(2015) Die Europäische Union ‘vor Ort’ erleben. Politische Exkursionen nach Brüssel. In: Oberle

(ed.) Die Europäische Union erfolgreich vermitteln – Perspektiven der politischen EU-Bildung heute. Wiesbaden: Springer VS, pp.251–263.

57.

Wehling

H-G

(1977) Konsens à la Beutelsbach? In: Schiele

Schneider

(eds) Das Konsensproblem in der politischen Bildung. Stuttgart: Klett, pp.173–184.

58.

Weinert

(1999) Concepts of Competence. Paris: OECD.

59.

Weinert

(2000) Lehren und Lernen für die Zukunft – Ansprüche an das Lernen in der Schule. Pädagogische Nachrichten Rheinland-Pfalz 2: 1–16.

60.

Weinert

(2001) Vergleichende Leistungsmessung in Schulen – eine umstrittene Selbstverständlichkeit. In: Weinert

(ed.) Leistungsmessungen in Schulen. Weinheim: Beltz, pp.17–32.

61.

Weisseno

Landwehr

(2015) Knowledge about the European Union in political education. What are the effects of motivational predispositions and cognitive activation? McGill Journal of Education/Revue des sciences de l’éducation de McGill, 50 (2/3), pp.413–432.

62.

Westle

(2006) Politisches Interesse, subjektive politische Kompetenz und politisches Wissen – eine Fallstudie mit Jugendlichen im Nürnberger Raum. In: Roller

(ed.) Jugend und Politik – ‘Voll normal!’. Wiesbaden: VS Verlag, pp.209–240.

63.

Zeff

(2003) Negotiating in the European council: A model European Union format for individual classes. International Studies Perspectives 4: 265–274.

Simulation games on the European Union in civics: Effects on secondary school pupils’ political competence

Abstract

Keywords

Theoretical background

Research aims and questions

Study design

The short EU simulation game

Sample and data collection

Survey instruments

Results

Outlook

Footnotes

Declaration of Conflicting Interests

Funding

Notes

References