Changing civil servants’ behaviour concerning the opening of governmental data: evaluating the effect of a game by comparing civil servants’ intentions before and after a game intervention

Abstract

Open data policies are increasingly being adopted by governments. However, civil servants find it challenging to comply with open data policies. Gaming can help civil servants to practise opening data and can change their behaviour to support the opening of more data. In this article, the effect of playing a game is evaluated in an experiment in which several factors that influence the opening of data are compared before and after the game. The benefits appeared in unexpected ways and areas. Data management, privacy and security knowledge was transferred using the game, the perception of benefits showed significant changes, and behavioural intention was positively affected.

Points for practitioners

Civil servants’ behaviour influences how public policies are enacted. The release of open data by governments is related by many as crucial for increasing public transparency and civic participation, and generating new economic opportunities. Games can influence the attitude of civil servants and, consequently, change governments’ decisions. Transferring knowledge and providing insights from new experiences can influence civil servants’ attitudes to open data. Moreover, governments can use games to influence civil servants’ attitudes.

Keywords

behaviour behavioural change data management freedom of information gaming open data open government

Introduction

Open government is a recent trend in public administration that aims to strengthen the relationship between governments and their populations (Wirtz et al., 2017). The opening of governmental data is an important part of open government policies, which aim to make public administration information available for firms, citizens and other governmental units (Ruijer et al., 2018; Zuiderwijk et al., 2018). Examples of such data include data on pollution, traffic, health, environment, justice and economics, which are often collected for policy development and decision-making. Open government data can be defined as raw data that are published on the Internet by governments or publicly funded research organizations (Janssen et al., 2012; Matheus and Janssen, 2015). They are non-sensitive, non-personal data that do not violate data protection or other regulations. They can be freely processed, reused or distributed by others, which therefore democratizes data.

Governments are opening their data to the public to increase transparency and participation, to improve public services, and to stimulate innovation (Hardy and Maurushat, 2017; Pasquier and Villeneuve, 2007). However, many data sets remain closed for various reasons, including inappropriate data infrastructures, lack of knowledge and skills (Janssen et al., 2012), and the will of top-level decision-makers. Lower-echelon civil servants who support decisions to disclose data sets have an enormous impact on the number of data sets opened (Wirtz and Piehler, 2015). Civil servants can gain from opening data as others might provide suggestions based on the data or create helpful insights.

However, the benefits to civil servants are often limited as the opening of data can result in more work. They also risk being held accountable for opening data that should not be opened, even if they benefit society. The bureaucratic environment of public administration, with its strict rules and hierarchy, reduces the space for discretionary attitudes and actions (Lipsky, 1971; Lotta and Marques, 2019). Overestimating risks and/or being unable to assess whether data disclosure can lead to societal benefits may make civil servants reluctant to open data or even to resist their release (Pasquier and Villeneuve, 2007). For example, reports and tables used for internal decision-making are typically not recognized as sharable content, even though they could be useful. Civil servants have to use their scarce resources to ensure that societal benefits can be gained, whereas their knowledge about possible effects and how to realize this is limited.

Even benefits at the individual level, such as reducing red tape or freedom of information (FOI) requests, can be unknown (Janssen et al., 2012). All too often, civil servants fear that opening data might increase risks of potential data misuse or failure to meet data protection regulations. Civil servants are often found to be risk-averse (Lipsky, 1971) and, if in doubt, their default option is often not to support the opening of data. Hence, civil servants might sense not having any direct benefits and being at risk of doing the wrong thing.

Information through training, documentation or videos is used to influence civil servants’ behaviour in opening data. However, these passive communication methods often have limited influence (de Caluwé et al., 2012). Gaming can change the behaviour of participants (McGonigal, 2011). To change perceptions of open data, a game called Winning Data (Kleiman, 2019) was developed. The goal of this game is to improve participants’ understanding of the importance of data management policies in governments for disclosing data, to provide insight into the actual risks and benefits, and to increase knowledge of mitigation mechanisms to reduce the risks. The game has the following objectives:

increase participants’ knowledge of data origin and management;

improve participants’ assessment of privacy and security risks; and

allow participants to experience the benefits of opening data from the public point of view.

Although gaming is advocated as an instrument for influencing participants’ behavioural intention, knowledge is limited about the actual effects of games, particularly when applied to civil servants and data management policies (Kolek et al., 2018). Often, games are viewed as a fun experience but one that does not affect participants’ behaviour. The objective of this article is to analyse the effects of a game on the behavioural intention of civil servants. This will help to gain insight into factors influencing civil servants’ support of the opening of governmental data.

This article is structured as follows: open data and gaming backgrounds are presented to define the hypotheses to be tested in the game experiment, and the methods and tools used to test these hypotheses are then discussed. The context of the experiment and its setting are explained in the analysis section, followed by the findings for each influencing factor of behavioural change (risk perception, performance expectancy, social influence and data management knowledge). These findings are then discussed and conclusions and limitations are presented.

Background: hypothesis formulation

The hypotheses focus on whether playing the game leads to a change in participants’ behaviour and on the elements that are included in the game to influence this change. These elements are derived from the theory of planned behaviour (Warkentin et al., 2002) and technology adoption models (Zuiderwijk and Cligge, 2016; Zuiderwijk et al., 2015).

Behavioural intention is a common category in the behaviour adoption literature, and is seen as the most important predictor of actual behaviour. It can be defined as ‘indications of how hard people are willing to try, of how much of an effort they are planning to exert, in order to perform the behaviour’ (Ajzen, 1991: 181). However, in the open data literature, it is usually applied to users’ intentions and not to providers’ inclinations to release governmental data (Carter and Campbell, 2011). The presented gaming case focuses on the data providers’ perspective and should result in a change in behavioural intentions (hence, in behaviour). The first hypothesis is formulated as:

Hypothesis 1: behavioural intention to open up data increases after playing the game.

The need for knowledge is also well acknowledged in the literature. Janssen et al. (2012) highlighted the lack of knowledge to make use or sense of data, as well as the lack of accuracy of the information itself when using open data. Hossain, Dwivedi and Rana (2016) also described how lack of knowledge results in new barriers, such as less use of data, clearly affecting perceived usefulness – another related factor in the adoption literature. Therefore, knowledge about how to use open data is needed (Conradie and Choenni, 2014; Hardy and Maurushat, 2017):

Hypothesis 2: the game results in more knowledge about how to use open data.

From the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh et al., 2003) originates the concept of performance expectancy, which includes ‘perceived usefulness’ (Davis, 1989), ‘relative advantage’ (Moore and Benbasat, 1991) and ‘outcome expectations’ (Compeau et al., 1999). Regarding data usage, performance expectancy also concerns the expected outcomes of opening data to governments, companies and the general public (Bozeman and Kingsley, 1998; Zuiderwijk et al., 2015). From the data provision perspective, it can synthesize the perception of benefits and positive outcomes that result from an open data practice, both at the individual and institutional levels (Carter and Campbell, 2011; Janssen et al., 2012). Therefore, the game should show the data providers the benefits of opening data for data users:

Hypothesis 3: the game results in a better understanding of the expected benefits of opening data.

The amount of work and perceived risks are barriers to opening data. Venkatesh et al. (2003) suggested ‘effort expectancy’ as a synthesizing variable influencing behavioural intention. Effort expectancy can be defined as ‘the degree of ease associated with the use of an information system’ (Venkatesh et al., 2003: 533). Effort expectancy can be seen as the level of the perception of risk and technical complexity in the adoption of open data technology. These are the barriers faced by data publishers in opening data (Venkatesh et al., 2003; Weerakkody et al., 2017; Zuiderwijk and Cligge, 2016; Zuiderwijk et al., 2015), and are related to the effort required to avoid risk:

Hypothesis 4: the game decreases expectations of the effort needed to make data available.

How others open data can influence a civil servant’s attitude to opening data. This can be the formal influence exerted in the hierarchy but also coercive, mimetic and normative isomorphic pressures (DiMaggio and Powell, 1983). There are many aspects that might have such a social influence, from hierarchy and teams, to formal rules: ‘normative influence can be considered the result of integrating one’s own expectations and feelings with significant others’ perceived expectations and feelings with respect to the shared moral or social meaning of performing a prospective act’ (Venkatesh et al., 2003: 534). This is particularly important in the open data context for governments, which have hierarchies and legal frameworks that limit civil servants’ actions (Venkatesh et al., 2003; Weerakkody et al., 2017; Zuiderwijk and Cligge, 2016). Whereas in the private sector, regulation is used to constrain illegal practices, in governments, civil servants are only allowed to do what the law determines. This leaves little room for innovation and can result in lack of autonomy, even concerning support for different practices:

Hypothesis 5: the game reduces civil servants’ perceptions of constraints to open data practice as exerted by hierarchies and legal frameworks.

Research approach

This article aims to analyse the effects of gaming on the behavioural intention of civil servants to support open data. To our knowledge, besides existing games for e-government (Kelley and Johnston, 2012) for civil servants (Bharosa et al., 2010) and open data users (Wolff et al., 2017), no games have been developed specifically for civil servants to support open data release. Three prototypes were developed but failed in their learning outcomes when tested with students (Kleiman, 2019). Winning Data was used with real civil servants after successful gameplay was achieved in the academic environment (Kleiman et al., 2019) (Figure 1).

Game set-up

The game aims to change civil servants’ attitude to opening governmental data (Hypothesis 1). The game simulates a public office where the players deliver services to a citizen (Kleiman et al., 2019). While the services are provided, data sets are created with various levels of sensitivity regarding privacy and security issues. This ensures that players learn to deal with various types of data sets and data protection (Hypothesis 2). Different service delivery performances and data-labelling options (open/do not open certain data sets) lead to results. In a time-restricted analogical role-playing set-up, four players perform different roles in each round, so that all the participants experience all the functions and perspectives of the office in terms of service delivery and data provision (Hypothesis 3).

Figure 1.

Play session in Brasilia.

The chronological metaphor of the game simulates five weeks in the office. Each week corresponds to a round of play (around five minutes) in which the players have to deliver certain service demands. The first week has general demands related to information requests. The next three weeks concern defined topics: education, health, the environment and urban issues. The last round assesses themes of corruption and red-tape burden reduction. The data sets contain a variety of sensitive data so that some should not be opened, some partially and others fully.

There are four main roles:

citizen (demands services and gives recognition points for the services delivered);

boss (supervises the action and has the final word in the data-labelling process);

civil servant (manages the resources to deliver services to the citizen); and

colleague (processes the demands and helps the civil servant collect the data for labelling).

As the players have played all four roles by the end of the ‘five weeks’, they have a better understanding of the various perspectives of the need for and benefits of open data (Hypothesis 4). The differences in performance, the chance provided by using dice in the service-processing routines and the feedback system of recognition points immerses the players in the plot and ensures fun.

As games should be educating and entertaining, time pressure is used to generate fun in the game (Koster, 2013). A timer is used to represent Monday to Friday in the five-minute rounds. The service delivery ‘week’ is then followed by a data-labelling session in which the civil servant and colleague read the description of each data set produced in the week and suggest a sensitivity label to the boss. Whether data sets are closed, shared or opened to the public depends on the content and context of the data set, and has consequences for the next round, which leads the participants to engage in experiential learning (Kolb, 2000). As the game progresses, improvements are suggested depending on the group performance – these improvements can change the routines and the number of demands received in the upcoming round. Lastly, specific dice combinations (doubles or triples) can produce privacy or security crises, simulating the risks of making data management decisions.

The quasi-experiment

To check civil servants’ perceptions of open data in a quasi-experimental set-up (Shadish et al., 2002), a survey was developed (see Online Appendix 1). In total, 33 questions were used to evaluate the hypotheses discussed earlier.

The gameplay sessions were divided into three moments: (1) a pretest was applied immediately before the game was played and the game rules were explained; (2) the game was played in five predefined rounds; and (3) a post-test was given containing the same 33 questions. This enabled comparison of the change in participants’ behaviours as a consequence of the game (Olejniczak et al., 2020). The pretest included 14 questions about the players’ characteristics, including age and gender. These were used to analyse the demographics and representativeness of the civil servants.

Each game session lasted for two hours on average. In the municipality of Sao Paulo, 32 civil servants played the game in eight sessions in one week. Another 41 civil servants played the game in the National School for Public Service in Brasilia in another nine sessions in one week. Finally, four civil servants from the Accounting Court of Sao Paulo played the game using Skype.¹ In total, 77 civil servants played the game.

Quasi-experiments can make it difficult to isolate bias from researchers and the supporting organizations. As required for a game activity, participation was voluntary.² Nevertheless, to prevent orienting the participants towards certain opinions (and avoid bias), the pretest surveys were applied before any kind of communication took place between the facilitator and the players. Suggestive formulations were avoided in all correspondence with participants. Finally, the game mechanics and dynamics were balanced to include both the benefits and risks of open data release. All the sessions and all participant selections were conducted in a similar manner.

Demographics

A total of 77 civil servants played the game during March and June 2019. Their age ranged from 21 to 61 years old, with an average of 35.51 (SD = 9.5). Males accounted for 61% of the group, as shown in Table 1. Most participants claimed previous knowledge of open data and data management policies in government. This is not surprising as the game sessions focused on civil servants who are involved in opening data. Over half of the participants were permanent government staff.

Table 1.

Demographics.

Category	Values	Frequency	%
Gender	Male	44	61
	Female	28	39
	Total	72*	100
Age	21–30	23	32
	31–40	29	40
	41–50	16	22
	51+	4	6
		72*	100
Government level	Municipal	36	47
	Federal	41	53
	Total	77	100
Years in public sector	0–5	33	45
	5–10	17	23
	10–15	15	21
	15–20	5	7
	20+	3	4
	Total	73*	100
Contract	Politically appointed	18	25
	Permanent staff	40	55
	Hired	4	5
	Other	11	15
	Total	73*	100
Previous knowledge	Heard of open data	69	95
	Studied open data	51	70
	Used open data	62	86
	Comfortable sharing personal data on Internet	33	45

Note: * Five participants did not supply age or gender; four did not answer questions on experience in the public sector or previous knowledge.

Constructs

The hypotheses were tested by comparing the scores before and after the game. All the hypotheses were tested using 33 statements to which a response could be provided on a seven-point Likert scale. The 33 questions are grouped into five concepts: lack of knowledge, performance expectancy, effort expectancy, social influence and behavioural intention. As most of the data were not normally distributed, and because the scores are related to each other (as they come from the same person), the Wilcoxon signed rank test was performed (Field, 2009). First, the difference in pre- and post-game scores for each construct (i.e. a combinations of questions) was tested, followed by a comparison of each single question in the group and the construct.

The constructs, for example, performance expectancy and lack of knowledge, were composed of different statements. A construct that is developed based on multiple items is usually more reliable than a single item. To check the reliability of these constructs, the Cronbach’s alpha reliability test was performed for each construct. The tests confirmed the reliability of three of the original concepts: performance expectancy (Cronbach α = .704; nine items excluding one of its original 10 questions), social influence (Cronbach α = .721; eight items) and behavioural intention (Cronbach α = .769; three items), as shown in Online Appendix 2. Of the total of 33 items, 10 were negatively formulated (LK_15, LK_16, PE_20, EE_12, EE_13, EE_15, SI_12, SI_13, SI_14 and SI_16) and their scores were reversed. This was done to avoid acquiescence bias, that is, when participants agree with questions without reading them properly.

The reliability measurements for the concepts of lack of knowledge and effort expectancy were below 0.6 and could not be improved by omitting statements. Thus, the reliability of these concepts could not be established. Principal component analysis (PCA) (Oblimin rotation because the factors were allowed to correlate) (Hof, 2012) was therefore performed on the remaining 12 items to find underlying concepts. The PCA resulted in two factors that could be interpreted in a logical way. Risk-related topics, labelled ‘risk perception’ (Cronbach α = .66), loaded onto five survey items: LK_15 (the public sector data that result from my work cannot be shared for privacy issues), LK_16 (the public sector data that result from my work cannot be shared for security issues), EE_12 (providing public sector data is a threat), EE_13 (I fear individual privacy by providing public sector data) and EE_15 (I fear people will have false conclusions if public sector data are provided). General data management topics, named ‘data management knowledge’ (Cronbach α = .699), loaded onto three items: LK_13 (I know how to make the public sector data available for others to access), EE_11 (I clearly understand how to provide open public sector data) and EE_16 (Learning to provide open public sector data will be easy for me). Four items from the original lack of knowledge and effort expectancy concepts – namely, LK_11, LK_12, LK_14 and EE_14 – were treated as separate variables as they did not load (in reliability terms) onto the defined constructs.

After establishing the reliability of the constructs, the pre- and post-game scores were compared to test the five hypotheses. The effects of the game on each of these concepts and their individual items are discussed in the next section.

Findings

The Wilcoxon signed rank test indicated that three of the five constructs displayed significant changes from before to after the game. The greatest change was in the questions related to risk, followed by behavioural intention and performance expectancy. Differences in data management knowledge and social influence were not statistically significant.

Behavioural intention (Hypothesis 1: behavioural intention increases after playing the game)

The results of the Wilcoxon signed rank test show that the behavioural intention to open data increases significantly after playing the game: the game increases the willingness of participants to support open data policymaking (see Table 2).

Table 2.

Behavioural intention differences.

	BI_1 (mean of BI_11, BI_12, BI_13)	BI_11: I already provide open public sector data in my work	BI_12: I intend to provide open public sector data in the future	BI_13: I predict that I will provide open public sector data in the future
Mean pre	4.97	3.67	5.65	5.58
Std. dev. pre	1.562	2.227	1.698	1.690
Mean post	5.33	4.13	6.03	5.83
Std. dev. post	1.374	2.166	1.337	1.601
Mean diff.	0.36	0.46	0.38	0.25
Wilcoxon, p-value	0.001	0.001	0.106	0.034
Statistically significant	Y	Y	N	Y

The increase in behavioural intention is probably related to differences in the statements. First, the increase in scores for question BI_11 (from 3.67 to 4.13) might indicate that the game creates awareness in the participants that they were already producing and sharing data in ways they did not realize. By performing the routines in the game and understanding that opening data is less complex than they imagined, their perception change indicates that this is likely to be a relevant effect of the game. Question BI_13 also changed positively (from 5.58 to 5.83) and influenced the increase in scores for the general measures of behavioural intention. Both participants’ individual intention and future perception are likely to be influenced by the game. An increase in the intention to provide open public sector data in the future is also observed for BI_12, though this is not statistically significant. In conclusion, Hypothesis 1 was confirmed.

Data management knowledge (Hypothesis 2: the game results in more knowledge about ways to open data)

Data management knowledge did not significantly differ after playing the game. As Table 3 shows, its constituting statements also showed no statistically significant differences, except for EE_11, which increased from 4.26 to 4.68. This hypothesis could therefore not be confirmed.

Table 3.

Knowledge differences.

	Data knowledge_1 (mean of LK_13, EE_11, EE_16)	LK_13: I know how to make the public sector data available to others	EE_11: I clearly understand how to provide open public sector data	EE_16: Learning to provide open public sector data will be easy for me
Mean pre	4.64	4.50	4.26	5.17
Std. dev. pre	1.363	1.957	1.650	1.540
Mean post	4.84	4.60	4.68	5.26
Std. dev. post	1.231	1.779	1.482	1.542
Mean diff.	0.20	0.10	0.41	0.09
Wilcoxon, p-value	0.153	0.123	0.041	0.788
Statistically significant	N	N	Y	N

Performance expectancy (Hypothesis 3: the game results in a better understanding of the expected benefits of opening data)

I don’t work on the topic, but there are data sets that could be better used if they were opened to the public. (Federal-level participant after game session)

The construct that showed the second-largest difference between pre- and post-testing was performance expectancy, from 5.76 to 5.96, as shown in Table 4. The increase reaches borderline statistical significance (p = 0.055). As the starting score for this item was already high, this might be a consequence of a ceiling effect, meaning the scales do not allow participants to express an increase as their perception was already high. This might indicate that the game provides its participants with a better understanding of the positive effects of open data as they experience the benefits in the game.

Table 4.

Performance expectancy differences.

	PE_1 (mean of PE_11 to 20)	PE_12: Providing open public sector data improves policy-making processes	PE_13: Providing open public sector data creates trust in government	PE_14: Providing open public sector data promotes citizen participation	PE_15: Providing open public sector data increases transparency	PE_16: Providing open public sector data is of benefit to me	PE_17: Providing open public sector data will help me do my job	PE_18: Providing open public sector data will increase my productivity	PE_19: Providing open public sector data improves my performance in my job	PE_20: Providing open public sector data has benefits which are difficult to explain
Mean pre	5.76	6.75	6.41	6.07	6.76	6.19	5.88	5.20	5.25	3.34
Std. dev. pre	0.736	0.598	0.980	1.250	0.628	1.182	1.530	1.822	1.635	1.857
Mean post	5.96	6.75	6.44	6.44	6.64	6.14	6.13	5.65	5.79	3.65
Std. dev. post	0.716	0.610	0.993	0.966	0.759	1.285	1.092	1.528	1.380	2.217
Mean diff.	0.20	0.00	0.04	0.37	–0.12	–0.04	0.25	0.45	0.54	0.31
Wilcoxon, p-value	0.055	0.338	0.361	0.007	0.055	0.531	0.323	0.167	0.024	0.279
Statistically significant	≈ Y	N	N	Y	≈ Y	N	N	N	Y	N

Of the nine statements that make up the concept of performance expectancy, only two show a statistically significant increase when individually analysed (PE_14, PE_19) and one shows borderline significance (PE_15). PE_19 increased from 5.25 to 5.79, indicating that participants’ perceptions of benefits are more influenced by the game’s in-office direct benefits: red-tape reduction and decrease in FOI requirements. Also, PE_14 increased from 6.07 to 6.44. This is highly statistically significant, despite the possible ceiling effect. Interestingly, PE_15 was negatively influenced by the game (a decrease from 6.76 to 6.64 (p = 0.05)). This may be due to the lack of consequences for citizens when data are opened in the game – as qualitative feedback provided in some game sessions indicated.

Risk perception (Hypothesis 4: the game decreases expectations of the effort needed to make data available)

Excellent game, helped me understand the procedures to open data and its risks. (Municipal-level participant)

As explained in the previous section, the new loadings obtained through the PCA resulted in the ‘risk perception’ construct. The scores obtained for this construct decreased (from 5.09 to 3.82) and were statistically significant (p < 0.001), as shown in Table 5. All the questions in the risk perception construct showed a statistically significant decrease. The score for question EE_12 decreased from 6.15 to 3.86, indicating that the game is likely to improve participants’ perceptions of what constitutes a risk for public data provision. As the score before the game was high on average (6 out of 7), the game may reduce participants’ perceptions of risk related to the release of public data, resulting in opening more data.

Table 5.

Risk perception differences.

	Risk (mean of LK_15, LK_16, EE_12, EE_13, EE_15)	LK_15: The public sector data that result from my work cannot be shared for privacy reasons	LK_16: The public sector data that result from my work cannot be shared for security reasons	EE_12: Providing public sector data is a threat	EE_13: I fear individual privacy by providing public sector data	EE_15: I fear people will have false conclusions if public sector data are provided
Mean pre	5.09	4.84	5.23	6.15	4.45	4.77
Std. dev. pre	1.130	1.860	1.872	1.273	1.782	1.762
Mean post	3.82	3.94	3.87	3.86	3.68	3.79
Std. dev. post	1.296	2.035	2.117	2.383	1.568	1.915
Mean diff.	–1.26	–0.91	–1.36	–2.29	–0.76	–0.98
Wilcoxon, p-value	< 0.001	0.02	< 0.001	< 0.001	0.006	0.011
Statistically significant	Y	N	Y	Y	Y	N

The decreases in LK_16 (from 5.23 to 3.87), LK_15 (from 4.84 to 3.94) and EE_15 (from 4.77 to 3.79) may result from the same effect as for question EE_12, though more specifically concerning security, privacy and misinterpretation risks as the game included privacy and security challenges. As described earlier, specific dice combinations produced crises that were increased or reduced by previous data set labelling options. It is likely that the mechanics metaphor of increasing risks by opening more data had an effect on players. Finally, EE_13, which was also a reversed score, decreased from 4.45 to 3.68. We therefore conclude that the game reduces participants’ perception of risks concerning the opening of governmental data.

Social influence (Hypothesis 5: the game reduces civil servants’ perceptions of constraints to open data practice as exerted by hierarchies and legal frameworks)

Social influence did not significantly change through gameplay, though some of its constituting questions did show significant changes (see Table 6). One explanation for this is that the participants played the game voluntarily, so there was no institutional pressure or change in social influence in the game. The game may also have been perceived as neutral.

Table 6.

Social influence differences.

	SI_1 (mean of SI_11 to 8)	SI_11: People who are important to me think that I should provide open public sector data	SI_12: Licensing and legal frameworks make it difficult to provide public sector data	SI_13: Providing public sector data is not a priority for me	SI_14: Providing public sector data is not a priority for the office I work for	SI_15: I have the necessary autonomy to provide public sector data	SI_16: My work does not require me to provide open public sector data	SI_17: My superiors expect me to provide open public sector data	SI_18: I have assistance available concerning the provision of open public sector data
Mean pre	3.85	4.74	3.52	5.46	4.44	2.26	4.09	3.39	2.90
Std. dev. pre	1.078	1.865	1.679	1.500	2.061	1.675	2.248	1.916	1.762
Mean post	4.06	5.22	4.64	3.95	4.43	2.97	4.12	3.71	3.48
Std. dev. post	1.017	1.774	1.813	2.102	2.168	1.933	2.127	2.045	2.004
Mean diff.	0.21	0.48	1.12	–1.51	0.00	0.71	0.02	0.33	0.58
Wilcoxon, p-value	0.159	0.268	< 0.001	< 0.001	0.784	0.003	0.718	0.113	0.005
Statistically significant	N	N	Y	Y	N	Y	N	N	Y

Question SI_13 was a reversed score and showed a significant decrease (from 5.46 to 3.95), indicating that opening data becomes a higher priority in future work. On the other hand, SI_12 (also a reversed score) increased significantly from 3.52 to 4.64. This suggests that respondents perceived more difficulties in opening governmental data after the game than before. The in-game discussions might have increased participants’ willingness to share more governmental data, while also making them more aware that governments might not be as supportive in real situations. The game mechanics probably increased participants’ perception of the potential for opening data, which is reflected in the in-game discussions for labelling data.

Increases were seen in SI_15 (from 2.26 to 2.97) and SI_18 (from 2.90 to 3.48). Interestingly, both questions had a very low benchmark on the pre-survey, indicating an increase in awareness of autonomy and support through the gameplay. Participants were allowed to make choices and convince the boss to label data more openly, which might result in a greater perception of autonomy and support. The perceived direct influence of familiar people (SI_11) or superiors (SE_17) does not seem to be influenced by the game.

In summary, the statistical differences found demonstrate that the game has effects on its participants, particularly on their tendency to support the opening of data. Based on the hypotheses, it is likely that participating in a Winning Data game session changes civil servants’ behavioural intentions, and therefore probably also their future behaviour towards supporting the opening of data.

Discussion

In this article, we have evaluated the effects of playing a game using five constructs: (1) behavioural intention, (2) data management knowledge, (3) performance expectancy, (4) social influence and (5) risk perception. A survey was developed to test the constructs before and after the quasi-experimental gameplay, and the results were presented based on five hypotheses. The outcomes indicate that the game significantly influences the behavioural intention of participants to support the opening of more data. Thus, it is likely that the game produces an effect on its participants and increases their willingness to support open data policymaking.

The underlying constructs taken from the literature (behavioural intention, lack of knowledge, performance expectancy, effort expectancy and social influence) were checked for their loadings in terms of reliability of Cronbach alphas. The concepts of social influence, performance expectancy and behavioural intention loaded sufficiently to progress with the analysis. The original lack of knowledge and effort expectancy groups needed to be reorganized into two new concepts: data management knowledge and risk perception. With this new set, the change observed in behavioural intention was compared to that observed in the defined constructs. Based on these, the hypotheses were formulated and tested.

Starting with H1 (behavioural intention increases after playing the game), the observed behavioural intention change was found to be statistically significant. This change, and its relation with the change in the other tested constructs, shows that the game is likely to have an effect and that it is likely that by playing Winning Data, more civil servants will support the opening of data by governments.

Concerning H2 (the game results in more knowledge about how to open data), we found that although playing Winning Data increased participants’ knowledge of data management, the increase was not statistically significant. However, our participants started with high levels of knowledge (almost 90% had used open data before (see Table 1)), and other civil servants with less previous knowledge and experience might profit more from this aspect of the game.

The next hypothesis (Hypothesis 3: the game results in a better understanding of the expected benefits of opening data) merged expected outcomes of opening data to others, including partners from government or the private sector (Bozeman and Kingsley, 1998; Zuiderwijk et al., 2015). This perception of benefits and positive outcomes was expected to increase at the individual and the institutional levels due to the open data practice simulated in the game (Carter and Campbell, 2011; Janssen et al., 2012). The results show that the increase was, in fact, statistically significant, even for such an experienced audience.

The difficulties faced by civil servants in making data accessible (Venkatesh et al., 2003; Weerakkody et al., 2017; Zuiderwijk and Cligge, 2016; Zuiderwijk et al., 2015) are synthesized in Hypothesis 4 (the game decreases expectations of the effort needed to make data available). Specifically regarding the risks involved in open data release, we – unexpectedly – found a statistically significant decrease. It is therefore likely that game participants improved their understanding of the actual risks and some of the possible mitigating mechanisms. It would be interesting to further explore the relationship between the decrease in risk perception and the increase in civil servants’ behavioural intention to support open data.

Finally, through Hypothesis 5 (the game reduces civil servants’ perceptions of constraints to open data practice as exerted by hierarchies and legal frameworks), hierarchies, legal frameworks and other social pressures are expressed as social influences. This is particularly important in the governmental context of open data as this can limit civil servants’ actions (Venkatesh et al., 2003; Weerakkody et al., 2017; Zuiderwijk and Cligge, 2016). Again, a change was found in the Winning Data participants’ perceptions, though this was not statistically significant. Once more, testing the game with a less experienced group could produce new outcomes.

Conclusions

All too often, public servants are reluctant to open data due to a lack of knowledge about how to do so and its benefits and risks. The effects of gaming on the behavioural intention of public servants were evaluated in this article. Using a survey to compare the situation before and after the game was played confirmed that it is likely that gaming alters the behaviours of civil servants concerning expected performance and risks. The outcomes suggest that gaming is a suitable instrument for knowledge transfer and for creating awareness of possibilities for opening governmental data.

The analysis makes it clear that interacting with the benefits and risks of open data in the game helps civil servants to develop a more realistic perspective of opening governmental data. The game seems to increase participants’ awareness of elements of risk for public data provision, both regarding individual privacy and institutional security. As all the items in the risk perception concept showed a statistically significant decrease, the game might balance participants’ perceptions of risks related to the release of public data.

There were also significant changes in benefits perception as the concept of performance expectancy resulted in the second-largest difference between pre- and post-test measurements. The game therefore gave participants a better understanding of the positive outcomes of data opening. It should be noted that the starting score for some items was already high, which may have caused a ceiling effect as the scales did not allow participants to express further increases. Items such as PE_19 (providing open public sector data improves my performance in my job) indicate that the game’s in-office direct benefits (a reduction in red tape and FOI requirements) had more of an influence in this regard. The negative influence found in item PE_15 (providing open public sector data increases transparency) might be due to the lack of an effect on citizens when data are opened in the game, as indicated by a qualitative point of feedback provided in some sessions.

Despite the fact that the concepts of social influence and data management knowledge did not result in statistically significant changes through gameplay, some of their constituting items did. On the one hand, as the participants played the game voluntarily, the game might have been perceived as neutral. On the other hand, data management, privacy and security knowledge were declared to have been transferred through the game. The decrease in item SI_13 (providing public sector data is not a priority for me) indicates that participants might be considering opening more data in their future work. Other items with a low benchmark, such as SI_15 (I have the necessary autonomy to provide public sector data) and SI_18 (I have assistance available concerning the provision of open public sector data), suggest an increase in awareness of autonomy and support through the gameplay. The in-game discussions may increase participants’ willingness to share more governmental data, while also increasing their awareness that the government might not be as supportive.

Finally, behavioural intention to share open data significantly increased after playing the game, showing that the game had effects on participants in terms of their willingness to support open data policymaking. Items BI_11 (I already provide open public sector data in my work), BI_12 (I intend to provide open public sector data in the future) and BI_13 (I predict that I will provide open public sector data in the future) all indicate that the game creates awareness in the participants that they both already produce and share data in a way that they did not realize before and will do so in the future. The in-game routines might help people to understand that opening data is less complex than they thought. The statistical differences found indicate that the game is effective for changing civil servants’ support of the opening of data.

These results can be explored in further studies as extending the repeated measurements and testing the long-term effects of behavioural intention change can increase understanding of the effects of the game. Although it would have been interesting to test the participants’ perceptions a third time and check the mid-term effects of the game, this was not feasible. Nevertheless, the collected data provide some interesting results.

Also, correlating the constructs and processing regressions may result in an integrated behaviour model. Such a model can be used to understand which factors influence civil servants’ behaviours, and to develop more effective game interventions. Furthermore, the effects of the game can be compared with other learning methods, such as training and documentation. We also support the idea of progressing with the gameplay and testing this intervention in more diverse groups in terms of experience in public service, governmental level and municipality.

Supplemental Material

sj-pdf-1-ras-10.1177_0020852320962211 - Supplemental material for Changing civil servants’ behaviour concerning the opening of governmental data: evaluating the effect of a game by comparing civil servants’ intentions before and after a game intervention

Supplemental material, sj-pdf-1-ras-10.1177_0020852320962211 for Changing civil servants’ behaviour concerning the opening of governmental data: evaluating the effect of a game by comparing civil servants’ intentions before and after a game intervention by Fernando Kleiman, Marijn Janssen, Sebastiaan Meijer and Sylvia JT Jansen in International Review of Administrative Sciences

Supplemental Material

sj-pdf-2-ras-10.1177_0020852320962211 - Supplemental material for Changing civil servants’ behaviour concerning the opening of governmental data: evaluating the effect of a game by comparing civil servants’ intentions before and after a game intervention

Supplemental material, sj-pdf-2-ras-10.1177_0020852320962211 for Changing civil servants’ behaviour concerning the opening of governmental data: evaluating the effect of a game by comparing civil servants’ intentions before and after a game intervention by Fernando Kleiman, Marijn Janssen, Sebastiaan Meijer and Sylvia JT Jansen in International Review of Administrative Sciences

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Fernando Kleiman

Supplemental material

Supplemental material for this article is available online.

Notes

Fernando Kleiman is a doctoral candidate at the Department of Systems and Services Engineering in the Delft University of Technology, The Netherlands. He received a bachelor’s degree in Economics from the University of Sao Paulo, Brazil. He holds an MSc in Sociology from the University of Brasilia, Brazil. His current research interests lie in digital government, open data and serious gaming.

Marijn Janssen is a Full Professor in ICT & Governance at the Technology, Policy and Management Faculty of Delft University of Technology, The Netherlands. He was ranked as one of the leading e-government researchers and has published over 500 refereed publications. For more information, see:

Sebastiaan Meijer is Full Professor in healthcare logistics and department head of Biomedical Engineering and Health Systems at KTH Royal Institute of Technology, Sweden. He works particularly on large-scale system change with the support of gaming and simulation methods, with applications in health and transport. For more information, see:

Sylvia JT Jansen works as Assistant Professor in the Department of Management in the Faculty of Architecture and the Built Environment at Delft University of Technology, The Netherlands. Her teaching activities include courses on research skills and statistics. Her research focuses on cognitive aspects and well-being, such as preferences, satisfaction, quality of life, attitude and values.

References

Ajzen

I (

1991) The theory of planned behavior. Organizational Behavior and Human Decision Processes 50(2): 179–211.

Bharosa

Janssen

Klievink

, et al. (2010) Guiding integrated service delivery: Synthesizing and embedding principles using role-playing games. Electronic Journal of e-Government 8(2): 83–92.

Bozeman

Kingsley

(1998) Risk culture in public and private organizations. Public Administration Review 58(2): 109–118.

Carter

Campbell

(2011) The impact of trust and relative advantage on Internet voting diffusion. Journal of Theoretical and Applied Electronic Commerce Research 6(3): 28–42.

Compeau

Higgins

Huff

S (

1999) Social cognitive theory and individual reactions to computing technology: A longitudinal study. MIS Quarterly 23(2): 145–158.

Conradie

Choenni

S (

2014) On the barriers for local government releasing open data. Government Information Quarterly. Available at: https://www.sciencedirect.com/science/article/pii/S0740624X14000513

Davis

FD (

1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13(3): 319–340.

De Caluwé

Geurts

Kleinlugtenbelt

WJ (

2012) Gaming research in policy and organization. Simulation & Gaming 43(5): 600–626.

DiMaggio

Powell

WW (

1983) The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review 48(2): 147–160.

10.

Field

(2009) Discovering Statistics Using SPSS (3rd edn). London: SAGE Publications.

11.

Hardy

Maurushat

A (

2017) Opening up government data for Big Data analysis and public benefit. Computer Law & Security Review 33(1): 30–37.

12.

Hof

M (

2012) Questionnaire Evaluation with Factor Analysis and Cronbach?s Alpha: An Example. Available at: http://www.let.rug.nl/nerbonne/teach/rema-stats-meth-seminar/student-papers/MHof-QuestionnaireEvaluation-2012-CronbachFactAnalysis.pdf

13.

Hossain

Dwivedi

Rana

NP (

2016) State-of-the-art in open data research: Insights from existing literature and a research agenda. Journal of Organizational Computing and Electronic Commerce 26(1/2): 14–40.

14.

Janssen

Charalabidis

Zuiderwijk

A (

2012) Benefits, adoption barriers and myths of open data and open government. Information Systems Management 29(4): 258–268.

15.

Kelley

Johnston

E (

2012) Discovering the appropriate role of serious games in the design of open governance platforms. Public Administration Quarterly 36(4): 504–554.

16.

Kleiman

F (

2019) Engaging governments in open data policies through gaming. Paper presented at the 12th International Conference on Theory and Practice of Electronic Governance, Melbourne, VIC, Australia.

17.

Kleiman

Janssen

Meijer

S (

2019) Evaluation of a pilot game to change civil servants’ willingness towards open data policy making. In: Wardaszko

(ed.) Simulation & Gaming Through Times and across Disciplines. Past and Future, Heritage and Progress. Proceedings of the 50th Anniversary ISAGA Conference. Warsaw, Poland: Kozminski University.

18.

Kolb D (2000) The process of experiential learning. In: Cross RL and Israelit S (eds), Strategic Learning in a Knowledge Economy: Individual, Collective and Organizational Learning Process. Oxford, UK: Butterworth-Heinemann.

19.

Kolek

Šisler

Brom

(2018) Video games and attitude change – Can we reliably measure this? The challenges of empirical study design. Paper presented at the International Conference on Games and Learning Alliance, Palermo, Italy, 5--7 December 2018.

20.

Koster

(2013) Theory of Fun for Game Design. Sebastopol, CA: O'Reilly Media, Inc.

21.

Lipsky

(1971) Street-level bureaucracy and the analysis of urban reform. Urban Affairs Quarterly 6(4): 391–409.

22.

Lotta

Marques

(2019) How social networks affect policy implementation: An analysis of street‐level bureaucrats’ performance regarding a health policy. Social Policy & Administration 54(3): 345–360.

23.

Matheus

Janssen

(2015) Transparency dimensions of big and open linked data. In: Janssen M, Mäntymäki M, Hidders J et al. (eds) Open and Big Data Management and Innovation: 14th IFIP WG 6.11 Conference on e-Business, e-Services, and e-Society, I3E 2015, Delft, The Netherlands, October 13–15, 2015, Proceedings. Cham: Springer International Publishing, pp. 236–246.

24.

McGonigal

(2011) Reality Is Broken: Why Games Make Us Better and How They Can Change the World. City of Westminster, London: Penguin.

25.

Moore

Benbasat

I (

1991) Development of an instrument to measure the perceptions of adopting an information technology innovation. Information Systems Research 2(3): 192–222.

26.

Olejniczak

Newcomer

Meijer

SA (

2020) Advancing evaluation practice with serious games. American Journal of Evaluation 41(3): 339–366.

27.

Pasquier

Villeneuve

J-P (

2007) Organizational barriers to transparency: A typology and analysis of organizational behaviour tending to prevent or restrict access to information. International Review of Administrative Sciences 73(1): 147–162.

28.

Ruijer

Grimmelikhuijsen

van den Berg

, et al. (2018) Open data work: Understanding open data usage from a practice lens. International Review of Administrative Sciences 86(1): 3–19.

29.

Shadish

Cook

Campbell

DT (

2002) Experimental and Quasi-Experimental Designs for Generalized Causal Inference (0395615569). Wadsworth: Cengage Learning.

30.

Venkatesh

Morris

Davis

, et al. (2003) User acceptance of information technology: Toward a unified view. MIS Quarterly 27(3): 425–478.

31.

Warkentin

Gefen

Pavlou

, et al. (2002) Encouraging citizen adoption of e-government by building trust. Electronic Markets 12(3): 157–162.

32.

Weerakkody

Kapoor

Balta

, et al. (2017) Factors influencing user acceptance of public sector big open data. Production Planning & Control 28(11/12): 891–905.

33.

Wirtz

Piehler

(2015) eGovernment applications and public personnel acceptance: An empirical analysis of the public servant perspective. International Journal of Public Administration 39(3): 238–247.

34.

Wirtz

Weyerer

Rösch

(2017) Open government and citizen participation: An empirical analysis of citizen expectancy towards open government data. International Review of Administrative Sciences 85(3): 566–586.

35.

Wolff

Barker

Petre

(2017) Creating a datascape: A game to support communities in using open data. In: Proceedings of the 8th International Conference on Communities and Technologies, ACM, New York, NY, pp. 135--138.

36.

Zuiderwijk

Cligge

(2016) The acceptance and use of open data infrastructures – Drawing upon UTAUT and ECT. Paper presented at the Electronic Government and Electronic Participation: Joint Proceedings of Ongoing Research, PhD Papers, Posters and Workshops of IFIP EGOV and EPart.

37.

Zuiderwijk

Janssen

Dwivedi

YK (

2015) Acceptance and use predictors of open data technologies: Drawing upon the unified theory of acceptance and use of technology. Government Information Quarterly 32(4): 429–440.

38.

Zuiderwijk

Shinde

Janssen

M (

2018) Investigating the attainment of open government data objectives: Is there a mismatch between objectives and results? International Review of Administrative Sciences 85(4): 645–672.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.19 MB

0.16 MB