Students’ attitudes towards fundamental democratic values: The construction of a measurement instrument

Abstract

Schools carry the important task of fostering democratic values like equality and tolerance among students. This study examined to what extent a valid and reliable measurement instrument can be developed to gain insight into students’ attitudes towards democratic values and what can be concluded based on this measurement. We constructed the Fundamental Democratic Values Questionnaire (FDVQ) based on existing conceptualisations of democratic values. After that, we conducted two pilot studies (N = 302 and N = 227) and a first measurement (N = 3278) among Dutch students in grades 5 and 6 (approximately 10–12 years old), resulting in an impression of the usefulness of the instrument for empirical research and practical use by schools. The results support the FDVQ’s validity and reliability to measure students’ attitudes towards democratic values. Support for democratic values among students was generally high. These findings and the implications of the FDVQ are further discussed.

Introduction

In pluriform societies, democracy is a form of government that supports the humane and structured co-living of (groups of) individuals who differ in terms of religion, culture, ethnic background, political ideas, and so on (De Winter, 2004). It does so by being responsive to the majority’s views while providing space to and protecting the rights of minorities (cf. Council of Europe, 2018). For democracy to persevere, it is important that citizens have an interest and active involvement in the democratic society and underscore and have knowledge of democratic values like equality, tolerance and non-discrimination (Levinson, 1999). In addition, citizens of pluriform democracy must also be able to make critical contributions (Wardekker, 2001), for example, by critically exploring strategies, perspectives and situations of (in)equality and (in)justice (Westheimer, 2008). This is not only relevant for effective representation, but an open and responsive decision-making process also strengthens the problem-solving capacity of groups. Without actively involved and informed citizens who comply with such a ‘democratic way of life’ (cf. Dewey, 1923), democracy becomes fragile. A democratic way of life presumes citizens to acknowledge and accept that others may have different interests and to be willing to deal with conflicts via dialogue and negotiation or, ultimately, the rule of law. Democratic societies benefit in terms of resilience when (young) citizens live up to a democratic way of life (Dekker and Den Ridder, 2016; Welzel and Dalton, 2014).

The importance of democratic values, such as equality, non-discrimination and respecting the rule of law, for the resilience of a pluriform democracy, is underscored by the fact that they are anchored in many national constitutions and the Universal Declaration of Human Rights (UDHR) (United Nations, 1948). Among citizens, the importance of underscoring democratic values is also acknowledged. Using decades of World Survey Data, Welzel (2021), for example, argued that support for so-called emancipative values (i.e. a subset of values that combines an emphasis on freedom of choice and equality of opportunities) among citizens throughout the world is on the rise, leading to a stronger commitment to democratic principles among younger generations of citizens. In addition, research shows that adult citizens perceived various democratic values (e.g. knowledge of values and norms, rules of conduct, and human rights, including children’s rights) to be important for young citizens to learn (Dekker and Den Ridder, 2016; van Goethem et al., 2022). Despite their importance, the acceptance and underscoring of democratic values is not a given. Or, as White (1999) puts it: ‘democrats are made, not born’ (p. 59).

Instead, democratic values require to be educated, promoted and maintained. Since teaching students the competences needed to partake in a democratic society is an important goal of education (Dewey, 1923), and education is a common socialisation context shared by all young citizens (Beane, 2013), it is important that schools carry out the task of fostering these competences in students (Council of Europe, 2018). In line with that, as a response to societal changes such as increased diversity, individualisation, and polarisation (Mattei and Broeks, 2018; Organisation for Economic Co-operation and Development, 2010; Tyler and Iyengar, 2023), many countries have appealed to schools with the task of fostering democratic citizenship competences in students via citizenship education (cf. Eurydice, 2017). Citizenship competences are generally understood as the knowledge, attitude and skills young citizens need to participate in a democratic and pluriform society (Schulz et al., 2018). To gain citizenship knowledge and skills, the underscoring of democratic values can be seen as an important basis: as a student, you have to be open, motivated and willing to learn about the perspectives of others and let others learn from your perspectives (De Winter, 2004; White, 1999).

In summary, democratic values are widely conceived as important for democratic and pluriform societies, require active fostering (i.e. ‘democrats are made, not born’) and (citizenship) education is one of the socialising agents that plays a role in this. Even though there is no fixed or demarcated set of values that should be promoted in education (e.g. differences may be revealed between a liberal and communitarianism perspective, also see Eidhof et al., 2016; Miller, 2000), common ground can be found in the regula aurea or Golden Rule (Wattles, 1996). The Golden Rule is a widely held philosophical and religious principle better known as ‘to treat others as you want others to treat you’, shimmering through in core values guaranteed in national constitutions or international declarations guarding human rights. As such, this norm of reciprocity can be seen as an important foundation for value formation and may help schools to reflect on the commonly held values they want to teach.

To better understand the quality and effectiveness of education focused on promoting democratic values among students, it is important to have insight into students’ citizenship competences. A standardised measurement instrument that provides insight into students’ attitudes towards democratic values is meaningful to educational practice because it helps schools adjust their education to what students need and supports schools in achieving learning goals regarding fostering democratic values. Although instruments for measuring students’ attitudes towards (some) democratic values are available (e.g. the instruments used in the International Civics and Citizenship Education Study (ICCS) (Schulz et al., 2018) and the Citizenship Education Longitudinal Study (CELS) (Keating et al., 2010), no measurement instrument is currently available i.e. not only suitable for research purposes but is also purposefully designed for direct application in educational practice with the main goal of capturing students’ democratic attitudes. Such an instrument is meaningful for educational practice because it helps schools to better understand the quality and effectiveness of education focused on promoting democratic values among students. Hence, the present study aimed to examine to what extent a reliable and valid measurement instrument suitable for educational practice can be developed to gain insight into students’ attitudes towards democratic values. The following research question is central in this study: ‘How can students’ attitudes towards democratic values be reliably and validly measured, and what can be concluded based on this measurement?’

Conceptual framework

The conceptualisation of values that are key to a democratic way of living, guarding strong and vibrant democratic societies, received consistent attention in the scholarly literature. According to Beane (2013), the competences needed for participation in a democratic society include students becoming aware of human rights and values such as respect for human dignity, equity, freedom, and social responsibility. Moreover, it includes that students become competent in skills such as critical thinking, problem-solving, collaboration, information and data gathering, reflection, and participatory planning. Eidhof et al. (2016) raise the label consensus goals, described as general democratic goals reflecting the values necessary to maintain democratic societies, based on what the majority of a population perceives as important democratic principles to foster in youth. Synthesising the literature, consensus goals generally consist of three main democratic values: tolerance of diversity (Barber, 1984; Galston, 2001; Kymlicka, 1989; Van Gunsteren, 1998), non-violence (Galston, 2001) and civic engagement (Almond and Verba, 1963; Putnam, 2000). In addition, Almond and Verba (1963) argued that, for democratic societies to last, it is not only important for democratic principles in the political and public sphere (e.g. support for democratic institutions, acceptance of policy decisions) to be anchored in society but also for values that belong to the social and cultural roots of democracy, such as tolerance, rejection of violence, sense of belonging and solidarity towards a community, openness towards individuals with other faiths and beliefs. In the UK context, Janmaat (2018) examined the extent to which young people supported so-called ‘fundamental British values’, which the UK Department for Education (2014) defined as democracy, the rule of law, individual liberty, and mutual respect and tolerance of those with different faiths and beliefs. Even though labelling these beliefs as ‘fundamentally British’ is criticised for being too narrowly specified (cf. Lander, 2016; Tomlinson, 2015), these values can be considered indeed fundamental for democratic societies and show resemblance with the values mentioned in national constitutions and in documents such as the UDHR.

In line with their importance for non-violent, cooperative co-living of different groups in a pluriform democracy, various values are captured in the UDHR. In the UDHR, among others, non-discrimination, non-violence, freedom of speech, thought and religion, equality, personal integrity, and political participation are included (United Nations, 1948). Moreover, the UDHR is alleged to be a universal document with universal compliance regardless of individuals’ sex, culture, ethnicity, religion, political ideas or other. The European Convention on Human Rights (ECHR) (Council of Europe, 1953) and United Nations Convention on the Rights of the Child (UNCRC) (United Nations, 1989) both show great similarities concerning the incorporated values with the UDHR but are, respectively, oriented towards member states of the Council of Europe and citizens younger than 18 years old.

Besides their role in the ECHR, the Council of Europe (Council of Europe, 2018) identified, among other competences, three main values and six attitudes in the Reference Framework of Competences for Democratic Culture (RFCDC). In this respect, values are ‘general beliefs that individuals hold about the desired goals that should be striven for in life’ (Council of Europe, 2018: 38). Attitudes are ‘overall mental orientations which an individual adopts towards someone or something’ (Council of Europe, 2018: 41). The RFCDC was designed to provide a clear focus and understanding of citizenship goals in education in Europe. The values in the RFCDC are human dignity and human rights; cultural diversity; and democracy, justice, fairness, equality and the rule of law (Council of Europe, 2018). The attitudes in the RFCDC are openness to cultural otherness and other beliefs, world views and practices; respect; civic-mindedness (i.e. an attitude of belonging to, mindfulness of, a sense of solidarity with, interest in, willingness to contribute to, commitment to, and a sense of accountability towards a larger community or social group); moral responsibility; self-efficacy; and tolerance of ambiguity (i.e. an attitude towards objects, events and situations which are perceived to be uncertain and subject to multiple conflicting or incompatible interpretations) (Council of Europe, 2018).

In sum, it can be concluded that both scientific reasoning and highly influential policy documents anchoring democratic values mention the following as fundamental to democracy: being open, tolerant and respectful towards individuals with other beliefs, world views, faiths and practices and, related, the principles of non-discrimination, non-violence and equality. In addition, human dignity, personal integrity, individual liberty, and freedom (of speech, thought and religion) are important elements. The same holds for human rights, justice, fairness, the rule of law, support for democratic institutions, and acceptance of policy decisions. At last, being part of a larger community or social group was mentioned, such as solidarity, accountability and (moral and social) responsibility towards this group.

The present study

In many countries, education is appointed by law to foster democratic values among young citizens. This is, for example, visible in the UK (Janmaat, 2018), in Norway, Scotland, Sweden, the Netherlands (Dijkstra and De la Motte, 2014) and many other countries throughout Europe (Eurydice, 2017) and around the world (Schulz et al., 2018). The present study is focused on the Netherlands, where the national government identified fundamental democratic values (FDVs) that should be fostered in citizenship education. Specifying freedom, equality and solidarity as core values, eight FDVs are included in education law, that is, equality of all people, individual autonomy, freedom of speech, tolerance and the rejection of intolerance, an attitude geared towards an understanding of others, the rejection of discrimination, and a sense of responsibility towards the common good (Ministry of Education, Culture and Science, 2021). The Dutch government describes these FDVs as universal values widely legitimated as visible in the national constitution and the UDHR. Moreover, the statutory citizenship obligation for schools, including the task of promoting these FDVs, leaves autonomy with schools to determine what values they want to foster or emphasise in their students but, at the same time, underscore that these eight values are essential for participation in a pluriform and democratic society – like the Dutch society – and, hence, should not be undermined in education. To adequately prepare young citizens for participation in society, the minimum requirement to ask from schools is to actively foster these FDVs in students and ensure that the education they give does not conflict with them.

While this set of FDVs specified by the Dutch government is not exhaustive, it resembles universal values, as visible in its great overlap with conceptualisations of FDVs in international policy documents such as the UDHR, ECHR, UNCRC and RFCDC, as well as in the scholarly literature (e.g. Almond and Verba, 1963; Beane, 2013; Eidhof et al., 2016; Janmaat, 2018). In determining what values should be included in the Dutch education law, the legislator took their relevance for and connection to the development phase of young students. In that respect, the FDV of equality is, for example, considered more overarching than the value of ensuring that all citizens have a fair judicial trial. More specifically, the FDV of equality is regarded as a more fundamental and grounding element for students and their contexts, compared to, for instance, a more specific value like ensuring a fair trial for all citizens. Although values such as the latter are not directly included in the legislation, they are still indirectly addressed, and their elaboration into related values relevant to the schools’ context and student population is left to the schools. Likewise, non-specified values like freedom of thought, consciousness and religion are, for example, indirectly addressed in the value of freedom of speech. Like in other countries, Dutch schools carry the task of fostering democratic values in students. Insight into students’ attitudes towards democratic values is meaningful to schools because it helps them adjust their education to what students need and achieve learning goals regarding democratic values. Hence, a standardised measurement instrument was developed for direct application by schools, and can additionally be used for empirical research. The remainder of this paper describes the construction of this measurement instrument, which can be used to gain insight into students’ attitudes towards FDVs as specified before. As such, this measurement instrument was designed to align with the context in which the study took place and followed the conceptualisation of FDVs as visible in the legislation that applies to Dutch schools.

Method

This section consecutively describes the participants, materials and procedure of two pilot studies and one first measurement conducted to construct the Fundamental Democratic Values Questionnaire (FDVQ). This study was approved by the Ethics Review Board of the Faculty of Social and Behavioural Sciences of the University of Amsterdam [2021-CDE-12966].

Participants

The scope of this paper was delineated to students in grades 5 and 6 of regular primary education (RPE) and students in the final (‘leaving’) grade of special needs primary education (SNPE) ¹ (approximately 10–12 years old). The descriptive statistics of the samples in the two pilot studies and the first measurement using the final instrument can be found in Table 1. In the first measurement, 7% of the students had a migration background, and 19% indicated that their home language was other than Dutch, Frisian or another regional Dutch language.

Table 1.

Characteristics of participants.

	Educational sector	Grade	N	Sex	Mean age
Pilot 1	Regular primary education	5	122	53% boys	9.79
		6	150	47% boys	10.69
	Special needs primary education	5/6	30	83% boys	10.87
Pilot 2	Regular primary education	5	125	46% boys	10.4
		6	102	49.0% boys	11.5
Measurement 1	Regular primary education	5	1367	50% boys 50% girls <0.1% other/I’d rather not say	10.77
		6	1606	47% boys 50.7% girls <0.1% other/I’d rather not say	11.72
	Special needs primary education	5/6	305	68.5% boys 29.2% girls <0.1% other/I’d rather not say	12.04

Materials

The FDVs, as conceptualised in the Dutch legislation and in this study, consist of equality (i.e. all individuals are of equal value, despite beliefs or faith), understanding of others (i.e. understanding why individuals or communities have, for instance, certain beliefs, faiths or identities, constituting their being), freedom of speech (i.e. all individuals are allowed to say and write what they think or go against conceptions of others, as long as it is in accordance with the law), autonomy (i.e. all individuals may decide for themselves who they want to be and how they want to live their live, as long as it is in accordance with the law), tolerance (i.e. accepting and providing space to the beliefs, faiths or behaviour of others even if you do not agree with it, as long as it is in accordance with the law), rejection of intolerance (i.e. rejecting behaviour in which other individuals beliefs, faiths or behaviours are not accepted or allowed to take space), rejection of discrimination (i.e. rejecting behaviour in which other individuals are being discriminated against based on their beliefs, faiths or behaviours), and sense of responsibility (i.e. taking responsibility for what you say or do, and for what you do not say and do not do, and taking into account what that means for others, while trying not to harm others and help democracy to function well) (Dijkstra et al., 2014; Ministry of Education, Culture and Science, 2021).

To establish the FDVQ to measure students’ attitudes towards FDVs, the FDVs originating from the Dutch law on citizenship education (cf. Ministry of Education, Culture and Science, 2021) were carefully operationalised into eight Likert-type scales. Each item had four answer options: ‘totally disagree’, ‘somewhat disagree’, ‘somewhat agree’, and ‘totally agree’. This was decided to force students to choose between a more negative (‘totally disagree’; ‘somewhat disagree’) or positive (‘somewhat agree’; ‘totally agree’) answer. Even though research is inconclusive on whether or not to include a neutral midpoint (e.g. Chyung et al., 2017), scholars have posed that excluding a neutral midpoint minimises socially desirable responses of respondents (Garland, 1991; Johns, 2005), which is important to take into account when studying citizenship competences of students (Ten Dam et al., 2013).

In addition to their attitude towards these FDVs, information on students’ sex, age, grade, educational sector, home language, and migration background was gathered. For home language, students indicated whether their home language was Dutch; Frisian or another Dutch regional language; Turkish; English; Moroccan or Berber; Surinamese or an Antillean language; or something different. For migration background, students indicated whether they lived in the Netherlands for less than a year; 1–3 years; 4–5 years; longer than 5 years; or whether they have always lived in the Netherlands.

Procedure

As part of the construction phase, two pilot studies were conducted in different groups of students using a digital survey: one between April and June 2021; and one between February and March 2022. After each pilot, the psychometrics of the proposed items and scales were checked by examining descriptive statistics (e.g. mean, standard deviation, answer pattern, skewness, kurtosis), Cronbach’s reliability, and results of exploratory and confirmatory factor analyses. Furthermore, students, teachers, school leaders and researchers were consulted before and after each pilot. Based on the psychometrics and consultations, the items were improved into a final version of the FDVQ (see Table 2). The changes consisted mostly of maximising linguistic accessibility by simplifying the language and shortening items as much as possible. Additionally, in the final questionnaire, when asked for their sex, students could also opt for ‘other/I’d rather not say’, whereas, in the pilot studies, this option was not available. The final version of the FDVQ was used for the first time between March and July 2022 as part of an annual measurement of citizenship knowledge, attitude, and skills using the Citizenship Competences Questionnaire (Ten Dam et al., 2011), in which schools voluntarily participate. The survey was delivered digitally.

Table 2.

Example items in the final measurement instrument.

FDVs	Example item
Equality	All people are equal, including people who live very differently from me.
Understanding of others	When somebody has a different opinion than I do, I try to understand why.
Freedom of speech	If you disagree with the government, you are allowed to say so.
Autonomy	You may decide for yourself whether you are in love with a boy or a girl.
Tolerance	It is important to respect people with a different opinion than mine.
Rejection of intolerance	I think it is wrong when somebody is not respected because of their religion.
Rejection of discrimination	I think it is wrong when somebody is discriminated against because of their disability.
Sense of responsibility	People should try not to harm others.

Before each (pilot) measurement occurred, parents received an information letter including an opt-out option. Before students filled in the questionnaire, they were ensured anonymity and allowed to quit participation at all times. After giving the instruction, students took approximately half an hour to complete the questionnaire.

Missing observations

The data that were gathered as part of the first measurement after the pilot studies (using the final measurement instrument) consisted of 3373 students (<1% missing values) from 69 schools for primary education, representing a broad range of identities (like public/private, religious), student populations and (sub)urban and rural regions. As the percentage of missing values was very small and manifested more towards the end of the questionnaire, incomplete cases were deleted from the dataset instead of using imputation methods. This resulted in a dataset consisting of 3278 students.

Results

This section reports the findings of several psychometric analyses based on advised thresholds. As such, it provides information on the quality of the instrument, such as an analysis of descriptive statistics (e.g. mean, standard deviation, answer pattern, skewness, kurtosis), correlation analyses, reliability analyses, the factor structure of the exploratory factor analyses, fit indices of the confirmatory factor analyses, an assessment of measurement invariance, response frequencies and relationships with external constructs.

Descriptive analyses

Pearson correlation coefficients between the eight scales were examined to provide insight into the relations between FDVs (Table 3). What stands out is that the correlations are all positive and predominantly moderate to high. Furthermore, the correlation coefficients between ‘rejection of intolerance’ and ‘rejection of discrimination’ and the other FDVs, particularly ‘understanding of others’ and ‘freedom of speech’, are somewhat lower than the other correlations.

Table 3.

Correlation coefficients fundamental democratic values.

	1.	2.	3.	4.	5.	6.	7.	8.
1. Equality	1.00
2. Understanding of others	0.44	1.00
3. Freedom of speech	0.40	0.31	1.00
4. Autonomy	0.62	0.32	0.47	1.00
5. Tolerance	0.61	0.40	0.47	0.68	1.00
6. Rejection of intolerance	0.39	0.26	0.24	0.30	0.35	1.00
7. Rejection of discrimination	0.43	0.21	0.26	0.35	0.40	0.54	1.00
8. Sense of responsibility	0.54	0.43	0.37	0.50	0.62	0.37	0.45	1.00

Correlations between 0.1 and 0.3 are considered low; correlations between 0.3 and 0.5 are considered medium; correlations between 0.5 and 1.0 are considered high.

Table 4 depicts the reliability coefficients, the number of items, the mean scale scores, standard deviations, and values for skewness and kurtosis. For all scales, values for Cronbach’s alpha were above the advised threshold of 0.70 (Cortina, 1993). The mean scores indicate a ceiling effect, which is unsurprising since the students who participated in this study live in a society where FDVs are generally broadly shared (van Goethem et al., 2022). Furthermore, values between −2 and +2 for skewness and −7 and +7 for kurtosis indicate a normal distribution (Byrne, 2010; Hair et al., 2010). Some values for skewness are smaller than −2, which underscores the importance of choosing a robust maximum likelihood as an estimator in the confirmatory factor analyses to account for this nonnormality (Knekta et al., 2019).

Table 4.

Reliability coefficients (Cronbach’s alpha), number of items (N), mean scale scores, standard deviations (sd) and skewness and kurtosis.

	Alpha	N	Mean	Sd	Skewness	Kurtosis
Equality	0.83	4	3.64	0.57	−1.85	3.43
Understanding of others	0.85	4	3.15	0.68	−0.69	0.28
Freedom of speech	0.71	5	3.46	0.52	−1.23	2.14
Autonomy	0.77	4	3.67	0.53	−2.01	4.20
Tolerance	0.80	4	3.67	0.49	−1.94	4.35
Rejection of intolerance	0.92	4	3.43	0.83	−1.67	1.94
Rejection of discrimination	0.95	4	3.70	0.70	−2.67	6.59
Sense of responsibility	0.83	5	3.61	0.52	−1.88	4.32

Exploratory factor analysis

Exploratory factor analysis was performed to examine how well the data fit the intended eight-factor structure. This was done by appointing eight factors and examining on what factor each item has the highest factor loading using varimax rotation. The results (Appendix 1) reveal that the data follow the intended factor structure of eight distinguishable FDVs. In two cases, the intended scale does not entirely group together: for ‘equality’, the item ‘Everybody is equal, including men who like men, women who like women and men who like women’ has a higher factor loading (0.499) outside the intended scale. In addition, for ‘autonomy’, the item ‘You may decide for yourself whether you are in love with a boy or a girl’ also has a higher factor loading (0. 884) outside the intended scale. Both items are about sexual diversity and group better together. In addition, the scales ‘autonomy’ and ‘tolerance’ have the highest factor loadings on the same factor. This may be the case because the items measuring ‘autonomy’ are about having the right to make your own decisions, whereas for ‘tolerance’, the items are about respecting others to make their own decisions.

Confirmatory factor analysis

Confirmatory factor analyses examine how well the data fit the intended factor structure. As opposed to the procedure in exploratory factor analysis, the items are fixed in the intended scales in confirmatory factor analysis. As Kline (2010) recommended, it was assessed whether the model fit was adequate before interpreting the model estimates. This was done by looking at the comparative fit index (CFI) and root mean square error of approximation (RMSEA). For CFI, a value above 0.95 is preferred (Hu and Bentler, 1999); for RMSEA, a value lower than 0.050 indicates close fit (Browne and Cudeck, 1993).

First, a first-order factor model with 34 items as indicators of eight uncorrelated FDVs was fitted to the data and yielded an unsatisfactory model fit (χ²(527) = 8067.417, N = 3278, p < 0.001, CFI = 0.782, RMSEA = 0.089, 90% CI [0.088, 0.091]). Therefore, the same model was fitted in which all FDVs were allowed to correlate. This yielded satisfactory to close model fit (χ²(499) = 2310.117, N = 3278, p < 0.001, CFI = 0.948, RMSEA = 0.045, 90% CI [0.043, 0.047]) and fitted the data significantly better than the first-order factor model with uncorrelated FDVs (p ⩽ 0.001).

Despite the satisfactory to close fit of the previous model, it was explored whether a second-order factor model with 34 items as indicators of eight FDVs as indicators of one latent construct ‘democratic attitude’ fitted the data better. This model yielded unsatisfactory to acceptable model fit (χ²(526) = 2911.581, N = 3278, p < 0.001, CFI = 0.931, RMSEA = 0.050, 90% CI [0.048, 0.052]) and fitted the data significantly worse than the first-order factor model with correlated FDVs (p ⩽ 0.001). Next, a first-order factor model was fitted with eight mean scores of FDVs as indicators of one latent construct ‘democratic attitude’. This yielded poor model fit (χ²(20) = 628.137, N = 3278, p < 0.001, CFI = 0.902, RMSEA = 0.123, 90% CI [0.115, 0.131]) and also fitted the data significantly worse than the first-order factor model with correlated FDVs (p ⩽ 0.001). These results indicate that neither the second-order factor model with items as indicators of FDVs as indicators of a latent construct nor the first-order factor model with mean scores as indicators of a latent construct fitted the data better than the first-order factor model with items as indicators of correlated FDVs.

At last, as the correlation matrix (Table 3) pointed out that ‘rejection of intolerance’ and ‘rejection of discrimination’ were less correlated to the other FDVs, a second-order factor model was constructed with 34 items as indicators of all FDVs, in which ‘rejection of intolerance’ and ‘rejection of discrimination’ served as indicators of a separate latent construct as opposed to the remaining FDVs. This yielded satisfactory to close model fit (χ²(524) = 2607.950, N = 3278, p < 0.001, CFI = 0.940, RMSEA = 0.047, 90% CI [0.045, 0.049]) but fitted the data significantly worse than the first-order factor model with correlated FDVs (p ⩽ 0.001). Therefore, the first-order factor model with 34 items as indicators of eight correlated FDVs was accepted and proceeded with in further analyses.

Assessment of measurement invariance

Multiple-group confirmatory factor analysis (MGCFA) was used to assess the measurement invariance (Meredith, 1993; Putnick and Bornstein, 2016) of the FDVQ to ensure meaningful comparisons across different groups of a population (Chen et al., 2005; Isac et al., 2019; Steinmetz et al., 2009). To do so, increasingly restricted models (i.e. configural, threshold, metric and scalar models) were fitted to the data. Achieving (partial) metric invariance is a minimal prerequisite for meaningful cross-group comparisons (Little, 2013). It justifies the comparison of latent variables and their associations across groups (Isac et al., 2019). Reaching (partial) scalar invariance justifies comparing latent means across groups (Reeskens and Hooghe, 2010). The Chi-square test of the difference in model fit (Δ χ²) was used to test the null hypothesis of equal model fit for two models. If the p-value of the Chi-square test of difference is significant, it indicates a meaningful difference between the two models in comparison, and the more restricted model is rejected. In addition, partial metric and partial scalar invariance were tested for. In these partial models, some (but not all) of the equality constraints were released again. The standardised mean residuals and significance of modification indices (using a Bonferroni-adjusted alpha level) were consulted to find out what constraints needed to be released.

Research on citizenship competences examined and found robust differences based on sex, socioeconomic position and migration background (Geijsel et al., 2012; Ten Dam et al., 2020). Scholars and schools will likely be interested in examining whether such cross-group differences also occur in students’ attitudes towards FDVs. Hence, the measurement invariance was tested across sex, home language, educational sector, grade and migration background. The students who answered the question about their sex with ‘other/I’d rather not say’ had to be excluded from the analyses because they were too small as a separate category. For sex, home language, educational sector and grade, (partial) metric invariance was reached (Appendix 2). This allows for the comparison of associations between the FDVs and the underlying items across boys versus girls, across students with Dutch or a dialect of Dutch as a home language versus another home language, across students in RPE versus SNPE, and across students in grade 5 versus grade 6 (Appendix 3). For migration background, scalar invariance was reached (Appendix 2). This allows for comparing mean scores of the FDVs across students with and without a migration background (Appendix 4).

Response frequency

The non-missing response frequency for each questionnaire item demonstrated, among others, a so-called ‘ceiling effect’: most students opted for ‘somewhat agree’ or ‘totally agree’ to indicate their attitude towards FDVs. Furthermore, the response frequencies of one of the items of ‘freedom of speech’ (i.e. ‘Journalists are allowed to decide for themselves what they write about’) and all four items of ‘understanding of others’ (e.g. ., ‘When someone has a very different opinion than I do, I try to understand why’) somewhat stand out because the positive answer options are more equally divided between ‘somewhat agree’ and ‘totally agree’, whereas for most other questionnaire items, the share of ‘totally agree’ within the positive answer options is larger than ‘somewhat agree’. The response frequencies for ‘totally disagree’ are largest for the aforementioned item of ‘freedom of speech’ and all items of ‘rejection of intolerance’ (e.g. ‘I think it is wrong when somebody is not respected because of their religion’).

Relation between FDVs and civic knowledge, attitudes and Skills

As a last analysis, the correlation coefficients between students’ attitudes towards FDVs and civic knowledge, civic attitudes and self-estimated civic skills ² were examined (see Table 5). Civic knowledge, attitudes and self-estimated skills are assumed to be related to students’ underscoring of FDVs but are measuring aspects of civic competences as operationalised and measured using the Citizenship Competences Questionnaire (CCQ) (Ten Dam et al., 2011). Students’ civic attitude, as measured by the CCQ, focuses on students’ attitudes towards four so-called social tasks: acting democratically, acting in a socially responsible manner, dealing with conflicts and dealing with differences.

Table 5.

Correlation coefficients fundamental democratic values and citizenship competences.

	Civic knowledge	Civic attitude	Self-estimated civic skills
1. Equality	0.49	0.47	0.39
2. Understanding of others	0.24	0.63	0.55
3. Freedom of speech	0.29	0.34	0.36
4. Autonomy	0.46	0.37	0.37
5. Tolerance	0.44	0.39	0.42
6. Rejection of intolerance	0.34	0.28	0.25
7. Rejection of discrimination	0.40	0.27	0.22
8. Sense of responsibility	0.42	0.48	0.48

Correlations between 0.1 and 0.3 are considered low; correlations between 0.3 and 0.5 are considered medium; correlations between 0.5 and 1.0 are considered high.

What stands out in Table 5 are the relatively lower correlation coefficients between ‘rejection of intolerance’ and ‘rejection of discrimination’ with civic attitudes and self-estimated civic skills. Furthermore, the relatively lower correlation coefficients for ‘understanding of others’ and ‘freedom of speech’ with civic knowledge’ indicate that civic knowledge does not necessarily relate to a positive attitude towards ‘understanding of others’ and ‘freedom of speech’. For ‘understanding of others’, the correlation coefficients with civic attitudes and self-estimated civic skills are relatively high, indicating a strong relationship between these constructs. The interpretation of these interrelations is discussed further in the discussion.

Discussion

For democracy to persevere, its citizens must actively maintain it. Citizens can do so by underscoring democratic values like equality, tolerance and non-discrimination (Levinson, 1999). Democratic values are widely conceived as important for democratic and pluriform societies and are anchored in international policy documents such as the UDHR. However, democratic values require active fostering (cf. White, 1999), and schools are one of the socialising agents that play a role in this. To improve the quality and effectiveness of education aimed at fostering democratic values, insight into students’ learning outcomes is important. Hence, this study aimed to examine to what extent a valid and reliable measurement instrument can be developed to gain insight into students’ attitudes towards democratic values. This was done by examining the following research question: ‘How can students’ attitudes towards democratic values be reliably and validly measured, and what can be concluded based on this measurement?’

The current study set out with a theoretical synthesis of different conceptualisations of democratic values. Most conceptualisations included (aspects of) openness towards diversity in a broad sense; human dignity and freedom; support for democratic institutions and principles, policy decisions and human rights; and communality. In the Netherlands, where the present study took place, democratic values are conceptualised as equality, understanding of others, freedom of speech, autonomy, tolerance, the rejection of intolerance, rejection of discrimination, and a sense of responsibility (Ministry of Education, Culture and Science, 2021). This set of democratic values shows similarities with conceptualisations of democratic values in the scholarly literature (Almond and Verba, 1963; Beane, 2013; Eidhof et al., 2016; Janmaat, 2018) and international policy documents such as the UDHR, ECHR, UNCRC, and RFCDC from the Council of Europe (Council of Europe, 2018). Hence, the eight democratic values were used as the conceptual framework for constructing the FDVQ.

The results provide empirical support for the validity and reliability of the measurement of democratic values in students as conceptualised in the FDVQ. Most FDVs had moderate correlations with other FDVs. This confirms that each FDV taps upon a different but related part of FDVs as an overarching construct. Overall, ‘rejection of intolerance’ and ‘rejection of discrimination’ were less related to other FDVs. A possible explanation may be that rejecting an undemocratic value of others is slightly different because it entails both yourself and (an action towards) other individuals, as opposed to the other FDVs, which only involve an individual’s compliance with democratic values. Moreover, whereas the underlying questions of ‘rejection of intolerance’ and ‘rejection of discrimination’ were purposely positively formulated to create alignment with the positive formulation of questions belonging to other FDVs, the constructs themselves do hold what can be considered a ‘negative’ (actively rejecting) component that may have mitigated the correlations with other FDVs. However, a model structure that treated ‘rejection of intolerance’ and ‘rejection of discrimination’ separately from the other FDVs did not yield the best possible model fit.

Most FDVs had moderate correlations with related constructs like civic knowledge, attitude, and self-estimated skills. The FDVs ‘understanding of others’ and ‘freedom of speech’ stood out somewhat because they were less related to civic knowledge. This indicates that students who highly underscore these FDVs do not necessarily obtain high scores on civic knowledge and, vice versa, that students who obtain high scores on civic knowledge do not necessarily highly underscore ‘understanding of others’ and ‘freedom of speech’. In this case, ‘understanding of others’ particularly stands out because its low correlation with civic knowledge contrasts its high correlation with civic attitude and skills. A possible explanation may be that the other FDVs are more content-related to fundamental rights and obligations of democratic citizens, whereas ‘understanding of others’ entails more of a ‘prosocial’ component that does not necessarily require civic knowledge and, moreover, seems strongly related to a democratic attitude and self-estimated civic skills. In addition, an explanation for the low correlation of both ‘understanding others’ and ‘freedom of speech’ with civic knowledge may be that the development of civic knowledge follows a different pattern than, for example, civic attitude. Whereas knowledge generally builds up, attitudes can shift in either direction. Nonetheless, this finding contradicts studies that show that knowledge and corresponding attitudes are positively related (Lauglo, 2013). It could be that correlations between FDVs and civic knowledge become stronger when the sample consists of older (secondary school) students who have gained a more solid basis of civic knowledge – like in the study of Lauglo (2013).

Future studies may want to compare students’ support for FDVs across different groups. Hence, measurement invariance was assessed to determine what information can be meaningfully compared across groups. The results indicated that for all FDVs, the minimum requirements were met (cf. Little, 2013). This means that the FDVQ can be used to gain information on interrelations between FDVs (e.g. students who score high on ‘autonomy’ also obtain higher scores on ‘tolerance’), to compare the associations between overarching constructs and the underlying questions across groups (e.g. the relationship between the first question and ‘equality’ is stronger for boys than for girls), and to interpret the latent mean scores for a group as a whole (e.g. on average, students in primary education obtained a mean score of 3.61 on ‘sense of responsibility’). For migration background, the findings additionally allowed for comparison of mean scores of the FDVs across students with and without a migration background. In that respect, it could be concluded that students with a migration background underscored the values of ‘equality’, ‘freedom of speech’, ‘tolerance’, ‘rejection of intolerance’ and ‘rejection of discrimination’ significantly less compared to students without a migration background. For the other FDVs, there was no difference between students with and without migration backgrounds. As there is no additional information available, such as whether the country of origin is equally or less democratic, caution is needed when interpreting these results.

The findings also pointed out that students in primary education generally highly support FDVs, with the highest support for ‘rejection of discrimination’ and the lowest support for ‘understanding others’. The high support for FDVs also manifested in other studies examining democratic values in students (Janmaat, 2018; Munniksma et al., 2017). A study in the British context, for example, found that students of approximately 23 years old supported the values democracy, individual liberty, the rule of law, and respect for people of different backgrounds and religions to a great extent (Janmaat, 2018). In addition, a longitudinal study conducted in secondary education in the Netherlands indicated that students in grade 8 (approximately 14 years old) highly supported FDVs like freedom, equality and solidarity – like they did in grade 7 in a previous measurement (Van der Meer et al., 2021). Concerning the generally high support for FDVs, it should be noted that the current study took place in the Netherlands, which is, according to the Democracy Index, ³ considered a ‘full democracy’ since the index started in 2006 (Economist Intelligence Unit, 2022). Underscoring FDVs to a high extent may be more obvious among students living in a country considered a ‘full democracy’ compared to students living in countries marked as a ‘flawed democracy’, a ‘hybrid regime’ or an ‘authoritarian regime’.

In CIVED, one of the first large-scale international citizenship education studies of the International Association for the Evaluation of Educational Achievement (IEA), a chapter is devoted to 14-year-old-students’ concepts of democracy, citizenship and government across 28 countries with varying degrees of democracy (Torney-Purta et al., 2001). The results of this study indicated that students generally considered it good for democracy to have many different organisations for people to join and to have elections. According to these students, democracy is weakened when the government is influenced by wealthy people or when the judiciary is influenced by politicians. In addition, students indicated that it is bad for democracy when citizens are not free to criticise the government. An important difference between this report and the present study is that the report asked students whether certain situations are either good or bad for democracy (e.g. ‘Is it good or bad for democracy when people who are critical of the government are forbidden from speaking at public meetings?’), whereas the present study asked students’ attitudes towards values considered fundamental for democracy (e.g. If you disagree with the government, you are allowed to say so’). Despite the difference in measurement and participants, combined, these studies indicate that students are not only able to grasp some of the most fundamental principles to the ideal functioning of democracy (Torney-Purta et al., 2001) but also have an attitude that is generally highly supportive of values fundamental for democracy.

Despite the overall high support for FDVs, our findings also shed light on the manifestation of disagreeing responses. To illustrate, in this study, ‘rejection of intolerance’ had the highest percentage of students who ‘totally disagreed’, indicating that when asked about the inclusion and acceptance of students with a different opinion, belief, behaviour or world view, relatively more students did not underscore this as compared to the other FDVs. Previous research using IEA’s ICCS data also pointed out that, despite high support in absolute terms, Dutch secondary school students underscored equal rights across sex, migration background and ethnicity to a lesser extent than in surrounding countries (Dijkstra et al., 2021; Munniksma et al., 2017; Schulz et al., 2018). The manifestation of more disagreeing responses to ‘rejection of intolerance’ in this study, as well as the differences in support between the Netherlands and surrounding countries in previous research, both underscore the importance of monitoring support for FDVs among students periodically (or even longitudinally) to detect possible changes in support – in line with the presumption that democracy is not a fixed state of government but needs to be actively maintained. The latter is also underscored by De Winter (2004), who poses that in a democracy, there is always a lingering risk of negligence and disinterest of citizens or even unvarnished attacks by those who seek to force their totalitarian value system onto everyone.

It is important to note that when asking students to indicate their attitude towards FDVs, no information is gathered on situations in which FDVs are applied in practice. Such situations may require students to choose between conflicting FDVs. In future research, the measurement can be elaborated by asking students to choose between contradictory democratic values, for example, by providing a dilemma that reflects the tension between ‘freedom of speech’ and ‘rejection of discrimination’ or between ‘autonomy’ and ‘sense of responsibility’. Such dilemmas were also used in research by Vaessen et al. (2022), who discussed three democratic issues (i.e. majority rule and minority rights, freedom of speech and non-discrimination, conditional political representation and equal political representation) with adolescents in vocational education. By qualitatively investigating FDVs in relation to each other, the findings provide useful insight into students’ consideration of democratic dilemmas when applied to everyday practice – in addition to quantitatively asking students to indicate to what extent they underscore FDVs.

This study examined whether it was possible to measure students’ attitudes towards FDVs in a reliable and valid way. It can be concluded that the FDVQ is a reliable and valid instrument that can be used to measure and monitor students’ attitudes towards eight FDVs – which are widely conceived to be important for a pluriform and democratic society to persevere. Furthermore, the findings demonstrated that students underscored FDVs to a great extent. These insights are relevant for research and educational practice and contribute to improving the quality and effectiveness of citizenship education, which prepares today’s youth to make a positive contribution to a sustainable and resilient future democracy.