Abstract
This paper discusses rationales for comparative work in education and draws on two projects on analysing publications in educational journals internationally. It uses the cases of Germany and England to illustrate the points made. The paper outlines some of the major developments in education in these two countries and identifies their implications for the work of academics in this discipline. The main argument is that the analysis of journal publications provides for tracking of the development of academic disciplines, and that the comparisons of publication patterns in two countries offers insights into the particular starting points and trajectories of a field of study such as education.
Introduction
This paper reports on the work of two projects, funded by the German Research Foundation (Deutsche Forschungsgemeinschaft), on the analysis of publication patterns in international journals of education. The detailed results of these projects have been presented elsewhere (Ertl et al., 2015; Zierer et al., 2013; Zierer and Ertl, 2014); the focus of this paper is thus on the methodological approaches and challenges that have guided the comparison of journals published in different national and international jurisdictions.
As a result of the previous work of the present authors and the funding source of the projects, the analytical framework that guided the analysis of German, English, US and European journals originated in the German tradition of ‘science research’ (Wissenschaftsforschung) and particularly in previous work on categorising papers in German journals. This starting point meant that existing analytical categories needed to be questioned and adapted carefully, in order to prevent a German-centric viewpoint occurring because of findings that were not relevant for non-German journals. The challenge was to enable the analysis to capture the specific characteristics of journals as important media for the dissemination of educational research, while still allowing meaningful comparison across the borders of different academic traditions.
In order to achieve this aim it was necessary to develop an understanding of different traditions and functions of academic journals in different academic systems. In addition, the research needed to consider which national jurisdictions could and should be compared – a decision that became particularly challenging when ‘European’ journals were included in the comparison. The question of the extent of the inclusiveness and meaningfulness of the analytical categories, which were developed for the work on the four jurisdictions mentioned, has recently come to the fore, as we have started to expand the comparison to journals in Australia and China.
Another challenging area is that of selecting specific journals in a given jurisdiction. While we have used the ranking of journals according to the impact factor of journals in the Social Science Citation Index, it is clear that this criterion has severe limitations, in particular with regard to the fact that the Index is dominated by journals published in the English language, mainly originating from the USA and the UK. This means that educational journals from most other parts of the world are hardly represented in the Index and therefore that other selection criteria needed to be developed.
The methodological discussion is inherently linked with the wider question about how far publication patterns can contribute to our understanding of the development of education as a discipline in different academic systems. To provide an answer to this question an understanding of the development of education as a social science in the past is required. Against this background this paper will also introduce some of the findings of the comparisons conducted.
Reflections on national comparisons: England and Germany
The mutual attraction between England and Germany in the area of education has a long and rich history, highlighted elsewhere (see, for instance, Ertl 2006a; Kuhlee et al., 2015). Comparisons comprise all sectors and levels of education, including teacher education, citizenship education, vocational education and training, compulsory schooling, educational reform, language learning and higher education. David Phillips in particular has made a highly important contribution to Anglo-German comparisons (see, for instance, Phillips 2011, 2015), not least by developing a structural typology of cross-national attraction in education from his own work which has become an influential conceptual framework for comparative work more generally (Ochs and Phillips, 2002).
Based on this scholarly foundation and the discussions in the wider context of the Anglo-German Educational Research Group, 1 we became interested in the question of how education as an academic discipline has developed in Germany and England. Very much in the tradition of the ‘founding father’ of modern comparative education, Marc-Antoine Jullien, the comparative perspective seems highly relevant in investigating the ways in which fields and methods of inquiry are shaped and change over time: ‘Research in comparative education must provide new means of perfecting the science of education’ (Jullien, quoted in Phillips and Schweisfurth, 2008: 1).
The development of education as a discipline in different national contexts has been the subject of discussion for a long time, with researchers arguing that scholarly self-reflection is a pre-requisite of disciplines being able to tackle current challenges (Keiner, 2011). It can be assumed that self-reflection is a key contributor to the generation of new conceptual lenses and research agendas in all academic disciplines, and that there are a number of different traditions in this respect. For instance, the German field of Wissenschaftsforschung (research on science and research), has a long history in the field of education and has been focused on analysing the different outcomes of educational research.
The question of whether education can be regarded as an academic discipline in its own right, or should be regarded as a ‘field’ of inquiry, mainly drawing on the concepts, methods and theories of a number of ‘contributory’ or ‘foundational’ disciplines, has been a matter of debate (Furlong, 2013; Keiner, 2002). The work of Wagner and Wittrock (1991) has highlighted the influence of national policy and societal structures more generally on the formation of academic disciplines, including education. According to their analysis, national differences in the formation of social science disciplines appeared in the 19th century when the role of professionals working in these disciplines became very divergent in the context of nation-building processes. Based on the analysis they identified three different patterns of development of social science formation processes along the lines of dimensions such as the extent of formalisation of disciplinary discourses, of institutionalisation of disciplines, and of structuring of disciplinary knowledge (Wagner and Wittrock, 1991: 340–351). According to this classification, education in Germany is characterised by a highly formalised disciplinary discourse concerning a systematic structure of knowledge which, in turn, contributes to a clear, independent disciplinary identity. The emphasis on theorised knowledge results in problems of relevance of disciplinary knowledge for educational practice. Keiner (2011) highlighted the importance of humanist thinking and hermeneutic approaches to knowledge generation which also contribute to the often-cited ‘theory–practice problem’ of German educational research.
In this categorisation, education in England is part of the ‘pragmatically specialised’ tradition of social sciences in the Anglo-Saxon world (Wagner and Wittrock, 1991: 345). Education as a field of inquiry is focussed on identifying and solving particular societal problems, often located in the area of educational practice. Linked to this overall orientation is an emphasis on empirical research aimed at the development of practically-relevant knowledge. An important consequence of this pragmatic and utilitarian focus (Becher, 1989: 16) is that the theoretical foundations of educational research are weaker (Keiner, 2011) and, therefore, that disciplinary identity is relatively weak. 2
Germany and England can therefore be regarded as examples of divergent disciplinary traditions in education, well suited to provide insights into the development of the discipline in the light of globalisation and internationalisation pressures. An underlying question this research has tried to answer is whether common global trends and challenges result in the convergence of similar national reactions and education systems and inquiry. This ‘world-system’ argument (see Ramirez and Meyer, 1980) is opposed by proponents of ‘path-dependency’, according to which national traditions in policy, practice and research in education will prevail; the different traditions of education in Germany and England thus appear to be a good starting point for answering this question.
However, focussing on these two national contexts also provides for deeper engagement with specific contextual developments affecting education as a discipline. For England, regular exercises conducted to evaluate of the quality of research since the 1980s have arguably been the single most decisive factor for shaping the discipline. The Research Assessment Exercise (RAE) and from 2014 the Research Excellence Framework (REF) have not only provided ‘benchmarking information and (…) reputational yardsticks’ (REF, 2012) but have also determined funding levels of university departments of education up and down the country. Since preparation for, and participation in, the exercise is a costly and, in reputational terms, a potentially risky endeavour, the number of participating departments of education has fallen significantly in recent exercises. This has led to a concentation of high-quality educational research in fewer universities. There seems to be a correlation between the size of educational departments and the quality of the research they produce, according to RAE/REF scores, with smaller departments predominantly located in the lower half of the table (see analysis in Ertl, 2016). This has contributed to the vertical differentiation of institutions active in educational research (Oancea, 2010; Furlong, 2013).
There are several other issues that have figured strongly in the work of educational researchers in England, mainly related to regulatory changes and reform in the English education system. One of these is the re-orientation of initial teacher education, from being primarily located in higher education institutions to being mainly carried out by schools themselves. This change can be traced back to Conservative government policies in the late 1980s and has been the subject of major debates in the educational research literature since that time (e.g. Hulme and Menter, 2015).
Another issue that has been influential in educational research is an almost constant reform of qualifications outside the main academic routes. A good example of this is the project of the Labour government in the mid-2000s to introduce so-called 14–19 Diplomas. This initiative must be seen in the context of a long-standing discussion on qualifications at secondary level, underpinned by concerns about low achievement in pre-16 education, traditionally low rates of staying in full-time education after the age of 16, a perceived lack of choice for young people in terms of programmes available, and a concern that there were not enough viable ‘vocational’ options for those not well served by ‘academic’ options (Ertl and Stasz, 2010). Like so many attempts to reform qualifications outside the academic route in the UK, the Diplomas never took off and funding for them was withdrawn immediately when the Coalition government of Conservatives and Liberal Democrats came to power in 2010.
In the German context, an important point in the discussion about the future development of education was the OECD Programme for International Student Assessment (PISA), first published in 2001. Due to the fact that the German school system was only in the middle of the rankings, education at all stages and in all contexts became the subject of severe criticism from politicians and society at large. One of the consequences was a stronger orientation towards the international discourse about schooling, teaching and learning. In the following, a few examples are provided for illustration.
In the field of elementary education in Germany, systemic reforms to practice were initiated – for instance, the increased introduction of concepts that enabled schools to offer teaching and other activities for pupils for the whole day, rather than only the traditional half day. In the academic area, more places for elementary education were created in higher education in order to develop a stronger research base for this sector. In secondary education, the traditional tripartite system was reformed in a number of the 16 German Federal States. In most cases this meant creating a two-track system, with the academic Gymnasium keeping its position as the main route into higher education and increasingly into the attractive parts of dual system vocational training. In a number of Federal States the duration of Gymnasium education was reduced from nine to eight years (following four years of primary education). For the second track, the traditional structures of Hauptschule and Realschule were merged into a new school type, preparing pupils mainly for the vocational component of the upper secondary and tertiary system.
Interestingly, the change into a two track system has proven less controversial than the reduction of years it takes pupils to achieve the Abitur via the Gymnasium route. This latter change is still highly controversial and some Federal States, for example Bavaria, have decided to move back from eight to nine years in the Gymnasium.
While PISA can be seen as the root cause of these far-reaching reforms of the education system, its impact on education as an academic discipline has been similarly dramatic. In the aftermath of PISA there was an increased emphasis on empirical approaches to educational research and the political relevance of the findings of these kinds of studies has changed fundamentally the landscape of research in Germany. While traditionally the so-called geisteswissenschafliche Pädagogik dominated the discourse, with Wolfgang Klafki and Hartmut von Hentig arguably the most well-known recent researchers in this tradition, the ‘empirical change’ (reminiscent of the ‘realistic change’ proclaimed by Heinrich Roth) in German educational research is in full swing. The post-PISA academic discourse in Germany can be characterised by the re-orientation of educational studies towards a greater emphasis on empirical research of pedagogic practice (empirische Unterrichtsforschung).
One of the primary subjects of the empirically-oriented research in education is that of educational standards. In nearly all fields and contexts educational standards have been developed and implemented, and are subject to regular evaluation. Traditionally, curricula for German schools prescribed the aims and content of instruction in great detail. This is due to the heritage of didactical theory in education (bildungstheoretische Didaktik) for which the selection of content was decisive. The introduction of national educational standards breaks with this tradition and aims to build a new guiding principle for educational control which also entails a new role for the curriculum. 3 In order to develop and evaluate educational standards a whole new suite of institutions and organisations was set up, in most cases connected to, but not part of, higher education institutions; the balance of power in the German academic discourse on education has changed dramatically.
Another important issue that has shaped the discourse on education in Germany in the last two decades has been the impact of the Bologna Process on German higher education. After reluctance on the part of some universities and some disciplines to engage with the suggested changes, the introduction of a new degree structure and accompanying measures such as study credit frameworks has changed the higher education landscape in Germany markedly. Apart from changes that are directly linked to the aims mentioned in the 1999 Bologna Declaration itself, it can be argued that the Bologna Process revitalised earlier initiatives to modernise German higher education, but had not produced successful tangible outcomes (see Ertl, 2013). Developments such as the introduction of accreditation agencies and measures to increase institutional differentiation (most importantly through the Exzellenzinitiative) and the strengthening of institutional autonomy of universities have been the subjects of research and development work of academics at higher education institutions and specialised research institutes. More recently, the same can be said for strongly increasing participation rates in higher education, questioning the relationship between this form of tertiary education and vocational education and company-based career pathways.
These are just a very few examples demonstrating some of the characteristics of education as a policy area in England and in Germany and the implications of changes for education as an academic discipline. In England there seems to be a tendency to undertake far-reaching, relatively quick and repeated policy interventions, often following party politics and ideology. For education as an academic field it can be expected that these characteristics will have far-reaching consequences. England therefore seems to be a good national context for comparison with the German system which has, arguably, proved to be more stable, more deliberate and politically consensus-oriented. However, as is demonstrated here, Germany has also experienced some fundamental re-orientation in its educational set-up, providing plentiful challenges and questions for educational researchers to work on.
Why compare educational journals, and how to select them?
Academic journals are important means of academic communication and dissemination of research findings in all disciplines. Articles in high quality journals are used by academics to demonstrate their productivity and standing in their disciplinary field and, in turn, these are important factors in academics’ career progression. Publication-based theses are replacing the traditional research monograph as the main product of doctoral education.
In the institutional context of higher education, the quantity of papers published in journals is regarded as a quality indicator of research quality and is used in national and international university rankings as a proxy for research output and dissemination. In this context, the number of times a piece of work is referred to by other researchers is an important indicator, often operationalised in citation indices.
Research in this area has shown that academics increasingly regard journal publications as a precondition for winning external research grants and being successful in the competition for prestigious university chairs (Söll et al., 2014). Observers have noted that this trend is part of the bigger picture of the ‘commercialisation of universities’ (Kieser, 2010: 347) and of academic research. In this context, newly established and implemented university reward systems seek to increase the research output at institutional and individual level. Klusmeyer et al. (2011) have highlighted some of the problems linked with measuring the amount and, more importantly, the quality of research output.
Academic journals play a particularly important role in the assessment of research output; they have almost normative power over what is accepted, and is important, as relevant, legitimate and important knowledge in academic disciplines. It is therefore no surprise that the number of academic journals has increased substantially (see evidence provided by the Australian Research Council, 2010), often in newly developed sub-disciplines.
Focusing on what gets published in academic journals therefore seems to be highly relevant in any attempt to understand dominating themes and discourses in an academic discipline. The comparative analysis of what gets published in educational journals can shed light on the differences between research approaches and foci in different countries, of changes in publication patterns over time, and of gaps in the research literature. Conceptually, the comparative investigation provides for assessment of the extent to which education research has either become an international endeavour or remains very much focused on national systems, of the extent to which dominant schools of thoughts are emerging, and to what extent convergence pressures (initiated, for example, by the PISA studies or the Bologna Process) have changed publication patterns in this field.
There has been some comparative analysis of journal publication patterns internationally. For instance, the analysis conducted by Leschinsky and Schoepflin (1991) showed clear differences in the conventions and self-perceptions of academic journals in different cultural spheres, very much in line with the traditions of social science disciplines in different countries identified by Wagner and Wittrock (1991). This path dependency is challenged by proponents of the world-system approach (Schriewer et al., 1999) who argue that the pressures of internationalisation, at work in similar ways in all national contexts, are necessitating similar responses, resulting in convergence of educational systems globally.
Visibility and impact of journal publications is measured by the citation indices. The journals covered by the projects reported here were selected on the basis of their high ranking in the Social Science Citation Index (SSCI). The SSCI is produced by Thomson Reuters and is widely regarded as an indicator of high international visibility, and therefore appears to be particularly important (Schmidt and Weishaupt, 2008; Zierer, 2011).
The journals selected for this study were the highest-ranked journals published in the two countries at the beginning of 2009. Table 1 shows the impact factors in 2014 of the six selected journals and the associated ranking. Impact factors can change rapidly from year to year – as an illustration Figures 1 and 2 show the impact factors of the highest ranked journals in the two countries for five consecutive years. In order to allow for better long-term comparison, a five-year (2009–2013) impact factor average is also provided in Table 1. It can readily be seen that the three English journals have higher impact factors than their German counterparts. Indeed, it needs to be noted that at the start of this work in 2009 there were only three German journals listed in the SSCI. This in turn highlights one of the limitations of citation indices such as SSCI, namely the underdeveloped coverage of journals not published in English.
Citation data for selected journals*.
Source: Web of Science.
The Impact Factor (IF) of Journal A for 2014 is calculated by dividing the number of times articles of Journal A published in 2012 and 2013 were cited in other journals in 2014 by the number of papers published by Journal A (in 2012 and 2013). See: www.webofknowledge.com.
The 5-year IF applies the same formula as used for calculating the IF but includes citations for papers published in the five previous years (2009–2013).
ZfE impact factor (2010–2014) (source: Web of Science)
BERJ impact factor (2010–2014) (source: Web of Science)
However, the SSCI provides for further analysis of citations, including the identification of journal self-citations – that is, authors of a paper citing papers in the same journal, which can be seen as a measure of a self-referential ‘inward-looking’ attitude of authors. Table 1 provides these data for the six selected journal, showing that all three German journals have a much higher proportion of self-citations. If these self-citations are taken out of the equation, the differences in impact factors between English and German journals becomes even bigger.
The criterion of high SSCI ranking was applied in combination with the aim of the study to cover general journals of education that encourage submission of articles on a broad range of educational topics and research approaches (rather than journals with a narrower thematic and/or methodological focus). The Oxford Review of Education, for instance, includes in its editorial statements the journal’s commitment to ‘a wide range of academic disciplines in the service of educational scholarship, and the Editors welcome articles reporting significant new research as well as contributions of a more analytical or reflective kind’ and that the journal is ‘seeking to publish articles which will be of interest and utility to a wider public’. 4 Equally, the Zeitschrift für Erziehungswissenschaft describes itself as ‘a representative scientific journal for the entire discipline of education’. 5 The mission statement of the Zeitschrift für Pädagogik highlights the relevance of the topic covered for the general public. 6 The British Journal of Sociology of Education and the Zeitschrift für Soziologie der Erziehung und Sozialisation seem to be comparable because of their focus on sociological aspects of education.
It is acknowledged that the SSCI can only provide a crude quantitative measure for the importance of journals and that it is subject to a number of limitations. For instance, the dominance of journals published in English raises the question of the extent to which different publication languages limit the validity of comparative analysis. However, SSCI did provide a first indication of journals widely received and used by educational researchers. In addition, the context of the project meant that only a small number of journals could be selected for analysis and so it was not possible to cover the entire range of educational research represented in the ever-growing number of journals in the field. However, by selecting well-established, high-profile journals dedicated to cover the entire spectrum of education as a discipline and analysing all (rather than a sample of) papers published over a ten-year period it is hoped that the data generated provide a good basis for a broad overview of publication patterns and trends in different jurisdictions. It was beyond the remit and scope of the project to aim for representativeness of journal publications in a more general way.
Other challenges for comparing journal publications internationally come from the definitional issues – such as what count as English, as opposed to UK or British, journals. The evolving process of devolution in the UK has increased the independence of educational policy-making in Wales and Northern Ireland in recent years; Scotland has always had wide-ranging autonomy in the area of education. We therefore decided to refer to ‘English’ journals, though this does not mean that the articles covered in the English journals do not refer to topics that are relevant for the UK more generally. In a similar way, the three ‘German’ journals have a significant share of authors who work in other German-speaking countries, particularly Switzerland (see Zierer et al., 2013).
Coding material written in two different languages and published in two different national contexts raises the challenge of ensuring culturally-neutral analysis. In order to face this challenge, the project was conducted by means of collaboration between two research teams, one based in Germany and one in England. Furthermore, contrary to most other studies of this kind, inter-coder reliability tests were conducted, for which papers in a sample were coded independently by two coders from different countries. On the basis of the number of incidences in which the coding of the two coders matched, Cohen’s kappa was calculated, resulting in a value of 0.87 overall. According to Hedderich and Sachs (2012) this value indicates a good level of clarity of the analytical categories and reliability of coding. 7
Conclusions: what comparison can tell us – some findings on the selected journals
Embedded in social worlds, education systems and research reflect distinct national trajectories. The aim of this paper was to highlight some of the challenges in comparing two contrasting traditions of educational research. Whereas educational research in England exhibits a multidisciplinary and pragmatic character, German educational research has traditionally been focused on pedagogy based on a strong notion of humanities-oriented research. However, there is evidence in both countries of a growing demand for evidence in educational research, this demand coming from, inter alia, the sphere of policy-making. This dynamic has resulted in increased funding for specific types of educational research, often organised in large programmes of inquiry. Many of these programmes are associated with conducting and evaluating newly implemented, mandatory large-scale assessments of learners’ achievement at different stages of education.
These changes have reshaped the field of educational research internationally and suggest similar research agendas in different countries. In order to track research agendas in Germany and England, our project created a new and comprehensive original dataset of publications in six educational journals over a ten-year period (2001–2010), comprising 978 articles written by 1209 authors in the three German journals and 1156 articles written by 1576 authors in the three English journals. We conducted our analysis in three areas: authors, methods and themes (see Zierer et al., 2013). Our findings provide some evidence for research contexts becoming more similar internationally. For instance, the strong emphasis in the German context on the single, male author writing in a non-empirical tradition has been decreasing over the ten years of our investigation. One might see this as an indication of the pressures on academia initiating changes to academic work patterns and contexts. In contrast, there are also clear indications of path-dependency, with some of the differences in publication patterns in the two countries, for instance in terms of the main themes covered in journal articles, not diminishing at all. The partial nature of convergence is characterised by a tendency towards more investigation of educational patterns transcending singular educational stages, the appearance of a clear focus on multidisciplinary work, and an emphasis on issues related to learning, and the teaching of learning processes, in different contexts. At the same time, there are still signs of the traditions of pragmatically specialised social science disciplines in Anglo-Saxon countries, and formalised disciplinary discourses in the German-speaking countries (Wagner and Wittrock, 1991) reflected in the journal publication patterns we found.
To conclude, we want to return to the question posed in the title of this paper: Why comparing? Our argumentation tries to illustrate a famous thought of Wilhelm von Humboldt – the well-known philosopher, linguist and educational politician whose 250th birthday anniversary we are celebrating this year. One of his most cited sentences is: ‘He who does not know foreign languages, does not know his own language’. The analysis of journal papers in different countries is a point in case: without looking at scholarly activities and their outcomes in another country the strengths and weaknesses of one’s own academic context may not be readily apparent. Comparing helps to provide insights into educational research which do not otherwise become visible. Comparing structures, process and outcomes opens the mind for reflection and cooperation across cultural boundaries. The most powerful resource possessed by people in general, and thus also academics, is their collective intelligence.
Four main conclusions can be drawn from this work. First, it seems more important than ever to accept the opportunity of comparing. In a world full of segregation and nationalistic tendencies comparing can help in creating common understanding and togetherness. Second, the young generation must be given the opportunities to develop the competences and attitudes required for conducting comparisons. In the area of educational research, this means that comparing theory and practice has to be a central theme of learning in higher education. Third, work on the questions of how and why comparing must be an ongoing task. Comparative education has to be responsive to changes in societies and the world of science, and these changes need to be reflected in the methods used for comparisons. Finally, fourth, we need to create structures that make all of these aspects of comparative work possible. If money for educational research is channelled exclusively into narrow, empiricist, quantitative investigations into ‘what works’ in educational practice, the fruits offered by comparative insights will not be harvested.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research that underpins this paper was funded by two awards from the German Research Foundation (Deutsche Forschungsgemeinschaft).