“It’s the Method,Stupid.” Interrelations between Methodological and Theoretical Advances: The Example of Comparing Higher Education Systems Internationally

Abstract

This article argues that strong interrelations between methodological and theoretical advances exist. Progress in, especially comparative, methods may have important impacts on theory evaluation. By using the example of the “Varieties of Capitalism” approach and an international comparison of higher education systems, it can be shown that the use of improved data and of advanced analysis techniques changes the acceptance or rejection of the theoretical approach. As a conclusion, the article advocates data improvements, the use of advanced methods, more generally a methodologically reflexive interpretation of comparative research results as well as a closer cooperation between theory and method development.

Keywords

Higher education Varieties of Capitalism international comparison comparative methods structural equation modeling OECD student figures theory development methodology falsificationism

Introduction

International comparisons in higher education have been gaining in importance over recent years. The reasons are manifold. An important one is globalization and an increase in international exchange. A second reason is the improved availability of comparative data; for example, from administrative sources (e.g. the Organisation for Economic Co-operation and Development (OECD), World Bank or the United Nations Educational, Scientific and Cultural Organization (UNESCO)) or from international surveys (Eurostudent, REFLEX). Policy, thirdly, is increasingly referring to findings from these data as a base of evidence-based policymaking or for legitimation purposes. Against this background, it is becoming ever more important to be aware of the methodological problems of quantitative international comparisons. The same is, however, also true for theory development. As most methodologists argue, theories in a field of study cannot be better than the applied methods. This holds at least in empirical sciences, such as research into higher education. The improvement of methods and data availability are, therefore, important ongoing requirements.

The following sections exemplify this claim by using the example of the “Varieties of Capitalism” (VoC) approach (Hall and Soskice, 2001b). It will be shown that the rejection or approval of hypotheses, derived from this theory, are heavily dependent on the data and analysis techniques used. Before delving into the data in more detail, however, first some general remarks on the importance of and problems with international comparisons will contextualize these data analyses.

The next section will, therefore, very briefly address some methodological issues on the relation between theories and data in general and introduce international comparison as a method, followed by some general problems of this method. This lays the ground for the empirical analyses in the subsequent section. The paper ends with some conclusions on further improvements with regard to the usage of data in international comparative research.

Theoretical and methodological advancements in international comparison

As already mentioned, research into higher education is an empirical science. Its theories and data want to describe and explain adequately, or one could even say “make sense of” a certain aspect of reality (higher education) and even aim in many cases at influencing this reality. As such, its statements (theories, hypotheses) relate to an empirical world and may fail when confronted with it. This understanding of empirical science refers back to Karl Popper’s (1935) idea of falsification and its further development by Imre Lakatos (1970), whose idea of “research programmes” claims, inter alia, that the evaluation of a theory and its validity also requires the evaluation of the methods applied for its measurement. Following from this, many methodologists see measurement instructions necessarily as being part of a theory (e.g. Kromrey, 1998). Ergo: advancements in methods can contribute substantially to advancement in our theories. On the other hand, theories are needed to make sense of empirical data, which are meaningless without interpretation. One important argument of this article is, therefore, that we need advancements in data and methods to improve our theories and vice versa, or what Marsh and Hau (2007) have called “substantial-methodological synergies”, to improve overall progress in (comparative) higher education research. It is, therefore, reasonable to look at the potential and problems of comparative methods in more detail to also improve theories in comparative and international education.

International comparison as a method and its problems

Comparison is one of the most important methods of empirical social research. Indeed, classics of sociology call the comparative method “the sole one suitable for sociology” (Durkheim, [1895] 1982: 147). In sociology, a comparison has to fulfil at least three requirements to be called a method in its own right (Dogan and Pelassy, 1990):

two or more units that each represent a specific context are explicitly compared;

the cognitive interest is directed towards an additional benefit through this comparison;

the used methods take into account the specific requirements of comparisons.

With regard to the first point, while one can compare quite different types of units such as organizations, regions and so on, the international comparison of countries or states is especially important in (higher) education. One reason is that (higher) education systems are still heavily regulated by their nation states (e.g. Teichler, 2007), despite globalization and internationalization trends (Hoelscher, 2012a). Another reason is the important role that education played, and probably still plays, in the creation of modern nation states (Gellner, 1983).

The second point above refers to an “additional benefit” of comparisons. Immerfall (1995) mentions four such benefits or functions: the first one is description. Higher education participation rates, for example, of 30% can only be assessed as high, medium or low when compared to other figures. However, what is often not fully understood is that such relatively easy comparisons are often more demanding with regard to representativeness of data than most kinds of relational analyses.

A second function, generalization, tests whether empirical relationships (approved hypotheses) found in one context can be observed in other contexts as well. If that is not the case, comparisons, third, can be used to specify hypotheses. Different contexts or their characteristics can then become moderator variables (e.g. Andersen and Van de Werfhorst, 2010). However, such tests need a clear theoretical basis to be able to reduce the number of potential influences. Last but not least, comparisons can be used for elucidation and for showing alternatives. This function is strongly related to David Phillips’ work on policy borrowing and transfer (Phillips, 2005; Phillips and Ertl, 2003; Phillips and Ochs, 2004) and is probably best served by qualitative comparisons that can elaborate in much more detail on the different alternatives and their inherent mechanisms.

Problems in quantitative comparative research

If, as argued so far, international comparisons are highly relevant for theory building as well as policy formulation in higher education, the potential problems also have to be taken into account. Because the literature on the problems of quantitative international comparisons is extensive, only two very general problems are discussed here: data availability and the question of equivalence (or invariance).¹

International quantitative comparisons are resource intensive, and the better they are, the more resources they normally need. A good example for such a huge research infrastructure is the European Social Survey. Following from this is, first, a lack of comparative data overall. Data availability has improved considerably during the last few years in higher education research (Teichler, 2008; Trahar, 2013); for example, through the additional efforts of international organizations such as the OECD, Eurostat and UNESCO, or through comparative surveys such as REFLEX (Allen and Van der Velden, 2007), CAP (Teichler et al., 2013) or EUROSTUDENT (Hauschildt et al., 2015). Nevertheless, it remains impossible to address many empirical and theoretical questions due to missing data. Many studies, therefore, resort to secondary analyses of the few available datasets. While this is in principle a useful strategy, Roose (2013) mentions two important shortcomings of when a research discipline is confronted with the problem of only few data sources: error multiplication and path dependency. Error multiplication denotes the issue that problems with the data (e.g. biased samples) are reproduced over and over again if many different studies draw on the same dataset. What might look like a new interesting research finding by different authors might then only be a data issue. Path dependency denotes the problem that research and related theory development progresses only in a direction where data are available. Therefore, in a bad-case scenario, data availability, and not relevance or scientific curiosity, might become the driver of scientific progress.

The second problem of comparative research, equivalence or invariance, is equally severe.² While we are used to comparative classifications of educational degrees (e.g. International Standard Classification of Education [ISCED], Comparative Analysis of Social Mobility in Industrial Nations [CASMIN]), many other issues are difficult to compare, and even with these classifications, it is questionable to what extent, for example, something as seemingly similar as bachelor degrees are really comparable across (or even within) countries. Similar problems exist with comparative survey data, where we are well aware that small changes in the instrument (questionnaire) or in sampling may result in huge biases (e.g. Harkness et al., 2010). Different forms of harmonization can be applied, but do have their problems as well (e.g. Hoffmeyer-Zlotnik and Warner, 2012). While all of these problems are well known, in practice, international comparative data is too often assumed to be already comparable. However, as the next section will show, this assumption is often not met.

Testing the Varieties of Capitalism (VoC): different data, different methods, different results

As elaborated above, empirical sciences such as education use theories to describe, explain and maybe even influence phenomena observed in the real world.³ In the following, a specific comparative theory, the VoC approach by Hall and Soskice (2001b), will be tested using different data and different methods. By showing that the acceptance or rejection of the approach depends on the used data and method, the interrelatedness between (comparative) methodological issues and theory development becomes apparent.

Introducing the VoC approach

The VoC approach by Hall and Soskice (2001b) is the most popular theory in the field of Comparative Capitalisms (see Jackson and Deeg (2006) for an overview). Their idea is that different types of organizing a capitalist market economy may be equally successful. The main characteristic for their differentiation are the predominant means for the coordination of companies and their environment (other companies, the labor force, customers, finance, etc.). In their analyses, they differentiate especially between two types, the Liberal Market Economies (LMEs) and the Coordinated Market Economies (CMEs). While in LMEs (e.g. USA, UK, Australia) prices, competition and the free market are the main instruments for coordination, in CMEs (e.g. Germany, Austria, Switzerland, but also the Scandinavian countries) contracts, networks, etc. additionally play an important role. It is not possible to give full credit to this theory here, but two more things are important to mention. First, complementarities across different societal subsystems (e.g. finance, welfare state, education) exist. “Two institutions can be said to be complementary if the presence (or efficiency) of one increases the returns from (or efficiency of) the other” (Hall and Soskice, 2001a: 17). Second, due to these complementarities, the corresponding countries gain comparative institutional advantages in certain economic fields.

While this theory is quite popular, it is by no means undisputed. Important questions refer, for example, to the number of different “varieties”, to the stability of these types and to the kind of complementarities observable (e.g. Hancké et al., 2007). It is, therefore, important to see to what extent this theory is compatible with empirical evidence and able to explain differences across countries.

In the field of higher education, if complementarities also exist within this field, which seems to be a reasonable assumption, then one should be able to differentiate higher education systems along their respective VoC (see Hoelscher, 2016). To date, research on education in the context of the VoC approach has mainly concentrated on vocational education and training (VET) (e.g. Culpepper and Finegold, 1999), but the theory also allows to deduce hypotheses about expected differences in higher education systems, which can be tested empirically. While Graf (2009), Hoelscher (2016) and Leuze (2010) find more or less strong support for a strong relationship, Wentzel (2011) is more skeptical.

The next sections cannot provide a full test of the VoC approach, but they want to illustrate the dependency of decisions regarding this theory on the methods and data applied. This will be exemplified by two different hypotheses.

Student figures in international comparison

One hypothesis that can be derived from the VoC approach is that one would expect there to be more Higher Education (HE) students in LMEs than in CMEs.⁴ The main reason for this is that CMEs “typically make extensive use of labour with high industry-specific or firm-specific skills” and therefore “depend on education and training systems capable of providing workers with such skills” (Hall and Soskice, 2001a: 25). LMEs, on the other hand, encourage individuals to invest in “general skills, transferable across firms” (Hall and Soskice, 2001a: 30). Industry- or firm-specific skills are normally attributed to VET, while HE is supposed to deliver more general and transferable skills (see also Amable, 2003: chapter 4.5; Brown et al., 2001).

To test this hypothesis, student figures from the OECD can be used.⁵ The first question, however, is, how to operationalize the dependent variable exactly, as the OECD provides many different figures. For example, data on study entrants (again differentiated in “first-time” and “first-degree” students) or on graduates are available. Because the main focus of the VoC approach is the labor market and not the educational system, it could be argued that figures on graduates are more appropriate.⁶ Table 1 presents the figures of the years 1995, 2000, 2005 and 2008 for all LMEs and CMEs that are mentioned by Hall and Soskice and where data is available.

Table 1.

Tertiary graduation rates in international comparison (in % of the relevant age cohort).

	Tertiary graduation rates (type-A level)				Adjusted for international students
	1995	2000	2005	2008	2009
Austria	9.7	15.3	20.4	25.0	26.4
Denmark	25.2	37.3	45.5	46.8	44.0
Finland	20.3	40.8	47.3	62.6	-
Germany	13.9	18.4	19.9	25.5	26.7
Japan	25.4	29.4	36.7	39.4	39.6
Netherlands	28.5	35.1	42.1	41.4	39.9
Norway	26.2	37.4	40.7	41.5	39.0
Sweden	24.0	28.1	37.7	39.9	33.0
Switzerland	9.5	11.9	27.4	32.4	-
CME ^a	20.3	28.2	35.3	39.4	35.5
Australia	–	35.7	49.7	48.5	33.9
Canada	27.2	27.2	29.0	36.6	34.3
Ireland	–	30.5	37.7	46.1	-
New Zealand	32.7	50.3	51.3	48.3	40.3
UK	–	42.0	46.6	47.9	35.6
USA	32.7	34.4	34.2	37.3	-
LME ^a	30.9	36.7	41.4	44.1	36.0

The unweighted country means are given.

Source: OECD (2011a) and own calculations; data for Belgium and France are missing totally, and for Australia, Ireland and the UK in 1995.

If we examine the first three columns of Table 1, the data support the hypothesis. While there is a convergence over time (the difference decreases from 10% in 1995 to 5% in 2008) and quite some heterogeneity within the groups (especially for the CME), the expected differences between LMEs and CMEs persist.⁷

Since 2009, however, the OECD has provided more specific data, allowing for the consideration of international students. As the hypothesis is mainly aimed at national students, it makes sense to use these improved data without international students. If one adjusts graduation figures in this way (shown in the last column), the difference between LME and CME totally disappears.⁸

The assessment of the hypothesis on graduation rates derived from the VoC approach is, therefore, dependent on whether one uses the “old” or the “new” data, with the latter leading to a preliminary rejection of the VoC approach. However, the now visible larger share of international students in LMEs would also be concordant with the VoC approach (Graf, 2009). Therefore, the new data can lead to the formulation of new hypotheses, which could be seen as a kind of theory refinement.

Comparing work orientations of graduates

Another hypothesis that can be derived from the VoC approach is that graduates from LMEs should emphasize “career” as a work orientation more strongly than graduates from CMEs. Without being able to explain this here in more detail, one reason is that studying is seen much more as an investment in one’s individual human capital in the former countries that should pay out by better career opportunities.

One data source to test this hypothesis is the REFLEX study (see www.reflexproject.org). It is a large comparative survey of more than 30,000 respondents from 12 countries from the year 2005.⁹ Graduates from higher education (ISCED Level 5a) were surveyed five years after their graduation in the academic year 1999/2000 to get a picture of their route of study as well as their transition into the labor market (Allen and Van der Velden, 2007; Arthur et al., 2007; Schomburg and Teichler, 2006).

The REFLEX study contains 10 items on work orientations. Allen and Van der Velden (2007: 242 ff.) combine these into three distinct factors via an exploratory factor analysis (see Appendix 1): career and status orientation; professional/innovative orientation; and social-oriented values.¹⁰ This is, in principle, a useful and common way of a) reducing data complexity and b) measuring complex issues, such as values, in a more reliable way. In a second step, Allen and Van der Velden compare the countries with regard to their average scores on these three factors.

Unfortunately, the REFLEX study contains data on only one LME, the UK. On the other hand, data on seven CMEs (Austria, Belgium, Finland, Germany, Netherlands, Norway and Switzerland) are provided. Comparing these countries once more supports the hypothesis: the UK exhibits an average score of around 0.2, while all CME countries have below-average scores (Austria ≈ −0.07, Belgium ≈ −0.11, Finland ≈ −0.01, Germany ≈ −0.21, Netherlands ≈ −0.12, Norway ≈ −0.21, Switzerland ≈ −0.28).¹¹ With regard to the other factors, Switzerland and Austria have especially high scores in professional values, and the UK has relatively low scores; Norway has relatively high figures in the social values. The other values are relatively similar across these countries.¹² On the basis of Allen and Van der Velden’s analysis, one would again accept the VoC approach.

While the OECD data in the first example stem from official administrative sources, surveys such as the REFLEX study in this second example are another important means of making international comparisons. However, survey data, and especially international comparative surveys, are prone to different kinds of error. Our hypothesis acceptance should, therefore, only be provisional.

One specific problem of comparative surveys is the question of equivalence or invariance (Deth, 1998; Harkness et al., 2010; Przeworski and Teune, 1966): does the instrument really measure the same in different countries? There are different techniques that allow testing for measurement invariance (Braun and Johnson, 2010). The standard procedure is a Confirmatory Factor Analysis (CFA) or Multigroup Structural Equation Model (SEM) (Byrne, 2010; French and Finch, 2006), which tests whether a set of manifest indicators (e.g. items in a questionnaire) measure a latent construct (e.g. work orientations) in a similar way across different groups (in this case, countries).

Comparing groups … with respect to hypothetical constructs requires that the measurement models are equivalent across groups. Otherwise, conclusions drawn from the observed indicators regarding differences at the latent level (mean differences, differences in the structural relations) might be severely distorted. (Temme and Hildebrandt, 2009: 138)

Equivalence is especially relevant in cases where the different contexts of the groups may potentially moderate the relationship between the manifest indicator and the latent construct (Temme and Hildebrandt, 2009), as explained previously with regard to the third function of comparisons.

Work orientations can be seen as such complex latent constructs. The question, then, is, whether the REFLEX items measure the same in the different countries and whether mean comparisons, as described above, are allowed and appropriate.

With regard to invariance, different levels are distinguished in the context of CFA (e.g. Temme and Hildebrandt, 2009): configural invariance means that the number of latent constructs and the assignment of manifest indicators to these latent constructs are the same in all groups. Metric invariance requires, furthermore, that factor loadings of the manifest indicators are equal across groups. Scalar invariance, also needs equivalent constants of each manifest indicator across countries. This is the prerequisite for mean comparisons of the latent concepts.

Figure 1 shows the results of a pooled CFA with the three latent concepts career, professional, and social values and the assignment of the 10 items (corresponding to the three factors by Allen and Van der Velden). Latent concepts are represented by ellipses, the manifest indicators by rectangles. The “e1”–“e10” on the right-hand side are the error terms, as the indicators are not fully explained by the latent constructs. The factor loadings for the pooled sample (respondents from all countries put together) are given at the single-headed arrows between ellipses and rectangles. The double-headed arrows between the latent concepts display their correlations. For example, “professional orientation” is best represented by the item “new challenges” and less well represented by “work autonomy”.

Figure 1.

Pooled CFA model for work orientations.

To check for configural invariance, this model is estimated in all countries separately. Factor loadings, etc. are set free and can, therefore, be different for each country. CFA offers a range of measures to evaluate whether a theoretical model fits the data. Traditionally, a chi-square test is provided. However, this test statistic is extremely sensitive to sample size (e.g. Lei and Wu, 2007: 36). Therefore, alternative goodness-of-fit indices are usually also given. Two of the most common ones are the comparative fit index (CFI) and the root mean square error of approximation (RMSEA). As a rule of thumb, values above 0.9 for the CFI and values below 0.06 for the RMSEA are seen as indicators of a good fit between model and data.

The fit indices for the model of configural invariance are CMIN/DF = 190.8, CFI = 0.839 and RMSEA = 0.085. While the chi-square test is, as expected, extremely high (test theory suggests that the value should not exceed 3), CFI and RMSEA are only slightly lower/higher than the threshold. Therefore, the model does not fit very well to the data, but due to further inspection (no single parameter or modification index stands out in particular) one might accept a reasonably similar structure in all countries.

Adding similar factor loadings across all countries as additional constraints results in metric invariance. To compare different nested models, Byrne (2010: 221) recommends, with reference to Cheung and Rensvolds (2002), to use the increase in the CFI (ΔCFI) for evaluation. The ΔCFI should not lie above 0.01. The increase for the model of work orientations analyzed here is 0.012, exceeding this threshold only slightly. Taking into account the large number of countries, this might again be acceptable. Metric invariance of the model might, therefore, be assumed as well.

Scalar invariance additionally requires similar constants for each manifest indicator across all countries. Testing this model results in ΔCFI = 0.309, well above the threshold, which means that the assumption of scalar invariance has to be rejected.

As a result, mean comparisons of the latent constructs across countries, as presented above (factors or work orientations), are, therefore, not adequate. A decision about a preliminary acceptance or rejection of the second hypothesis is, therefore, not possible on the basis of these data (or at least needs additional analyses, i.e. in the context of “partial invariance”).

Conclusion

The question of the special issue “Why compare?” was addressed in a double sense in this paper. First, it was argued that comparisons are necessary for theory development and improvement. Only by confronting our hypotheses with empirical data can we reject, (preliminarily) accept or refine our theoretical assumptions. (International) comparisons especially allow for transferring or specifying hypotheses to different contexts. On the other hand, observed differences between countries can best be interpreted in the light of theory. The VoC approach was used to exemplify this interrelation between theory and international comparative data in the field of higher education.

However, on a second level, it was argued that it is also necessary to compare different data sources and different techniques of data analysis, as international comparisons are especially prone to data difficulties. Two of these problems, general data availability and equivalence, were shown to have an important impact on the above-mentioned theory assessment.

With regard to the example of hypotheses derived from the VoC, in one case new and better data reversed the interpretation overall. While the old data supported the hypothesis, improved data (for a reduced sample of countries, though) suggested a rejection. However, the new data also led to the formulation of a new hypothesis. In a second case, it was argued that the availability of comparative data does not necessarily mean that the data is also comparable. While traditional techniques (exploratory factor analysis) supported the hypothesis again, improved techniques of data analysis (Multigroup SEM) found measurement inequivalence of the instrument, thereby impeding a straightforward test of the hypothesis. It is important to mention, though, that while these two examples are critical about the applicability of the VoC approach to a comparison of higher education systems, other analyses support a relationship between the VoC and their respective higher education systems (Graf, 2009; Hoelscher, 2012b, 2013, 2016; Leuze, 2010).

What are the conclusions from these considerations? Comparisons are highly important, but are also demanding with regard to theory and data. While a lot of progress has been achieved in the last years already, further improvements in data availability as well as in our capacity to analyze the data with advanced techniques are necessary. In spite of, or maybe even especially because of, the high relevance of comparisons, one should be extremely careful, and ideally combine different approaches and data sources. In the case of the above-presented assessment of the VoC, the quantitative data should be complemented by qualitative approaches to allow for the triangulation of results, otherwise false conclusions may be drawn; for example, in the field of evidence-based policymaking, when, again for example, US policies in the field of higher education are transferred to European systems too easily. Therefore, when interpreting (comparative) data, we should always be full of doubt – to refer back to Russell’s statement presented at the beginning of this article.

Footnotes

Appendix 1

This article uses data from the study REFLEX – International Survey Higher Education Graduates (www.reflexproject.org). The Swiss data was provided by the “Bundesamt für Statistik der Schweizerischen Eidgenossenschaft”. I would like to thank all primary researchers for access to this data.

Work values (Allen and Van der Velden, 2007: 242 ff.; REFLEX master questionnaire)

The question was “Please indicate how important the following job characteristics are to you personally?”. Each item could be rated on a 5-point scale ranging from “1 = not at all important” to “5 = very important”.

The items and their respective factors were as follows:

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author biography

Michael Hoelscher is a professor for Science and Higher Education Management at the German University of Administrative Sciences Speyer. His main fields of interest are higher education, international comparisons, as well as cultural, economic and urban sociology more broadly. Recent publications include “Spielarten des akademischen Kapitalismus” (Springer 2016), Measuring Social Innovation (Historical Social Research 2015; with Bund, Gerhard & Mildenberger) and Potential and Problems of International Creativity Indices (Creativity Research Journal 2015; with Schubert).

References

Allen

van der Velden

(2007) The Flexible Professional in the Knowledge Society: General Results of the REFLEX Project. Maastricht: Research Centre for Education and the Labour Market, Maastricht University, The Netherlands.

Amable

(2003) The Diversity of Modern Capitalism. Oxford: Oxford University Press.

Andersen

van de Werfhorst

(2010) Education and occupational status in 14 countries: The role of educational institutions and labour market coordination. British Journal of Sociology 61(2): 336–355.

Arthur

Brennan

Weert

(2007) REFLEX – the Qualitative Study: Employer and Higher Education Perspectives on Graduates in the Knowledge Society. Milton Keynes/Enschede: Centre for Higher Education Research and Information/Center for Higher Education Policy Studies.

Barro

Lee

J-W

(2001) International data on educational attainment: Updates and implications. Oxford Economic Papers 53(3): 541–563.

Behnke

(2005) Lassen sich Signifikanztests auf Vollerhebungen anwenden? Einige essayistische Anmerkungen. Politische Vierteljahresschrift 46(1): O1–O15.

Braun

Johnson

(2010) An illustrative review of techniques for detecting inequivalences. In: Harkness

Braun

Edwards

et al . (eds) Survey Methods in Multinational, Multiregional, and Multicultural Contexts. Hoboken: Wiley, pp. 375–393.

Broscheid

Gschwend

(2005) Zur statistischen Analyse von Vollerhebungen. Politische Vierteljahresschrift 46(1): O16–O26.

Brown

Green

Lauder

(2001) High Skills. Globalization, Competitiveness and Skill Formation. Oxford: Oxford University Press.

10.

Byrne

(2010) Structural Equation Modeling With AMOS: Basic Concepts, Applications, and Programming. New York: Routledge.

11.

Cheung

Rensvold

(2002) Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling 9(2): 233–255.

12.

Culpepper

Finegold

(1999) The German Skills Machine: Sustaining Comparative Advantage in a Global Economy. New York: Berghahn.

13.

Deth

JWv

(1998) Equivalence in comparative political research. In: Deth

JWv

(ed) Comparative Politics: The Problem of Equivalence. London: Routledge, pp. 1–19.

14.

Deth

JWv

Scarbrough

(1995) The concept of values. In: Deth

JWv

Scarbrough

(eds) The Impact of Values. Oxford: Oxford University Press, pp. 21–47.

15.

Dogan

(2004) The quantitative method in comparative research. In: Kennett

(ed) A Handbook of Comparative Social Policy. Cheltenham: Edward Elgar, pp. 324–340.

16.

Dogan

Pelassy

(1990) How to Compare Nations. Strategies in Comparative Politics. Chatham: Chatham House Publications.

17.

Durkheim

([1895] 1982) The Rules of Sociological Method. New York: Free Press.

18.

Ebbinghaus

(2009) Vergleichende Politische Soziologie: Quantitative analyse – oder qualitative Fallstudiendesigns? In: Kaina

Römmele

(eds) Politische Soziologie: Ein Studienbuch. Wiesbaden: Springer VS / GWV Fachverlage GmbH, pp. 481–501.

19.

French

Finch

(2006) Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modelling 13(2): 378–402.

20.

Gellner

(1983) Nations and Nationalism. Ithaca, NY: Cornell University Press.

21.

Goldthorpe

(1997) Current issues in comparative macrosociology: A debate on methodological issues. In: Mjøset

Engelstad

Brochmann

et al . (eds) Methodological Issues in Comparative Social Sciences. Greenwich and London: Jai Press, pp. 1–26.

22.

Graf

(2009) Applying the varieties of capitalism approach to higher education: Comparing the internationalisation of German and British universities. European Journal of Education 44(II): 569–585.

23.

Hall

Soskice

(2001a) An introduction to varieties of capitalism. In: Hall

Soskice

(eds) Varieties of Capitalism. The Institutional Foundations of Comparative Advantage. Oxford: Oxford University Press, pp. 1–68.

24.

Hall

Soskice

(2001b) Varieties of Capitalism. The Institutional Foundations of Comparative Advantage. Oxford: Oxford University Press.

25.

Hancké

Rhodes

Thatcher

(2007) Introduction: Beyond varieties of capitalism. In: Hancké

Rhodes

Thatcher

(eds) Beyond Varieties of Capitalism: Conflict, Contradictions, and Complementarities in the European Economy. Oxford: Oxford University Press, pp. 3–38.

26.

Harkness

Braun

Edwards

et al . (2010) Survey Methods in Multinational, Multiregional, and Multicultural Contexts. Wiley Series in Survey Methodology. Hoboken, NJ: Wiley.

27.

Hauschildt

Gwosć

Netz

et al . (2015) Social and Economic Conditions of Student Life in Europe. Synopsis of Indicators Eurostudent V 2012–2015. Bielefeld: Bertelsmann.

28.

Hoelscher

(2012a) Universities and higher learning. In: Anheier

Juergensmeyer

(eds) Encyclopedia of Global Studies. London: SAGE, pp. 1713–1718.

29.

Hoelscher

(2012b) Spielarten des Kapitalismus und Kompetenzen von Hochschulabsolventen. Eine komparative empirische Analyse. Kölner Zeitschrift für Soziologie und Sozialpsychologie 64(3): 479–505.

30.

Hoelscher

(2013) Hochschullehre zwischen Varieties of Capitalism und internationaler Konvergenz. Ein europäischer Vergleich. Swiss Journal of Sociology 39(Special Issue): 405–427.

31.

Hoelscher

(2016) Spielarten des Akademischen Kapitalismus. Hochschulsysteme im Internationalen Vergleich. Wiesbaden: Springer VS.

32.

Hoffmeyer-Zlotnik

JHP

Harkness

(2005) Methodological aspects in cross-national research. In: ZUMA (ed.) ZUMA Nachrichten Spezial (Special Issue 11). Mannheim: ZUMA.

33.

Hoffmeyer-Zlotnik

JHP

Warner

(2012) Harmonisierung demographischer und sozio-ökonomischer Variablen: Instrumente für die international vergleichende Surveyforschung. Wiesbaden: Springer VS.

34.

Immerfall

(1995) Einführung in den Europäischen Gesellschaftsvergleich. Ansätze – Problemstellungen – Befunde. Passau: Wissenschaftsverlag Rothe.

35.

Jackson

Deeg

(2006) How many varieties of capitalism? Comparing the comparative institutional analyses of capitalist diversity. MPIfG Discussion Papers 06/2. Cologne: Max-Planck-Institut für Gesellschaftsforschung.

36.

Jowell

(1998) How comparative is comparative research? American Behavioral Scientist 42(2): 168–177.

37.

Kivinen

Rinne

(1998) Methodological challenges for comparative research into higher education. Interchange 29(2): 121–136.

38.

Koucky

Meng

van der Velden

(2007) Reflex Country Study. Maastricht: Research Centre for Education and the Labour Market, Education Policy Centre.

39.

Kromrey

(1998) Empirische Sozialforschung. Modelle und Methoden der Datenerhebung und Datenauswertung. Opladen: Leske+Budrich.

40.

Lakatos

(1970) Falsification and the methodology of scientific research programmes. In: Lakatos

Musgrove

(eds) Criticism and the Growth of Knowledge. New York: Cambridge University Press, pp. 91–195.

41.

Lei

(2007) Introduction to structural equation modeling: Issues and practical considerations. Educational Measurement Issues and Practice 26(3): 33–43.

42.

Leuze

(2010) Smooth Path or Long and Winding Road? How Institutions Shape the Transition from Higher Education to Work. Farmington Hills, MI: Budrich UniPress Ltd.

43.

Little

(1997) Mean and covariance structures (macs) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research 32(1): 53–76.

44.

Marsh

Hau

K–T

(2007) Applications of latent-variable models in educational psychology: The need for methodological-substantive synergies. Contemporary Educational Psychology 32: 151–171.

45.

Mjøset

et al . (eds) (1997) Methodological issues in comparative social science. Greenwich and London: Jai Press.

46.

Mullen

(1995) Diagnosing measurement equivalence in cross-national research. Journal of International Business Studies 26(3): 573–596.

47.

OECD (2011a) Education at a Glance 2011. OECD Indicators. Paris: OECD Publishing.

48.

OECD (2011b) Education at a Glance 2011. Glossary. Paris: OECD Publishing.

49.

OECD (2011c) Education at a Glance 2011. Annex 3: Sources, methods and technical notes. Paris: OECD Publishing.

50.

Pfau-Effinger

(2009) International vergleichende Sozialforschung: Ansätze und Messkonzepte unter den Bedingungen der Globalisierung. Wiesbaden: Springer VS.

51.

Phillips

(2005) Policy borrowing in education: Frameworks for analysis. In: Zajda

(ed) International Handbook on Globalisation, Education and Policy Research: Global Pedagogies and Policies. Heidelberg: Springer, pp. 23–34.

52.

Phillips

Ertl

(2003) Implementing European Union Education and Training Policy. A Comparative Study of Issues in Four Member States. Dordrecht: Kluwer.

53.

Phillips

Ochs

(2004) Researching policy borrowing: Some methodological challenges in comparative education. British Educational Research Journal 30(6): 773–784.

54.

Pickel

Lauth

H-J

et al . (2009) Methoden der vergleichenden Politik- und Sozialwissenschaft: Neue Entwicklungen und Anwendungen. Wiesbaden: Springer VS.

55.

Popper

(1935) Logik der Forschung. Wien: Springer.

56.

Przeworski

Teune

(1966) Equivalence in cross-national research. Public Opinion Quarterly 30(4): 551–568.

57.

Przeworski

Teune

(1970) The Logic of Comparative Social Inquiry. New York: Wiley-Interscience.

58.

Roose

(2013) Fehlermultiplikation und Pfadabhängigkeit. Ein Blick auf Schattenseiten von Sekundäranalysen standardisierter Umfragen. Kölner Zeitschrift für Soziologie und Sozialpsychologie 65(4): 697–714.

59.

Rössel

(2012) Methodological nationalism. In: Anheier

Juergensmeyer

(eds) Encyclopedia of Global Studies. London: SAGE, pp. 1151–1153.

60.

Schomburg

Teichler

(2006) Higher Education and Graduate Employment in Europe. Results from Graduate Surveys from Twelve Countries. Dordrecht: Springer.

61.

Teichler

(2007) Higher Education Systems. Conceptual Frameworks, Comparative Perspectives, Empirical Findings. Rotterdam: Sense Publishers.

62.

Teichler

(2008) Hochschulforschung international. In: Zimmermann

Kamphans

Metz-Göckel

(eds) Perspektiven der Hochschulforschung. Wiesbaden: Springer VS, pp. 65–85.

63.

Teichler

Arimoto

Cummings

(2013) The Changing Academic Profession: Major Findings of a Comparative Survey. Dordrecht: Springer.

64.

Temme

Hildebrandt

(2009) Gruppenvergleiche bei hypothetischen Konstrukten – Die Prüfung der Übereinstimmung von Messmodellen mit der Strukturgleichungsmethodik. Schmalenbachs Zeitschrift für betriebswirtschaftliche Forschung 61(2): 138–185.

65.

Trahar

(2013) Introduction. Contemporary methodological diversity in European higher education research. European Educational Research Journal 12(3): 301–309.

66.

Vijver

FJRvd

Leung

(1998) Methods and Data Analysis for Cross-Cultural Research. Thousand Oaks, CA: SAGE.

67.

Wentzel

(2011) An Imperative to Adjust? Skill Formation in England and Germany. Wiesbaden: VS Verlag.

68.

Øyen

(1990) Comparative Methodology: Theory and Practice in International Social Research (SAGE Studies in International Sociology). London/Newbury Park: SAGE.