Sage Journals: Discover world-class research

Abstract

We revisit and empirically evaluate crucial yet under-examined arguments articulated in “God Gave Physics the Easy Problems” (2000), the authors of which emphasized that, in International Relations (IR) predictions, predominant nomothetic approaches should be supplemented with concrete scenario thinking. We test whether the IR predictive toolkit is in fact dominated by nomothetic generalizations and, more broadly, map the methodological profile of this subfield. We build on the TRIP database, supplementing it with extensive original coding to operationalize the nuances of predictive research. In particular, we differentiate between nomoscopic predictions (predictive generalizations) and idioscopic predictions (predictions for concrete situations), showing that this distinction is not reducible to other methodological cleavages. We find that even though in contemporary IR an increasing number of articles seek to provide predictions, they consistently avoid predictions about concrete situations. The proportion of idioscopic predictions is stably small, with an even smaller proportion of predictions that develop concrete narratives or specify any determinate time period. Furthermore, those idioscopic studies are mostly limited to a niche with specialized themes and aims. Thus, our research shows that the critical claims from 20 years ago are still relevant for contemporary IR, as the “difficult problem” of developing predictive scenarios is still consistently overlooked in favor of other objectives. Ultimately, the types of predictions that IR scholars develop depend on their specific aims and constraints, but the discipline-wide result is a situation in which international studies’ ambition to provide predictions grows, but they tend to reproduce the same limitations as they did in 2000.

Keywords

Future forecasting scenarios methodology predictions International Relations discipline

Providing precise and nontrivial predictions is one of the most challenging tasks both for International Relations (IR) and for social science in general. The ability of a particular field of research to provide predictive findings is often seen as one of the main proofs of its scientific status, yet at the same time fundamental features of social reality make predictive research in IR quite problematic. In the influential article “God Gave Physics the Easy Problems: Adapting Social Science to an Unpredictable World,” Steven Bernstein, Richard Ned Lebow, Janice Gross Stein, and Steven Weber made a powerful case for the limitations IR faces when it comes to predictions. In this paper, we revisit their methodological ideas and provide empirical insights on how contemporary IR as a discipline deals with the “difficult problems” of predicting the “unpredictable world” of global politics (Bernstein et al., 2000).

Bernstein and his coauthors argued that attempts to develop predictive theories in IR are often fundamentally flawed due to the fact that they are entrenched in the questionable assumption of the similarity between social and physical phenomena. The authors of “God Gave Physics the Easy Problems” insisted that while it is possible to provide predictions for the physical world on the basis of discoverable nomothetic laws, quantifiable variables and measurements, the same approaches would often be inapplicable in IR because of the complexity of social systems and their ability to react to experts’ theories and predictions. For these reasons, in social research we can never extract the objects of analysis so that they are absolutely independent of each other, of past experience, future expectations, and of the knowledge about them that may be acquired by the researcher. Moreover, laws in physics often describe probabilities concerning trillions of atoms and molecules, incomparable to the number of units in studies of IR. Furthermore, the complexity of social processes, in which even marginal fluctuations in one of the many variables lead to substantial changes of the whole system, makes it difficult to provide reliable projections (Bernstein et al., 2000: 45–47).

Bernstein et al. emphasized the divergencies between social and physical phenomena and criticized IR researchers who ignore that difference and are, thus, mostly attracted to “deductive logic, falsifiable hypothesis and large-n statistical ‘tests’” (Bernstein et al., 2000: 45), in which “the ‘scientific means’ has become more an end in itself, and the ‘science’ of the social, a jeu d’esprit” (Bernstein et al., 2000: 44). It was the search for a way out of this situation that drove the authors of “God Gave Physics the Easy Problems” to address the issue of IR forecasts and predictions. While providing argumentation against attempts to fashion IR predictive work along the standards of nomothetic knowledge, they, nevertheless, did not rule out the possibility of looking into the future of international affairs. They proposed an innovative approach to forecasting that implied constructing versions of narratives of how the future may unfold rather than searching for general probabilistic predictive models. These scenarios would integrate the predetermined “certainties,” as well as critical “uncertainties” into plot lines with particular indicators and driving forces. They assumed that implementation of this so-called forward reasoning approach would compensate for the imbalance toward deductive probabilistic theories and facilitate interaction between the academic IR and the analysis of pressing policy problems. Ultimately, as the authors of “God Gave Physics the Easy Problems” claimed, this would contribute to IR better fitting to its purpose of providing concrete practical insights about the world’s problems (Bernstein et al., 2000: 44, 48).

With a large number of citations¹ and reads since 2000, Bernstein et al.’s work attracted considerable attention in the IR research community. For example, in The Power of International Theory: Reforging the Link to Foreign Policy-Making through Scientific Enquiry Fred Chernoff follows Bernstein et al. in opposing “the Newtonian analogy” in international studies and advocating “scenario analysis” as an alternative to replace theory-based predictions in IR (Chernoff, 2005: 153). Similarly, support for ideas from “God Gave Physics the Easy Problems” can be found in Richard Meissner’s “Paradigms and Theories: Popular Labels and Their Delimitation” (Meissner, 2016: 148). In his discussion of IR research programs, Jack Levy also emphasized that by being critical of the traditional understanding of ontology, epistemology, and methodology, Bernstein et al. made an attempt “to broaden the conception of science and thus of what constitutes scientific progress” (Levy, 2007: 178). The arguments advanced by Steven Bernstein and his coauthors resonated not only with authors of broad theoretical works, but also influenced even more studies dedicated to more specific aspects of international interactions (e.g., conflict resolution, trade and trade policy, cybersecurity, terrorism etc.²).

Two decades since the publication of “God Gave Physics the Easy Problems,” it seems appropriate to revisit the arguments presented in that article and to pose new questions on the topic. Has the methodological landscape of predictive research in IR changed since Bernstein and his colleagues published their paper? Has there been anything like a “forward reasoning turn”? Or is the field of IR today still dominated by the nomological approaches and statistical hypothesis testing to the same extent as 20 years ago? Have they become even more dominant?³

Although the diagnosis of the IR discipline provided by Bernstein, Lebow, Stein, and Weber rings true, it was mostly based on the authors’ intuitions about the field. Such intuitions, as Daniel Maliniak et al. have shown, do not always correspond with the actual picture that emerges from a more systematic quantitative empirical analysis of IR publications (Maliniak et al., 2011: 442). So, today it makes sense not only to explore the impact of “God Gave Physics the Easy Problems” but also to test its claims empirically. Luckily, now, unlike 20 years ago, we have access to quite credible data about the state of IR as a discipline, collected by the Teaching, Research, and International Policy (TRIP) Project (Maliniak et al., 2018; Teaching, Research, and International Policy Project, 2017). In this article we use the TRIP database of IR articles as our evidentiary starting point and supplement it with extensive original coding of articles in order to provide better operationalization for the types of predictions provided in each of them.

We will leverage this data to evaluate Bernstein et al.’s intuitions empirically, probing their relevance for today’s IR. In particular, we seek answers to two main questions. First, was predictive research in IR dominated by the nomological models and statistical methods at the end of the 20th century and, if so, how has the situation changed since? Second, more broadly, what is the methodological profile of IR predictions in general — what paradigms, approaches, and methods dominate the field and what type of predictions do researchers usually seek to provide? We interpret our findings in the context of how research goals and constraints that are specific for each topic and even for each individual study are mutually coupled with discipline-level methodological inclinations, as well as with IR’s overall raison d’être.

Nomoscopic and idioscopic predictions

Scholars have proposed various typologies of IR predictions — including but hardly limited to the contrast between nomological deductive approaches and narrative scenario-oriented ones discussed above. Notable among these is the remarkable typology developed by Nazli Choucri (1974), who distinguished different kinds of forecasts based on different levels of organization, and structured the approaches into a spectrum from the most organized to the least organized ones. In this typology the predictions that are based on the use of artificial intelligence are qualified as the most systematized ones and normative forecasts as the least systematized ones. Between these two extremes there appear well systematized simulation-based forecasts, less systematized model-based forecasts, and even less systematized exploratory projections.

A different approach was more recently proposed by Gerald Schneider, Nils Petter Gleditsch, and Sabine Carey (Schneider et al., 2011) who came up with the three-part typology of approaches to IR forecasting that includes structural approach,⁴ time-series analysis,⁵ and game-theoretic modeling.⁶ The structural approach is designed to “predict the risk of a geographical unit (whether a country, a region, or a town) experiencing a certain behavior in subsequent time periods given important characteristics of the unit at present” (Schneider et al., 2011: 6), while the time-series approach deals with relatively short time periods, when the situation changes rapidly. Game-theoretic forecasting is primarily suitable for predictions of decision-making patterns as the empirical basis for the forecaster mostly consists of area experts’ evaluations (Schneider et al., 2011: 6–7).

Moreover, as Nils Metternich and Kristian Gleditsch demonstrate, one can also distinguish between data- and model-driven approaches to forecasting (Metternich and Gleditsch, 2016). Highly data-driven models mostly evolve around fitting to observable data with very few assumptions about the subject matter. The opposite end of the spectrum is represented by extreme model-driven forecasts that are based on a priori assumptions built into simplified mathematical or theoretical models that are sometimes totally devoid of any empirical data. In between these two poles there are various approaches that combine a priori modeling with some use of empirical data. Based on the type of the model used, those model-driven approaches can be divided into informal modeling, formal modeling, and agent-based and computational modelling approaches (Metternich and Gleditsch, 2016).

Even though our analysis is based on TRIP database of IR articles, which is not detailed enough to grasp all these nuances of existing predictive techniques, this dataset includes enough information to distinguish between quantitative predictions, predictions based on formal modeling, and predictions deriving from qualitative research. However, there is one important distinction that is crucial for our analysis and is more difficult to trace with existing data: the distinction between general predictive statements and specific predictions about particular situations. The differentiation between these two types of predictions largely derives from the criticisms of the “nomological” approach to IR forecasting that were presented in “God Gave Physics the Easy Problems.” Moreover, this distinction is reminiscent of the Nils Metternich’s and Kristian Gleditsch’s dichotomy between approaches that “are highly case-specific and predict outcomes for a single unit and particular issue area” and those that “seek to assess the risks of some general phenomenon across a wide range of units” (Metternich and Gleditsch, 2016).

Here we develop and deploy a toolkit that allows us to analyze both of these types of predictions while keeping the difference between them clear. For that purpose, we will use the notion of prediction that is intentionally defined very broadly — as any research that has statements about future⁷ states of affairs as its key findings. This conceptual tool is designed to grasp all kinds of predictions — both those appearing as general assessments of probabilities that predict some general phenomenon to be more or less likely across a wide range of units and concrete predictions for a single unit and particular situation. It is in order to differentiate between these two approaches that we introduce the distinction between nomoscopic predictions and idioscopic⁸ predictions.

Thus, our definition of prediction is quite similar to the one proposed by Chernoff (2005: 8) but differs from the narrower understanding of predictions used in Bernstein et al.’s work (2000). Even though we acknowledge the importance of distinction between “[concrete] scenarios” and “[deductive-nomological] predictions” (Bernstein et al., 2000: 54), we still needed a very general term that would denote both these forms of talking about future. So, we are defining the word prediction in the broadest way but then introduce the contrast between nomoscopic and idoscopic predictions⁹ to account for the differences similar to the ones emphasized by Bernstein et al. (2000).

Our twofold typology of predictions is an attempt to turn the critical claim made by Steven Bernstein and his coauthors against the “nomological” approaches to IR predictions into an analytical tool. Thus, the concept of nomoscopic prediction is an attempt to outline the type of predictive statements that generally appear as probabilistic middle-range models or generalized regularities. Such predictions seek to foresee what types of events are more likely in particular types of situations, attempting to draw predictive statements in the form of probabilistic laws.

By contrast, idioscopic predictions are more specific predictions that rather seek to provide narratives about the futures of concrete actors, regions, and situations. This type of prediction includes Bernstein et al.’s “forward reasoning” approach but is not limited to it. It also covers qualitative and normative predictive projections that are quite common in interpretive, applied IR analytics, as well as in publications that not only develop some general predictive models but also apply them to concrete situations.

Although our analysis was largely a reaction to the ideas outlined by Bernstein and his coauthors, we had an intuition that the whole idea of “forward reasoning” has not really caught on. At the same time the critical claims made in the article “God Gave Physics the Easy Problems” have been quite influential.¹⁰ For this reason, we have not designed our conceptual framework around the notion of forward reasoning but made it capable of grasping a much broader range of different idioscopic approaches that are alternative to the presumably dominant nomoscopic predictions.

Even though the distinction between nomoscopic and idioscopic predictions is by design reminiscent of that between nomothetic and idiographic approaches to knowledge (Windelband, 1894, 1998), these two dichotomies are not identical. While the difference between nomothetic and idiographic approaches separates the two different forms of knowledge sought by empirical sciences retrospectively (nomothetic knowledge of general laws and idiographic knowledge of particular historically determined forms), the distinction between nomoscopic and idioscopic predictions identifies two distinct ways of speaking and thinking about future: predictions about general regularities and predictions about particular events. Although nomoscopic predictions always appear as predictive projections of nomothetic knowledge, idioscopic ones can derive both from the knowledge of the general (nomothetic) and from the knowledge of the particular (idiographic).

The difference between idioscopic and nomoscopic predictions also cannot be reduced to the distinction of grand theories, middle-range theories, and narrow-gauge theories (Chilcote, 2018: 7), as this typology characterizes not the scope of the theory or the relations between the theory and empirical evidence, but the level of concreteness inherent in the resulting predictive statement that is seen as the goal of prediction. We, thus, characterize a prediction as idioscopic if it is aimed at predicting the course of events in a concrete situation for concrete set of actors — regardless of whether those predictions derive from grand theories, middle-range theories, or just from the analysis of one particular case. Moreover, in IR it is quite common to use grand theories like realism, liberalism, and constructivism as analyticist¹¹ interpretive frameworks in the predictions for specific situations providing concrete narratives about particular units and situations.

Examples of how the distinction of nomoscopic and idioscopic predictions work will help situate the analysis below. An idioscopic prediction can be, for instance, found in the article “Pyongyang’s Survival Strategy: Tools of Authoritarian Control in North Korea” by Daniel Byman and Jennifer Lind (Byman and Lind, 2010). In that paper the authors address the problem of the future of the North Korean regime by researching the range of tools that can be used by the North Korean leader to prevent possible attempts at a coup and analyzing the potential of various tools of external coercion that may be used against the North Korean leader. In their analysis, Byman and Lind refer to some elements of middle-range theories about how dictators stay in power, comparing Kim Jong-Il to other autocrats and arguing that rulers of that kind tend to protect their selectorate while shifting the burden of sanctions to the people (Byman and Lind, 2010: 72). This research, however, has to be characterized as an idioscopic prediction due to the fact that the goal of the study is the analysis of the future of one particular situation, not the development of the middle-range theory.

Contrastingly, an article by Abel Escribà-Folch and Joseph Wright called “Dealing with Tyranny: International Sanctions and the Survival of Authoritarian Rulers” that is devoted to a similar topic can be referred to as an example of nomoscopic prediction (Escribà-Folch and Wright, 2010). In this paper, the authors study the influence of economic sanctions on the stability of authoritarian regimes of different types by using quantitative analysis of data on sanctions that were imposed in the period from 1960 to 1997. Their study conveys almost nothing about the future of specific international actors, but it provides an understanding of some common patterns and general phenomena, which lead to generalized probabilistic predictions across a wide range of units — for instance, to the idea of how in general personalistic regimes will tend to react to sanctions.

Escribà-Folch and Wright draw the conclusions that are to a great extent similar to Byman and Lind’s, and are, indeed, rooted in the same middle-range theories; however, the two papers choose two different perspectives. Escribà-Folch and Wright try to determine general patterns, while Byman and Lind seek to answer quite specific questions about concrete political actors. These two cases are illustrative for our analytical framework, as they show that mere distinction of grand theories, middle-range theories, and narrow-gauge theories is not enough to grasp important nuances in the design of predictive research.

Analyzing predictions: materials and methods

As mentioned above, the main source of empirical evidence for our study is the TRIP project database of IR research articles published between 1992 and 2014.¹² For each article the TRIP database provides manually coded information on 29 parameters, which describe the key epistemological, methodological, and thematic features of the publication.¹³

The key objective of our research is in general very close to that of the TRIP project, as we also attempt to create a methodological portrait of international studies. However, our analysis is aimed at investigating not the IR discipline as a whole, but only the part of it that focuses on predictions. In order to select articles from the TRIP database that contained predictions, we developed a list of keywords that were used as verbal indicators of predictive research.¹⁴ The articles that contained those keywords in their titles or abstracts were automatically included in the preliminary list of predictive ones. Then, additional manual filtering was performed to refine the dataset. As a result, from 5559 articles of the TRIP database, 817 were identified as publications containing some kind of predictive statements as their key findings.¹⁵

The distinction of nomoscopic and idioscopic predictions as a separate analytical category was especially important for our analysis, as none of the variables already present in the TRIP database allows us to discriminate between nomoscopic predictive generalizations (what factors will make some type of future courses of events more likely than the others) and concrete idioscopic predictions (what specific events will happen to particular international actors in the future). Thus, we had to recode articles of TRIP database in order to add variables that capture that distinction.

Additionally, in order to take a more detailed account of the methodological nuances of idioscopic predictions we coded the idioscopic articles with two extra variables. One of those variables indicated whether the time span of predictive statements is explicitly indicated in the article or can only be inferred based on indirect cues. Another variable reflected what the time span of those predictive statements was based on either direct indicators or indirect cues. As the predictive time span variable that was used in this study allowed for each article to refer to several different periods in the future, in our analysis we were able to distinguish between minimal and maximal time spans for each article.

The rest of the variables used in our analysis were taken from the original TRIP project database (Maliniak et al., 2013).

The full list of parameters analyzed in our study is described in Table 1.

Table 1.

Variables used in the study.

Variables	Details	Source
is_prediction	Does the article contain statements about future trends or future states of affairs?	Coded in this study.Coding guide is provided in Appendix 1.
is_nomoscopic	Does the article describe future states of affairs by mentioning what types of conditions in general make particular types of future events more/less likely in some types of situations for some types of actors?
is_idioscopic	Does the article describe future states of affairs for concrete situations, concrete emerging trends, and/or concrete actors?
explicit_period	Does the article explicitly indicate the time period to which the predictive statements refer? (Only for idioscopic predictions.)
prediction_period	How far in the future do the predictive statements of the article refer to? (Only for idioscopic predictions. Multiple answers are possible for articles that contain several predictive statements referring to different time periods.)
journal	The journal in which the article was published.	TRIP Project
year	The year in which the article was published.
Methodology_. . .[8 variables]	This is a measure of what methodologies the study uses: quantitative, qualitative, formal modeling, counterfactual, nonformal conceptual, descriptive, policy analysis, or experimental. (Several methodologies can be selected for each article.)
PolicyPrescription	Does the author make explicit policy prescriptions in the article?
Paradigm	Which paradigm is used to frame the research question and answer, is advocated by the author, or is used to guide analysis? (Only one paradigm (realist, liberal, constructivist, Marxist, nonparadigmatic, atheoretic) can be selected for each article.)
Focus_. . . [34 variables]	Substantive focus of the article. (Several foci can be selected for each article.)

We used descriptive statistics¹⁶ of these variables¹⁷ to build the methodological profile of contemporary IR predictions by exploring four types of IR publications:

articles that provide no predictions,¹⁸

articles that provide nomoscopic predictions,¹⁹

articles that provide idioscopic predictions,²⁰

articles that provide narrative idioscopic predictions.²¹

We add the narrative²² idioscopic predictions (idioscopic predictions that use descriptive, qualitative, or analytic methodologies) as a separate subtype under study in order to take a closer look at the research works that are the most similar to Bernstein et al.’s concept of “forward reasoning” — developing “scenarios, or narratives with plot lines that map a set of causes and trends in the future time” (Bernstein et al., 2000: 53).

Results: adapting to the unpredictable world?

Our analysis shows, first and foremost, that only 15% of the IR articles analyzed had some kind of predictive statements as their main outcomes and less than one-third of those predictive studies developed predictions that were idioscopic (Figure 1). In most journals, moreover, the proportion of nomoscopic predictions was much larger in comparison to idioscopic ones, with International Security being the only exception (Figure 2). As to the dynamics, the distribution of predictive articles by year (Figure 3) shows that in general the proportion of IR papers that seek to provide some kind of predictions increases. However, this growth is due solely to nomoscopic predictions, while the niche of idioscopic predictions remains quite narrow and stable (Figure 4).

Figure 1.

Percentages of different types of predictions from all articles analyzed.

Figure 2.

Percentages of predictions in the total number of articles of each journal.

Figure 3.

Percentage of predictive articles by year.

Figure 4.

Percentages of different types of predictions from all articles by year.

A closer look at the subset of narrative idioscopic predictions reveals a similar trend. Only about 2% of all IR articles (13% of all predictions) develop this kind of prediction, but their number neither increases nor decreases. Thus, Bernstein et al.’s criticism about the nomological inclinations inherent in the IR predictive toolkit is fully supported by empirical evidence. Moreover, the data shows those criticisms remain relevant today, indeed, perhaps even more relevant, as the number of nomoscopic predictions has considerably increased since 2000 with no signs of growth either among narrative predictions or among idioscopic predictions in general (Figures 4 and 5).

Figure 5.

Percentages of different types of predictions from all predictions by year.

Our analysis of predictive time spans yields a more nuanced understanding of idioscopic predictions. Even though the idioscopic predictions are more concrete than nomoscopic ones, the study of time spans shows that even among the idioscopic predictions there are only 13% of publications (3% of all predictions) that indicate explicitly²³ the time period to which their predictions apply (Figure 6). For other 26% of idioscopic articles (3% of all predictions), it is possible to infer some particular time span based on indirect cues in the text,²⁴ but the rest of idioscopic predictions refer only to some indeterminate future. It seems that for most researchers it is usually too difficult or too academically risky to explicitly identify the time spans for IR predictions, even if they seek to develop concrete, idioscopic predictions.

Figure 6.

Percentages of predictions with and without explicitly indicated time span of predictions from all idioscopic predictions.

Additionally, we find that among the predictions that identify their time span explicitly or indirectly, the absolute majority of predictions refers to the minimal periods at least 10 years (Figure 7) and to maximal periods of no more than 20 years (Figure 8). This is, probably, due to the fact that the short-time predictions can be easily audited and proved wrong; thus, only a small number of researchers risk making such claims. At the same time, developing predictions for the periods of more than 20 years ahead seem to be just too difficult a task.

Figure 7.

Percentages of idioscopic predictions with different minimal time spans.

Figure 8.

Percentages of idioscopic predictions with different maximal time spans.

If we relate our data to some of the TRIP’s variables, we can get insights about the methodological portrait of different types of predictions. In particular, one can notice that, in contrast to other types of articles, idioscopic predictions and especially narrative idioscopic predictions more frequently seek to develop policy prescriptions (Figure 9). Furthermore, in terms of the thematic foci, nomoscopic and idioscopic types of predictions tend to have specific spheres of interest (Figure 10). In contrast to IR articles in general, idioscopic predictions, especially the narrative ones, are more often focused on studies of alliance politics, economic interdependence, regional integration, and intergovernmental organizations (IGOs). They more frequently discuss issues in terms of the balance of power and put special emphasis on foreign policy analysis. In comparison to other types of IR research, idioscopic predictions are more often interested in the problems of weapons of mass destruction (WMD) proliferation and in the studies of weapon systems in general. Nomoscopic predictions also have specific thematic foci that include the studies of interstate conflicts and regime types, as well as bargaining and deterrence strategies.

Figure 9.

Percentages of studies with and without policy prescriptions.

Figure 10.

Percentages of studies with particular thematic foci in the total number of predictions of each type.²⁵

Moreover, idioscopic predictions over different time spans also tend to have specific thematic foci (Figure 11). In particular, predictions of short (less than 10 years) and medium (10–30 years) spans are often focused on studies of IGOs and on the analysis of regional integration. There are also relatively large percentages of medium-term predictions that discuss balance of power, deterrence, interdependence and weapon systems (including WMD). Long-term predictions, by contrast, often explore environmental, developmental, and humanitarian issues as well as intrastate conflicts and interstate crises.

Figure 11.

Percentages of studies with particular thematic foci in the total number of predictions of each time span.²⁶

Based on these distinctive thematic profiles, we can hypothesize different explanations for their configuration. First, we can suggest that short-term idioscopic predictions focus on WMDs, interdependence, and integration because these topics require concrete predictions and specific policy prescriptions to a greater degree than others do. Second, it is possible that there are certain thematic domains that allow different toolkits — some can be predicted idioscopically, while others are more approachable from the nomoscopic perspective. For example, nobody wants to risk predicting wars and regime changes, but they are ready to predict shifts in the balance of power and changes in the structures of alignments.

Finally, one can also suggest that it is not the specific topics themselves that determine the type of predictions but rather the differences in the research goals and available data that are more common in particular thematic domains. For example, the predictions that deal with WMD proliferation tend more often to be narrative ones, due to the fact that their designs are often aimed at just a handful of “nuclear” states, with just too few cases for large-N analysis. For now, all these explanations seem plausible and consistent with our data, but they have to be tested in future research in which the study of our quantitative data will be supplemented with a more detailed qualitative analysis of predictive works.

Additional insights about the distinction of nomoscopic and idioscopic approaches to prediction can be provided by relating it to other known methodological cleavages that are common in IR. The most valuable of these insights, however, is a negative finding: our analysis demonstrates that the difference between nomoscopic and idioscopic predictions cannot be fully reduced to the standard methodological distinctions. In particular, it shows that in terms of paradigms (Figure 12), predictive research does not seem to be very different from the IR discipline in general. In particular, both among idioscopic and nomoscopic predictions more than 50% of predictions are provided in nonparadigmatic frameworks. One could have probably expected that the share of traditional paradigms of realism, liberalism, and constructivism in idioscopic predictions would be larger, but the actual data shows that these grand theories hardly dominate this subfield and have in fact only a slight advantage in comparison to nonparadigmatic approaches. It is only among the narrative idioscopic predictions that the majority of predictions are provided on the basis of traditional paradigms; however, even in this group 40% of articles use nonparadigmatic perspectives.²⁷ Furthermore, even though the field of nomoscopic predictions, quite expectedly, is dominated by quantitative analysis and formal modeling, while idioscopic predictions tend to more extensively use qualitative research tools and are more often focused on policy analysis, in general the idioscopic predictions rather often apply the same instruments as nomoscopic ones (Figure 13).

Figure 12.

Percentages of studies using different paradigms.

Figure 13.

Percentages of studies using different methodologies.

Thus, even though the distinction between idioscopic and nomoscopic predictions can be partially mapped onto well-known debates and cleavages, it cannot be fully reduced to them. In particular, it is likely due to the fact that the choice between seeking predictive generalizations and tracking concrete scenarios of the future is not only a “methodo-logical” question of predictive tools, but also, and above all, a “teleo-logical” one. It is not so much about how the prediction is developed, but for what purpose.

Conclusion

The article from 20 years ago that inspired this paper had the subtitle: “Adapting Social Science to an Unpredictable World” (Bernstein et al., 2000). Our study shows that in a way the discipline of International Relations today does try to “adapt to an unpredictable world” but definitely not in the sense suggested by Steven Bernstein and his coauthors. While IR’s ambition to provide predictions grows, attempts to develop concrete scenarios and to predict particular situations do not become more numerous.

In our study we analyzed over 5500 IR research articles that appeared in top academic journals since 1992. More than 800 of those articles turned out to be, very broadly speaking, predictive ones. But we discovered that even though the number of predictive IR articles is rather large and tends to grow, less than one-third contains concrete predictions, referring to particular situations and actors (idioscopic predictions), with an even smaller proportion of studies constructing narratives and scenarios of possible futures. The remaining predictions are formulated as predictive generalizations (nomoscopic predictions), describing what types of events are more possible in some types of situations, without referring to concrete states of affairs. Moreover, even when concrete predictions are provided, they rarely are bold enough to clearly articulate time spans. And when they do, they usually avoid risky short-term predictions for periods of less than 6 years ahead and almost never develop predictions beyond a 20-year horizon.

Thus, our analysis suggests that narrative idioscopic predictions, as well as idioscopic predictions in general, exist as a very narrow, but stable niche in IR studies. This niche differs from the rest of the discipline, as it is characterized by a visible focus on policy analysis and on developing policy prescriptions. Moreover, this type of prediction more often applies qualitative methods and uses traditional paradigms. At the same time, however, its toolkit is far from being limited to those instruments, as nonparadigmatic approaches and quantitative methods are also commonly used to develop idioscopic predictions. So, this narrowness of the niche of idioscopic predictions is not so much a problem of methodological limitations, paradigmatic trends, or epistemological sects. It is rather a question of what goals researchers seek to pursue, what counts as a good outcome of their work, and, ultimately, what is considered to be the aim of the IR as a discipline.

Nomoscopic predictions predominate in contemporary international studies, and their proportion in the corpus of IR publications is on the rise. Quite expectedly, they are mostly based on nonparadigmatic analysis with the use of quantitative methods and formal modeling approaches. So, one can be certain that all the critical claims about the state of the IR discipline that were articulated in “God Gave Physics the Easy Problems” are still valid. And they are even more appropriate today, because IR researchers are more inclined to generate predictions than ever before, but they still mostly develop them as quantitative models that imply general regularities.

Importantly, our analysis has also shown that each type of predictions has a distinctive thematic profile. In particular, nomoscopic predictions tend to be more focused on studies of interstate conflicts and regime types, as well as on the analysis of bargaining and deterrence strategies. Contrastingly, idioscopic predictions, especially the narrative ones, are more often focused on alliance politics, intergovernmental organizations, economic interdependence, and regional integration. They more frequently explore the issues in terms of the balance of power and pay more attention to foreign policy analysis. They also seem to be more often interested in WMD proliferation and in the studies of weapon systems in general. Moreover, predictions of different time spans also tend to differ in their thematic foci.

The existence of these distinctive thematic profiles can have different explanations. It either shows that there are certain topics that require concrete predictions about particular events or simply indicates that different issues allow different limited toolkits — some can be predicted idioscopically, while others are more approachable from nomoscopic perspective. Furthermore, it is also possible that it is not the particular topics themselves that determine the prediction types but rather the differences in the research aims and available evidence that are more common in particular thematic areas.

When Steven Bernstein and his colleagues criticized IR approaches to dealing with predictions, they emphasized that that IR scholars should consider supplementing the predominant nomothetic approaches based on probabilistic predictive models with alternative narrative tools based on concrete scenario thinking. What seemed to be lacking were the attempts to do a “forward tracking” of international dynamics, synthesizing general and concrete knowledge into narratives and scenarios of possible futures. As our study shows, Bernstein et al.’s claims were completely justified 20 years ago and they are still relevant for the current state of the IR discipline. In 1990s–2000s the number of narrative idioscopic predictions (similar to “forward reasoning”) was very small and the discipline was dominated by nomoscopic approaches. Today, our study shows, it is still so. At the same time, however, we were also able to show that, even though we do not really see any significant changes in what type of predictions IR studies generally seek to provide, the idioscopic predictions, even though they remain a minority, are not becoming much less common. While their proportion in the broader corpus of predictive international studies decreases, their share in the full corpus of IR publications has not fallen.

To sum up our analysis, let us once again go back to the challenge of “adapting” IR to the difficult problems of “unpredictable world.” The results of this study strongly suggest that all the 20-year-old agenda of this challenge is still valid and still mostly untouched. Considering the stably growing number of predictive IR articles, it would probably be unfair to say that nothing in this respect has been done. However, we also have to admit that what was done was mostly just an extensive buildup of old solutions with no visible attempts to develop new ones. So, if the “difficult problem” of “forward tracking” the scenarios of world politics was not intentionally avoided, it was at least consistently overlooked in favor of other objectives.

However, our research also revealed that there is a quite stable, if narrow, niche of IR studies that do seek to develop idioscopic predictions. Furthermore, we have shown that that niche has a quite noticeable prescriptive inclination and a distinct thematic profile that can arguably be traced to thematically specific aims of research. This raises a question: how can the goals of the mainstream nomoscopic studies and the aims of the idioscopic niche research both fit within IR’s raison d’être writ large? What is the overarching purpose of the discipline of International Relations?

Thus, as we go back to the ideas formulated by Bernstein and his colleagues 20 years ago, we should revisit not only the conceptual aspect of their argument but also the purposive one. Fundamentally, the question of how we as researchers think about the future is not only about having a more or less appropriate conceptual toolkit, but also about our understanding of the goal of our work. Bernstein and his colleagues insisted that the science of International Relations (and social science in general) “aspires — or should aspire — to provide insight into practical world problems that are generally part of a small or very small n” (Bernstein et al., 2000: 48). However, our study shows that this just does not seem to be the case for most academic IR predictions. This may be because most IR researchers just have a different understanding of what the purpose of international studies is. Or they agree with this understanding but differ concerning its implications in terms of their own research. For example, some of them can be proponents of the opinion that the central goal of international studies is providing generalizations about regularities of world politics, while the application of this knowledge to concrete situations is either a secondary task of the discipline or an aim that, though important, lies beyond the realm of IR science per se. Finally, at least some fraction of our colleagues might simply be epistemologically agnostic or driven by inconsistent sets of aims.

In 2000 Bernstein, Lebow, Stein, and Weber made “a plea for constructive humility” (Bernstein et al., 2000: 45). In turn, a plea that we want to make today is for individual and collective reflexivity, for awareness of our own constraints that often derive not only from our concepts and methods, but also from our aims. The task of this paper was not to make our fellow researchers change their ways but rather to empower them with the ability to attend to how the goals, inclinations, and limitations of their individual research are mutually coupled with the advantages, disproportions, and blind spots of IR discipline as a whole. All in all, it is our individual aims and means in their entanglement with the raison d’être of our discipline that determine how we see the future of world politics, and, ultimately, what this future will be.

Supplemental Material

Appendix_1._IR_Predictions_Coding_Guide – Supplemental material for International studies in an unpredictable world: still avoiding the difficult problems?

Supplemental material, Appendix_1._IR_Predictions_Coding_Guide for International studies in an unpredictable world: still avoiding the difficult problems? by Ivan Fomin, Konstantin Kokarev, Boris Ananyev, Nikita Neklyudov, Anzhelika Bondik, Pavel Glushkov, Aliya Safina, Svetlana Stolyarova, Dmitry Tkach, Oksana Vedernikova, Daria Korobkova, Daria Kovaleva, Ekaterina Kuzina, Darya Voronina, Alexander Chekov, Andrey Sushentsov and William Wohlforth in European Journal of International Relations

Supplemental Material

Appendix_2._IR_Predictions_Database1 – Supplemental material for International studies in an unpredictable world: still avoiding the difficult problems?

Supplemental material, Appendix_2._IR_Predictions_Database1 for International studies in an unpredictable world: still avoiding the difficult problems? by Ivan Fomin, Konstantin Kokarev, Boris Ananyev, Nikita Neklyudov, Anzhelika Bondik, Pavel Glushkov, Aliya Safina, Svetlana Stolyarova, Dmitry Tkach, Oksana Vedernikova, Daria Korobkova, Daria Kovaleva, Ekaterina Kuzina, Darya Voronina, Alexander Chekov, Andrey Sushentsov and William Wohlforth in European Journal of International Relations

Supplemental Material

titles_and_descriptions_for_supplemental_materials1 – Supplemental material for International studies in an unpredictable world: still avoiding the difficult problems?

Supplemental material, titles_and_descriptions_for_supplemental_materials1 for International studies in an unpredictable world: still avoiding the difficult problems? by Ivan Fomin, Konstantin Kokarev, Boris Ananyev, Nikita Neklyudov, Anzhelika Bondik, Pavel Glushkov, Aliya Safina, Svetlana Stolyarova, Dmitry Tkach, Oksana Vedernikova, Daria Korobkova, Daria Kovaleva, Ekaterina Kuzina, Darya Voronina, Alexander Chekov, Andrey Sushentsov and William Wohlforth in European Journal of International Relations

Footnotes

Acknowledgements

We would like to show our gratitude to Daniel Maliniak, Eric Parajon, and all other TRIP project participants for sharing their data with us. We also thank Nikolai Silaev, Igor Istomin, Christopher Korten, and Evgeniia Belskaia for their valuable feedback at various stages of the development of this paper. Finally, we are grateful to the anonymous reviewers as well as EJIR editors for critically reading our manuscript and suggesting substantial improvements.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is funded by the Ministry of Science and Higher Education of the Russian Federation (grant agreement number 14.641.31.0002).

ORCID iD

Fomin, Ivan

Supplemental material

Supplemental material for this article is available online.

Notes

References

Arikan

Shenkar

(2019) Nation-dyadic history and cross-border corporate deals: role of conflict, trade, generational distance, and professional education. Strategic Management Journal 41(3): 422–466.

Auguie

(2017) gridExtra: miscellaneous functions for “Grid” graphics. R package version 2.3. Available at: https://CRAN.R-project.org/package=gridExtra (accessed 18 October 2019).

Beck

King

Zeng

(2000) Improving quantitative studies of international conflict: a conjecture. American Political Science Review 94(1): 21–35.

Bentham

(1841) The Works of Jeremy Bentham, Now First Collected; Under the Superintendence of His Executor, John Bowring. Part XV. Bowring

(ed.) Edinburgh: William Tait; London, Simpkin, Marshall, & Co.; Dublin: John Cumming.

Bernstein

Lebow

Stein

, et al. (2000) God gave physics the easy problems: adapting social science to an unpredictable world. European Journal of International Relations 6(1): 43–76.

Bryan

Wickham

(2019) readxl: read excel files. R package version 1.3.1. Available at: https://CRAN.R-project.org/package=readxl (accessed 18 October 2019).

Bueno de Mesquita

(2002) Predicting Politics. Columbus, OH: Ohio State University Press.

Bueno de Mesquita

(2004) Decision-making models, rigor and new puzzles. European Union Politics 5(1): 125–138.

Bueno de Mesquita

(2009) The Predictioneer’s Game: Using the Logic of Brazen Self-Interest to See and Shape the Future. New York: Random House.

10.

Bueno de Mesquita

(2011) A new model for predicting policy choices: preliminary tests. Conflict Management and Peace Science 28(1): 65–87.

11.

Bueno de Mesquita

Newman

Rabushka

(1985) Forecasting Political Events: The Future of Hong Kong. New Haven, CT: Yale University Press.

12.

Byman

Lind

(2010) Pyongyang’s survival strategy: tools of authoritarian control in North Korea. International Security 35(1): 44–74.

13.

Chernoff

(2005) The Power of International Theory: Reforging the Link to Foreign Policy-Making Through Scientific Enquiry. London: Routlege.

14.

Chilcote

(2018) Comparative Inquiry in Politics and Political Economy: Theories and Issues. New York: Routledge.

15.

Choucri

(1974) Forecasting in international relations: problems and prospects. International Relations 1(2): 63–86.

16.

Cooper

(2018) NCmisc: miscellaneous functions for creating adaptive functions and scripts. R package version 1.1.6. Available at: https://CRAN.R-project.org/package=NCmisc (accessed 18 October 2019).

17.

Elff

(2020) memisc: management of survey data and presentation of analysis results. R package version 0.99.22. Available at: https://CRAN.R-project.org/package=memisc (accessed 11 April 2020).

18.

Escribà-Folch

Wright

(2010) Dealing with tyranny: international sanctions and the survival of authoritarian rulers. International Studies Quarterly 54(2): 335–359.

19.

Fomin

Kokarev

Ananyev

, et al. (2018) Forecasting practices in academic IR: methodological mainstream and unsolved problems. MGIMO Review of International Relations 11(6): 159–193.

20.

François

Henry

Müller

, et al. (2019) dplyr: a grammar of data manipulation. R package version 0.8.3. Available at: https://CRAN.R-project.org/package=dplyr (accessed 18 October 2019).

21.

Gerbing

(2019) lessR: less code, more results. R package version 3.8.9. Available at: https://cran.r-project.org/web/packages/lessR/index.html (accessed 18 October 2019).

22.

Google Scholar (2020) Bernstein: god gave physics the easy problems: adapting. . . - Google Scholar. Available at: https://scholar.google.ru/scholar?cites=14078468138717109149 (accessed 10 July 2020).

23.

Huling

(2019) jcolors: colors palettes for R and ‘ggplot2’, additional themes for ‘ggplot2’. R package version 0.0.4. Available at: https://CRAN.R-project.org/package=jcolors (accessed 11 April 2020).

24.

Jackson

(2010) The Conduct of Inquiry in International Relations. London: Routledge.

25.

Kaufman

Diep

Kaufman

(2018) Sociophysics of intractable conflicts: three-group dynamics. Physica A: Statistical Mechanics and its Applications 517: 175–187.

26.

Kaufman

Diep

(2019) Multi-group conflict paths: anticipatory scenarios of attitudes and outcomes. Journal on Policy and Complex Systems 5(2): 5–21.

27.

Kristensen

(2018) International relations at the end: a sociological autopsy. International Studies Quarterly 62(2): 245–259.

28.

Levy

(2007) Theory, evidence, and politics in the evolution of international relations research programs. In: Lebow

Lichbach

(eds) Theory and Evidence in Comparative Politics and International Relations. New Visions in Security. New York: Palgrave Macmillan, 177–197.

29.

Mahmoudian

(2020) varhandle: functions for robust variable handling. R package version 2.0.5. Available at: https://CRAN.R-project.org/package=varhandle (accessed 11 April 2020).

30.

Maliniak

Oakes

Peterson

, et al. (2011) International relations in the US academy. International Studies Quarterly 55(2): 437–464.

31.

Maliniak

Peterson

Powers

, et al. (2013) Codebook and User’s Guide for TRIP 2.0 Journal Article Database. Teaching, Research, and International Policy Project. Williamsburg, VA: Global Research Institute.

32.

Maliniak

Peterson

Powers

, et al. (2018) Is international relations a global discipline? Hegemony, insularity, and diversity in the field. Security Studies 27(3): 448–484.

33.

Meissner

(2016) Paradigms and theories: popular labels and their delimitation. In: Paradigms and Theories Influencing Policies in the South African and International Water Sectors. New York, NY: Springer International Publishing, 131–153.

34.

Metternich

Gleditsch

(2016) Forecasting in international relations. In: [Oxford Bibliographies in] International Relations. Oxford University Press. Available at: https://www.oxfordbibliographies.com/view/document/obo-9780199743292/obo-9780199743292-0179.xml (accessed 18 October 2019).

35.

Meyer

De Franco

Otto

(2019) Warning about War: Conflict, Persuasion and Foreign Policy. Cambridge: Cambridge University Press.

36.

Moritz

Bartz-Beielstein

(2017) imputeTS: time series missing value imputation in R. The R Journal 9(1): 207–218.

37.

Neuwirth

(2014) RColorBrewer: ColorBrewer palettes. R package version 1.1-2. Available at: https://CRAN.R-project.org/package=RColorBrewer (accessed 11 April 2020).

38.

Oatley

(2019) Toward a political economy of complex interdependence. European Journal of International Relations 25(4): 957–978.

39.

Peirce

(1902) Minute Logic: Chapter II. Prelogical Notions. Section I. Classification of the Sciences (Logic II). [MS [R] 427].

40.

R Core Team (2019) R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. Available at: https://www.R-project.org/ (accessed 18 October 2019).

41.

Raymond

(2018) Cyber futures and the justice motive: avoiding Pyrrhic victory. Military Cyber Affairs 3(1): 1–23.

42.

RStudio Team (2018) RStudio: Integrated Development for R. Boston, MA: RStudio, Inc. Available at: http://www.rstudio.com/ (accessed 18 October 2019).

43.

Schneider

Gleditsch

Carey

(2011) Forecasting in international relations. Conflict Management and Peace Science 28(1): 5–14.

44.

Schrodt

Gerner

(2000) Cluster-based early warning indicators for political change in the contemporary Levant. American Political Science Review 94(4): 803–817.

45.

Teaching, Research, and International Policy Project (2017) TRIP Journal Article Database Release (Version 3.1). Available at: https://trip.wm.edu/charts/ (accessed 18 October 2019).

46.

Torres-Soriano

Toboso-Buezo

(2019) Five terrorist dystopias. The International Journal of Intelligence, Security, and Public Affairs 21(1), 49–65.

47.

Whyte

(2019) Can we change the topic, please? Assessing the theoretical construction of International Relations scholarship. International Studies Quarterly 63(2): 432–447.

48.

Wickham

(2016) ggplot2: Elegant Graphics for Data Analysis. New York: Springer-Verlag.

49.

Wickham

Henry

(2019) tidyr: easily tidy data with ‘spread()’ and ‘gather()’ functions. R package version 0.8.3. Available at: https://CRAN.R-project.org/package=tidyr (accessed 18 October 2019).

50.

Wilke

(2019) cowplot: streamlined plot theme and plot annotations for ‘ggplot2’. R package version 1.0.0. Available at: https://CRAN.R-project.org/package=cowplot (accessed 18 October 2019).

51.

Windelband

(1894) Geschichte und Naturwissenschaft: Rede zum Antritt des Rectorats der Kaiser-Wilhelms-Universität Strassburg. Strassburg: Heitz.

52.

Windelband

(1998) History and natural science. Theory & Psychology 8(1): 5–22.

53.

Xiao

(2018) ggsci: Scientific Journal and Sci-Fi Themed Color Palettes for ‘ggplot2’. R package version 2.9. Available at: https://CRAN.R-project.org/package=ggsci (accessed 11 April 2020).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

1.12 MB

0.01 MB