Lessons from near real-time forecasting of irregular leadership changes

Abstract

Since 2014, we have been producing regular six-month forecasts of the probability of irregular leadership changes – coups, rebellions, protests that result in state leader changes – for most countries in the world for the Political Instability Task Force (PITF). During 2015, we issued new forecasts each month, with a delay as short as five days and no longer than two weeks into each six-month forecasting window. This article describes the approach we use to generate our forecasts and presents several examples of how we present forecasts. The forecasts are derived from a statistical ensemble of seven thematic models, each based on a split-population duration model that aims to capture a specific argument or related set of covariates. This approach is modular in that thematic models can be swapped out or new models integrated, and it lessens the need for generalist ‘kitchen sink’ models. Together, the models achieve high out-of-sample accuracy. Based on our experience, we draw conclusions about the practical, policy, and scientific aspects of this and similar undertakings. These include thoughts on how to evaluate and present forecasts, the potential role of ensembles in model comparison, the role of ensembles and prediction in causal research, and considerations for future efforts in forecasting and predictive modeling.