International Comparison of Public Sector Performance: The Use of Anchoring Vignettes to adjust Self-Reported Data

Abstract

International comparison of performance has become an influential lever for change in the provision of public services. For health care, patients’ views and opinions are increasingly being recognized as legitimate means for assessing the provision of services, to stimulate quality improvements, and more recently, in evaluating system performance. This has shifted the focus of analyses towards the use of individual-level surveys of performance from the perspective of the user and raises the issue of how to compare appropriately self-reported data across institutional settings and population groups. Using data on health systems responsiveness across 17 EU countries contained within the World Health Survey, this paper outlines the issues that arise in comparative inference that relies on respondent self-reports. The problem of systematic reporting behaviour is described and illustrated together with potential solutions brought about through the use of anchoring vignettes. Our results show large differences in the rankings of country performance once adjustment for systematic country-level reporting behaviour has been undertaken compared to the ranking observed in the raw unadjusted data. The use of anchoring vignettes as a means to obtain comparability in self-reported survey instruments of performance promises to be a fundamental tool for international comparative research.