Extremity Bias in Online Reviews: The Role of Attrition

Abstract

In a range of studies across platforms, researchers have shown that online ratings are characterized by distributions with disproportionately heavy tails. The authors of this study focus on understanding the underlying process that yields such “J-shaped” or “extreme” distributions. They propose a novel theoretical mechanism behind the emergence of J-shaped distributions: differential attrition, or the idea that potential reviewers with moderate experiences are more likely to leave the pool of active reviewers than potential reviewers with extreme experiences. The authors present an analytical model that integrates this mechanism with two extant mechanisms: differential utility and base rates. They show that although all three mechanisms can give rise to extreme distributions, only the utility-based and attrition-based mechanisms can explain the authors’ empirical observation from a large-scale field experiment that an unincentivized solicitation email from an online travel platform reduces review extremity. Subsequent analyses provide clear empirical evidence for the existence of both differential attrition and differential utility.