The Moral Limits of Predictive Practices: The Case of Credit-Based Insurance Scores

Abstract

Corporations gather massive amounts of personal data to predict how individuals will behave so that they can profitably price goods and allocate resources. This article investigates the moral foundations of such increasingly prevalent market practices. I leverage the case of credit scores in car insurance pricing—an early and controversial use of algorithmic prediction in the U.S. consumer economy—to unpack the premise that predictive data are fair to use and to understand the conditions under which people are likely to challenge that moral logic. Policymaker resistance to credit-based insurance scores reveals that contention arises when predictions depend on mathematical distinctions that do not align with broader understandings of good and bad behavior, and when theories about why predictions work point to the market holding people accountable for actions that are not really their fault. Via a de-commensuration process, policymakers realign the market with their own notions of moral deservingness. This article thus demonstrates the importance of causal understanding and moral categorization for people accepting markets as fair. As data and analytics permeate markets of all sorts, as well as other domains of social life, these findings have implications for how social scientists understand the novel forms of stratification that result.