Whose Variance Is It Anyway? Interpreting Empirical Models with State-Level Data

Abstract

Researchers commonly apply inferential statistical procedures to population data from the 50 U.S. states as if they were estimating population parameters from sample statistics. This method is incorrect because with population data there is no need to make inferences about quantities that are already known. Instead, authors should simply provide evidence that their specified model provides a good fit to the data. Summary measures of variance as well as the full engine of Bayesian statistics perform this function. This research note demonstrates why the current practice of making inferences from population data with the null hypothesis significance test is wrong, provides some specific examples of problems in the literature, and gives prescriptive advice about correctly assessing and conveying empirical model results.