P-Values are not Sufficient but they are Necessary: Probing the Role and Application of Statistical Significance Testing in Public Administration Research

Abstract

P-values are important to assert the relevance of findings. They should be reported in statistical studies. Although presented differently, they rely on the same sufficient statistics as confidence intervals, and they provide more information. They are a necessary, but not sufficient, condition to communicating practical significance. Effect size, or magnitude analysis, is missing in current Public Administration studies and should be at the core of ensuing discussions. Practitioners should indeed be able to know if the effects on the dependent variable are important in magnitude, but also if they can be acted upon through a cause-to-effect relationship. As we have yet to fight the old battle of p-values, Public Administrationists might be repeating the same errors that led psychologists and economists to delay their use of sound experimental designs instead of reported results from underpowered studies. In other words, the methodological turn that we should discuss and (we hope) embrace is the combined use of proper identification strategies and larger sample sizes.