The statistic Prep estimates the probability of replicating an effect. It captures traditional publication criteria for signal-to-noise ratio, while avoiding parametric inference and the resulting Bayesian dilemma. In concert with effect size and replication intervals, Prep provides all of the information now used in evaluating research, while avoiding many of the pitfalls of traditional statistical inference.
Get full access to this article
View all access options for this article.
References
1.
BergerJ.O.SelkeT. (1987). Testing a point null hypothesis: The irreconcilability of P values and evidence. Journal of the American Statistical Association, 82, 112–122.
2.
BruceP. (2003). Resampling stats in Excel [Computer software]. Retrieved February 1, 2005, from http://www.resample.com
3.
BurnhamK.P.AndersonD.R. (2002). Model selection and multimodel inference: A practical information-theoretic approach (2nd ed.). New York: Springer-Verlag.
4.
CohenJ. (1969). Statistical power analysis for the behavioral sciences. New York: Academic Press.
5.
CohenJ. (1994). The earth is round (p < .05). American Psychologist, 49, 997–1003.
6.
CooperH.HedgesL.V. (Eds.). (1994). The handbook of research synthesis. New York: Russell Sage Foundation.
CoxR.T. (1961). The algebra of probable inference. Baltimore: Johns Hopkins University Press.
9.
CummingG.FinchS. (2001). A primer on the understanding, use and calculation of confidence intervals based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532–575.
10.
CummingG.WilliamsJ.FidlerF. (2004). Replication, and researchers' understanding of confidence intervals and standard error bars. Understanding Statistics, 3, 299–311.
11.
DarwinC.. (1994). The correspondence of Charles Darwin (Vol. 9; BurkhardtF.BrowneJ.PorterD.M.RichmondM., Eds.). Cambridge, England: Cambridge University Press.
12.
EaglyA.H.Johannesen-SchmidtM.C.van EngenM.L. (2003). Transformational, transactional, and laissez-faire leadership styles: A meta-analysis comparing men and women. Psychological Bulletin, 129, 569–591.
13.
EstesW.K. (1997). On the communication of information by displays of standard errors and confidence intervals. Psychonomic Bulletin & Review, 4, 330–341.
14.
FisherR.A. (1925). Theory of statistical estimation. Proceedings of the Cambridge Philosophical Society, 22, 700–725.
15.
FisherR.A. (1959). Statistical methods and scientific inference (2nd ed.). New York: Hafner Publishing.
16.
GeisserS. (1992). Introduction to Fisher (1922): On the mathematical foundations of theoretical statistics. In KotzS.JohnsonN.L. (Eds.), Breakthroughs in statistics (Vol. 1, pp. 1–10). New York: Springer-Verlag.
17.
GreenwaldA.G.GonzalezR.GuthrieD.G.HarrisR.J. (1996). Effect sizes and p values: What should be reported and what should be replicated?Psychophysiology, 33, 175–183.
18.
GrissomR.J.KimJ.J. (2001). Review of assumptions and problems in the appropriate conceptualization of effect size. Psychological Methods, 6, 135–146.
19.
HarlowL.L.MulaikS.A.SteigerJ.H. (Eds.). (1997). What if there were no significance tests?Mahwah, NJ: Erlbaum.
20.
HedgesL.V. (1981). Distribution theory for Glass's estimator of effect sizes and related estimators. Journal of Educational Statistics, 6, 107–128.
21.
HedgesL.V.OlkinI. (1985). Statistical methods for meta-analysis. New York: Academic Press.
22.
HedgesL.V.VeveaJ.L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486–504.
23.
JaynesE.T.BretthorstG.L. (2003). Probability theory: The logic of science. Cambridge, England: Cambridge University Press.
24.
KrantzD.H. (1999). The null hypothesis testing controversy in psychology. Journal of the American Statistical Association, 44, 1372–1381.
25.
KruegerJ. (2001). Null hypothesis significance testing: On the survival of a flawed method. American Psychologist, 56, 16–26.
26.
LoftusG.R. (1996). Psychology will be a much better science when we change the way we analyze data. Current Directions in Psychological Science, 5, 161–171.
27.
LorberM.F. (2004). Psychophysiology of aggression, psychopathy, and conduct problems: A meta-analysis. Psychological Bulletin, 130, 531–552.
28.
LouisT.A.ZeltermanD. (1994). Bayesian approaches to research synthesis. In CooperH.HedgesL.V. (Eds.), The handbook of research synthesis (pp. 411–422). New York: Russell Sage Foundation.
29.
MeehlP.E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806–834.
30.
MeehlP.E. (1997). The problem is epistemology, not statistics: Replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions. In HarlowL.L.MulaikS.A.SteigerJ.H. (Eds.), What if there were no significance tests? (pp. 393–425). Mahwah, NJ: Erlbaum.
31.
MillerN.PollockV.E. (1994). Meta-analytic synthesis for theory development. In CooperH.HedgesL.V. (Eds.), The handbook of research synthesis (pp. 457–484). New York: Russell Sage Foundation.
32.
MostellerF.ColditzG.A. (1996). Understanding research synthesis (meta-analysis). Annual Review of Public Health, 17, 1–23.
33.
MoyerC.A.RoundsJ.HannumJ.W. (2004). A meta-analysis of massage therapy research. Psychological Bulletin, 130, 3–18.
34.
NeymanJ.PearsonE.S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London, Series A, 231, 289–337.
35.
NickersonR.S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241–301.
36.
ParkinsonS.R. (2004). [Levels of processing experiments in a methods class]. Unpublished raw data.
37.
RaudenbushS.W. (1994). Random effects models. In CooperH.HedgesL.V. (Eds.), The handbook of research synthesis (pp. 301–321). New York: Russell Sage Foundation.
38.
RichardF.D.BondC.F.Jr.Stokes-ZootaJ.J. (2003). One hundred years of social psychology quantitatively described. Review of General Psychology, 7, 331–363.
39.
RosenthalR. (1994). Parametric measures of effect size. In CooperH.HedgesL.V. (Eds.), The handbook of research synthesis (pp. 231–244). New York: Russell Sage Foundation.
RossiJ.S. (1997). A case study in the failure of psychology as a cumulative science: The spontaneous recovery of verbal learning. In HarlowL.L.MulaikS.A.SteigerJ.H. (Eds.), What if there were no significance tests? (pp. 175–197). Mahwah, NJ: Erlbaum.
42.
RubinD.B. (1981). Estimation in parallel randomized experiments. Journal of Educational Statistics, 6, 377–400.
SteigerJ.H.FouladiR.T. (1997). Noncentrality interval estimation and the evaluation of statistical models. In HarlowL.L.MulaikS.A.SteigerJ.H. (Eds.), What if there were no significance tests? (pp. 221–257). Mahwah, NJ: Erlbaum.
ThompsonB. (2002). What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher, 31(3), 25–32.
47.
TrafimowD. (2003). Hypothesis testing and theory evaluation at the boundaries: Surprising insights from Bayes's theorem. Psychological Review, 110, 526–535.
48.
van den NoortgateW.OnghenaP. (2003). Estimating the mean effect size in meta-analysis: Bias, precision, and mean squared error of different weighting methods. Behavior Research Methods, Instruments, & Computers, 35, 504–511.
49.
WilkinsonL.the Task Force on Statistical Inference. (1999). Statistical methods in psychology: Guidelines and explanations. American Psychologist, 54, 594–604.
