Registered Reports,Replication,and the Canadian Journal of School Psychology : Improving the Evidence in Evidence-Based School Psychology

Abstract

The Canadian Journal of School Psychology (CJSP) is offering scholars the opportunity to register research reports and make research protocols publicly available to promote replication, transparency, credibility, and utility for clinical practice. The purpose of this article is to outline the challenges regarding replication, reproducibility, and evidence-based practices, as well as describe the submission protocol and criteria for acceptance of registered reports. Advances and criticisms of the registered reports model are discussed. Although CJSP will accept submissions through the traditional peer-review model, registered reports and support of replication studies have the objective of promoting high-quality research to improve the research foundation for evidence-based practices in the profession of school psychology.

Keywords

editorial evidence-based interventions improving research registered reports replications

Get full access to this article

View all access options for this article.

References

Allen

C. P. G.

Mehler

D. M. A.

(2018, October 18). Open Science challenges, benefits and tips in early career and beyond. doi:10.31234/osf.io/3czyt

Anderson

S. F.

Kelley

Maxwell

S. E.

(2017). Sample-size planning for more accurate statistical power: A method adjusting sample effect sizes for publication bias and uncertainty. Psychological Science, 28, 1547-1562. doi:10.1177/0956797617723724

Baker

(2016). 1,500 scientists lift the lid on reproducibility. Nature, 533, 452-454. doi:10.1038/533452a

Chambers

C. D.

(2013). Registered reports: A new publishing initiative at Cortex. Cortex, 49, 609-610. doi:10.1016/j.cortex.2012.12.016

Chambers

C. D.

(2017). The seven deadly sins of psychology: A manifesto for reforming the culture of scientific practice. Princeton, NJ: Princeton University Press.

Chambers

C. D.

Dienes

McIntosh

R. D.

Rothstein

Willmes

(2015). Registered reports: Realigning incentives in scientific publishing. Cortex, 66, A1-A2. doi:10.1016/j.cortex.2015.03.022

Dwan

Gamble

Williamson

P. R.

Kirkham

J. J.

(2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias: An updated review. PLoS ONE, 8, e66844. doi:10.1371/journal.pone.0066844

Ferguson

C. J.

Heene

(2012). A vast graveyard of undead theories publication bias and psychological science’s aversion to the null. Perspectives on Psychological Science, 7, 555-561. doi:10.1177/1745691612459059

Fraley

R. C.

Vazire

(2014). The N-pact factor: Evaluating the quality of empirical journals with respect to sample size and statistical power. PLoS ONE, 9, e109019. doi:10.1371/journal.pone.0109019

10.

Garfield

(2006). The history and meaning of the journal impact factor. The Journal of the American Medical Association, 295, 90-93. doi:10.1001/jama.295.1.90

11.

Greenhalgh

Howick

Maskrey

(2014). Evidence based medicine: A movement in crisis? British Medical Journal, 348, Article g3725. doi:10.1136/bmj.g3725

12.

Guyatt

Cairns

Churchill

Cook

Haynes

Hirsh

. . . Tugwell

(1992). Evidence-based medicine: A new approach to teaching the practice of medicine. Journal of the American Medical Association, 268, 2420-2425. doi:10.1001/jama.1992.03490170092032

13.

Ioannidis

J. P. A.

(2005). Why most published research findings are false. PLoS Medicine, 2, e124. doi:10.1371/journal.pmed.0020124

14.

Ioannidis

J. P. A.

(2016a). The reproducibility wars: Successful, unsuccessful, uninterpretable, exact, conceptual, triangulated, contested replication. Clinical Chemistry, 63, 943-945. doi:10.1373/clinchem.2017.271965

15.

Ioannidis

J. P. A.

(2016b). Why most clinical research is not useful. PLoS Medicine, 13, e1002049. doi:110020490.1371/journal.pmed

16.

Ioannidis

J. P. A.

(2017). Acknowledging and overcoming nonreproducibility in basic and preclinical research. Journal of the American Medical Association, 317, 1019-1020. doi:10.1001/jama.2017.0549

17.

Jefferson

Alderson

Wager

Davidoff

(2002). Effects of editorial peer review: A systematic review. Journal of the American Medical Association, 287, 2784-2786. doi:10.1001/jama.287.21.2784

18.

John

L. K.

Loewenstein

Prelec

(2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524-532.

19.

Kühberger

Fritz

Scherndl

(2014). Publication bias in psychology: A diagnosis based on the correlation between effect size and sample size. PLoS ONE, 9, e105825. doi:10.1371/journal.pone.0105825

20.

La Greca

A. M.

Silverman

W. K.

Lochman

J. E

. (2009). Moving beyond efficacy and effectiveness in child and adolescent intervention research. Journal of Consulting and Clinical Psychology, 77, 373-382. doi:10.1037/a0015954

21.

Levant

R. F.

Hasan

N. T.

(2008). Evidence-based practice in psychology. Professional Psychology: Research and Practice, 39, 658-662. doi:10.1037/0735-7028.39.6.658

22.

Lilienfeld

S. O.

(2017). Psychology’s replication crisis and the grant culture: Righting the ship. Perspectives on Psychological Science, 12, 660-664. doi:10.1177/1745691616687745

23.

Makel

M. C.

Plucker

J. A.

(2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43, 304-316. doi:10.3102/0013189X14545513

24.

Maxwell

S. E.

Lau

M. Y.

Howard

G. S.

(2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70, 487-498. doi:10.1037/a0039400

25.

McGill

Ward

T. J.

Canivaz

G. L.

(2018). On the evidential value of school psychology intervention research. The School Psychologist, 72, 48-57.

26.

Mellor

(2017). Promoting reproducibility with registered reports. Nature Human Behaviour, 1, Article 0034. doi:10.1038/s41562-016-0034

27.

Nosek

B. A.

Alter

Banks

G. C.

Borsboom

Bowman

S. D.

Breckler

S. J.

. . . Yarkoni

(2015). Promoting an open research culture. Science, 348, 1422-1425. doi:10.1126/science.aab2374

28.

Nosek

B. A.

Ebersole

C. R.

DeHaven

A. C.

Mellor

D. T.

(2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115, 2600-2606. doi:10.1073/pnas.1708274114

29.

Nosek

B. A.

Lakens

(2014). Registered reports: A method to increase the credibility of published results. Social Psychology, 45, 137-141. doi:10.1027/1864-9335/a000192

30.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, Article aac4716. doi:10.1126/science.aac4716

31.

Rosenthal

(1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638. doi:10.1037/0033-2909.86.3.638

32.

Rycroft-Malone

Seers

Titchen

Harvey

Kitson

McCormack

(2004). What counts as evidence in evidence-based practice? Journal of Advanced Nursing, 47, 81-90. doi:10.1111/j.1365-2648.2004.03068.x

33.

Shaw

S. R.

Lysenko

Wang

Nieva Bitencourt

A. C

. (2018, June). Improving the scientific foundations of evidence-based practices in school psychology: The role of registered reports and replications. Paper presented to the International Congress of Applied Psychology/Canadian Psychologist Association, Montreal, Quebec, Canada.

34.

Simmons

J. P.

Nelson

L. D.

Simonsohn

(2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366. doi:10.1177/0956797611417632

35.

Tajika

Ogawa

Takeshima

Hayasaka

Furukawa

T. A.

(2015). Replication and contradiction of highly cited research papers in psychiatry: 10-year follow-up. The British Journal of Psychiatry, 207, 357-362. doi:10.1192/bjp.bp.113.143701

36.

van der Zee

Reich

. (2018). Open education science. AERA Open, 5, 1-15. doi:10.1177/2332858418787466