Sage Journals: Discover world-class research

Abstract

Replication studies contribute to scientific self-correction and assessing the credibility of a given effect. This study examined the prevalence of replication studies published in Human Factors from 2020 to 2024 (N = 550). Of the studies coded, 8.59% explicitly identified replication as an aim. Of these, 19.51% were direct replications, 24.39% were conceptual replications, and 56.10% were direct replications with extensions. About three-quarters of replication efforts were conducted by non-independent teams (i.e., at least one author shared with the original publication) while the rest were independent (i.e., no authors in common with the original publication). These findings suggest that replication studies are more common in ergonomics and human factors than in some related fields, but that more independent replication efforts might be valuable.

Keywords

replication concepts methods open science meta-science

Get full access to this article

View all access options for this article.

References

Boekel

Wagenmakers

E.-J.

Belay

Verhagen

Brown

Forstmann

B. U.

(2015). A purely confirmatory replication study of structural brain-behavior correlations. Cortex, 66, 115–133. https://doi.org/10.1016/j.cortex.2014.11.019

Camerer

C. F.

Dreber

Forsell

T.-H.

Huber

Johannesson

Kirchler

Almenberg

Altmejd

Chan

Heikensten

Holzmeister

Imai

Isaksson

Nave

Pfeiffer

Razen

(2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433–1436. https://doi.org/10.1126/science.aaf0918

Clarke

Lee

P. Y. (Katherine)

Schiavone

S. R.

Rhemtulla

Vazire

(2024). The prevalence of direct replication articles in top-ranking psychology journals. American Psychologist. Advance online publication. https://doi.org/10.1037/amp0001385

de Winter

J. C. F.

Eisma

Y. B.

(2024). Ergonomics & human factors: Fade of a discipline. Ergonomics, 1–9. https://doi.org/10.1080/00140139.2024.2416553

Devezer

Navarro

D. J.

Vandekerckhove

Ozge Buzbas

(2021). The case for formal methodology in scientific reform. Royal Society Open Science, 8(3), 200805. https://doi.org/10.1098/rsos.200805

Edworthy

J. R.

DeLucia

P. R.

(2018). Whither replication? Human Factors: The Journal of the Human Factors and Ergonomics Society, 60(6), 741–742. https://doi.org/10.1177/0018720818790933

Errington

T. M.

Mathur

Soderberg

C. K.

Denis

Perfito

Iorns

Nosek

B. A.

(2021). Investigating the replicability of preclinical cancer biology. eLife, 10, e71601. https://doi.org/10.7554/eLife.71601

Jones

K. S.

Derby

P. L.

Schmidlin

E. A.

(2010). An Investigation of the Prevalence of Replication Research in Human Factors. Human Factors: The Journal of the Human Factors and Ergonomics Society, 52(5), 586–595. https://doi.org/10.1177/0018720810384394

Klein

R. A.

Ratliff

K. A.

Vianello

Adams

R. B.

Bahník

Š.

Bernstein

M. J.

Bocian

Brandt

M. J.

Brooks

Brumbaugh

C. C.

Cemalcilar

Chandler

Cheong

Davis

W. E.

Devos

Eisner

Frankowska

Furrow

Galliani

E. M.

. . . Nosek

B. A.

(2014). Investigating Variation in Replicability: A “Many Labs” Replication Project. Social Psychology, 45(3), 142–152. https://doi.org/10.1027/1864-9335/a000178

10.

Klein

R. A.

Vianello

Hasselman

Adams

B. G.

Adams

R. B.

Alper

Aveyard

Axt

J. R.

Babalola

M. T.

Bahník

Š.

Batra

Berkics

Bernstein

M. J.

Berry

D. R.

Bialobrzeska

Binan

E. D.

Bocian

Brandt

M. J.

Busching

. . . Nosek

B. A.

(2018). Many Labs 2: Investigating variation in replicability across samples and settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. https://doi.org/10.1177/2515245918810225

11.

Makel

M. C.

Plucker

J. A.

Hegarty

(2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7(6), 537–542. https://doi.org/10.1177/1745691612460688

12.

Moussaoui

J. R.

Marra-Dotts

Groetsch

Lichter

K. A.

Hyk

Johnson

Hoxha

Howell

F. G.

Baker, Doane

Munoz Gomez Andrade

McCarley

J. S.

(in press). Sample size justifications and open science practices in human factors (2020-2024). Proceedings of the Human Factors and Ergonomics Society Annual Meeting.

13.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716

14.

Pashler

Harris

C. R.

(2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531–536. https://doi.org/10.1177/1745691612463401

15.

Schmidt

(2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13(2), 90–100. https://doi.org/10.1037/a0015108

16.

Simons

D. J.

(2014). The value of direct replication. Perspectives on Psychological Science, 9(1), 76–80. https://doi.org/10.1177/1745691613514755

17.

Van Dongen

Van Bork

Finnemann

Haslbeck

J. M. B.

Van Der Maas

H. L. J.

Robinaugh

D. J.

De Ron

Sprenger

Borsboom

(2025). Productive explanation: A framework for evaluating explanations in psychological science. Psychological Review, 132(2), 311–329. https://doi.org/10.1037/rev0000479

18.

Warm

J. S.

Finomore

V. S.

Vidulich

M. A.

Funke

M. E.

(2015). Vigilance: A Perceptual Challenge. In Hoffman

R. R.

Hancock

P. A.

Scerbo

M. W.

Parasuraman

Szalma

J. L. E.

(Eds.), The Cambridge handbook of applied perception research (pp. 241–283). Cambridge University Press.

19.

Wickens

C. D.

(2008). Multiple Resources and Mental Workload. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(3), 449–455. https://doi.org/10.1518/001872008X288394

The Prevalence of Replication Studies in Human Factors (2020-2024)

Abstract

Keywords

Get full access to this article

References