We analyze the trajectory of Organizational Research Methods (ORM) during the first 20 years of its existence (i.e., 1998-2017). First, beginning with the inaugural volume, we review the editorials to create a qualitative account regarding ORM’s journey as narrated by the journal’s leaders in their own voices. Second, we examine the composition of the five senior editorial teams (i.e., editors and associate editors), including their qualitative-quantitative, micro-macro, and disciplinary orientation, as well as the types of articles published by ORM along the qualitative-quantitative and micro-macro distinctions. Third, we describe the 27 feature topics (i.e., set of articles addressing a common issue) published by ORM. Fourth, we offer information regarding ORM’s impact and influence based on impact factor data, journal lists, and other indicators (e.g., ORM articles that have received awards from professional organizations, most cited ORM articles out of a total of 484). Fifth, we identify the most frequently published ORM authors (and their disciplinary background) out of a total of 884 who have published at least one article. Finally, we discuss implications and outline opportunities and challenges as well as possible future directions for ORM. Overall, our review and analysis of the first 20 years of ORM allowed us to create a historical record for future generations, gain qualitative and quantitative insights into ORM’s trajectory and its impact and influence over time, and make predictions for the future of the journal and, more broadly, research on methodological issues.
AdlerN. J.HarzingA. (2009). When knowledge wins: Transcending the sense and nonsense of academic rankings. Academy of Management Learning & Education, 8, 72–95.
2.
AguinisH. (2005). Editorial. Organizational Research Methods, 8, 7–8.
3.
AguinisH. (2006). Editorial. Organizational Research Methods, 9, 3–4.
4.
AguinisH. (2007a). Editorial. Organizational Research Methods, 10, 3–4.
5.
AguinisH. (2007b). Organizational Research Methods update: New editor appointed. Organizational Research Methods, 10, 191.
6.
AguinisH.BoydB. K.PierceC. A.ShortJ. C. (2011). Walking new avenues in management research methods and theories: Bridging micro and macro domains. Journal of Management, 37, 395–403.
7.
AguinisH.BradleyK. J.BrodersenA. (2014). Industrial-organizational psychologists in business schools: Brain drain or eye opener?. Industrial and Organizational Psychology, 7, 284–303.
8.
AguinisH.CascioW. F.RamaniR. S. (2017). Science’s reproducibility and replicability crisis: International business is not immune. Journal of International Business Studies, 48, 653–663.
9.
AguinisH.GottfredsonR. K.JooH. (2013). Best-practice recommendations for defining, identifying, and handling outliers. Organizational Research Methods, 16, 270–301.
10.
AguinisH.JiY. H.JooH. (2018). Gender productivity gap among star performers in STEM and other scientific fields. Journal of Applied Psychology, 103, 1283–1306.
11.
AguinisH.Molina-AzorínJ. F. (2015). Using multilevel modeling and mixed methods to make theoretical progress in microfoundations for strategy research. Strategic Organization, 13, 353–364.
12.
AguinisH.PierceC. A.BoscoF. A.MuslinI. S. (2009). First decade of Organizational Research Methods: Trends in design, measurement, and data-analysis topics. Organizational Research Methods, 12, 69–112.
13.
AguinisH.RamaniR. S.AlabduljaderN. (2018). What you see is what you get? Enhancing methodological transparency in management research. Academy of Management Annals, 12, 1–28.
14.
AguinisH.RamaniR. S.CampbellP. K.Bernal-TurnesP.DrewryJ. M.EdgertonB. T. (2017). Most frequently cited sources, articles, and authors in industrial-organizational psychology textbooks: Implications for the science-practice divide, scholarly impact, and the future of the field. Industrial and Organizational Psychology, 10, 1–51.
15.
AguinisH.ShapiroD. L.AntonacopoulouE.CummingsT. G. (2014). Scholarly impact: A pluralist conceptualization. Academy of Management Learning & Education, 13, 623–639.
16.
AguinisH.Suarez-GonzálezI.LannelongueG.JooH. (2012). Scholarly impact revisited. Academy of Management Perspectives, 26, 105–132.
17.
AguinisH.VandenbergR. J. (2014). An ounce of prevention is worth a pound of cure: Improving research quality before data collection. Annual Review of Organizational Psychology and Organizational Behavior, 1, 569–595.
18.
BanksG. C.O’BoyleE. H.PollackJ. M.WhiteC. D.BatchelorJ. H.WhelpleyC. E.…AdkinsC. L. (2016). Questions about questionable research practices in the field of management: A guest commentary. Journal of Management, 42, 5–20.
19.
BerghD. D.AguinisH.HeaveyC.KetchenD. J.BoydB. K.SuP.…JooH. (2016). Using meta-analytic structural equation modeling to advance strategic management research: Guidelines and an empirical illustration via the strategic leadership-performance relationship. Strategic Management Journal, 37, 477–497.
20.
BettisR. A. (2012). The search for asterisks: Compromised statistical tests and flawed theories. Strategic Management Journal, 33, 108–113.
21.
BoscoF. A.AguinisH.FieldJ. G.PierceC. A.DaltonD. R. (2016). HARKing’s threat to organizational research: Evidence from primary and meta-analytic sources. Personnel Psychology, 69, 709–750.
22.
ButlerN.DelaneyH.SpoelstraS. (2017). The gray zone: Questionable research practices in the business school. Academy of Management Learning & Education, 16, 94–109.
23.
CertoS. T.BusenbarkJ. R.WooH.SemadeniM. (2016). Sample selection bias and Heckman models in strategic management research. Strategic Management Journal, 37, 2639–2657.
24.
CertoS. T.WithersM. C.SemadeniM. (2017). A tale of two effects: Using longitudinal data to compare within- and between-firm effects. Strategic Management Journal, 38, 1536–1556.
25.
CheungG. W.LauR. S. (2008). Testing mediation and suppression effects of latent variables: Bootstrapping with structural equation models. Organizational Research Methods, 11, 296–325.
26.
CombsJ. G.2010. Big samples and small effects: Let’s not trade relevance and rigor for power. Academy of Management Journal, 53, 9–13.
27.
CortinaJ. M. (2011). Editorial. Organizational Research Methods, 14, 6–9.
28.
CortinaJ. M.AguinisH.DeShonR. P. (2017). Twilight of dawn or of evening? A century of research methods in the Journal of Applied Psychology. Journal of Applied Psychology, 102, 274–290.
29.
EderR. W. (1998). Welcome to Organizational Research Methods. Organizational Research Methods, 1, 5.
30.
GardnerD. M.RyanA. M.SnoeyinkM. (2018). How are we doing? An examination of gender representation in I-O psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 11, 369–388.
31.
GioiaD. A.CorleyK. G.HamiltonA. L. (2013). Seeking qualitative rigor in inductive research: Notes on the Gioia methodology. Organizational Research Methods, 16, 15–31.
32.
HaytonJ. C.AllenD. G.ScarpelloV. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7, 191–205.
33.
HenselerJ.DijkstraT. K.SarstedtM.RingleC. M.DiamantopoulosA.StraubD. W.CalantoneR. J. (2014). Common beliefs and reality about PLS: Comments on Rönkkö and Evermann (2013). Organizational Research Methods, 17, 182–209.
34.
HinkinT. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104–121.
35.
HowardG. S.ColeD. A.MaxwellS. E. (1987). Research productivity in psychology based on publication in the journals of the American Psychological Association. American Psychologist, 42, 975–986.
36.
KacmarK. M.WhitfieldJ. M. (2000). An additional rating method for journal articles in the field of management. Organizational Research Methods, 3, 392–406.
37.
LeBaronC.JarzabkowskiP.PrattM. G.FetzerG. (2018). An introduction to video methods in organizational research. Organizational Research Methods, 21, 239–260.
38.
LeBretonJ. M. (2014). Editorial. Organizational Research Methods, 17, 113–117.
39.
LeBretonJ. M. (2015). Editorial. Organizational Research Methods, 18, 575–577.
40.
LeBretonJ. M. (2016a). Editorial. Organizational Research Methods, 19, 3–7.
41.
LeBretonJ. M. (2016b). Editorial. Organizational Research Methods, 19, 351.
42.
LeBretonJ. M. (2017). Editorial. Organizational Research Methods, 20, 547–548.
43.
LeBretonJ. M.SenterJ. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852.
44.
Molina-AzorínJ. F. (2012). Mixed methods research in strategic management: Impact and applications. Organizational Research Methods, 15, 33–56.
45.
Molina-AzorínJ. F.BerghD. D.CorleyK. G.KetchenD. J. (2017). Mixed methods in the organizational sciences. Organizational Research Methods, 20, 179–192.
46.
O’BoyleE. H.BanksG. C.Gonzalez-MuléE. (2017). The chrysalis effect: How ugly initial results metamorphize into beautiful articles. Journal of Management, 43, 376–399.
ShookC. L.KetchenD. J.CycyotaC. S.CrockettD. (2003). Data analytic trends and training in strategic management. Strategic Management Journal, 24, 1231–1237.
49.
SiemsenE.RothA.OliveiraP. (2010). Common method bias in regression models with linear, quadratic, and interaction effects. Organizational Research Methods, 13, 456–476.
50.
SpectorP. E. (2006). Method variance in organizational research: Truth or urban legend?Organizational Research Methods, 9, 221–232.
51.
TaylorA. B.MacKinnonD. P.TeinJ. Y. (2008). Tests of the three-path mediated effect. Organizational Research Methods, 11, 241–269.
52.
TsuiA. S. (2013). Editorial: The spirit of science and socially responsible scholarship. Management and Organization Review, 9, 375–394.
53.
Van FleetD. D.BedeianA. J. (2016). The Journal of Management’s first 40 years: A look back. Journal of Management, 42, 349–356.
54.
VandenbergR. J. (2008). Editorial. Organizational Research Methods, 11, 6–8.
55.
VandenbergR. J. (2009). Editorial. Organizational Research Methods, 12, 3–5.
56.
VandenbergR. J. (2010a). Announcement. Organizational Research Methods, 13, 223.
57.
VandenbergR. J. (2010b). Editorial. Organizational Research Methods, 13, 3–5.
58.
VandenbergR. J. (2011). Editorial. Organizational Research Methods, 14, 3–5.
59.
VandenbergR. J.LanceC. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70.
60.
WilliamsL. J. (1998). From the editor. Organizational Research Methods, 1, 3–4.
61.
WilliamsL. J. (1999). Editor’s note. Organizational Research Methods, 2, 3–4.
62.
WilliamsL. J. (2001). Editor’s update. Organizational Research Methods, 4, 193.
63.
WilliamsL. J. (2005). Editorial. Organizational Research Methods, 8, 5–6.
64.
WilliamsL. J. (2008). Reflections on creating a new scholarly journal: Perspectives from a founding editor. In BaruchY.KonradA. M.AguinisH.StarbuckW. H. (Eds.), Opening the black box of editorship (pp. 188–196). New York, NY: Palgrave Macmillan.
65.
ZhangZ.ZyphurM. J.PreacherK. J. (2009). Testing multilevel mediation using hierarchical linear models: Problems and solutions. Organizational Research Methods, 12, 695–719.
66.
ZickarM. J.HighhouseS. (2001). Measuring prestige of journals in industrial-organizational psychology. The Industrial-Organizational Psychologist, 38(4), 29–36.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
