We examine the criticisms and subsequent changes that arise in the course of peer review. Fifty-two scholars who had recently published in Administrative Science Quarterly were surveyed regarding their peer review experience and how their article changed from initial journal submission to eventual publication. Papers that challenged theoretical perspectives faced distinctively high levels of criticism and change, particularly with attention to methodology, while those that offered a new perspective or that extended or combined established perspectives were less criticized and changed. The number of challenge-oriented publications was small as well, suggesting that either few such submissions survive the review process or few are submitted in the first place. Overall, peer review appears open to expansion of the variety of theoretical argument but does little to aid in the winnowing out of established perspectives.
AudiaPino G.GoncarloJack A.. 2007. “Past Success and Creativity over Time: A Study of Inventors in the Hard Disk Drive Industry.” Management Science53 (1): 1–15.
4.
BedeianArthur G.2003. “The Manuscript Review Process: The Proper Roles of Authors, Referees, and Editors.” Journal of Management Inquiry12 (4): 331–38.
BeyerJanice M.ChanoveRoland G.FoxWilliam B.. 1995. “The Review Process and the Fates of Manuscripts Submitted to AMJ.” Academy of Management Journal38 (5): 1219–60.
7.
BornmannLutzDanielHans-Dieter. 2008. “What Do Citation Counts Measure? A Review of Studies on Citing Behavior.” Journal of Documentation64 (1): 45–80.
8.
BoudreauKevin J.GuinanEvaLakhaniKarimRiedlChistoph. 2016. “Looking across and Looking beyond the Knowledge Frontier: Intellectual Distance and Resource Allocation in Science.” Management Science62 (10): 2765–83.
9.
BourdieuPierre. 1988. Homo Academicus. Stanford, CA: Stanford University Press.
10.
BrembsBjörnButtonKatherineMunafòMarcus. 2013. “Deep Impact: Unintended Consequences of Journal Rank.” Frontiers in Human Neuroscience7:1–12.
11.
BurtRonald S.2004. “Structural Holes and Good Ideas.” American Journal of Sociology110 (2): 349–99.
12.
DasguptaParthaDavidPaul. 2002. “Toward a New Economics of Science.” In Science Bought and Sold, edited by MirowskiP.SentE., 219–48. Chicago: University of Chicago Press.
13.
DavisGerald F.2015. “Editorial Essay: What Is Organizational Research For?” Administrative Science Quarterly60 (2): 179–88.
14.
DavisGerald F.MarquisChristopher. 2005. “Prospects for Organizational Theory in the Early 21st Century: Institutional Fields and Mechanisms.” Organization Science16 (4):332–43.
15.
DavisMurray S.1971. “That’s Interesting! Towards a Phenomenology of Sociology and a Sociology of Phenomenology.” Philosophy of the Social Sciences1 (4):309–44.
16.
DenrellJerkerMarchJames G.. 2001. “Adaptation as Information Restriction: The Hot Stove Effect.” Organization Science12 (5): 523–38.
17.
DornbuschSanford M.Richard ScottW.. 1975. Evaluation and the Exercise of Authority. San Francisco, CA: Jossey-Bass.
FlynnFrancisChatmanJennifer A.. 2001. “Strong Cultures and Innovation: Oxymoron or Opportunity?” In International Handbook of Organizational Culture and Climate, edited by CartwrightS.CooperC.EarleyC.ChatmanJ.CummingsT.HoldenN.SparrowP.StarbuckW., 263–87. Sussex, UK: John Wiley.
20.
FosterJacob G.RzhetskyAndreyEvansJames A.. 2015. “Tradition and Innovation in Scientists’ Research Strategies.” American Sociological Review80 (5): 875–908.
21.
FoxCraig R.TverskyAmos. 1995. “Ambiguity Aversion and Comparative Ignorance.” Quarterly Journal of Economics110 (3): 585–603.
22.
GansJoshua S.ShepherdGeorge B.. 1994. “How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists.” Journal of Economic Perspectives8 (1): 165–79.
23.
GrossAlan. 1988. “On the Shoulders of Giants: Seventeenth-century Optics as an Argument Field.” Quarterly Journal of Speech74:1–17.
24.
GuetzkowJoshuaLamontMichèleMallardGrégoire. 2004. “What Is Originality in the Humanities and the Social Sciences?” American Sociological Review69 (2): 190–212.
25.
HambrickDonald C.2007. “The Field of Management’s Devotion to Theory: Too Much of a Good Thing?” Academy of Management Journal50 (6): 1346–52.
26.
HamermeshDaniel S.1994. “Facts and Myths about Refereeing.” Journal of Economic Perspectives8 (1): 153–63.
27.
HorrobinDavid F.1990. “The Philosophical Basis of Peer Review and the Suppression of Innovation.” Journal of the American Medical Association263 (10): 1438–41.
28.
IoannidisJ. P. A. 2005. “Why Most Published Research Findings Are False.” PLoS One Medicine2 (8): 696–701.
29.
JuhaszStephenCalvertEarlJacksonTomKronickDavid A.ShipmanJoseph. 1975. “Acceptance and Rejection of Manuscripts.” IEEE Transactions on Professional Communication, 18 (3): 177–85.
KerrS.TolliverJ.PetreeD.. 1977. “Manuscript Characteristics Which Influence Acceptance for Management and Social Science Journals.” Academy of Management Journal20 (1): 132–41.
32.
KuhnThomas. 1962. The Structure of Scientific Revolutions. Chicago: University of Chicago Press.
33.
KuhnThomas. 1977. The Essential Tension. Chicago: University of Chicago Press.
34.
LamontMichèle. 2009. How Professors Think. Cambridge, MA: Harvard University Press.
35.
LatourBruno. 1987. Science in Action. Cambridge, MA: Harvard University Press.
LockeKarenGolden-BiddleKaren. 1997. “Constructing Opportunities for Contribution: Structuring Intertextual Coherence and “Problematizing” in Organizational Studies.” Academy of Management Journal40 (5): 1023–62.
39.
LordCharles G.RossLeeLepperMark R.. 1979. “Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence.” Journal of Personality and Social Psychology37 (11):2098–109.
40.
MahoneyMichael J.1977. “Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System.” Cognitive Therapy and Research1 (2): 161–75.
41.
MahoneyMichael J.KimperT. P.. 1976. “From Ethics to Logic: A Survey of Scientists.” In Scientist as Subject, edited by MahoneyM. J., 187–93. Cambridge, MA: Ballinger.
42.
MarchJames. 1991. “Exploration and Exploitation in Organizational Learning.” Organization Science2 (1): 71–87.
43.
MarchJames. 1996. “Learning to be Risk Averse.” Psychological Review103 (2):309–19.
44.
MertonRobert K.1948. “The Self-fulfilling Prophecy.” The Antioch Review8 (2): 193–210.
45.
MertonRobert K.1968a. “The Matthew Effect in Science.” Science159 (3810):56–63.
46.
MertonRobert K.1968b. “On Sociological Theories of the Middle Range.” In Social Theory and Social Structure, edited byMertonR. K., 39–72. New York: Free Press.
47.
MoneMark A.McKinleyWilliam. 1993. “The Uniqueness Value and its Consequences for Organization Studies.” Journal of Management Inquiry2 (3): 284–96.
48.
MuellerJennifer S.MelwaniShimulGoncaloJack A.. 2012. “The Bias against Creativity: Why People Desire but Reject Creative Ideas.” Psychological Science23 (1): 13–17.
49.
Nature. 2003. “Editorial: Coping with Peer Rejection.” 425: 645.
50.
PacanowskyM.StrineM. S.. 1985. “The Grand Scrivener: Text and Commentary.” In Publishing in the Organizational Sciences, edited by CummingsL. L.FrostP. J., 290–98. Homewood, IL: Irwin.
51.
PerrowCharles. 1995. “Journaling Careers.” In Publishing in the Organizational Sciences, edited by CummingsL. L.FrostPeter J., 203–15. Thousand Oaks, CA: Sage.
52.
PetersD. P.CeciS. J.. 1982. “Peer Review Practices of Psychological Journals: The Fate of Published Articles, Submitted again.” Behavioral and Brain Sciences5 (2):187–95.
53.
PfefferJeffrey. 1993. “Barriers to the Advance of Organizational Science: Paradigm Development as a Dependent Variable.” Academy of Management Review18 (4):599–620.
54.
PillutlaM. M.ThauS.. 2013. “Organizational Sciences’ Obsession with ‘that’s Interesting!’ Consequences and an Alternative.” Organizational Psychology Review3 (2):187–94.
55.
ReschK. I.ErnstE.GarrowJ.. 2000. “A Randomized Controlled Study of Reviewer Bias against an Unconventional Therapy.” Journal of the Royal Society of Medicine93 (4):164–67.
56.
RzhetskyAndreyFosterJacob G.FosterIan T.EvansJames A.. 2015. “Choosing Experiments to Accelerate Collective Discovery.” Proceedings of the National Academy of Sciences112 (47): 14569–74.
57.
SaltonGerardMcGillMichael. 1983. Introduction to Modern Information Retrieval. Columbus, OH: McGraw-Hill.
58.
ShepherdGeorge B.1994. Rejected: Leading Economists Ponder the Publication Process. Sun Lakes, AZ: Thomas Horton & Daughters.
59.
SilerKyleLeeKirbyBeroLisa. 2015. “Measuring the Effectiveness of Scientific Gatekeeping.” Proceedings of the National Academy of Sciences112 (2): 360–65.
60.
Sparck-JonesKaren. 1972. “A Statistical Interpretation of Term Specificity and its Application in Retrieval.” Journal of Documentation28 (1):11–21.
61.
StawBarry M.1995. “Why No-one really Wants Creativity.” In Creative Action in Organizations: Ivory Tower Visions and Real Voices, edited by FordCameronGioiaDennis, 161–66. Thousand Oaks, CA: Sage.
62.
StrangDavidSilerKyle. 2015. “Revising as Reframing: Original Submissions versus Published Papers in Administrative Science Quarterly, 2005 to 2009. Sociological Theory33 (1): 71–96.
63.
SulsJerryMartinRené. 2009. “The Air We Breathe: A Critical Look at Practices and Alternatives in the Peer-review Process.” Perspectives on Psychological Science4 (1): 40–50.
64.
TravisG. D. L.CollinsH. M.. 1991. “New Light on Old Boys: Cognitive and Institutional Particularism in the Peer Review System.” Science, Technology, & Human Values16 (3):322–41.
WeickKarl E.1995. “Editing Innovation into Administrative Science Quarterly.” In Publishing in the Organizational Sciences, edited by CummingsL. L.FrostPeter J., 284–96. Thousand Oaks, CA: Sage.
67.
WijnbergNachoem M.GemserGerda. 2000. “Adding Value to Innovation: Impressionism and the Transformation of the Selection System in Visual Arts.” Organization Science11 (3):323–29.
68.
YalowRosalyn S.1982. “Competency Testing for Reviewers and Editors.” Behavioral and Brain Sciences5 (2): 244–45.
