This study seeks to examine the impact of alumni connections between the evaluators and evaluatees on the results of peer review ratings for the Korean national R&D project and selection success rate. Specifically, this study analyzed the evaluation results of 8,402 research proposal entries submitted between 2007 and 2011 for the “general researchers support project,” all in the Natural Science and Engineering areas and sponsored by the National Research Foundation of Korea. Each proposal entry was evaluated by three evaluators, and approximately 39 percent of the proposals had at least one evaluator from the same university. The results of this study showed that evaluators have a tendency to give relatively high scores to research proposals submitted by the alumni of the same universities as their alma mater. Also, when an evaluator from the same university as an evaluatee was included in the evaluator group, the results of this study showed that the percentage of entry submissions was higher compared to the contrary. Such results show that in the process of peer review–based research proposal evaluations for national R&D projects, alumni connections have significant influence on evaluation results in South Korea.
AbdoulH.PerreyC.TubachF.AmielP.Durand-ZaleskiI.AlbertiC.. 2012. “Non-financial Conflicts of Interest in Academic Grant Evaluation: A Qualitative Study of Multiple Stakeholders in France.” PLoS ONE7 (4): e35247.
BaegJ. Y. 2009. “Does Peer Review for Journals Predict Performance of Research?” Korean Public Management Review23 (4): 383–99.
4.
BornmannL.2011. “Scientific Peer Review.” Annual Review of Information Science and Technology45 (1): 197–245.
5.
BornmannL.2012. “The Hawthorne Effect in Journal Peer Review.” Scientometrics91 (3): 857–62.
6.
BornmannL.DanielH. D.. 2005. “Selection of Research Fellowship Recipients by Committee Peer Review: Reliability, Fairness and Predictive Validity of Board of Trustees’ Decisions.” Scientometrics63 (2): 297–320.
7.
BornmannL.MutzR.DanielH. D.. 2007. “Gender Differences in Grant Peer Review: A Meta-analysis.” Journal of Informetrics1 (3): 226–38.
8.
BraffJ. P.2010. “Conflict of Interest in Research: Towards a Greater Transparency.” The Permanente Journal14 (2): 31–34.
BuddenA. E.TregenzaT.AarssenL. W.KorichevaJ.LeimuR.LortieC. J.. 2008. “Double-blind Review Favours Increased Representation of Female Authors.” Trends in Ecology & Evolution23 (1): 4–6.
11.
CeciS. J.WilliamsW. M.. 2011. “Understanding Current Causes of Women’s Underrepresentation in Science.” Proceedings of the National Academy of Sciences108 (8): 3157–62.
12.
ChoM. H.HongH. D.KimG. S.. 2004. “Comparison Analysis of Research Proposal Evaluation Systems of Korea and the U.S.” Korean Society and Administration Study15 (3): 581–604.
13.
ChoiY. H.LeeG. C.. 2009. “Analysis of the Forms of Joint Academic Dissertation Writing: the Korean Review of Public Administration (1989-2008).” Korean Review of Public Administration43 (3): 51–72.
14.
ChubinD. E.HackettE. J.. 1990. Peerless Science: Peer Review and US Science Policy. Albany: SUNY Press.
15.
ChubinD. E.JasanoffS.. 1985. “Peer Review and Public Policy.” Science, Technology, & Human Values10 (3): 3–5.
16.
CollinsH. M.1981. “The Place of the Core-set in Modern Science: Social Contingency with Methodological Property in Science.” History of Science19 (1): 6–19.
17.
CroninB.2009. “Vernacular and Vehicular Language.” Journal of the American Society for Information Science and Technology60 (3): 433.
18.
DanielH. D.1993. Guardians of Science: Fairness and Reliability of Peer Review. Weinheim: VCH.
GilbertJ. R.WilliamsE. S.LundbergG. D.. 1994. “Is There Gender Bias in JAMA’s Peer Review Process?” Journal of the American Medical Association272 (2): 139–42.
22.
GruberA.2009. Asia-Special Report. Princeton, NJ: Secondary School Admission Test Board.
23.
HanD.2011. An Empirical Study on the Effects of Blind Review in the General Researcher Program. Daejeon, South Korea: National Research Foundation of Korea.
24.
HojatM.GonnellaJ. S.CaelleighA. S.. 2003. “Impartial Judgment by the “Gatekeepers” of Science: Fallibility and Accountability in the Peer Review Process.” Advances in Health Sciences Education8 (1): 75–96.
25.
HolbrookJ. B.2010a. “The Use of Societal Impacts Considerations in Grant Proposal Peer Review: A Comparison of Five Models.” Technology and Innovation12 (3): 213–24.
26.
HolbrookJ. B.2010b. “Peer Review.” In The Oxford Handbook of Interdisciplinarity, edited by FrodemanR.KleinJ. T.MitchamC., 321–32. Oxford, UK: Oxford University Press.
27.
HolbrookJ. B.2012. “Re-assessing the Science-Society Relation: The Case of the US National Science Foundation’s Broader Impacts Merit Review Criterion (1997 - 2011).” In Peer Review, Research Integrity, and the Governance of Science: Practice, Theory, and Current Discussions, edited byFrodemanR.HolbrookJ.B.MitchamC.XiaonanH., 328–62. Beijing: People’s Publishing House. Accessed January 19, 2016. http://digital.library.unt.edu/ark:/67531/metadc77119.
28.
HolbrookJ. B.FrodemanR.. 2011. “Peer Review and the Ex Ante Assessment of Societal Impacts.” Research Evaluation20 (3): 239–46.
29.
HoodC.2007. “What Happens When Transparency Meets Blame-avoidance?” Public Management Review9 (2): 191–210.
30.
HornerK. L.RushtonJ. P.VernonP. A.. 1986. “Relation between Aging and Research Productivity of Academic Psychologists.” Psychology and Aging1 (4): 319–24.
31.
HyunK. J.2001. “Sociocultural Change and Traditional Values: Confucian Values among Koreans and Korean Americans.” International Journal of Intercultural Relations25 (2): 203–29.
32.
JangD.KangG. M.DohS.HanD. S.YangJ. M.. 2015. “An Analysis of the Influence of School Ties on Peer Review of Research Proposal.” Korean Policy Studies Review24 (2): 263–87.
33.
KimL.1997. Imitation to Innovation: The Dynamics of Korea’s Technological Learning. Brighton, MA: Harvard Business Press.
34.
LaFolletteM. C.1994. “Measuring Equity: The U.S. General Accounting Office Study of Peer Review.” Science Communication16 (2): 211–20.
35.
LariviereV.GingrasY.CroninB.. 2013. “Global Gender Disparities in Science.” Nature504 (7479): 211–13.
36.
LeeC. J.SugimotoC. R.ZhangG.CroninB.. 2013. “Bias in Peer Review.” Journal of the American Society for Information Science and Technology64 (1): 2–17.
37.
LeeM.OmK.KohJ.. 2000a. Scientific Technology and Evaluation: Theory and Facts about Peer Review. Seoul, Korea: PD.
38.
LeeM.OmK.KohJ.. 2000b. “The Bias of Sighted Reviewers in Research Proposal Evaluation: A Comparative Analysis of Blind and Open Review in Korea.” Scientometrics48 (1): 99–116.
39.
LeeS.BrintonM. C.. 1996. “Elite Education and Social Capital: The Case of South Korea.” Sociology of Education69 (3): 177–92.
40.
LinkA. M. (1998). “US and non-US Submissions.” Journal of the American Medical Association280 (3): 246–47.
41.
LoonenM. P.HageJ. J.KonM.. 2005. “Who Benefits from Peer Review? An Analysis of the Outcome of 100 Requests for Review by Plastic and Reconstructive Surgery.” Plastic and Reconstructive Surgery116 (5): 1461–72.
42.
MatthewsC. M.2012. U.S. National Science Foundation: Experimental Program to Stimulate Competitive Research (EPSCoR). Congressional Research Service. Accessed January 17, 2015. https://www.fas.org/sgp/crs/misc/RL30930.pdf.
43.
MarginsonS.2011. “Higher Education in East Asia and Singapore: Rise of the Confucian Model.” Higher Education61 (5): 587–611.
44.
MarshH. W.JayasingheU. W.BondN. W.. 2008. “Improving the Peer-review Process for Grant Applications: Reliability, Validity, Bias, and Generalizability.” American Psychologist63 (3): 160–68.
45.
MartinB.2000. “Research Grants: Problems and Options.” Australian Universities’ Review43 (2): 17–22.
46.
MertonR. K.1968. “The Matthew Effect in Science.” Science159 (3810): 56–63.
OwenR.1982. “Reader Bias.” Journal of the American Medical Association247 (18): 2533–534.
49.
ParkK. B.2011. “Examination of Improvement Plans for the Project Evaluator Selection System for the Research Projects of the National Research Foundation of Korea.” National Research Foundation of Korea Policy Report (Policy Study-2010-014-Survey Analysis). Daejeon, Korea: NRF.
50.
PetersD. P.CeciS. J.. 1982. “Peer-review Practices of Psychological Journals: The Fate of Published Articles, Submitted Again.” Behavioral and Brain Sciences5 (2): 187–95.
51.
PolanyiM.1962. “The Republic of Science: Its Political and Economic Theory.” Minerva1 (1): 54–73.
52.
PruthiS.JainA.WahidA.MehraK.NabiS.A.. 1997. “Scientific Community and Peer Review System: A Case Study of a Central Government Funding Scheme in India.” Journal of Scientific and Industrial Research56 (7): 398–407.
53.
ReesT.2011. “The Gendered Construction of Scientific Excellence.” Interdisciplinary Science Reviews36 (2): 133–45.
54.
RossP. F.1980. The Sciences’ Self-management: Manuscript Refereeing, Peer Review, and Goals in Science. Lincoln: The Ross Company.
55.
RossJ. S.GrossC. P.DesaiM. M.HongY.GrantA. O.DanielsS. R.KrumholzH. M.. 2006. “Effect of Blinded Peer Review on Abstract Acceptance.” Journal of the American Medical Association295 (14): 1675–80.
56.
SchroterS.GrovesT.HøjgaardL.. 2010. “Surveys of Current Status in Biomedical Science Grant Review: Funding Organizations’ and Grant Reviewers’ Perspectives.” BMC Medicine8 (62): 1–9.
57.
SharpD. W.1990. “What Can and Should Be Done to Reduce Publication Bias: The Perspective of an Editor.” Journal of the American Medical Association263 (10): 1390–391.
ShinJ. C.2012. “Higher Education Development in Korea: Western University Ideas, Confucian Tradition, and Economic Development.” Higher Education64 (1): 59–72.
60.
SmithS.WardW.HouseA.. 2011. “Impact in the Proposals for the UK’s Research Excellence Framework: Shifting the Boundaries of Academic Autonomy.” Research Policy40 (10): 1369–79.
61.
StephanP. E.1996. “The Economics of Science.” Journal of Economic Literature34 (3): 1199–235.
62.
StephanP. E.LevinS. G.. 1993. “Age and the Nobel Prize Revisited.” Scientometrics28 (3): 387–99.
63.
SungY.TinkhamS. F.. 2005. “Brand Personality Structures in the United States and Korea: Common and Culture-specific Factors.” Journal of Consumer Psychology15 (4): 334–50.
64.
TregenzaT.2002. “Gender Bias in the Refereeing Process?” Trends in Ecology & Evolution17 (8): 349–50.
65.
UmS. H.LeeC. W.JangD. H.ChoiY.. 2012. “Analysis of Difference in Research Funding and Research Performance between Person with Domestic Degree and Person with Foreign Degree.” The Journal of Korean Policy Studies12 (1): 21–43.
66.
WeberM.1987. “Legitimate Authority and Bureaucracy.” In The Great Writings in Management and Organizational Behavior, edited by BooneL. E.BowenD. D., 5–19. New York: Random House.
67.
WenneråsC.WoldA.. 1997. “Nepotism and Sexism in Peer-review.” In Women, Science, and Technology: A Reader in Feminist Science Studies, edited by WyerM., 46–52. New York, NY: Taylor & Francis.
68.
WhittakerR. J.2008. “Journal Review and Gender Equality: A Critical Comment on Budden et al.” Trends in Ecology & Evolution23 (9): 478–79.
69.
ZuckermanH.MertonR. K.. 1971. “Patterns of Evaluation in Science: Institutionalisation, Structure and Functions of the Referee System.” Minerva9 (1): 66–100.
