This article discusses the method of calculating the disruption index (D index) based on COCI (the OpenCitations Index of Crossref open DOI-to-DOI citations), breaks through the difficulties brought by the acquisition of massive citation data and verifies the reliability of the method of calculating the disruption index based on open citation data based on empirical research. Through empirical research, we found that (1) there is little difference in the number of citation data of focus papers in Web of Science (WoS) and COCI; (2) the levels of disruptive innovation of the papers calculated based on the WoS and COCI are significantly strongly correlated; (3) among the D index and related extended indicators calculated based on COCI, has the strongest correlation with peer-review indicators, which is consistent with the calculation results based on WoS. Given the broad disciplinary coverage of COCI, although it has not yet been able to fully replace the function of commercial citation databases in research assessment, it can undoubtedly serve as an important source of citation data for further the disruption index and other scientometrics research thereafter.
ChristensenCMRosenbloomRS. Explaining the attacker’s advantage: technological paradigms, organizational dynamics, and the value network. Res Policy1995; 24: 233–257.
2.
YangWZhengYDongC. How to review basic research with a disruptive innovation nature. Bull Natl Nat Sci Found China2017; 31: 313–315.
3.
FunkRJOwen-SmithJ. A dynamic network measure of technological change. Manage Sci2017; 63: 791–817.
4.
WuLWangDEvansJA. Large teams develop and small teams disrupt science and technology. Nature2019; 566: 378–382.
5.
LeydesdorffLBornmannL. Disruption indices and their calculation using web-of-science data: indicators of historical developments or evolutionary dynamics?J Informetr2021; 15: 101219.
6.
SongPFengCLongC, et al. Study on discovery of outstanding scientific and technological talents in specific domains based on optimized disruptive index. J Intell2022; 41: 61–65.
7.
LyuDRuanXLiJ, et al. Research on the relationship between the interdisciplinarity and radicalness of papers from the perspective of knowledge integration. J China Soc Sci Tech Inf2022; 41: 263–274.
8.
LyuDGongKRuanX, et al. Does research collaboration influence the ‘disruption’ of articles? Evidence from neurosciences. Scientometrics2021; 126: 287–303.
9.
RuanXLyuDGongK, et al. Rethinking the disruption index as a measure of scientific and technological advances. Technol Forecast Soc2021; 172: 121071.
10.
RuanXAoWLyuD, et al. Effect of the topic-combination novelty on the disruption and impact of scientific articles: evidence from PubMed. J Inf Sci. Epub ahead of print 15 March 2023. DOI: 10.1177/01655515231161133.
11.
BornmannLTeklesA. Disruption index depends on length of citation window. Prof Inform. Epub ahead of print 25 March 2019. DOI: 10.3145/epi.2019.mar.07.
12.
BornmannLDevarakondaSTeklesA, et al. Disruptive papers published in Scientometrics: meaningful results by using an improved variant of the disruption index originally proposed by Wu, Wang, and Evans (2019). Scientometrics2020; 123: 1149–1155.
13.
BornmannLDevarakondaSTeklesA, et al. Are disruption index indicators convergently valid? The comparison of several indicator variants with assessments by peers. Quant Sci Stud2020; 1: 1242–1259.
14.
OsórioABornmannL. On the disruptive power of small-teams research. Scientometrics2021; 126: 117–133.
15.
LiuXShenZLiaoY, et al. The research about the improved disruption index and its influencing factors. Libr Inform Serv2020; 64: 84–91.
16.
LiuXLiaoYZhuM. A preliminary study on the application of disruption index to scientific research evaluation. Inform Stud: Theor Appl2021; 44: 34–40.
17.
LiuXShenZLiaoY, et al. Research on the stable time window of disruption index. Libr Inform Serv2021; 65: 49–57.
18.
LiuXZhuM. Research on the relationship between the disruption of scientific papers and the age of references. Inform Stud: Theor Appl2023; 46: 60–66.
19.
Martín-MartínAThelwallMOrduna-MaleaE, et al. Correction to: Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations. Scientometrics2021; 126: 907–908.
20.
PeroniSShottonD. OpenCitations, an infrastructure organization for open scholarship. Quant Sci Stud2020; 1: 428–444.
21.
ShottonD. Funders should mandate open citations. Nature2018; 553: 129.
22.
Teixeira da SilvaJAHuangC-K(K)OrtegaJL. Open citations as a tool for bibliometric verification and transparency and for correcting erroneous references. J Scholarly Publ2023; 54: 60–79.
23.
HeibiIPeroniSShottonD. Software review: COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. Scientometrics2019; 121: 1213–1228.
KrügerAKPetersohnS. ‘I want to be able to do what I know the tools will allow us to do’: practicing evaluative bibliometrics through digital infrastructure. Res Evaluat2022; 31: 475–485.
26.
HendricksGTkaczykDLinJ, et al. Crossref: the sustainable source of community-owned scholarly metadata. Quant Sci Stud2020; 1: 414–427.
27.
LammeyR. Using the Crossref Metadata API to explore publisher content. Sci Ed2016; 3: 109–111.
28.
PeroniSDuttonAGray JonesT, et al. Setting our bibliographic references free: towards open citation data. J Doc2015; 71: 253–277.
29.
LightRPPolleyDEBörnerK. Open data and open code for big science of science studies. Scientometrics2014; 101: 1535–1551.
30.
LeydesdorffLTeklesABornmannL. A proposal to revise the disruption index. Prof Inform2021; 30: e300121.
31.
BuYWaltmanLHuangY. A multidimensional framework for characterizing the citation impact of scientific publications. Quant Sci Stud2021; 2: 155–183.
32.
WangSMaYMaoJ, et al. Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities. J Assoc Inf Sci Tech2023; 74: 150–167.
33.
ZhouYXuX-LYangX-H, et al. The influence of disruption on evaluating the scientific significance of papers. Scientometrics2022; 127: 5931–5945.
34.
WangRZhouYZengA. Evaluating scientists by citation and disruption of their representative works. Scientometrics2023; 128: 1689–1710.
35.
JiangYLiuX. A construction and empirical research of the journal disruption index based on open citation data. Scientometrics2023; 128: 3935–3958.
36.
DengNZengA. Enhancing the robustness of the disruption metric against noise. Scientometrics2023; 128: 2419–2428.
37.
LammeyR. Crossref at 20 years: what do the community need?Sci Ed2020; 7: 125–129.
38.
LammeyR. CrossRef developments and initiatives: an update on services for the scholarly publishing community from CrossRef. Sci Ed2014; 1: 13–18.
39.
WuSWuQ. A confusing definition of disruption. SocArXiv. Epub ahead of print 18 April 2019. DOI: 10.31235/osf.io/d3wpk.
40.
VisserMVan EckNJWaltmanL. Large-scale comparison of bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. Quant Sci Stud2021; 2: 20–41.
41.
ChudlarskýTDvořákJ. Can crossref citations replace web of science for research evaluation? The share of open citations. J Data Inf Sci2020; 5: 35–42.
42.
BornmannL. Scientific peer review. Annu Rev Inform Sci2011; 45: 197–245.
43.
JiangYLiuX. The relationship between absolute disruption index, peer review index and CNCI: a study based on virology papers. Libr Inform Serv2023; 67: 96–105.
44.
BrezisESBirukouA. Arbitrariness in the peer review process. Scientometrics2020; 123: 393–411.
