The Majestic’s Trust Flow (TF) is a link-based score aimed at measuring the influence of online objects (e.g. scientific publications) by considering the weighted number of links received from trusted websites. This study describes the bibliographic characteristics and impact of those publications with the highest TF score. In order to do this, 20,810 URL-based Digital Object Identifiers (DOIs) were identified and analysed. The results show that these DOIs mainly represent recent publications (57.1% of publications were published between 2010 and 2020), journal articles (93.75%) published in the first SCImago Journal Rank (SJR) quartile (81.7%), written with international collaboration (40.4%) and biased towards the field of medicine (36.9%). While the TF score is a discovering tool with the potential to be used in webometric studies to find influential publications, a few technical limitations jeopardise the general applicability of this indicator for research evaluation at the publication level.
ThelwallM.The influence of highly cited papers on field normalised indicators. Scientometrics2019; 118: 519–537.
2.
AksnesDW.Characteristics of highly cited papers. Res Eval2003; 12: 159–170.
3.
ChenSQiuJArsenaultCet al. Exploring the interdisciplinarity patterns of highly cited papers. J Infor2011; 15: 101124.
4.
TijssenRJVisserMSVan LeeuwenTN.Benchmarking international scientific excellence: are highly cited research papers an appropriate frame of reference?Scientometrics2002; 54: 381–397.
5.
BornmannLLeydesdorffL.Count highly-cited papers instead of papers with h citations: use normalized citation counts and compare ‘like with like’!Scientometrics2018; 115: 1119–1123.
BornmannL.How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature. Res Eval2014; 23: 166–173.
8.
GlänzelWSchubertA.Some facts and figures on highly cited papers in the sciences, 1981–1985. Scientometrics1992; 25: 373–380.
9.
GlänzelWCzerwonHJ.What are highly cited publications? A method applied to German scientific papers, 1980–1989. Res Eval1992; 2: 135–141.
10.
Martín-MartínAOrduna-MaleaEDelgado López-CózarE.Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: a multidisciplinary comparison. Scientometrics2018; 116: 2175–2188.
11.
CroninB.The citation process: the role and significance of citations in scientific communication. Oxford: Taylor Graham, 1984.
12.
MoedHF.Citation analysis in research evaluation. Dordrecht: Springer Science & Business Media, 2006.
13.
OppenheimCRennSP.Highly cited old papers and the reasons why they continue to be cited. JASIS1978; 29: 225–231.
14.
MertonRK.The Matthew effect in science: the reward and communication systems of science are considered. Science1968; 159: 56–63.
15.
Martín-MartínAOrduna-MaleaEAyllónJMet al. A two-sided academic landscape: snapshot of highly-cited documents in Google Scholar (1950-2013). Rev Esp Doc Cient2016; 39: e149.
16.
Orduna-MaleaEMartín-MartínADelgado López-CózarE. Classic papers: using Google Scholar to detect the highly-cited documents. In: Proceedings of the 23rd international conference on science and technology indicators – STI 2018, Leiden, pp. 1298–1307. Leiden: Centre for Science and Technology Studies (CWTS), http://hdl.handle.net/1887/65275 (accessed 17 April 2022).
17.
Van NoordenRMaherBNuzzoR. The top hundred papers. Nature2014; 514: 550–553.
18.
LowryOHRosebroughNJFarrALet al. Protein measurement with the Folin phenol reagent. J Biol Chem1951; 193: 265–275.
19.
KresgeNSimoniRDHillRL. The most highly cited paper in publishing history: protein determination by Oliver H. Lowry. J Biol Chem2005; 280: e25, http://www.jbc.org/content/280/28/e25 (accessed 17 April 2022).
20.
Orduña-MaleaEDelgado-López-CózarE.Dimensions: re-discovering the ecosystem of scientific information. El Prof Inf2018; 27: 420–431.
21.
ThackerPD.Covid-19: researcher blows the whistle on data integrity issues in Pfizer’s vaccine trial. BMJ2021; 375: 1–3.
22.
BornmannL.Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. J Inf2014; 8: 895–903.
23.
SugimotoCRWorkSLarivièreVet al. Scholarly use of social media and altmetrics: a review of the literature. JASIST2017; 68: 2037–2062.
WoutersPZahediZCostasR. Social media metrics for new research evaluation. In: WGlänzelHFMoedUSchmochet al. (eds) Handbook of science and technology indicators. Cham: Springer, 2019, pp. 687–713.
26.
OrtegaJL.Reliability and accuracy of altmetric providers: a comparison among Altmetric. Com, Plumx and Crossref Event Data. Scientometrics2018; 116: 2123–2138.
27.
ZahediZCostasR.General discussion of data quality challenges in social media metrics: extensive comparison of four major altmetric data aggregators. PLoS ONE2018; 13: e0197326.
28.
Más-BledaAAguilloIF.Can a personal website be useful as an information source to assess individual scientists? The case of European highly cited researchers. Scientometrics2013; 96: 51–67.
29.
Más-BledaAThelwallMKoushaKet al. Do highly cited researchers successfully use the social web?Scientometrics2014; 101: 337–356.
30.
ThelwallMKoushaK.Web indicators for research evaluation. Part 2: social media metrics. Prof Inform2015; 24: 607–620.
31.
OrtegaJLAguilloIF.Mapping world-class universities on the web. Inf Proc Manag2009; 45: 272–279.
32.
ThelwallMZuccalaA.A university-centred European Union link analysis. Scientometrics2008; 75: 407–420.
33.
ThelwallM.Journal impact evaluation: a webometric perspective. Scientometrics2012; 92: 429–441.
34.
ThelwallM. The story of a colony: theory development in webometric research. In: DHSonnenwald (ed.) Theory development in the information sciences. Austin, TX: University of Texas Press, 2021, pp. 164–182.
35.
ThelwallMKoushaK.Web indicators for research evaluation. Part 1: citations and links to academic articles from the web. Prof Inform2015; 24: 587–606.
36.
BrinSPageL.The anatomy of a large-scale hypertextual web search engine. Comput Networks ISDN1998; 30: 107–117.
37.
KleinbergJM.Authoritative sources in a hyperlinked environment. In: MNewmanA-LBarabásiDJWatts (eds) The structure and dynamics of networks. Princeton, NJ: Princeton University Press, 2006, pp. 604–632.
38.
Orduña-MaleaE.Dot-science top level domain: academic websites or dumpsites?Scientometrics2021; 126: 3565–3591.
39.
Orduña-MaleaECostasR.Link-based approach to study scientific software usage: the case of VOSviewer. Scientometrics2021; 126: 8153–8186.
40.
PaskinN.Digital object identifiers for scientific data. Data Sci J2005; 4: 12–20.
41.
SpearmanC.The proof and measurement of association between two things. Am J Psychol1904; 15: 72–101.
42.
DucutELiuFFonteloP.An update on Uniform Resource Locator (URL) decay in MEDLINE abstracts and measures for its mitigation. BMC Med Inf Decis Mak2008; 8: 23.
43.
FranceschiniFMaisanoDMastrogiacomoL.Errors in DOI indexing by bibliometric databases. Scientometrics2015; 102: 2181–2186.
44.
Valderrama-ZuriánJCAguilar-MoyaRMelero-FuentesDet al. A systematic analysis of duplicate records in Scopus. J Inf2015; 9: 570–576.
45.
XuSHaoLAnXet al. Types of DOI errors of cited references in Web of Science with a cleaning method. Scientometrics2019; 120: 1427–1437.
46.
HuangMLiuW.Substantial numbers of easily identifiable illegal DOIs still exist in Scopus. J Inf2019; 13: 901–903.
47.
ZhuJHuGLiuW.DOI errors and possible solutions for Web of Science. Scientometrics2019; 118: 709–718.
48.
BradfordMM.A rapid and sensitive method for the quantitation of microgram quantities of protein utilizing the principle of protein-dye binding. Anal Biochem1976; 72: 248–254.
49.
ThelwallMSudP.Scopus 1900–2020: growth in articles, abstracts, countries, fields, and journals. Quan Sci Stud2022; 3: 37–50.
50.
CostasRZahediZWoutersP.Do ‘altmetrics’ correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. JASIST2015; 66: 2003–2019.
51.
ThelwallMHausteinSLarivièreVet al. Do altmetrics work? Twitter and ten other social web services. PLoS ONE2013; 8: e64841.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
