Sage Journals: Discover world-class research

Abstract

Purpose:

The standard bibliometric indexes (“m-quotient “H-,” “H2-,” “g-,” “a-,” “m-,” and “r-” index) do not considered the research’ position in the author list of the paper. We proposed a new methodology, System of Authorship Best Assessment (SABA), to characterize the scientific output based on authors’ position.

Material and Methods:

Four classes S1A, S1B, S2A, and S2B include only papers where the researcher is in first, first/last, first/second/last, and first/second/second-last/last position respectively were used for the calculation of H-index and number of citations The system was tested with Noble prize winners controlled with researchers matched for H-index. The different in percentage between standard bibliometric index and S2B was calculated and compared.

Results:

The percentage differences in Noble prize winners between S2B-H-index versus Global H-index and number of citations is very lower comparing with control group (median 4.15% [adjusted 95% CI, 2.54–5.30] vs 9.00 [adjusted 95% CI, 7.16–11.84], p < 0.001; average difference 8.7% vs 20.3%). All different in percentage between standard bibliometric index and S2B except two (H2- and m-index) were significantly lower among Noble prize compared with control group.

Conclusion:

The SABA methodology better weight the research impact by showing that for excellent profiles the S2B is similar to global values whereas for other researchers there is a significant difference.

Keywords

Abstracting and indexing bibliometrics

Introduction

The identification of correct metric system to objectively assess the impact and visibility in literature of a researcher represents a critical need in academic and non-academic perspectives. These are related to the academic and professional progression, to the transnationality, to the commercial effects, to the diffusion of the products and to the relapses that his research has induced in terms of knowledge as well as to the probability to obtain funds and research grants.¹

In the past years several methodologies have been suggested in order to quantify the value of the researcher in particular the total number of citations, the Hirsh-index.^2,3 These systems have several strengths in their application, and these are accepted as good systems to quantify the impact and visibility in literature of a researcher. However, this strength (the impact of the research papers of a researcher) could be at the same time the weakness of these systems because it is not the production of the researcher (in terms of papers written or leaded) to be computed but the papers he/she authored or co-authored.⁴

This fact is linked to a new phenomenon: the increase in the number of authors included in a research paper as showed in Nature by Greene.⁵ Some authors speculated that this increase in the number of authors in a research paper could be explained by the new level of complexity the research where “fewer and fewer people know enough to work and write alone”^5,6 whereas other authors hypothesize that the increased number of authors included for a paper is also linked to bibliometric needs and/or honorary authorship^7,8 even if there are the International Committee of Medical Journal Editors (ICMJE) criteria that address what the rules that allow a researcher to be consider author or not.⁹ More recently other indexes such as m-quotient, g index, H2 index, an index, m index and r index¹⁰ were introduced to compensate some drawbacks of H-index and number of citations (the details of the indexes and their advantages over H-index are displayed in Table 1).

Table 1.

Various indexes in literature.¹⁰

Indexes	description	Advantages
M-quotient	H-index/year last publication-year first publication	Compensate based length of career
g index	The highest number g of papers that together received g2 or more citations	Much more weight for high citation paper
H2 index	Highest natural number such that his h(2) most cited papers received each at least [h(2)]2 citations	Reduced precision problem
a index	Average citation of Hirsch core	Evaluate the most productive core
m index	Median citation of Hirsch core	Evaluate the central tendency of the most productive core
r index	The square root of the sum of citations in the Hirsch core	Evaluate the citations intensity

Moreover, another debatable point is that search and database engine of authors/papers such as Scopus or Google scholar, widely used for bibliometric analysis, include in their output quantification analysis not only the “authors” but also “contributors”: this means that it is possible to see papers with 30 authors and 200 contributors and the system of analysis compute in the same way the authors and the contributors (!) with consequent impact to citations and H-index. Interestingly for multi-authors papers some colleagues, such as Rennie et al.,¹¹ suggested that in our era of multi-author articles, the concept of authorship should be replaced by that of contributorship that is quite different from the authorship. It is noteworthy, that those type of papers, with several contributors, are usually highly cited¹² with the paradoxical effect that for some authors most of the research citations is generated by papers where they are not authors but, simply, contributors.

Therefore, we hypothesize that the H-index does not measure the impact of an author correctly because it does not take into account the actual intellectual property. A potential solution could be generating new metric system, that should not substitute the other already used, but could be useful to derive further information necessary to have a better insight of the researcher impact and not the that of the papers where is co-author/contributor and in this scenario the position of the author in the list plays a fundamental role.

The first author is usually the researcher who has made the most significant intellectual contribution to the paper, in terms of designing, acquiring and analyzing data from experiments, and writing the manuscript. The importance of the first author is reflected in the common practice of referring to a paper by the first author’s name for example, “Jones et al. report that. . .” The last author is commonly the senior (lead-group author) who has supervised the research whereas second and second last positions usually represent are attributed to the second most important contributor and second senior contributor. For this reason, we introduce a new criterion that “weighs” the order in the list of authors, because we believe that this balances the effective intellectual property better, and to check its effectiveness we evaluate if, unlike the H-index, it can discriminate the Nobel winners from a group of authors matched for discipline and H-index to the Nobel winners themselves.

The purpose of this paper is to present a new methodology, System of Authorship Best Assessment (SABA), that weight the impact of the position name of the authors by checking this system on two homogeneous (on the basis of traditional H-index) cohorts of high-level researchers but different according to the accomplishment of result of excellence (Nobel Prize). Furthermore, the SABA methodology was applied to other bibliometric indexes.

Method

The SABA methodology was tested into two different ways. Firstly, the four classes listed below were compared among Nobel prize winner and control group for H-index and total number of citations. Secondly, the difference in percentage between global and S2B in the other bibliometric indexes (m-quotient, a-index, m-index, H2 index, g-index and r-index) were confronted between Nobel prize winner and control group.

The SABA methodology considers the position of the authors, and the following groups were considered:

S1A = included in the analysis only papers with author in first position

S1B = included in the analysis only papers with author in first/last position

S2A = included in the analysis only papers with author in first/second/last position

S2B = included in the analysis only papers with author in first/second/second-last/last position

SnA = included in the analysis only papers with author in first/second//n/second-last/last position

SnB = included in the analysis only papers with author in first/second//n/n-last/second-last/last position

The system could be applied to the all the metric systems used:

• Hirsch index (H-Index)

• Total number of citations (Nc, tot).

• Impact factor (IF)

• Total number of papers (Np)

Study’s population

In order to test the effect of this metric systems, the System of Authorship Best Assessment was applied to a group of high-level researchers in biomedical field by testing the effects to tow of the most used parameters: H-index and citations number. Two homogeneous (on the basis of traditional H-index) cohort of high-level researchers but different according to the accomplishment of result of excellence (Nobel Prize) were selected. It is assumed that the Nobel Prize is a criterion for the impact of scientific production, or if one has won the Nobel Prize is an irrefutable element of the impact of its production and quality.

In the first phase of the analysis, the winners of the Nobel Prize in Physiology or Medicine from 1997 to 2017 for a total of 50 researchers were therefore included and another group matched for similar H index, possible, age, gender, and topic of research, were matched with the Nobel Prize winners. The global number of researchers assessed was 100.

Four classes analysis

In the second phase a Scopus database analysis was performed and the CSV files with the output was exported for each researcher. Therefore, for each one, all papers were classified according to five categories:

S1A: papers with researcher in first position

S1B: papers with researcher in first/last position

S2A: papers with researcher in first/second/last position

S2B: papers with researcher in first/second/second-last/last position

Global: papers with researcher in other positions.

Accordingly, the H factor was calculated for S1A, S1B, S2A, S2B, and Global categories (global category included all the papers and therefore represents the current H-index factor as indicated by Hirsch³). The absolute difference in H-index between the S2B and Global as well as the percentage differences was calculated. Moreover, the percentage difference was grouped in four classes (<5%, 5%–10%, 10%, 15%, and >15%).

Bibliometric indexes tests

For each bibliometric indexes including m-quotient, a-index, m-index, H2 index, g-index and r-index (Table 1) the percentage different between global and S2B calculation were calculated in Nobel prize winner and in control group. Subsequently, the differences were compared between the two groups

Outcomes

The primary outcome was to assess the effect of the research position in the author list of the paper on H-index and number of citations between Noble prize winner and control group depending on the four classes (S1A, S1B, S2A, S2B) and Global

The secondary outcome was to compare the percentages difference between Global and S2B class of all bibliometric indexes among Nobel prize winner and control group.

Statistical analysis

The normality of each continuous variable group was tested using Kolmogorov-Smirnov Z test and because the normality was rejected nonparametric tests were applied. Mann-Whitney test was used for comparing all bibliometric indexes between global and S2B class of all bibliometric indexes among Nobel prize winner and control group. A p value <0.05 was regarded to indicate statistical significance and all correlation values were calculated using a two-tailed significance level. R software (www.r-project.org) was employed for statistical analyses.

Results

The summary of H-index and citation analysis according to the System of Authorship Best Assessment for Nobel Winners and control group are given in the Tables 2 and 3 respectively.

Table 2.

Summary table for H factor analysis in Nobel prize winner groups and controls calculated for S1A, S1B, S2A, S2B, and Global categories including absolute difference in H-index between the S2B and Global and percentage difference. In the last column “difference class” the percentage difference between the S2B and Global H are grouped into four classes (<5%, 5%–10%, 10%, 15%, and >15%). The name of the Scientists of the control group are blinded for privacy but are at disposal previous authorization and upon specific request.

	Researcher	H-index	S1A	S1B	S2A	S2B	Gap H-index/S-index	% gap H N	Difference class (%)
Nobel prize winners	Jeffrey C. Hall	80	22	61	63	76	4	5.0	<5
	Michael Rosbash	96	18	80	81	93	3	3.1	<5
	Michael W. Young	54	11	44	45	53	1	1.9	<5
	Yoshinori Ohsumi	91	10	68	74	87	4	4.4	<5
	Tu Youyou	9	4	9	9	9	0	0.0	<5
	Satoshi Omura	77	35	58	60	70	7	9.1	5−10
	Campbell WC	25	18	22	23	23	2	8.0	5−10
	O’Keefe JM	56	18	50	52	55	1	1.8	<5
	Moser MB	56	9	31	36	54	2	3.6	<5%
	Moser EI	63	17	48	52	61	2	3.2	<5
	Sudhof T	158	32	122	124	145	13	8.2	5−10
	Schekman RW	94	17	85	89	92	2	2.1	<5
	Rothman JE	105	30	93	94	105	0	0.0	<5
	Yamanaka S	87	18	61	61	75	12	13.8	10−15
	Gurdon JB	74	43	73	73	74	0	0.0	<5
	Steinman RM	148	47	116	124	144	4	2.7	<5
	Hoffmann JA	91	16	51	53	79	12	13.2	10−15
	Beutler BA	102	43	78	80	92	10	9.8	5−10
	Edwards RG	56	35	45	47	48	8	14.3	10−15
	Szostak JW	83	14	77	80	82	1	1.2	<5
	Greider CW	66	18	56	58	65	1	1.5	<5
	Blackburn EH	87	27	74	76	83	4	4.6	<5%
	Montagnier L	68	17	40	43	59	9	13.2	10−15
	Barré-Sinoussi F	66	9	18	22	39	27	40.9	>15
	zur Hausen H	80	46	73	73	79	1	1.3	<5
	Smithies O	91	29	66	67	82	9	9.9	5−10
	Evans MJ	52	12	30	35	44	8	15.4	>15
	Capecchi MR	81	19	70	71	80	1	1.2	<5
	Mello CG	53	9	35	38	47	6	11.3	10−15
	Fire AZ	70	13	49	56	66	4	5.7	5−10
	Warren JR	11	2	3	7	9	2	18.2	>15
	Marshall BJ	45	26	37	42	42	3	6.7	5−10
	Buck LB	35	12	33	34	34	1	2.9	<5
	Axel R	92	13	68	71	88	4	4.3	<5
	Mansfield P	41	29	36	38	39	2	4.9	<5
	Lauterbur P	42	20	39	39	42	0	0.0	<5
	Sulston JE	51	16	31	35	39	12	23.5	>15
	Horvitz R	114	13	94	94	107	7	6.1	5−10
	Brenner S	80	24	57	63	73	7	8.8	5−10
	Nurse PM	94	22	81	83	92	2	2.1	<5
	Hunt T	67	20	49	53	64	3	4.5	<5
	Hartwell LH	69	30	63	67	68	1	1.4	<5
	Kandel ER	148	35	118	125	142	6	4.1	<5
	Greengard P	163	20	131	133	155	8	4.9	<5
	Carlsson A	87	52	75	83	85	2	2.3	<5
	Blobel G	116	23	103	108	115	1	0.9	<5
	Murad F	90	28	82	83	87	3	3.3	<5
	Ignarro LJ	98	52	80	85	93	5	5.1	5−10
	Furchgott RF	40	29	37	40	40	0	0.0	<5
	Prusiner SB	144	40	116	119	138	6	4.2	<5
Control group		95	24	62	66	85	10	10.5	10−15
	Blinded Name at disposal upon request	96	26	79	83	93	3	3.1	<5
		62	24	42	47	58	4	6.5	5−10
		103	33	97	98	102	1	1.0	<5
		59	11	38	43	52	7	11.9	10−15
		76	19	70	71	76	0	0.0	<5
		44	11	35	38	42	2	4.5	<5
		87	41	78	80	86	1	1.1	<5
		83	27	63	72	77	6	7.2	5−10
		68	36	60	61	65	3	4.4	<5
		140	38	124	128	135	5	3.6	<5
		116	11	80	84	104	12	10.3	10−15
		126	22	89	103	119	7	5.6	5−10
		100	22	69	71	86	14	14.0	10−15
		79	10	48	51	68	11	13.9	10−15
		131	39	86	92	120	11	8.4	5−10
		92	13	71	73	79	13	14.1	10−15
		103	29	68	59	86	17	16.5	>15
		79	29	57	69	75	4	5.1	10−15
		89	25	59	68	81	8	9.0	5−10
		64	14	60	61	64	0	0.0	<5
		82	25	67	71	77	5	6.1	5−10
		76	22	46	52	59	17	22.4	>15
		67	22	52	56	60	7	10.4	10−15
		89	22	61	62	73	16	18.0	>15
		107	19	60	70	79	28	26.2	>15
		66	22	41	49	61	5	7.6	5−10
		83	5	40	42	62	21	25.3	>15
		71	15	56	59	65	6	8.5	5−10
		140	14	70	76	121	19	13.6	10−15
		41	11	21	24	28	13	31.7	>15
		53	25	42	44	48	5	9.4	5−10
		88	20	73	74	79	9	10.2	10−15
		73	38	62	71	71	2	2.7	<5
		53	12	34	41	42	11	20.8	>15
		69	16	47	50	60	9	13.0	10−15
		65	36	55	58	65	0	0.0	<5
		150	38	111	114	138	12	8.0	5−10
		94	17	54	60	70	24	25.5	>15
		105	31	84	86	96	9	8.6	5−10
		93	17	46	66	81	12	12.9	10−15
		70	25	54	56	65	5	7.1	5−10
		140	50	90	101	114	26	18.6	>15
		171	92	155	157	164	7	4.1	<5
		89	15	67	67	81	8	9.0	5−10
		141	23	70	72	103	38	27.0	>15
		93	31	71	75	82	11	11.8	10−15
		115	27	88	94	109	6	5.2	5−10
		67	44	62	64	66	1	1.5	<5
		105	36	68	71	87	18	17.1	>15

Table 3.

Summary table for citation analysis in Nobel prize winner groups and controls calculated for S1A, S1B, S2A, S2B, and Global categories including absolute difference in H index between the S2B and Global and percentage difference. In the last column “difference class” the percentage difference between the S2B and total number of citations are grouped into four classes (<5%, 5%–10%, 10%, 15%, and >15%). The name of the Scientists of the control group are blinded for privacy but are at disposal previous authorization and upon specific request.

	Researcher	Citations	S1A	S1B	S2A	S2B	Gap of citations	% gap cit N	Difference class (%)
Nobel prize winners	Jeffrey C. Hall	19,308	2135	11,296	12,724	17,968	1340	6.9	5−10
	Michael Rosbash	27,910	1091	19,324	20,204	26,326	1584	5.7	5−10
	Michael W. Young	11,320	1332	8753	8944	10,687	633	5.6	5−10
	Yoshinori Ohsumi	42,567	1811	23,473	25,634	36,907	5660	13.3	10−15
	Tu Youyou	564	434	503	505	555	9	1.6	<5
	Satoshi Omura	30,376	5011	18,811	20,658	26,617	3759	12.4	10−15
	Campbell WC	3225	1789	2618	2687	2834	391	12.1	10−15
	O’Keefe JM	21,059	7710	19,843	20,301	20,855	204	1.0	<5
	Moser MB	16,327	2188	7569	8483	16,087	240	1.5	<5
	Moser EI	18,440	2403	12,663	13,967	17,766	674	3.7	<5
	Sudhof T	79,495	10,535	51,753	53,697	72,432	7063	8.9	5−10
	Schekman RW	27,350	1921	22,673	24,034	26,559	791	2.9	<5
	Rothman JE	40,827	7946	34,981	35,197	40,022	805	2.0	<5
	Yamanaka S	54,197	3842	43,211	43,276	48,364	5833	10.8	10−15
	Gurdon JB	16,130	7312	15,184	15,473	16,095	35	0.2	<5
	Steinman RM	92,661	20,808	63,101	69,180	86,911	5750	6.2	5−10
	Hoffmann JA	28,639	4643	16,541	16,915	25,107	3532	12.3	10−15
	Beutler BA	54,943	16,929	36,461	41,918	46,613	8330	15.2	>15
	Edwards RG	11,914	4862	8764	9410	9647	2267	19.0	>15
	Szostak JW	30,584	3634	27,044	29,065	30,250	334	1.1	<5
	Greider CW	30,587	6452	19,343	19,888	28,321	2266	7.4	5−10
	Blackburn EH	33,387	9223	26,239	28,505	30,933	2454	7.4	5−10
	Montagnier L	22,561	1918	13,644	14,115	19,876	2685	11.9	10−15
	Barré-Sinoussi F	17,859	4462	5837	7149	10,492	7367	41.3	>15
	zur Hausen H	32,877	14,446	28,606	29,488	32,322	555	1.7	<5
	Smithies O	46,847	4287	32,746	33,025	40,618	6229	13.3	10−15
	Evans MJ	16,995	5982	10,522	12,008	14,268	2727	16.0	>15
	Capecchi MR	25,686	3760	20,054	20,246	24,032	1654	6.4	5−10
	Mello CG	23,663	4251	20,409	20,883	22,700	963	4.1	<5
	Fire AZ	30,672	11,297	22,684	24,285	28,954	1718	5.6	5−10
	Warren JR	5765	70	3583	4910	5080	685	11.9	10−15
	Marshall BJ	12,752	929	10,939	11,559	11,984	768	6.0	5−10
	Buck LB	12,218	4144	11,545	11,781	11,830	388	3.2	<5
	Axel R	33,771	1260	24,950	25,942	31,195	2576	7.6	5−10
	Mansfield P	6721	4061	5667	5930	6267	454	6.8	5−10
	Lauterbur P	7558	3149	6701	6920	7511	47	0.6	<5
	Sulston JE	34,048	7065	9148	10,473	11,563	22,485	66.0	>15
	Horvitz R	55,864	2191	32,647	33,328	51,537	4327	7.7	5−10
	Brenner S	30,727	12,838	21,811	23,250	27,468	3259	10.6	10−15
	Nurse PM	31,444	6009	26,568	27,172	30,827	617	2.0	<5
	Hunt T	16,137	1794	8375	9337	14,832	1305	8.1	5−10
	Hartwell LH	23,290	12,601	21,824	22,276	22,722	568	2.4	<5
	Kandel ER	72,653	8660	46,934	50,915	67,280	5373	7.4	5−10
	Greengard P	92,951	4921	55,412	58,536	83,333	9618	10.3	10−15
	Carlsson A	29,637	13,538	20,839	25,671	27,306	2331	7.9	5−10
	Blobel G	44,365	6981	33,766	38,551	43,861	504	1.1	<5
	Murad F	29,551	3731	25,693	2136	28,453	1098	3.7	<5
	Ignarro LJ	38,417	18,797	29,990	32,569	36,746	1671	4.3	<5
	Furchgott RF	20,109	16,708	19,059	20,089	20,109	0	0.0	<5
	Prusiner SB	73,607	18,342	52,261	53,597	66,966	6641	9.0	5−10
Control group		38,256	3396	19,435	21,575	33,957	4299	11.2	10−15
	Blinded Name at disposal upon request	32,926	6045	22,241	23,793	28,901	4025	12.2	10−15
		13,771	1821	6087	7790	11,800	1971	14.3	10−15
		51,089	13,901	45,824	46,812	48,936	2153	4.2	<5
		11,608	508	4232	5198	8461	3147	27.1	>15
		20,377	1941	15,011	15,728	19,602	775	3.8	<5
		6551	706	3644	4072	5078	1473	22.5	>15
		25,586	10,895	22,291	23,469	25,450	136	0.5	<5
		26,424	5246	14,837	18,177	22,351	4073	15.4	>15
		16,576	7364	12,840	13,741	15,459	1117	6.7	5−10
		59,641	7244	44,743	48,422	56,923	2718	4.6	<5
		42,617	1585	22,782	24,125	35,515	7102	16.7	>15
		57,480	9274	36,992	44,676	54,276	3204	5.6	5−10
		45,260	3644	22,684	23,933	34,304	10,956	24.2	>15
		27,363	2389	12,769	13,222	22,356	5007	18.3	>15
		64,930	17,342	37,272	40,443	56,369	8561	13.2	10−15
		31,923	1613	15,801	16,308	21,478	10,445	32.7	>15
		45,894	7228	22,002	24,983	35,122	10,772	23.5	>15
		17,836	3429	9918	12,425	14,837	2999	16.8	>15
		39,273	5811	17,774	20,828	26,794	12,479	31.8	>15
		14,607	2230	13,792	14,013	14,326	281	1.9	<5
		26,090	5263	17,131	19,835	22,689	3401	13.0	10−15
		22,609	2441	7207	9240	11,696	10,913	48.3	>15
		16,737	2519	9290	11,481	13,100	3637	21.7	>15
		23,677	1963	12,027	12,450	16,469	7208	30.4	>15
		68,940	17,039	26,986	39,097	45,272	23,668	34.3	>15
		16,847	1486	5768	8613	14,351	2496	14.8	10−15
		22,935	834	7507	8208	13,711	9224	40.2	>15
		29,299	4918	24,005	25,306	26,926	2373	8.1	5−10
		101,068	6721	42,879	52,209	80,370	20,698	20.5	>15
		8621	412	1594	1858	2510	6111	70.9	>15
		11,169	3897	6860	7730	8472	2697	24.1	>15
		24,896	5545	17,165	17,622	19,738	5158	20.7	>15
		22,581	5795	16,272	19,909	21,398	1183	5.2	5−10
		11,782	940	4779	7263	7954	3828	32.5	>15
		16,832	930	6878	7513	13,305	3527	21.0	>15
		16,483	7754	12,743	13,639	15,828	655	4.0	<5
		99,709	19,199	59,798	61,277	86,759	12,950	13.0	10−15
		52,727	5726	20,005	22,916	29,124	23,603	44.8	>15
		34,488	6815	23,057	23,457	30,238	4250	12.3	10−15
		32,925	3631	14,332	19,290	26,828	6097	18.5	>15
		19,152	4445	12,176	13,064	15,792	3360	17.5	>15
		82,428	10,506	27,571	38,537	48,820	33,608	40.8	>15
		102,390	31,009	76,547	79,048	89,393	12,997	12.7	10−15
		29,125	4691	16,183	16,462	24,155	4970	17.1	>15
		90,016	2651	16,227	18,450	35,509	54,507	60.6	>15
		30,385	5654	18,656	20,457	23,993	6392	21.0	>15
		49,239	5236	32,563	34,885	43,286	5953	12.1	10−15
		16,456	6404	13,546	14,698	15,837	619	3.8	<5
		38,077	5311	16,135	1395	29,081	8996	23.6	>15

From the data analysis it is extremely clear that at the class S2B, the H values of the Noble winners of the are extremely close to the global H index with a mean % difference of 6.54% and 62% of the cases with a variation <5%, in 20% of cases a variation between 5% and 10%, in 10% of cases with a variation between 10% and 15% and only in 8% of cases with a variation >15%; with only 8% of the analyzed researchers with differences >15% between H-index with S2B correction and Global H index. In the control group the percentage differences between H measured with S2B correction and Global H-index showed a statistically significant difference (Wilcoxon analysis showed a p value = 0.0008) with 20% of cases with difference with a difference >15% and an average difference of 10.7%.

The same approach was applied by analyzing the effects to the number of citations and the results are summarized in the Table 3. In this case the mean % difference between those obtained in S2B and the total number is 9.13% with 36% of the cases with a variation <5%, in 34% of cases a variation between 5% and 10%, in 20% of cases with a variation between 10% and 15% and only in 10% of cases with a variation >15%; therefore, only 10% of Nobel prize winners have differences >15% between the number of citations with S2B correction and the total number of citations. In the control group the percentage differences between global number of citations and values obtained with S2B show a statistically significant difference (Wilcoxon analysis showed a p value = 0.0001) where 56% of control group researchers have difference is >15%; average difference 20.3%).

In the Table 4 the percentages of differences are showed in order to have an easy view of the differences in H-index and Citations for Nobel and non-Nobel group whereas in the Figure 1 the boxplot is given.

Table 4.

The percentages of differences are showed in order to have an easy view of the differences in H-index and citations for Nobel and non-Nobel group.

% difference H-index Nobel	% difference citations Nobel	% difference H-index control	% difference citation cit control
5.0	6.9	10.5	11.2
3.1	5.7	3.1	12.2
1.9	5.6	6.5	14.3
4.4	13.3	1.0	4.2
0.0	1.6	11.9	27.1
9.1	12.4	0.0	3.8
8.0	12.1	4.5	22.5
1.8	1.0	1.1	0.5
3.6	1.5	7.2	15.4
3.2	3.7	4.4	6.7
8.2	8.9	3.6	4.6
2.1	2.9	10.3	16.7
0.0	2.0	5.6	5.6
13.8	10.8	14.0	24.2
0.0	0.2	13.9	18.3
2.7	6.2	8.4	13.2
13.2	12.3	14.1	32.7
9.8	15.2	16.5	23.5
14.3	19.0	5.1	16.8
1.2	1.1	9.0	31.8
1.5	7.4	0.0	1.9
4.6	7.4	6.1	13.0
13.2	11.9	22.4	48.3
40.9	41.3	10.4	21.7
1.3	1.7	18.0	30.4
9.9	13.3	26.2	34.3
15.4	16.0	7.6	14.8
1.2	6.4	25.3	40.2
11.3	4.1	8.5	8.1
5.7	5.6	13.6	20.5
18.2	11.9	31.7	70.9
6.7	6.0	9.4	24.1
2.9	3.2	10.2	20.7
4.3	7.6	2.7	5.2
4.9	6.8	20.8	32.5
0.0	0.6	13.0	21.0
23.5	66.0	0.0	4.0
6.1	7.7	8.0	13.0
8.8	10.6	25.5	44.8
2.1	2.0	8.6	12.3
4.5	8.1	12.9	18.5
1.4	2.4	7.1	17.5
4.1	7.4	18.6	40.8
4.9	10.3	4.1	12.7
2.3	7.9	9.0	17.1
0.9	1.1	27.0	60.6
3.3	3.7	11.8	21.0
5.1	4.3	5.2	12.1
0.0	0.0	1.5	3.8
4.2	9.0	17.1	23.6

Figure 1.

Box-plot representing the percentage differences in Nobel prize winner groups and controls for H-index and citations.

In order to better understand the impact of the different categories to the H-index and total number of citations a bar-percentage graph was created (Figure 2) where the percentages of the contributions of H and total number of citations derived respectively from each class (S1A, S1B, S2A, S2B, Global) are computed for Nobel and non-Nobel group.

Figure 2.

Percentage effect to H index (panel a) and citations (panel b) of the S1A-S1B-S2A-S2B and Global positions in author list in Noble prize winners and controls. The name of the Scientists of the control group are blinded for privacy but are at disposal previous authorization and upon specific request.

Concerning the other bibliometric indexes all but two (H2- and m-indexes) percentages differences resulted significantly lower in Nobel prize winner comparing with control group. In particular, the median differences for a-index was 3.5% (adjusted 95% CI, 1.6–5.5), for g-index was 3.1% (adjusted 95% CI, 0.8−5.0), for H index was 4.2% (adjusted 95% CI, 1.7%–6.7%), for H2 index was 0.0 (95% CI,−4.5−0.0), for m-quotient was 4.2% (adjusted 95% CI, 1.5–6.7), for m-index 0.5 (adjusted 95% CI, −1.5–2.8) and for r-index was 4.2% (adjusted 95% CI, 2.0–6.0). The complete results are displayed in Table 5.

Table 5.

Analysis of the variations among different metric systems.

	Noble prize (n [CI95%])	Control group (n [CI95%])	Median difference	p
m-quotient	3.30 [2.22–5.78]	8.80 [6.92–11.02]	4.20 [1.50–6.70]	0.0025
g index	3.85 [3.00–5.64]	7.10 [6.10–9.92]	3.15 [0.80–5.00]	0.0096
H2 index	5.30 [2.53–7.09]	1.85 [0.00–5.30]	0.00 [4.50–0.00]	0.16
a index	1.35 [0.18–2.37]	4.50 [3.10–6.22]	3.50 [1.60–5.50]	0.0003
m index	3.60 [2.20–5.31]	4.30 [2.60–6.04]	0.50 [−1.50–2.80]	0.58
r index	2.80 [1.40–3.48]	6.75 [5.20–9.18]	4.20 [2.50–6.00]	<0.0001
H index	4.15 [2.54–5.23]	9.00 [7.16–11.84]	4.20 [1.70–6.70]	0.0014

Discussion

The purpose of this paper was to present a new methodology, the System of Authorship Best Assessment, that weight the impact of the position name of the authors as system to better characterize the scientific output of a researcher compared to other methods currently used. The need of such type of system relies on the fact that the increase in the number of authors included in a research paper⁵ together with the automated method of calculation of scientific output performed by tools such as Google scholar or Scopus, makes complex to distinguish the real impact of a researchers. The Nobel prize winner have less reduction of all bibliometric indexes respect to the control group between Global and S2B class.

The first phase was to test the difference of our model compared to traditional system (H-index and citations) in excellent researchers. The first question was: how we can identify quite “objectively” an excellent researcher? We decided to define such type of researcher as objectively excellent if he/she was awarded with the Nobel Prize. And we found that in this group of 50 people the System of Authorship Best Assessment shows results similar to the conventional H-index and global citation values: it is clear the impact derived from the papers not included in first/second/second-last/last position for the Nobel Prizes winners usually do not play a significant role whereas the most of the results came out from the first/last position and become almost complete by including also the papers in second and second-last position.

In the second phase we wanted to test if the results we found were generalizable or if on other groups of researchers, the system showed difference and quite surprisingly we found that in another cohort of excellent researcher (from bibliometric point-of-view) the system showed a statistically significant difference compared to the Nobel Prize winner groups by lowering the H-index and citations of the controls.

By applying our model seems to be possible to obtain a screenshot of the impact of a researcher by deleting the influences of papers where the author has not preeminent or significant position. Some authors, such as Kovacs,¹³ suggested to consider the “weight” of the contributions of each author in a paper and the model proposed in the current paper tries to easily optimize this concept with the target to differentiate the global output of a researcher from his/her original capacity/contributions. The use of position in the authors list is not a novel idea for weighting the contribution of a single authors on H-index^14,15 or in g-index,¹⁶ however, to out knowledge this is the first paper that use Nobel prize winner as standard of reference for excellent research.

The effect of S2B methodology is confirmed also by the other bibliometric indexes. In fact, in all but two indexes the percentages reduction between global and S2B was significantly lower in Nobel winner group comparing with control group. The H-index³ is a well-established bibliometric parameter for researcher evaluation. However, it is not free from drawbacks, in particular it is sensible to field of research, it is sensitive to scientific age, it does not taken into account the context of the citation and auto citations.³ For example, two of the most important physics of all time, Paul Dirac and Richard Feynman had only H-index, calculated by Scholar, of 62 and 58. Some other bibliometric indexes were proposed in literature.¹⁰ The present analysis confirmed that Nobel prize winner have more consistent research compared with control group with a more preeminent role in their articles. Interestingly, H2 and m-index have an inverse trend compared to the other ones. However, it could be reasonable that Noble prize winners have more citations in less articles (the articles that support the Nobel prize) respect to the control group in which the citations and the articles have a less skewed distribution. In this set, H2 and m-index could have a paradoxical effect. A finishing touch of bibliometric indexes seems to be important nowadays, in fact, Koltun and Hafner¹⁷ demonstrated that the correlation of H-index to physics scientific award decline from 0.39 in 2010 to 0.00 in 2019 mainly due to hyper-authorship. In this set S2B may a reliable tool to overcome this trend.

It is evident that also this new system has limitations because it is possible that some academic biases could occur in the authorship position and because some complex research needs several people working on it, but it is unquestionable that the value of a first position is different from the indeterminate position. Moreover, in this analysis a perfect match for H-index, gender, age, and topic of research was not possible between Nobel and non-Nobel winners. Another limitation is that in some cases some of the control researchers shared publications with the Noble prize winners by generating a bias into the model.

This new System of Authorship Best Assessment could help to better understand the research output and could be useful to compare, in an unbiased way, different researchers in the scientific achievement by further expanding the knowledge derived from the simple H-index.

In conclusion two key results could be derived from this analysis

The H-index and number of citations calculated with the S2B (first/second/second-last/last position) correction for high-level researcher is similar to the global H-index and global number of citations

The percentage difference between H-index and citations calculated with the S2B correction and global H-index and total number of citations is very small for high level researcher (Nobel prize winners) and this evidence was confirmed by the performance of the other bibliometric indexes.

It is hoped that metric database systems (such as Google scholar, Scopus, ISI web, ResearchGate et al) incorporate these parameters in the researcher output quantification options and that further studies are being performed to test this model.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Significancy for public health

Bibliometric index is a critical need for academic and non-academic aspects. A new method is necessary for correctly evaluate research. h-index is limited by several drawback. The System of Authorship Best Assessment may help for better assess the researcher literature impact.

References

Edwards

Roy

. Academic Research in the 21st century: maintaining scientific integrity in a climate of perverse incentives and hypercompetition. Environ Eng Sci 2017; 34(1): 51–61.

Choi

Fuller

Thomas

Jr.

Estimation of citation-based scholarly activity among radiation oncology faculty at domestic residency-training institutions: 1996-2007. Int J Radiat Oncol Biol Phys 2009; 74(1): 172–178.

Hirsch

. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A 2005; 102(46): 16569–16572.

Bornmann

Daniel

. The state of h index research. Is the h index the ideal way to measure research performance? EMBO Rep 2009; 10(1): 2–6.

Greene

. The demise of the lone author. Nature 2007; 450(7173): 1165–1165.

Adams

. The rise of research networks. Nature 2012; 490(7420): 335–336.

Kovacs

. Honorary authorship and symbolic violence. Med Health Care Philos 2017; 20(1): 51–59.

Tang

. Letter to the editor: on plurality and authorship in science. Account Res 2018; 25(4): 254–258.

ICMJE. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals, https://www.icmje.org/recommendations/ (2022).

10.

Bornmann

Mutz

Daniel

H-D

. Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. J Am Soc Inf Sci Technol 2008; 59: 830–837.

11.

Rennie

Yank

Emanuel

. When authorship fails. A proposal to make contributors accountable. JAMA 1997; 278(7): 579–585.

12.

Elango

. Does co-authored articles receive more citations? Int J Inf Stud Lib 2017; 2: 1–4.

13.

Kovacs

. Honorary authorship epidemic in scholarly publications? How the current use of citation-based evaluative metrics make (pseudo)honorary authors from honest contributors of every multi-author article. Med Ethics J 2013; 39(8): 509–512.

14.

Zhang

C-T

. A proposal for calculating weighted citations based on author rank. EMBO Rep 2009; 10(5): 416–417.

15.

Sekercioglu

. Quantifying coauthor contributions. Science 2008; 322(5900): 371.

16.

Liu

Fang

. Fairly sharing the credit of multi-authored papers and its application in the modification of h-index and g-index. Scientometrics 2012; 91(1): 37–49.

17.

Koltun

Hafner

. The h-index is no longer an effective correlate of scientific reputation. PLoS One 2021; 16(6): e0253397.