Quantifying scientists’ research ability by taking institutions’ scientific impact as priori information

Abstract

Scholar performance evaluation is extremely important in research assessment decisions, such as funding allocation, academic rankings, and academic promotion. In this article, we propose the institution Q model (IQ) and its two variants (IQ-2 and IQ-3), which aim to evaluate the individual-level research ability to publish high-quality scientific papers. Specifically, our models integrate scientists’ institutions, countries and collaborators as valuable prior information and jointly evaluate the research ability of scientists from different institutions. To estimate model parameters and hidden variables defined in our models, we propose a generic BBVI-EM algorithm. To test the effectiveness of our models, we examine their performance on the synthetic data and the empirical data (17,750/26,992 scientists in the computer science/physics field). We find that our models can more accurately quantify the research ability of scientists and institutions and more effectively predict scientists’ scientific impact (the h-index and total citations) than the Q model and common machine learning models. In conclusion, our models are effective evaluation and prediction tools for quantifying research ability and predicting the scientific impact, and the BBVI-EM algorithm is an effective variational inference algorithm. This study makes a theoretical contribution to broaden the idea of incorporating the academic environment into scientific evaluation.