The effect of co-opinion on the cocitation-based information retrieval systems’ effectiveness evaluated by semantic similarity

Abstract

The co-opinionatedness measure, that is, the similarity of cociting documents in their opinions about their cocited articles, has been recently proposed. The present study uses a wider range of baselines and benchmarks to investigate the measure’s effectiveness in retrieval ranking that was previously confirmed in a pilot study. A test collection was built including 30 seed documents and their 4702 cocited articles. Their citances and full-texts were analysed using natural language processing (NLP) and opinion mining techniques. Cocitation values, syntactical similarity and contexts similarity were used as baselines. The distributional semantic similarity and the linear and hierarchical Medical Subject Headings (MeSH) similarities served as benchmarks to evaluate the effect of the co-opinionatedness as a boosting factor on the performance of the baselines. The improvements in the rankings were measured by normalised discounted cumulative gain (nDCG). According to the findings, there existed significant differences between the nDCG mean values obtained before and after weighting the baselines by the co-opinionatedness measures. The results of the generalisability study corroborated the reliability and generalisability of the systems. Accordingly, the similarity in the opinions of the cociting papers towards their cocited articles can explain the cocitation relation in the scientific papers network and can be effectively utilised for improving the results of the cocitation-based retrieval systems.