In measuring the overlap between two sets A and B (e.g. libraries, databases) one is obliged to calculate the overlap O(A|B) of A with respect to B (i.e. the fraction of elements of B that are also in A) and of O(B|A) of B with respect to A (i.e. the fraction of elements in A that are also in B). Theoretically this requires two samples. In this paper we explain that one sample can suffice to determine confidence intervals for both O(A|B) and O(B|A). The paper closes with the example of measuring the overlap between the secondary sources in mathematics MathSciNet and Zentralblatt MATH and with a remark on the estimation of the Jaccard index.
W. Y. Arms, Duplication in union catalogues, Journal of Documentation29(4) (1973) 373-379.
2.
M. K. Buckland, A. Hindle and P. M. Walker, Methodological problems in assessing the overlap between bibliographical files and library holdings, Information Processing and Management11(3-4) (1975) 89-105.
3.
L. Egghe, Properties of the n-overlap vector and n-overlap similarity theory, Journal of the American Society for Information Science and Technology57(9) (2006) 1165-1177.
4.
L. Egghe and R. Rousseau, Introduction to Informetrics: Quantitative Methods in Library, Documentation and Information Science (Elsevier, Amsterdam, 1990).
5.
D. A. Grossman and O. Frieder, Information Retrieval Algorithms and Heuristics (Kluwer Academic Publishers, Boston, MA, 1998).
6.
L. Egghe and R. Rousseau, Elementary Statistics for Effective Library and Information Service Management (Aslib, London, 2001).
7.
M. Gluck, A review of journal coverage overlap with an extension to the definition of overlap, Journal of the American Society for Information Science41(1) (1990) 43-60.
8.
W. W. Hood and C. S. Wilson, Overlap in bibliographic databases, Journal of the American Society for Information Science and Technology54(12) (2003) 1091-1103.
9.
A. Giral and A. G. Taylor, Indexing overlap and consistency between the Avery Index to Architectural Periodicals and the Architectural Periodicals Index, Library Resources & Technical Services37(1) (1993) 19-44.
10.
N. Pravdic and V. Oluić-Vuković, Application of overlapping technique in selection of scientific journals for a particular discipline - methodological approach, Information Processing and Management23(1) (1987) 25-32.
11.
P. Vinkler, Direct and indirect literature overlap measures for information pools of research teams, Journal of Information Science23(6) (1997) 433-443.
12.
S. Wu and F. Crestani, Multi-objective resource selection in distributed information retrieval. In: Proceedings of IPMU 02, International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, Annecy, France, July 2002, 1171-1178. Available at www.cis.strath.ac.uk./%7Efabioc/papers/02-ipmu.pdf (accessed 30 November 2006).
13.
B. R. Boyce, C. T. Meadow and D. H. Kraft, Measurement in Information Science (Academic Press, New York, 1995).
14.
L. Egghe and R. Rousseau, Classical retrieval and overlap measures satisfy the requirements for rankings based on a Lorenz curve. Information Processing and Management42(1) (2006) 106-120.
15.
G. Salton and M. J. McGill, Introduction to Modern Information Retrieval (Mc Graw-Hill, New York, 1987).
16.
J. Tague-Sutcliffe, Measuring Information: an Information Services Perspective (Academic Press, New York, 1995).
17.
C. J. Van Rijsbergen, Information Retrieval (Butterworths, London, 1979).
18.
L. Egghe, A universal method of information retrieval evaluation: the “missing” link M and the universal IR surface, Information Processing and Management40(1) (2004) 21-30.
