The Set Theory Matching System: An Application to Ethnographic Research

Abstract

The Set Theory Matching System is a computer system application that matches data records from two independently collected data sources. The two data sources used in this study are the standard census enumeration results and an alternative enumeration using ethnographic field observation techniques. The research project this system was designed to assist is an effort to identify in more detail the kinds of people who are missed in a standard census enumeration and the circumstances that lead to less-than-complete enumeration (Brownrigg & Martin, 1989). The matching process is complicated by the occurrence of inaccurate, incomplete, and conflicting data in both data sources. The Set Theory Matching System attempts to resolve data differences by a complex set of algorithmic decisions. The system uses a hierarchical decision system that considers whole households as the primary unit of matching consideration and the evaluation of household members as the secondary matching criterion. The paper discusses the theoretical applications of the hierarchical decision theory involved in the Set Theory Matching System. It also presents an evaluation of the performance of the system in a large-scale ethnographic enumeration research project carried out in 28 sites throughout the United States for the 1990 Decennial Census.

Get full access to this article

View all access options for this article.

References

Brownrigg

(1990). 1990 guidelines for the Alternative Enumeration. U.S. Bureau of the Census, Center for Survey Methods Research.

Brownrigg

, & Fansler

(1990). Ethnographic evaluation of behavioral causes of undercount: Clerical matching review. U.S. Bureau of the Census, Centex for Survey Methods Research.

Brownrigg

, & Martin

(1989). Proposed study plan for ethnographic evaluation of behavioral causes of undercount. U.S. Bureau of the Census, Center for Survey Methods Research.

Fellegi

, & Sunter

(1969). A theory for record linkage. Journal of the American Statistical Association 40.

Horowitz

, & Sahni

(1987). Fundamentals of data structures in Pascal. Rockville, MD: Computer Science Press.

Kernighan

, & Ritchie

(1988). The C programming language. Englewood Cliffs, NJ: Prentice-Hall.

Social Security Administration. (1965a). Soundex code history. Central Office Bulletin, vol. 2 (12 February).

Social Security Administration. (1965b). DAO'S coded file. Central Office Bulletin, vol. 2 (19 February).

Statistics Canada/EDP Planning and Support Division. (1984). Record linkage software.

10.

Statistics Canada/Systems Development Division. (1982). Record linkage software.

11.

Winkler

(1985). Preprocessing of lists and string comparisons. In Alvey

, & Kilss

(eds.), Record Linkage Techniques–1985. U.S. Internal Revenue Service Publication 1299 (2–86), 181–87.

12.

Winkler

(1987a). Using the EM algorithm for weight computation in the Fellegi-Sunter model of record linkage. Technical report.

13.

Winkler

(1987b). Computational aspects of applying of the Fellegi-Sunter Model of Record Linkage to Lists of Businesses. Statistical Uses of the Administrative Data Proceedings.

14.

Wirth

(1976). Algorithms + data structures = programs. Englewood Cliffs, NJ: Prentice-Hall.