Challenges and opportunities in administrative records

Abstract

There are presently a number of factors conspiring to force statistical agencies to consider more fully the exploitation of administrative records for statistical purposes. Agencies face shrinking budgets, rising demands for their services, a need to reduce respondent burden and the need to better service an increasingly sophisticated and demanding clientele. At the same time developments in information technology and in statistical methodology present opportunities in record management, transfer and manipulation that did not exist even a year or two ago. Now is an opportune time to consider new directions.

A prerequisite to new directions, however, is the need for agencies to put their houses in order with regard to data management. Agencies do not now generally manage and exploit their own data resources well. Significant improvements will have to be made before agencies gain the credibility and the confidence that they can effectively manage the data holdings of others.

As a minimum to administrative record mining, overly restrictive privacy legislation needs to be modified with access authority provided to statistical agencies. With regard to the issue of the standardization of record keeping, three possible scenarios emerge. One is the introduction of legislation which provides the agency with the authority to influence the content and format of records. Another is the possible collaboration of record holders in developing standards (ideally in conjunction with the agencies) because of perceived benefits to themselves. A third is the possibility of the information technology industry developing generic products (with built-in standards) which become universal in application thereby introducing the standards needed for more effective data mining.