Using administrative records to increase quality and reduce burden in the Survey of Graduate Students and Postdoctorates in Science and Engineering

Abstract

The Survey of Graduate Students and Postdoctorates in Science and Engineering is an annual census of U.S. academic institutions granting research-based postsecondary degrees in science, engineering, and health fields. Sponsored by the National Center for Science and Engineering Statistics within the National Science Foundation and by the National Institutes of Health, the survey underwent a major redesign in 2017 with the goal of improving data quality while simultaneously reducing reporting burden. The key elements of the redesign effort included the use of digital file transfers for data reporting, separate collection by degree level (master’s and doctoral) of data for graduate students, and the move to a more common coding scheme to describe academic disciplines. The major challenges of this data collection involve the voluntary participation of respondents, the complexity in providing the requested data, variability in the availability of administrative records, and differences in the reporting capabilities of responding institutions. This paper focuses on these challenges and how they were addressed in the redesign process. Results from a 2016 pilot data collection as well as preliminary results of a 2017 data collection are presented to illustrate the impact of the redesign on data quality and respondent burden.

Keywords

Administrative data survey redesign data quality respondent burden

Get full access to this article

View all access options for this article.

References

Hand

. Statistical challenges of administrative and transaction data. J R Stat Soc Ser A Stat Soc. 2018; 181(Part 3): 1-24.

National Academies of Sciences, Engineering, and Medicine. Federal statistics, multiple data sources, and privacy protection: next steps. In: Harris-Kojetin

Groves

, eds. Washington, DC; 2017.

National Academies of Sciences, Engineering, and Medicine. Innovations in Federal statistics: combining data sources while protecting privacy. In: Harris-Kojetin

Groves

, eds. Washington, DC; 2017.

Bound

Krueger

. The extent of measurement error in longitudinal earnings data: do two wrongs make a right? J Labor Econ. 1991; 9: 1-24.

Kreuter

Presser

Tourangeau

. Social desirability bias in CATI, IVR, and web surveys: The effects of mode and question sensitivity. Public Opin Quart. 2008; 72(5): 847-65.

Kreuter

Muller

Trappmann

. Nonresponse and measurement error in employment research: making use of administrative data. Public Opin Quart. 2010; 74(5): 880-906.

Meyer

Mok

WKC

Sullivan

. Household surveys in crisis. J Econ Perspect. 2015; 29(4): 199-226.

Davies

Fisher

. Measurement issues associated with using survey data matched with administrative data from the Social Security Administration. Soc Secur Bull. 2009; 69(2): 1-12.

Mervis

. Plan for 2020 U.S. census is fatally flawed, critics say. Science. 2018 April 20: 250-1.

10.

Walejko

Miller

. The 2013 census test: piloting methods to reduce 2020 census costs. Survey Practice. 2015; 8(6).

11.

Bureau of Labor Statistics. Quarterly census of employment and wages: electronic data reporting. Washington, DC: U.S. Department of Labor; 2005 [Available from: https://www.bls.gov/cew/cewedr01.htm].

12.

National Science Foundation. Survey of Graduate Students and Postdoctorates in Science and Engineering fall 2016. Alexandria, VA: National Science Foundation; 2018 [Available from: https://ncsesdata.nsf.gov/gradpostdoc/2016/].