Which Outreach Modes Improve Response Rates to Physician Surveys? Lessons from an Experiment at the American Board of Internal Medicine

Abstract

Physicians are a notoriously difficult group to survey due to a low propensity to respond. We investigate the relative effectiveness of reminder phone calls, pre-notification postcards, mailed paper surveys, and $1 upfront incentives for boosting survey response rate by embedding a randomized experiment into a mixed-mode operational survey at the American Board of Internal Medicine in 2019. Expected response rates and average marginal effects for each follow-up method were computed from a logistic regression model. The control group which only received email reminders achieved a response rate of 18.2%, 95% CI: (15.0%, 21.9%). The intervention group which included reminder emails, pre-notification postcards, and mailed paper surveys with $1 incentives achieved a response rate of 43.1%, 95% CI: (38.8%, 47.5%). Mailed paper surveys yielded the largest percentage point increase in response rate of 11.2%, 95% CI: (7.3%, 15.2%), while $1 upfront monetary incentives and phone call reminders increased survey response rate by 5.9%, 95% CI: (1.6%, 10.2%) and 5.5%, 95% CI: (2.6%, 8.3%) respectively. Pre-notification postcards are associated with a 2.0%, 95% CI: (−1.7%, 5.6%) increase in survey response rate. Cost-effectiveness for each method is discussed. This research supports optimal decision making for researchers when planning a physician survey study.