Abstract
In 2016 to 2017, we surveyed primary care providers (PCPs) in upper Manhattan and the South Bronx, New York, on their knowledge, attitudes, and practices surrounding preexposure prophylaxis (PrEP) for HIV. Despite efforts to promote survey response, we were only able to obtain a meager response rate, limiting our ability to interpret results. In this short communication, we examine our survey’s methodology, as well as the methods used by other similar studies, in order to suggest how certain strategies appear to influence PCP response to PrEP surveys. Administering the survey in a variety of modes, sampling from a professional organization’s listserv, promoting the survey topic’s relevance to potential participants, and offering monetary incentives to each survey respondent all appear to be promising strategies for increasing response rates in PrEP provider surveys.
Introduction
Preexposure prophylaxis (PrEP) has been demonstrated to be effective in reducing HIV incidence among at-risk individuals and is now an important addition to the HIV prevention toolbox. 1 As gatekeepers for biomedical interventions, health-care providers play an essential role in the scale-up of PrEP for individuals at risk for HIV infection. 2,3 Understanding how medical providers view and prescribe PrEP is a first step in informing future scale-up efforts. For this reason, many researchers have recently surveyed providers on their knowledge, attitudes, and practices surrounding PrEP (Table 1). Some attempts at surveying providers about PrEP have resulted in low response rates (RRs). 16 –18 Because low response yield may result in a sample that does not adequately represent a study’s sampling frame, 20 a low RR can limit the utility of the survey’s findings. In 2016 to 2017, we surveyed primary care providers (PCPs) in upper Manhattan and the South Bronx to assess knowledge, attitudes, and practices regarding PrEP services. We obtained a poor RR, and we were thus unable to interpret the study’s results. In this brief report, we consider our survey methodology in the context of similar, recently published surveys in order to provide guidance to other researchers conducting Internet-based surveys of providers regarding PrEP adoption.
Characteristics of Online Physician Surveys on Pre-exposure Prophylaxis.
Abbreviations: Am Acad HIV Med, American Academy of HIV Medicine; AMA, American Medical Association; EIN, Emerging Infections Network; HMO, Health Maintenance Organization; ID, infectious diseases; MD, medical doctor; NE AETC, Northeast AIDS Education & Training Center; NP, nurse-practitioner; PCP, primary care physician; RR, response rate; Rs, respondents; SGIM, Society of General Internal Medicine.
a Authors note this estimate is likely inflated due to the inability to calculate denominator from the web site link and snowball referrals.
b Authors’ term: “Cooperation rate.”
Our Survey Methods
We designed a brief survey to examine PCPs’ adoption of PrEP into their practices. The survey assessed providers’ knowledge, attitudes, and clinical experience with PrEP and asked them to respond to hypothetical patient scenarios. We pilot tested the survey and found it took approximately 10 minutes to complete.
Our objective was to construct a sampling frame inclusive of providers most likely to encounter individuals who might not be aware of being at substantial risk for HIV but who would benefit from taking PrEP. We therefore approached 294 primary care medical providers affiliated with a Health Maintenance Organization (HMO) that accepts Medicaid and who serve areas of New York City with high HIV incidence, upper Manhattan and the South Bronx. Infectious disease specialists were excluded because they are presumed to have specialized access to training and resources about PrEP.
Because we were unable to obtain e-mail addresses from the online HMO database, recruitment letters were mailed to the providers’ offices. The recruitment letters included information about the survey and detailed measures taken to safeguard respondents’ confidentiality; an easy URL that linked to the survey; and description of incentives, that is, all respondents were entered into a lottery for three USD50 gift cards. Letters were printed using official letterhead and included personalized addresses and salutations, blue-ink signatures, stamped postage, and other methods to improve RRs. 21 Two reminder postcards were sent in follow-up 1 month apart in addition to periodic telephone reminders when phone numbers were available. The Columbia University institutional review board approved the survey (IRB-AAAO0852); e-consent was obtained.
Results and Discussion
Our RR was strikingly low. While few of the recruitment mailings were returned as undeliverable, only 16 providers responded to the survey after reminder mailings and calls, yielding a RR of 5.5%. To understand this low RR, we assessed our survey instrument and recruitment methods, drawing on published literature to inform our assessment.
Our questionnaire appeared to conform with criteria for appropriate and sound measurement tools: none of the respondents dropped out or had missing data, the average survey completion time was 13 minutes, the items elicited good variability in response options, and the skip and display logic worked appropriately. Additionally, no respondents reported difficulties or negative experiences in taking the survey.
Given the apparent soundness of our instrument, we examined other factors that may affect RRs. 20,21 In August 2017, we searched PubMed, Medline, and Google Scholar for recently published articles that report surveying health-care providers via the Internet on their knowledge, attitudes, and/or practices regarding PrEP in order to explore how methods used by these surveys may have affected their RRs. All articles that met these criteria and provided sufficient description of survey methodology were included. Table 1 shows the target sample size, survey administration and recruitment modes, use of incentives, and final RRs of reviewed surveys. In cases where study authors did not report RRs, they were calculated by dividing the number of respondents by the number of providers invited to participate.
While no conclusive best practices can be drawn, some strategies appeared to be more effective in eliciting high RRs than others. In general, using samples assembled from professional organizations’ membership lists were effective, especially if these organizations were smaller in size. 4,7,8,10 The one study that recruited members from a panel of individuals who have actively agreed to participate in surveys obtained an especially high RR (59.8%). 6 Such a strategy, which aims to recruit respondents who have already indicated interest in survey participation, has been suggested to be a more effective recruitment method than those that advertise a survey to a group of individuals whose interests are unknown. However, use of such a methodology may result in a less diverse and possibly less representative sample. 22
While all surveys were administered online, the approach by Castel et al 5 to also include a paper-and-pencil option appeared to be effective (RR = 61.5%). While most studies recruited participants by e-mailing a professional organization’s listserv, those that mixed recruitment modes appear to be most effective (mean RR = 38.13%). 4,5,8 –11,16,17 Additionally, while the 2 studies that included mailed letters in their larger recruitment plans 5,9 achieved strong RRs (61% and 30%), the only study that exclusively utilized a mailed letter recruitment strategy for all potential participants was our survey, which achieved a poor RR (5.4%); we were only able to follow-up by e-mail with about one quarter of our target list of providers. This suggests that mailed letters may help bolster e-mail-based recruitment efforts, but on their own may be an ineffective strategy for achieving an adequate RR. These findings are concordant with reviews suggesting mixed-mode recruitment and administration may be beneficial for increasing RRs. 23 –25
While reminders were not always needed to produce high RRs, 2 to 3 reminders at 1-week intervals appear to be the most effective way to increase the RR (RR = 48.8% and 30%). 7,9 As with distribution and recruitment, the most effective specified reminder strategy involved mixing physical and electronic modes (RR = 61.5%). 5
Incentives were not always necessary in order to obtain a high RR. Providing universal incentives to all those who complete the survey 5,9,10 –12,14 appears to be more effective than lotteries, 16,17,19 although we were not able to conduct a test for statistical significance due to low sample size and limited RR reporting. The studies that offered cash and gift card incentives 5,9 –12 appear to be more effective than those that offered lotteries for iPads, 16,17,19 but we had similar limitations in assessing whether this difference is significant. These findings are also consistent with past research indicating that monetary incentives can improve RRs in surveys of physicians. 26 –28 Other evidence suggests that unconditional incentives sent with the questionnaire may be best at promoting response even if they are small, as the respondent may feel obliged to reciprocate the gift. 21,29
Perhaps the most important factor to address when advertising a survey is stressing its relevance to the potential respondents. Whether or not a respondent perceives a survey’s subject matter to be personally relevant is one of the most critical factors determining his or her participation. 30 Indeed, a few of the PCPs who were asked to participate in our study replied that they have never prescribed PrEP and/or that their practice has nothing to do with PrEP; therefore, they declined to be surveyed. Low RRs precipitated by low perceived relevance would be particularly concerning because these surveys aim to measure whether such attitudes and practices exist among providers. Not capturing a lack of adoption could result in an overestimation of the amount of providers offering PrEP and an underestimation of the need for PrEP-related medical training and dissemination efforts.
Interestingly, the low response we received in our survey of PCPs may relate to the “purview paradox” of PrEP 31,32 —HIV specialists are comfortable prescribing antiretroviral medication but are unlikely to see HIV-negative patients; providers who care for people at substantial risk for HIV are unfamiliar with antiretroviral medication and may overestimate the difficulties in prescribing it as PrEP. This paradox may lead PCPs to underestimate their potential role in making PrEP more accessible. The low RR in our study may suggest that this paradox is present in our response catchment area. On the other hand, it may simply be indicative of “survey fatigue” due to the growing number of surveys made possible by the ease of Internet-based survey administration. We found no significant difference between studies that surveyed infectious disease and HIV specialists 5,7,8,10,11,13 –15 or providers who serve areas of high HIV incidence 9,18 and those surveying general practitioners 4,6,16,17 in achieving RRs above or below 14% (Fisher exact test P >.05).
It is also likely that publication bias significantly limits the survey efforts we were able to review. Many surveys that do not obtain RRs that are sufficient for representing their catchment area or powering their analyses are not published; therefore, it is difficult to conclusively determine which methods are effective and which are not. Nevertheless, due to the critical need for surveys of the PrEP attitudes of care providers, this review highlights the methods that have tended to produce higher RRs than others for the benefit of this future research.
Conclusion
Our survey experience illustrates how difficult it can be to effectively survey PCPs regarding their adoption of PrEP. Even when the researchers follow evidence-based administration and recruitment strategies, survey nonresponse can be an insurmountable limitation to this type of research. Nevertheless, this type of research is fundamental in assessing how PrEP is being used and how to improve its use, and some of the surveys reviewed achieved adequate response in spite of the challenges of survey research. We hope that this report can lend important information to researchers when planning surveys of providers on PrEP and similar topics. Our survey attempt and our review of other recent physician PrEP surveys suggests that when asking providers about their knowledge, attitudes, and practices surrounding PrEP, it may be prudent to utilize a mixed-mode administration strategy, utilize a professional organization’s listserv if possible, develop strategies to promote relevance, and offer a monetary incentive to all participants in order to achieve high RRs.
Footnotes
Authors’ Note
Research materials can be made available upon request to senior author, Paul Colson,
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported here was supported by the National Institute of Mental Health (R01MH098723). Y.H.-M. was supported by the National Institute of Allergy & Infectious Diseases under award number 1K01A104351.
