Abstract
This study reports how 3 design features (size of incentive, who is responsible, and target goal) affect clinical staff perceptions of pay-for-performance (P4P) for HPV vaccination. We conducted a national survey of clinical staff in 2022 (N = 2527; response rate = 57%). Respondents worked in pediatrics, family medicine, or general medicine specialties in the United States and had a role in HPV vaccination for children ages 9 through 12 years. Respondents were randomized to 1 of 8 P4P scenarios representing 3 design features with 2 levels each. We used ordered logistic regression to model respondents’ agreement with each of 11 statements about the P4P scenario. Statements represented domains of the Theory of Planned Behavior (attitudes, perceived behavioral control, and norms) plus 2 equity items. Relative to a $1000 incentive, a $5000 incentive was associated with favorable perceptions in 8/11 items: 5/5 attitude items and 1/2 items for each of control, norms, and equity. Incentives based on an individual provider’s patients, rather than the entire clinic’s patients, were associated with more agreement for 1 attitude item and 1 perceived behavioral control item. Relative to an absolute goal of 80%, a goal of 5% relative increase in HPV vaccination rates was associated with favorable perceptions for 3/5 attitude items and 1 equity item. Clinical staff perceptions of P4P for HPV vaccination were more favorable the larger the size of the incentive, when it was focused on a provider’s own patients, and when the goal was a relative, rather than absolute, target.
Highlights
Pay-for-performance (P4P), which ties financial incentives to improvements in quality measures, can increase child vaccination. However, use of P4P to promote human papillomavirus vaccination is rare. We know little about how incentive design features affect clinical staff’s perceptions of P4P.
Clinical staff perceptions of P4P for HPV vaccination were more favorable the larger the size of the incentive, when it was focused on a provider’s own patients, and when the goal was a relative, rather than absolute, target.
P4P designers can use this information to increase participation in P4P programs, participant effort, program effectiveness, and sustainability.
Introduction
The HPV vaccine can prevent over 90% of the infections associated with HPV cancers.1,2 There are an estimated 34 800 new cases of HPV-attributable cancers (cervical, vulvar, penile, anal, oropharyngeal, vaginal) each year in the United States (US). 3 Therefore, HPV vaccination has the potential to prevent 32 100 cancers per year in the US. 3 Oropharyngeal and cervical cancers are the most common. 3 As of 2024, 61% of adolescents 13 to 17 were fully vaccinated, well below the Healthy People 2030 goal of 80%. 4
The use of financial incentives to providers to increase quality and/or lower costs has increased as pay-for-performance (P4P) has proliferated.5-7 P4P ties financial incentives (eg, bonuses) to improvements in quality measures. 8 P4P programs have been shown to improve child vaccination rates.9-17 P4P programs targeting vaccines have been successful because vaccines are influenced by provider recommendation, can be completed in a few office visits, and are relatively easy to track.9,18-20 However, use of P4P to promote human papillomavirus (HPV) vaccination is rare. 21 While clinical staff perceptions of P4P are generally favorable22,23 and there is guidance for design of incentives,9,10,20,24-27 we know little about how incentive design features affect clinical staff’s perceptions of P4P, especially for HPV vaccination.
This study reports how variation in 3 key P4P design features (size of incentive, who is responsible, and target goal) affects clinical staff perceptions of a P4P incentive for HPV vaccination. The design of P4P programs is crucial for their effectiveness. Clinical staff perceptions of P4P programs, and their key design features, will affect the decision to participate in P4P programs, the amount of behavior change induced by the P4P programs, the P4P programs’ effectiveness, and the sustainability of the programs. P4P implementers can use the results to improve the design of P4P programs for HPV and other vaccines, leading to increased vaccination rates, improved financial performance for clinics in P4P contracts, and improved public health.
Methods
Survey Participants
This study compared perceptions of P4P for HPV vaccinations across randomized P4P design features in a cross-sectional national survey. The survey included clinical staff working in primary care clinics in the US that provided HPV vaccination to children. WebMD Market Research recruited participants through their Medscape Network. The online panel includes about 60% of US physicians. From May through July 2022, WebMD Market Research sent invitations to active participants in their Medscape Network. Respondents provided informed consent and completed the survey online, after which they received a $30 to 45 honorarium, depending on their clinical role. The inclusion criteria were that respondents were (1) certified to practice in the US; (2) practiced as a physician, physician assistant, nurse practitioner, advanced practice nurse including nurse practitioner, registered nurse, licensed practical/vocational nurse, medical assistant, or certified nursing assistant; (3) worked in pediatrics, family medicine, or general medicine specialties; and (4) had a role in HPV vaccination for children ages 9 through 12 years. The survey had 2527 responses and an American Association for Public Opinion Research response rate #3 of 57% (Supplemental Figure 1). 28 There were no missing responses among the 2527 completed surveys; all were included in the analyses.
Randomization
Respondents were provided a brief description of a P4P program for HPV vaccination to consider before answering several questions about the program. Respondents received the following description of a P4P program with the fields in brackets randomly assigned: “Imagine you will get a {$1000 OR $5000} bonus at the end of the year if the HPV vaccine completion rate among {your OR your clinic’s} 13-year-old patients {reaches 80% OR increases by 5%}.” Each respondent was randomized to 1 of 8 scenarios representing 3 design features with 2 levels each. The scenarios varied the amount of the annual financial bonus ($1000 or $5000), responsibility for the target HPV vaccine completion rate (the respondent’s patients or the clinic’s patients) and the target rate (13-year-old vaccine completion rate reaches 80% or increases by 5%). For the scope of the target, “your” 13-year-old patients was intended to be a smaller set of patients than the entire clinics’ patient pool. In pilot testing, respondents tended to interpret “your” patients to mean those patients they see personally or those assigned to the provider(s) under whom they work. The P4P target was selected because Healthy People 2030 and HEDIS adolescent immunization quality metrics target HPV vaccine series completion by age 13 to 15 and age 13, respectively. Healthy People 2030 target 80% completion; we chose a relative increase of 5% as an achievable goal based on other HPV vaccine interventions. 29
Outcome Variables
Each respondent was then asked to rate the extent to which they agreed or disagreed with a set of previously validated statements included in Meterko et al’s 30 evaluation of provider attitudes about P4P programs. To reduce respondent burden, we included 9 of 26 statements from the original study. We mapped the statements to domains of the Theory of Planned Behavior (TPB) 31 following Waddimba et al. 32 The TPB predicts an individual’s likelihood to change behavior to meet the P4P’s goals is based on 3 components: attitudes toward the behavior, subject norms associated with the behavior and perceived behavioral control. In addition to the TPB, we also added 2 statements focused on the perceived equity implications of P4P programs (Table 1).
Outcome Survey Items.
Note. Items adapted from Meterko et al. 30 Respondents saw the following prompt (fields in brackets randomly assigned): “Imagine you will get a {$1000 OR $5000} bonus at the end of the year if the HPV vaccine completion rate among {your OR your clinic’s} 13-year-old patients {reaches 80% OR increases by 5%}.” For each statement in the table above, respondents selected level of agreement from a 5 level Likert scale: “Strongly disagree,” “Somewhat disagree,” “Neither disagree nor agree,” “Somewhat agree,” and “Strongly agree.”
Reverse coded so agreement is favorable for P4P.
Respondents rated their agreement for each of the 11 statements on a five-point Likert scale from strongly disagree (1) to strongly agree (5). We re-coded the responses to 3 categories: “Strongly agree” or “Somewhat agree,” “Neither disagree nor agree,” and “Somewhat disagree” or “Strongly disagree.”
Covariates
The main explanatory variables were fully interacted indicators for the levels in each of the 3 design dimensions, representing each unique randomized scenario. We checked for balance across randomized design feature levels for characteristics of the clinic (financial incentives at the clinic, rurality, specialty, number of clinics in the system, ownership structure, number of prescribers, % of children using Vaccine for Children, and number of patients ages 9-12) and of the respondent (training, gender, race/ethnicity, and years in practice).
Statistical Analysis
We used ordered logistic regression to model respondents’ agreement (ie, “Strongly agreed” or “Agreed,” “Neither agree nor disagree,” “Disagree” or “Strongly disagreed”) with each of 11 statements about the P4P program. The 11 outcomes were analyzed separately but grouped for discussion as follows: attitudes (5 items), control (2), norms (2), and equity (2). The main specification relied on randomization and adjusted only for fully interacted indicators for the level of each design dimension. We tested the proportional odds assumption of the ordered logit using Hausman tests relative to multinomial logit models. We failed to reject the proportional odds assumption for all but one outcome; for consistency we estimated separate ordered logit models for all outcomes. As a robustness check, we also included models controlling for the full set of covariates listed above. All statistical analyses were performed using Stata version 15.1 (College Station, TX). We followed STROBE reporting guidelines for cross-sectional observational studies. 33 This study was approved by the University of North Carolina at Chapel Hill Institutional Review Board (IRB Study #21-2829).
Results
Most respondents worked in clinics with no use of financial incentives and in non-rural areas, evenly split between pediatrics and family medicine (Supplemental Tables 1–3). Respondents’ clinics tended to be either in large systems of 5 or more clinics or not in a system at all. The most common practice ownership type was group practice. Just under half of the respondents were physicians, about one quarter were nurses, and the rest of the sample were advanced practice nurses (~16%), physician assistants (8%), and medical and nursing assistants (4%). Most respondents were women (72%) and white (65%). There were no statistically significant differences in clinic and respondent characteristics between levels of the randomized P4P design features (Supplemental Tables 1–3).
Figures 1 to 3 present the raw response frequency of each outcome variable stratified by levels of each design feature and grouped by TPB domains (counts, percentages and tests for differences across design feature in Supplemental Tables 4–6). Across design features, most respondents agreed with 6/11 items, with the highest rates for the Goal is Good for their patients (84%-87%). Disagreement rates were roughly 30% or higher for Income Opportunity, Better Care, Actions within Control, and Equitable Care. The rates for neither agreeing nor disagreeing with the items were highest for Equitable Care (38%-40%) and Better Care (29%-31%).

(a) Attitudes outcomes by randomized amount of incentive ($5000 vs $1000) and (b) control, norms, and equity outcomes by randomized amount of incentive ($5000 vs $1000).

(a) Attitudes outcomes by randomized scope of target (clinic vs provider) and (b) control, norms, and equity outcomes by randomized scope of target (clinic vs provider).

(a) Attitudes outcomes by randomized goal (reaches 80% vs increases by 5%) and (b) control, norms, and equity outcomes by randomized goal (reaches 80% vs increases by 5%).
To summarize the main analyses, we consider a design feature leading to a favorable perception when there was a statistically significant marginal effect (5% level) for a decrease in the probability of disagreeing with the statement, an increase in the probability of agreeing with the statement, or both. Relative to a $1000 incentive, a $5000 incentive was associated with favorable perceptions in 8/11 items: 5/5 attitude items and 1/2 items for each of control, norms, and equity (Table 2). Incentives based on an individual provider’s patients, rather than the entire clinic’s patients, were associated with more agreement for one attitude item (Income Opportunity) and one perceived behavioral control item (Actions within Control). Relative to an absolute goal of 80%, a goal of 5% relative increase in HPV vaccination rates was associated with favorable perceptions for 3/5 attitude items (Change Behavior, Better Care, and Invest Time and Effort) and one equity item (Equitable Care). Estimated marginal effects provided in Supplemental Tables 7 to 11.
Significant Associations Between Pay-for-Performance (P4P) Design Features and Favorable Perceptions of P4P.
Note. “+” indicates a statistically significant marginal effect (5% level) for a decrease in the probability of disagreeing with the statement, an increase in the probability of agreeing with the statement, or both. Marginal effects from separate ordered logistic models for each outcome adjusting for 8 randomized combinations of design features.
In the robustness check that included clinic and respondent covariates in the ordinal logistic models, 2 marginal effects were similar in magnitude to the main analysis but were no longer statistically significant. Incentives based on an individual provider’s patients, rather than the entire clinic’s patients, were no longer statistically significantly associated with agreement with the Income Opportunity item (marginal effect for the Pr(Disagree) = −0.028 [95% confidence interval = −0.058, 0.0013]). Incentives targeting a 5% relative increase in HPV vaccination rates were no longer statistically significantly associated with agreement with the Better Care item (marginal effect for the Pr(Disagree) = −0.027 [95% confidence interval = −0.056, 0.0015]).
Discussion
This study examined how important design features of P4P programs, such as the amount of the financial incentives and how the target goals are defined, affect clinical staff’s perceptions of the P4P programs. We found that the amount of the incentive bonus for meeting quality improvement goals was associated with the most items, including statements from all dimensions of the TPB plus an equity dimension. Larger incentives led to increased favorability in perceptions of the P4P program. P4P incentives targeting clinical staff’s own patients, rather than the clinic’s patients, and framed as a relative increase over baseline, rather than an absolute target, led to increased favorability in fewer dimensions.
The main contribution of this study is to provide evidence of the impact of P4P design features on clinical staff’s perceptions of P4P. Prior research investigating provider perceptions of P4P programs tended to focus on the entirety of the programs rather than specific components. 32 It is also difficult for randomized trials to power head-to-head comparisons of different P4P program designs. Our study provides useful and novel information for payers and health systems designing P4P programs, especially those aiming to increase HPV vaccination.
Our results contribute to the literature on best practices for P4P.9,10,20,24-27 First, larger incentives improve favorability of P4P among clinical staff in many dimensions. Existing studies have used a variety of incentive sizes for providers. 34 Our incentive sizes are similar to those used by Fairbrother et al.11,12 While our results do not provide definitive guidance on selecting the size of P4P incentives for HPV vaccination, health systems and insurers may engender more buy in to P4P programs if they do not skimp on the financial incentives.
Second, our results support the recommendation to direct incentives at individuals or small teams.9,20 Specifically, incentive targets focusing on clinical staff’s own patients, rather than the entire clinic’s patients, increased clinical staff’s feelings of control over outcomes in the P4P program. Finally, while at least 1 review found that absolute quality targets (eg, 80% HPV vaccination rate) were more effective than relative improvement targets, 9 our results suggest that favorability among clinical staff is higher for P4P programs using relative targets. This may be because our relative target was only a 5% increase that may have been seen as easier to attain than the 80% absolute target. It is possible that the increased favorability among clinical staff would translate into higher effort to earn the incentives and better health outcomes for patients, but future research is needed to confirm that hypothesis.
Other strengths of our study include the participation of a wide range of clinical staff, not just physicians, who are typically targeted in P4P and related research. 22 Successful HPV vaccination involves a primary care team (team based approach), so we included broader clinical staff our study on P4P design.35,36 Including clinical staff in P4P programs focused on primary and preventive care metrics like vaccines may increase clinical team engagement and progress on primary care metrics. 37 Our randomization design also helps with causal interpretation of the relationship between the P4P design features and clinical staff perceptions. Specifically, our samples were balanced on observable characteristics across the randomized design features. If unobserved characteristics were similarly balanced, our results would not be biased from confounding.
This study had limitations. While the study provides new information on how clinical staff perceptions change with key design features, it does not link design features to effectiveness of the P4P program(s) to improve vaccine recommendations or uptake. Future randomized controlled trials are needed to determine the effectiveness of P4P programs for increasing HPV vaccination. The HPV P4P programs were hypothetical; responses may have been different if participants were enrolled in such programs. In addition, although the sample included clinical staff from across the nation, it may not be nationally representative. Lewis et al 38 provide a comparison of the sample demographics to a nationally representative sample of these professionals.
Another limitation of this study is that our P4P program only involved bonuses to respondents; we did not propose a P4P program with “downside” risk in which participants would pay a financial penalty for not meeting quality targets. It is plausible that perceptions about P4P design features would be different under a program with downside risk. For example, respondents might prefer clinic-level incentives over individual provider-level incentives with risk of a financial penalty. As these types of P4P programs become more popular, future research is needed to understand how provider perceptions change with the incorporation of downside risk.
Conclusion
This study provided useful new evidence about the role of P4P design on clinical staff perceptions of P4P. Clinical staff perceptions of a financial incentive for HPV vaccination were more favorable the larger the size of the incentive, when it was focused on a provider’s own patients, and when the goal was a relative, rather than absolute, target. The size of the incentive mattered most. P4P designers can use this information to increase participation in P4P programs, participant effort, program effectiveness, and sustainability.
Supplemental Material
sj-docx-1-inq-10.1177_00469580251353406 – Supplemental material for Design of Pay-for-Performance Programs Affects Clinical Staff Perceptions of HPV Vaccine Incentives
Supplemental material, sj-docx-1-inq-10.1177_00469580251353406 for Design of Pay-for-Performance Programs Affects Clinical Staff Perceptions of HPV Vaccine Incentives by Justin G. Trogdon, Kathryn Brignole and Tara Licciardello Queen in INQUIRY: The Journal of Health Care Organization, Provision, and Financing
Supplemental Material
sj-docx-2-inq-10.1177_00469580251353406 – Supplemental material for Design of Pay-for-Performance Programs Affects Clinical Staff Perceptions of HPV Vaccine Incentives
Supplemental material, sj-docx-2-inq-10.1177_00469580251353406 for Design of Pay-for-Performance Programs Affects Clinical Staff Perceptions of HPV Vaccine Incentives by Justin G. Trogdon, Kathryn Brignole and Tara Licciardello Queen in INQUIRY: The Journal of Health Care Organization, Provision, and Financing
Footnotes
Ethical Considerations
This study was approved by the University of North Carolina at Chapel Hill Institutional Review Board (IRB Study #21-2829).
Consent to Participate
Survey participants provided written informed consent.
Consent for Publication
Not applicable.
Author Contributions
JGT contributed to conceptualization, data curation, formal analysis, funding acquisition, investigation, methodology, supervision, visualization, writing the original draft, and editing and reviewing. KB contributed to data curation, methodology, project administration, visualization, writing the original draft, and editing and reviewing. TLQ contributed to data curation, formal analysis, investigation, methodology, and editing and reviewing.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number P01CA250989. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
