Expanding PrEP Services for Men Who Have Sex With Men and Transgender Persons Through Health Department Programs: Key Processes and Outcomes From Project PrIDE,2015-2019

Abstract

Objective:

Pre-exposure prophylaxis (PrEP) Implementation, Data to Care, and Evaluation (PrIDE) was a demonstration project implemented by 12 state and local health departments during 2015-2019 to expand PrEP services for men who have sex with men (MSM) and transgender persons at risk for HIV infection. We describe findings from the cross-jurisdictional evaluation of the project.

Methods:

We analyzed work plans, annual progress reports, and aggregate quantitative program data submitted by funded health departments (n = 12) to identify key activities implemented and summarize key project outcomes.

Results:

PrIDE jurisdictions implemented multiple health equity–focused activities to expand PrEP services to priority populations, including building program capacity, conducting knowledge and awareness campaigns, providing PrEP support services, and addressing barriers to PrEP use. Overall, PrIDE jurisdictions identified 44 813 persons with PrEP indications. Of these, 74.8% (n = 33 500) were referred and 33.1% (n = 14 821) were linked to PrEP providers, and 25.3% (n = 11 356) were prescribed PrEP. Most persons prescribed PrEP were MSM or transgender persons (87.9%) and persons from racial and ethnic minority groups (65.6%). However, among persons with PrEP indications, non-Hispanic Black/African American persons (14.9% of 18 782) were less likely than non-Hispanic White persons (31.0% of 11 633) to be prescribed PrEP (z = −33.57; P < .001).

Conclusions:

PrIDE jurisdictions successfully expanded PrEP services for MSM, transgender persons, and racial and ethnic minority groups by implementing health equity–focused activities that addressed barriers to PrEP services. However, PrEP prescription was generally low, with significant disparities by demographic characteristics. Additional targeted interventions are needed to expand PrEP services, achieve equity in PrEP use, and contribute to ending the HIV epidemic in the United States.

Keywords

Data to Care pre-exposure prophylaxis men who have sex with men transgender persons cross-jurisdiction monitoring and evaluation Project PrIDE

At the end of 2019, an estimated 1 189 700 persons aged ≥13 years were living with HIV infection in the United States.¹ Persons in sexual and gender minority groups, particularly gay, bisexual, and other men who have sex with men (MSM) and transgender persons, are disproportionately affected by HIV. Although MSM represent approximately 2% of the US population,² they accounted for 69% of new diagnoses of HIV in 2019.³ Similarly, transgender persons constitute about 0.4% of the US population,⁴ yet accounted for approximately 2% of new diagnoses of HIV in 2019.³ For MSM and transgender persons, HIV prevalence and health-related outcomes are worst among persons in racial and ethnic minority groups, particularly Black/African American and Hispanic/Latino persons.^5,6 Although the rate of new HIV diagnoses declined nationally during 2015-2019, subgroup analysis indicates stable or increasing trends among MSM and transgender persons aged 25-34.³

The disproportionate prevalence of HIV among MSM and transgender persons indicates the need for expanding high-impact HIV prevention strategies that focus on achieving health equity for these populations. Pre-exposure prophylaxis (PrEP)—the use of antiretroviral medications for prevention—is an effective biomedical intervention that reduces the risk of HIV infection among MSM and heterosexual men and women by more than 90% when used consistently.^7,8 A subgroup analysis from a clinical trial indicated that PrEP might also be effective in preventing HIV among transgender women when taken consistently.⁹

Expanding access to PrEP among populations at risk for HIV infection is one of the high-impact HIV prevention strategies identified by the Centers for Disease Control and Prevention (CDC)¹⁰ and the Ending the HIV Epidemic initiative in the United States.¹¹ CDC issued clinical practice guidelines to facilitate PrEP services in 2014 and 2017.⁷ Although the number of persons prescribed PrEP has recently increased,^12-14 US studies indicate only 24% to 35% of MSM^12,13 and 3% to 18% of transgender persons^15-17 with PrEP indications (ie, recent sexual or drug use behaviors that carry high risk for HIV infection) are using PrEP. Even lower levels of PrEP uptake are reported among racial and ethnic minority MSM and transgender persons.^18,19

Implementation of PrEP programs is conceptualized along a continuum. First, programs identify persons at risk for HIV infection, educate them about PrEP, and raise their willingness to take PrEP. Next, access to PrEP is facilitated through referral and linkage to PrEP providers. Finally, eligible persons are prescribed and initiate PrEP and, when possible, provided adherence and retention support.^20,21 Success in PrEP programs depends on addressing individual-level barriers (eg, limited PrEP knowledge), social barriers (eg, PrEP-related stigma), and structural barriers (eg, institutional racism, homophobia, transphobia) that may impede progress along the continuum.^22-25 Thus, efforts to expand PrEP services must be multipronged and include health equity–focused approaches that address these barriers.²⁶

Description of Project PrIDE

During 2015-2019, CDC funded 12 state and local health department jurisdictions (Baltimore, California, Chicago, Colorado, Houston, Los Angeles County, Louisiana, Michigan, New York City, San Francisco, Tennessee, and Virginia) to implement the demonstration project, PrEP Implementation, Data to Care, and Evaluation (PrIDE). Project PrIDE supported activities that strengthened jurisdictions’ ability to identify the project’s priority populations (ie, MSM and transgender persons at risk for HIV infection, particularly persons from racial and ethnic minority groups), refer them to PrEP providers, and increase the number of providers capable of offering PrEP to priority populations.²⁷ Project PrIDE also required the provision of PrEP-related support services, including referral and linkage to behavioral health, social, and other essential services.²⁷ Project PrIDE expected jurisdictions to incorporate into their work plans the following elements: health equity principles; program collaboration and service integration; data sharing across HIV, other sexually transmitted disease, and hepatitis programs; and sexual health principles. Project PrIDE developed a logic model that specifies the required activities and expected outcomes for PrEP programs (Figure 1).

Figure 1.

PrEP program logic model for Project PrIDE, 2015-2019. Abbreviations: MSM, gay, bisexual, or other men who have sex with men; PrEP, pre-exposure prophylaxis; PrIDE, Pre-exposure prophylaxis (PrEP) Implementation, Data to Care, and Evaluation; STD, sexually transmitted disease.

Purpose of Evaluation

The goals of the evaluation of Project PrIDE were to learn from the implementation of PrEP programs in funded jurisdictions and to share those lessons to inform future PrEP programs. The PrIDE evaluation team adopted a 2-pronged evaluation strategy to assess progress toward these goals: (1) cross-jurisdiction evaluation—a CDC-led activity to monitor program implementation processes and outcomes across the 12 jurisdictions and (2) local site-specific evaluation—a recipient-led evaluation whereby each jurisdiction conducted in-depth process or outcome evaluations of its PrEP activities with technical assistance from CDC. This article addresses 2 cross-jurisdiction evaluation questions: (1) What activities did PrIDE jurisdictions implement to expand PrEP services for priority populations? and (2) To what extent did uptake occur in screening for PrEP indications, referral and linkage to PrEP providers, and prescriptions for priority populations?

Methods

Data Sources and Performance Indicators

We used qualitative and quantitative data to answer the evaluation questions. We extracted qualitative data from the work plans and annual progress reports submitted to CDC. The work plans included project descriptions, implementation and evaluation timelines, and end-of-project performance targets. All jurisdictions, in consultation with CDC, set their own performance targets (eg, number of persons to be screened for PrEP) to monitor project progress. The annual progress reports included information on key activities implemented, accomplishments, challenges, and lessons learned during the reporting period.

PrIDE jurisdictions provided aggregate quantitative data (ie, summary counts at the jurisdiction level) on their efforts to expand PrEP services as part of annual progress reports. The key PrEP program performance indicators were the number of persons (1) screened for PrEP indications, (2) identified as eligible for referral to PrEP providers (ie, persons deemed at risk for HIV infection based on CDC’s eligibility criteria⁷ or locally defined criteria), (3) referred to PrEP providers, (4) linked to PrEP providers (ie, attended an appointment with a provider), and (5) prescribed PrEP. Jurisdictions’ approaches and ability to track persons referred to PrEP providers varied; for example, some jurisdictions used PrEP navigators and others used data-sharing agreements with providers to collect linkage and prescription information. Some jurisdictions were unable to track post-referral outcomes, particularly among persons referred to external PrEP providers.

We stratified the PrEP program performance indicators by age (13-19, 20-29, 30-39, and ≥40), race and ethnicity (Hispanic/Latino, non-Hispanic Black/African American, non-Hispanic White, and non-Hispanic other races, including American Indian/Alaska Native, Asian, Native Hawaiian/Other Pacific Islander, and persons with multiple races), assigned sex at birth (female and male), and key population categories (MSM, transgender persons, heterosexual persons, and “others,” which included persons with other HIV transmission risk [eg, hemophilia, blood transfusion] and persons for whom a specific risk was not identified). Data collection for this project constituted a routine program monitoring activity; therefore, CDC determined that institutional review board approval was not required.

Data Analysis

First, we used a simplified version of a qualitative description method^28,29 to identify and summarize PrEP-related activities implemented during the project. Our approach involved the following sequential steps: (1) two coauthors (M.M., S.B.) reviewed the work plans and identified key PrEP-related activities and performance targets, (2) four other coauthors (A.K., C.G., J.C., W.D.) who were also project officers reviewed the initial lists of planned activities and confirmed which activities were implemented in their respective jurisdictions, and (3) all coauthors reviewed and condensed specific activities into categories.

We conducted quality assurance on all aggregate quantitative data and determined that 4 jurisdictions did not report data on 1 or more of the PrEP program performance indicators:

Two jurisdictions did not conduct their own PrEP screening, relying instead on existing prevention or clinical data to identify persons eligible for referral to PrEP providers,

Three jurisdictions reported difficulty tracking referral information for persons prescribed PrEP by internal providers, and

Two jurisdictions did not track linkage and prescription outcomes for persons referred to external PrEP providers.

To conduct analyses, we operationalized several decision rules. If a jurisdiction reported data on a subsequent step in the PrEP continuum but was unable to report on a preceding step, we assumed at least the same number of persons were reached on both steps. For example, for jurisdictions that did not report screening but identified persons eligible for referral to PrEP providers, we assumed that all eligible persons were screened. For jurisdictions that did not report data on linkage to PrEP providers, we assumed that all persons prescribed PrEP were linked to PrEP providers. These assumptions are likely to underestimate the number of persons screened, identified as eligible for referral, and referred to PrEP providers. We made no adjustment for missing data on persons missing post-referral follow-up outcomes.

After quality assurance adjustments, we conducted a descriptive analysis on key PrEP program performance indicators by demographic characteristics. For some PrEP program outcomes, we used z tests to examine group differences between 2 proportions, where significance was ascertained at P < .05. We used SPSS statistical package version 21 (IBM Corp) to conduct analyses.

Results

Implemented PrEP Program Activities

Qualitative data analysis revealed that PrIDE jurisdictions implemented multiple health equity–focused activities to expand PrEP services to priority populations. The 6 categories of activities implemented were (1) engaging communities to inform PrEP program priorities and appropriateness or get buy-in; (2) developing organizational capacity for PrEP at the health department and partner agencies; (3) improving providers’ awareness, knowledge, and willingness to prescribe PrEP; (4) improving priority populations’ awareness, knowledge, and willingness to take PrEP; (5) providing PrEP and support services; and (6) conducting PrEP program monitoring and evaluation (Table 1). Program structure and implementation varied by jurisdiction, but all jurisdictions conducted at least 1 activity in each category.

Table 1.

Activities implemented to initiate and expand PrEP services in 12 Project PrIDE health department jurisdictions, 2015-2019^a

PrEP program domains and implemented activities/strategies	Examples of implemented activities	No. of jurisdictions that implemented activities (N = 12)
Engage communities to inform PrEP program priorities and appropriateness or get buy-in
Consult with MSM and transgender community members	Identify and engage members of priority population groups to get their input in the planning and implementation of the project	12/12
Conduct social innovation events with priority populations	Host house ball, photovoice, or storytelling events that facilitate the exchange of information about PrEP among peers	11/12
Create/engage community advisory board	Identify and engage influential community members to serve in a council that provides guidance for the project	10/12
Engage existing HIV planning groups or councils	Involve existing HIV prevention planning council or Ryan White planning council to support PrEP programs	5/12
Develop organizational capacity for PrEP at the health department and/or partner agencies
Hire or train staff to work on PrEP support programs	Hire health department staff to conduct PrEP navigation; provide training to partner agency staff on PrEP or delivery of PrEP services	12/12
Develop PrEP-related policies, procedures, or protocols	Develop PrEP linkage protocols for navigators or clinical guidelines for PrEP providers in the jurisdiction	12/12
Enhance, integrate, and use data for PrEP	Improve data systems by adding PrEP data elements or features that combine HIV and other STI prevention program data with clinical care data	10/12
Develop PrEP provider directories or referral systems	Develop or enhance an online database of PrEP providers such as PleasePrEPme.org (www.pleaseprepme.org/online-providers) that provides contact information, location, and other information about the providers	10/12
Develop PrEP-related toolkits/best practices	Develop and distribute a PrEP provider toolkit that includes information about how to take a sexual history, billing codes, and answers to most frequently asked questions by providers and consumers	11/12
Collaborate with community-based organizations (CBOs) or AIDS service organizations (ASOs)	Engage in contractual or noncontractual partnerships with CBOs/ASOs to expand PrEP, behavioral health, and/or social services	12/12
Collaborate with clinical care providers	Engage in contractual or noncontractual partnerships with sexual health or other health care providers to deliver PrEP care	12/12
Collaborate with local health departments	Fund local and county health departments to expand PrEP awareness, knowledge, or services	7/12
Collaborate with AIDS Education and Training Centers (AETCs)	Partner with the regional AETC to provide PrEP-related capacity-building technical assistance or training to project staff members	7/12
Collaborate with academic partners	Partner with universities to assist with program planning or to expand PrEP awareness, knowledge, or services	4/12
Improve PrEP providers’ awareness, knowledge, willingness, or practice
Provide training/education/resources on PrEP or implementation of PrEP	Provide workshops designed to enhance providers’ knowledge about efficacy of PrEP, prescribing guidelines, or improve their skills to conduct comprehensive sexual history	12/12
Provide training/education/resources on cultural competency	Provide trainings designed to enhance providers’ understanding and sensitivity to the needs of gender, sexual, and racial and ethnic minority communities	8/12
Provide training/education/resources on billing/financial assistance for PrEP	Provide information about existing PrEP medication assistance programs or billing options to cover the costs of providing PrEP services	7/12
Conduct social media/marketing campaign	Promote the efficacy of PrEP among providers through placement of messages on traditional media such as television and local newspapers and on social media platforms including Twitter, Facebook, or Grindr	9/12
Conduct public health or academic detailing	Engage PrEP outreach specialists to reach clinical providers such as infectious disease specialists with information to encourage them to offer PrEP services	9/12
Provide PrEP-related technical assistance	Provide a one-to-one expert consultation on prescribing and managing PrEP to providers with limited PrEP experience	9/12
Provide PrEP-related materials	Develop and share with providers PrEP-related brochures, palm cards, or pamphlets not included in public health or academic detailing or training/educational events	12/12
Improve priority populations’ awareness, knowledge, and willingness to use PrEP
Host/participate in outreach events	Community events that encourage open and nonjudgmental discussions about personal experiences with navigating the health care system, using PrEP, and addressing stigma about PrEP	12/12
Provide information/education on PrEP as prevention strategy	Engage PrEP navigators to educate priority populations about HIV risk and transmission, the benefits of using PrEP, and dispelling myths about PrEP	12/12
Provide information/education on where or how to get PrEP	Develop and launch an online database with information about PrEP providers and how priority populations can access PrEP services in the jurisdiction	12/12
Provide information/education on how to pay for PrEP	Share information with clients about existing PrEP assistance programs to cover the costs of PrEP-related services	10/12
Conduct social media/marketing campaign	Conduct a multimedia and multilanguage campaign featuring a diverse case of local spokespersons with lived experience of taking PrEP; audio, video, and print messages appearing on television, internet, or social media platforms	11/12
Provide PrEP and relevant support services
Provide PrEP services	Set up new or expand existing PrEP programs to screen, refer, link, and/or prescribe PrEP to priority populations	12/12
Provide behavioral health services	Screen, refer, navigate, or link PrEP clients with unmet substance abuse or mental health service needs to providers	10/12
Provide financial assistance/support	Screen, refer, navigate, or link PrEP clients without health insurance to PrEP financial assistance programs	9/12
Provide social services	Screen, refer, navigate, or link PrEP clients with unmet housing, transportation, or other social service needs to providers	9/12
Provide PrEP adherence support	Engage PrEP clinical coordinators or navigators to support adherence by following up on appointments and prescription pickup, identifying potential barriers to PrEP use, and providing peer and navigation support	6/12
Provide other PrEP-related services	Expand screening for HIV and other STIs; refer, navigate, or link PrEP clients with HIV/STI to care and treatment	9/12
Conduct PrEP program M&E and use and disseminate findings
Conduct activities to meet CDC’s cross-jurisdictional M&E expectations	Implement CDC’s M&E requirements, including data collection, data quality assurance and analysis, and data reporting to CDC, implementation partners, and other collaborators	12/12
Conduct PrEP-related local and site-specific evaluations, including cluster evaluations	Conduct a locally specified process or outcome evaluation on a PrEP activity, including data collection, data quality assurance and analysis, and data reporting to implementation partners, CDC, and other collaborators	12/12
Use M&E findings to inform partners and collaborators	Present key findings to community groups, health department staff, or federal funders	10/12
Use M&E findings to inform program improvement	Share performance feedback with implementation partners	10/12
Share findings widely	Present project findings at conferences or through reports and scientific publications	10/12
Collaborate with external (non-CDC) evaluator(s) for M&E activities	Partner with universities or other academic institutions to assist or independently conduct M&E activities	4/12

Abbreviations: CDC, Centers for Disease Control and Prevention; M&E, monitoring and evaluation; MSM, gay, bisexual, or other men who have sex with men; PrEP, pre-exposure prophylaxis; PrIDE, Pre-exposure prophylaxis (PrEP) Implementation, Data to Care, and Evaluation; STI, sexually transmitted infection.

The executive summaries of all 12 PrIDE jurisdictions’ PrEP project activities are available elsewhere.³⁰

Screening for PrEP Indications and Referral to PrEP Providers

PrIDE jurisdictions screened at least 69 438 persons for PrEP indications (Table 2), 31.8% more than the number of persons targeted to be reached by the end of the project (n = 52 689). Of those screened, 44 813 (64.5%) had PrEP indications and were eligible for referral to providers, and 33 500 (74.5%) were referred to providers.

Table 2.

Number of persons screened for PrEP, referred and linked to PrEP providers, and prescribed PrEP, by demographic characteristics, 12 Project PrIDE health department jurisdictions, 2015-2019

Demographic characteristic	Screened for PrEP,^ano. (%)	Eligible for PrEP referral,^ano. (%)	Referred to PrEP provider,^ano. (%)	Linked to PrEP provider,^bno. (%)	Prescribed PrEP,^bno. (%)
Age, y
13-19	4276 (6.2)	2377 (5.3)	1416 (4.2)	426 (2.9)	326 (2.9)
20-29	30 144 (43.4)	20 525 (45.8)	15 105 (45.1)	6822 (46.0)	5177 (45.6)
30-39	18 436 (26.6)	12 090 (27.0)	9302 (27.8)	4610 (31.1)	3572 (31.5)
≥40	16 062 (23.1)	9544 (21.3)	7463 (22.3)	2891 (19.5)	2224 (19.6)
Missing	520 (0.7)	277 (0.6)	214 (0.6)	72 (0.5)	57 (0.5)
Sex
Female	16 839 (24.3)	8033 (17.9)	6026 (18.0)	1486 (10.0)	619 (5.5)
Male	51 877 (74.7)	36 390 (81.2)	27 140 (81.0)	13 223 (89.2)	10 647 (93.8)
Missing	722 (1.0)	390 (0.9)	334 (1.0)	112 (0.8)	90 (0.8)
Race and ethnicity
Non-Hispanic White	15 792 (22.7)	11 633 (26.0)	8328 (24.9)	4250 (28.7)	3609 (31.8)
Non-Hispanic Black/African American	33 464 (48.2)	18 782 (41.9)	13 221 (39.5)	4277 (28.9)	2794 (24.6)
Hispanic/Latino (all races)	15 111 (21.8)	10 841 (24.2)	9171 (27.4)	4817 (32.5)	3721 (32.8)
Non-Hispanic other^c	3220 (4.6)	2316 (5.2)	1819 (5.4)	1115 (7.5)	934 (8.2)
Missing	1851 (2.7)	1241 (2.8)	961 (2.9)	362 (2.4)	298 (2.6)
Key population group
Heterosexual	25 291 (36.4)	10 853 (24.2)	8433 (25.2)	2068 (14.0)	801 (7.1)
MSM	34 090 (49.1)	27 158 (60.6)	20 232 (60.4)	11 137 (75.1)	9386 (82.7)
Transgender	2563 (3.7)	2097 (4.7)	1423 (4.2)	674 (4.5)	596 (5.2)
Other^d	563 (0.8)	504 (1.1)	240 (0.7)	139 (0.9)	73 (0.6)
Missing	6931 (10.0)	4201 (9.4)	3172 (9.5)	803 (5.4)	500 (4.4)
Total	69 438 (100.0)	44 813 (100.0)	33 500 (100.0)	14 821 (100.0)	11 356 (100.0)

Abbreviations: MSM, gay, bisexual, or other men who have sex with men; PrEP, pre-exposure prophylaxis; PrIDE, Pre-exposure prophylaxis (PrEP) Implementation, Data to Care, and Evaluation.

Level of data reporting varied by jurisdiction. Two jurisdictions did not conduct new screening for PrEP indications. Instead, they used existing clinical or prevention program data to identify persons who had indications for PrEP. As a result, the number of persons screened for PrEP in these jurisdictions was assumed to be the same as the number of persons identified as eligible for referral to PrEP providers. Three jurisdictions reported PrEP prescription data but were unable to accurately track the earlier steps of referral or linkage to providers; for those jurisdictions, the number of persons referred and linked to PrEP providers was assumed to be the same as the number of persons prescribed PrEP.

Linkage to PrEP providers and PrEP prescription was reported by all 12 health departments for persons referred to internal or external providers, or both. However, health departments reported challenges in obtaining linkage and/or prescription data for persons referred to external providers.

Includes non-Hispanic American Indian/Alaska Native, Asian, Native Hawaiian/Other Pacific Islander, and persons of multiple races.

Persons with other HIV transmission risk (eg, hemophilia, blood transfusion) or persons with no known risk for HIV infection.

The 2 priority populations accounted for 52.8% of those screened (MSM, 49.1%; transgender persons, 3.7%), 65.3% of those identified as eligible for referral (MSM, 60.6%; transgender persons, 4.7%), and 64.6% of those referred to providers (MSM, 60.4%; transgender persons, 4.2%). Persons from racial and ethnic minority groups accounted for more than two-thirds of all persons screened (74.6%), identified as eligible for referral (71.2%), and referred to providers (72.2%). Three-fourths or more of those screened, identified as eligible for referral, and referred to providers were males. Persons aged <30 years represented approximately half of those screened, identified as eligible for referral, and referred to providers.

The proportion referred to PrEP providers was lower among MSM (74.5%; z = −6.55; P < .001) and transgender persons (67.9%; z = −9.68; P < .001) than among heterosexual persons (77.7%). Hispanic/Latino persons (84.6%; z = 23.47; P < .001) and non-Hispanic/Latino persons of other races (78.5%; z = 6.86; P < .001) were more likely to be referred than non-Hispanic White persons (71.6%). Referral was lower among persons aged 13-19 years (59.6%; z = −14.40; P < .001) and higher among persons aged 30-39 (76.9%; z = 6.73; P < .001) and ≥40 (78.2%; z = 8.59; P < .001) compared with persons aged 20-29 (73.6%) (Table 3).

Table 3.

Differences in PrEP referral, linkage, and prescription among persons eligible for PrEP referral, by demographic characteristics, 12 Project PrIDE health department jurisdictions, 2015-2019

Demographic characteristic	Eligible for PrEP referral,^a no.	Of persons eligible for PrEP referral
		Referred to PrEP provider^a		Linked to PrEP provider^b		Prescribed PrEP
		No. (%)	z test (P value)^c	No. (%)	z test (P value)^c	No. (%)	z test (P value)^c
Age, y
13-19	2377	1416 (59.6)	−14.40 (<.001)	426 (17.9)	−15.20 (<.001)	326 (13.7)	−19.44 (<.001)
20-29	20 525	15 105 (73.6)	Reference	6822 (33.2)	Reference	5177 (25.2)	Reference
30-39	12 090	9302 (76.9)	6.73 (<.001)	4610 (38.1)	8.95 (<.001)	3572 (29.5)	8.51 (<.001)
≥40	9544	7463 (78.2)	8.59 (<.001)	2891 (30.3)	−5.09 (<.001)	2224 (23.3)	−3.60 (.001)
Missing	277	214 (77.3)	1.37 (.31)	72 (26.0)	−2.54 (.03)	57 (20.6)	−1.77 (.17)
Sex
Female	8033	6026 (75.0)	Reference	1486 (18.5)	Reference	619 (7.7)	Reference
Male	36 390	27 140 (74.6)	−0.81 (.57)	13 223 (36.3)	30.75 (<.001)	10 647 (29.3)	40.18 (<.001)
Missing	390	334 (85.6)	4.77 (<.001)	112 (28.7)	5.03 (<.001)	90 (23.1)	10.68 (<.001)
Race/ethnicity
Non-Hispanic White	11 633	8328 (71.6)	Reference	4250 (36.5)	Reference	3609 (31.0)	Reference
Non-Hispanic Black/African American	18 782	13 221 (70.4)	−2.23 (.07)	4277 (22.8)	−25.97 (<.001)	2794 (14.9)	−33.57 (<.001)
Hispanic/Latino (any race)	10 841	9171 (84.6)	23.47 (<.001)	4817 (44.4)	12.06 (<.001)	3721 (34.3)	5.27 (<.001)
Non-Hispanic other^d	2316	1819 (78.5)	6.86 (<.001)	1115 (48.1)	11.65 (<.001)	934 (40.3)	8.73 (<.001)
Missing	1241	961 (77.4)	4.37 (.001)	362 (29.2)	−5.14 (<.001)	298 (24.0)	−5.11 (<.001)
Key population group
Heterosexual	10 853	8433 (77.7)	Reference	2068 (19.1)	Reference	801 (7.4)	Reference
MSM	27 158	20 232 (74.5)	−6.55 (<.001)	11 137 (41.0)	40.60 (<.001)	9386 (34.6)	54.04 (<.001)
Transgender	2097	1423 (67.9)	−9.68 (<.001)	674 (32.1)	13.43 (<.001)	596 (28.4)	28.43 (<.001)
Other^e	504	240 (47.6)	−15.54 (<.001)	139 (27.6)	4.73 (<.001)	73 (14.5)	5.85 (<.001)
Missing	4201	3172 (75.5)	−2.88 (.01)	803 (19.1)	0.08 (.80)	500 (11.9)	8.86 (<.001)
Total	44 813	33 500 (74.8)	—	14 821 (33.1)	—	11 356 (25.3)	—

Abbreviations: MSM, gay, bisexual, or other men who have sex with men; PrEP, pre-exposure prophylaxis; PrIDE, Pre-exposure prophylaxis (PrEP) Implementation, Data to Care, and Evaluation.

P < .05 considered significant.

Includes non-Hispanic American Indian/Alaska Native, Asian, Native Hawaiian/Other Pacific Islander, and persons of multiple races.

Persons with other HIV transmission risk (eg, hemophilia, blood transfusion) or persons with no known risk for HIV infection.

Linkage to PrEP Providers and Prescription of PrEP

PrIDE jurisdictions linked 14 821 persons to providers, 29.1% more than their end-of-project target (n = 11 481). Of those with PrEP indications, only 33.1% (14 821 of 44 813) were linked to providers. Nearly 80% of those linked were MSM (75.1%) or transgender persons (4.5%). About two-thirds of persons linked to providers were non-Hispanic Black/African American (39.5%) or Hispanic/Latino (27.4%) persons. Males constituted 89.2% and persons aged <30 years represented 48.9% of all persons linked to PrEP providers (Table 2).

Linkage to PrEP providers was higher among MSM (41.0%; z = 40.60; P < .001) and transgender persons (32.1%; z = 13.43; P < .001) than among heterosexual persons (19.1%). Non-Hispanic Black/African American persons (22.8%; z = −25.97; P < .001) were less likely and Hispanic/Latino persons (44.4%; z = 12.06; P < .001) and non-Hispanic persons of other races (48.1%; z = 11.65; P < .001) were more likely to be linked than non-Hispanic White persons (36.5%). Linkage was lower among those aged 13-19 years (17.9%; z = −15.20; P < .001) and ≥40 (30.3%; z = −5.09; P < .001) and higher among those aged 30-39 (38.1%; z = 8.95; P < .001) than among those aged 20-29 (33.2%). Males (36.3%) were also more likely to be linked to providers than females (18.5%; z = 30.75; P < .001) (Table 3).

Overall, PrIDE jurisdictions reported that 11 356 persons were prescribed PrEP during the project period, which was 8.3% more than their end-of-project target (n = 10 483). The percentage of persons with PrEP indications who received a prescription was 25.3% (11 356 of 44 813) (Figure 2). Nearly 88% of persons prescribed were MSM (82.7%) or transgender persons (5.2%). More than half of persons prescribed were non-Hispanic Black/African American (24.6%) or Hispanic/Latino (32.8%) persons. Males constituted 93.8% and persons aged <30 years represented 48.5% of all persons prescribed PrEP (Table 2).

Figure 2.

Number and percentage of persons screened as eligible for referral at subsequent steps in the pre-exposure prophylaxis (PrEP) continuum in 12 Project PrIDE health department jurisdictions, 2015-2019. Level of data reporting varied by jurisdiction. Two jurisdictions did not conduct new screening for PrEP indications. Instead, they used existing clinical or prevention program data to identify persons who have indications for PrEP. As a result, the number of persons screened for PrEP in these jurisdictions was assumed to be the same as the number of those they identified as eligible for referral to PrEP providers. Three jurisdictions reported PrEP prescription data but were unable to accurately track the earlier steps of referral or linkage to providers. For those jurisdictions, the number of persons referred and linked to PrEP providers was assumed to be the same as the number of persons prescribed PrEP. Linkage to PrEP providers and PrEP prescription were reported by all 12 health departments for persons referred to internal or external providers or both. However, health departments reported challenges in obtaining linkage and/or prescription data for persons referred to external providers. Abbreviation: PrIDE, Pre-exposure prophylaxis Implementation, Data to Care, and Evaluation.

PrEP prescription was higher among MSM (34.6%; z = 54.04; P < .001) and transgender persons (28.4%; z = 28.43; P < .001) than among heterosexual persons (7.4%). Non-Hispanic Black/African American persons (14.9%; z = −33.57; P < .001) were less likely and Hispanic/Latino persons (34.3%; z = 5.27; P < .001) and non-Hispanic persons of other races (40.3%; z = 8.73; P < .001) were more likely to receive prescriptions than non-Hispanic White persons (31.0%). PrEP prescription was lower among those aged 13-19 years (13.7%; z = −19.44; P < .001) and ≥40 (23.3%; z = −3.60; P = .001) and higher among those aged 30-39 (29.5%; z = 8.51; P < .001) compared with those aged 20-29 (25.2%). Males (29.3%) were also more likely to receive prescriptions than females (7.7%; z = 40.18; P < .001) (Table 3).

Lessons Learned

PrIDE jurisdictions implemented various activities to build capacity, expand PrEP and support services, and monitor program outcomes. In doing so, jurisdictions used health equity–focused strategies to address barriers salient within their jurisdictions (eg, trainings to improve providers’ cultural competence, social marketing campaigns to improve PrEP awareness or reduce PrEP- or HIV-related stigma, essential support services to meet other needs). Although the outcomes of each activity were not independently evaluated, the project’s overall success is reflected in the magnitude of PrEP services provided to potential candidates, particularly to populations prioritized for Project PrIDE.

During the project (2015-2019), PrIDE jurisdictions reached nearly 70 000 persons, of whom 11 356 were eventually prescribed PrEP, exceeding project targets set by the jurisdictions. Analysis of pharmacy-based data in counties that constituted PrIDE jurisdictions indicated increases in the number of persons using PrEP from 25 124 in 2015, at the beginning of Project PrIDE, to 72 111 in 2019.³¹ Although PrEP use has generally increased in the United States,^12-14 it is likely that Project PrIDE may have accounted for up to 24% of the increase in PrEP uptake (11 356/[72 111 – 25 124]), assuming that all PrEP-prescribed PrIDE clients were retained in PrEP care in 2019.

PrIDE jurisdictions successfully targeted PrEP services to the priority populations. MSM and transgender persons accounted for the largest proportion of PrEP clients at each step in the PrEP continuum: 64.6% of those referred, 79.6% of those linked to PrEP providers, and 87.9% of those prescribed PrEP. MSM and transgender persons were less likely than heterosexual persons to be referred but more likely to be linked to providers and prescribed PrEP. The lower level of referral for MSM and transgender persons is consistent with studies that documented individual, social, and structural barriers to PrEP services among sexual and gender minority persons.^18,19 Contrarily, better linkage and prescription outcomes indicate the success of PrIDE jurisdictions’ efforts at delivering appropriate PrEP services for priority populations. In this regard, activities implemented by PrIDE jurisdictions that targeted PrEP-related awareness and knowledge among priority populations, cultural competence among providers, and stigma reduction at the community level may have contributed to better linkage and prescription outcomes.

Despite the expansion of PrEP services, only 33% of those with PrEP indications were linked to providers and only 25% of them were prescribed PrEP. These findings are consistent with findings of another study showing similar sizable drops in percentages of persons referred to PrEP providers who were linked (31%) and prescribed (29%).³² In contrast, another project demonstrated better linkage (60%) and prescription (43%) than ours among clients with PrEP indications.³³ Low linkage and prescription levels suggest that PrEP programs need to more effectively address barriers that prevent persons referred from getting to providers (eg, perceived stigma) and persons linked to providers from obtaining their prescriptions (eg, provider hesitation). Data reporting challenges may also account for the low levels of linkage and prescription. Health departments often partner with clinical providers for PrEP prescription and care. In this collaborative relationship, health departments may not have full or timely access to linkage or prescription data because of reporting delays or concerns with sharing confidential information.

Non-Hispanic Black/African American and Hispanic/Latino persons represented more than half of all PrIDE clients referred and linked to providers and subsequently prescribed PrEP. Although this finding shows that PrIDE programs have successfully engaged underserved population groups, non-Hispanic Black/African American persons had a noticeably lower level of PrEP prescription than non-Hispanic White persons (14.8% vs 31.0%). The lower level of PrEP prescription among Black/African American persons is consistent with the literature.^18,19 Studies indicate pervasive individual, social, and structural barriers to PrEP uptake among Black/African American persons, including fear of potential side effects, HIV and PrEP stigma, medical mistrust, and perceived racism in health care settings.^34-37 Although PrIDE jurisdictions implemented various strategies to facilitate PrEP uptake (eg, social media campaigns to address PrEP stigma, provider training to improve cultural competence), additional research is needed to identify and expand strategies that are particularly effective among Black/African American MSM and transgender persons. Interventions that offer PrEP services in accessible community settings (eg, local community-based organizations), provide appropriate support services (eg, PrEP navigation by peers), and foster a positive health care experience (eg, culturally competent and streamlined services) may help improve PrEP uptake among Black/African American persons.^34,36,37

Limitations

Our evaluation approach had several limitations. First, the qualitative analysis focused on the “what” of the activities implemented and may have missed other important dimension of PrEP programs (eg, the “how well”). However, local site-specific evaluation efforts described elsewhere³⁵ provide detailed contexts of the programs, complement the cross-jurisdictional findings, and enhance the lessons learned from Project PrIDE. Second, data reporting varied by jurisdiction, partly because of differences in program structure, program maturity, and evaluation capacity. As noted in the Methods section, we made adjustments that replaced missing data on a given step of the PrEP continuum with available data from a subsequent step. These assumptions may have underestimated the number of persons served in preceding steps or overestimated the percentages in subsequent steps. Third, some jurisdictions indicated difficulty in reporting linkage and prescription data from external partners that provided PrEP services. Therefore, data reported are likely to underestimate these outcomes. Lastly, our cross-jurisdiction evaluation design did not enable us to collect pre- and post-intervention data or include comparison sites to assess the relative effectiveness of the project, or individual strategies, compared with other interventions.

Conclusion

Although data presented in this evaluation were not based on research from which one can make causal inferences, several broad programmatic lessons learned can inform similar health department–initiated PrEP programs. A primary lesson of our analyses is the feasibility of expanding PrEP services for priority populations by adopting a health-equity focus in planning and implementing programs that address common individual, social, and structural barriers (eg, PrEP-related stigma, institutional racism, homelessness). Another lesson is that PrEP programs are complex and require health departments to invest in capacity building (eg, hire and train PrEP navigators, enhance data integration) and collaborations with multiple partners (eg, community-based organizations, local health departments, health care providers). Key strategies that were commonly implemented by PrIDE jurisdictions (eg, expanding HIV testing and PrEP navigation) and delivery of essential support services (eg, behavioral health and social services) are impossible without collaboration and leveraging of resources. Furthermore, despite reaching the intended priority populations, our analyses indicate the need for additional efforts to increase PrEP uptake and reduce persistent disparities in PrEP use. However, these analyses do not answer the question of what specific efforts are associated with increases in PrEP uptake. Detailed program-specific lessons from Project PrIDE are included in articles published in a special issue of Evaluation and Program Planning.^38-50

Our findings show that PrIDE jurisdictions successfully demonstrated the feasibility of health department–supported programs in expanding PrEP services to MSM and transgender persons at risk for HIV infection. PrIDE jurisdictions exceeded their own targets in the number of persons reached to identify PrEP service needs, referred, and linked to providers, and eventually prescribed PrEP. PrIDE jurisdictions effectively prioritized their services to MSM, transgender persons, and persons in racial and ethnic minority groups. These successes in reach and engagement of underserved populations may be attributed to the range of health equity–focused activities conducted to address individual, social, and structural barriers to PrEP use. However, the proportions of persons with PrEP indications linked to providers and prescribed PrEP were generally low, indicating continued challenges to sustained engagement of persons in PrEP services. More specifically, the lower levels of linkage and PrEP prescription among Black/African American persons than among other racial and ethnic groups are concerning and suggest the need for additional targeted interventions that address unique barriers to engagement in PrEP services among Black/African American persons.

Footnotes

Acknowledgements

The authors acknowledge Project PrIDE jurisdictions and their collaborating partners for their demonstrated commitment in implementing project activities and collecting and reporting program evaluation data. The authors also thank the following branches in the Division of HIV/AIDS Prevention at CDC for their contribution to the implementation, monitoring, and evaluation of the project: Prevention Research Branch, Program Evaluation Branch, Epidemiology Branch, Prevention Programs Branch, and Capacity Building Branch. The authors thank Arin Freeman, MPH; Adrienne R. Herron, PhD; Tamika Hoyte, MPH; Mary Neumann, PhD; Yamir Salabarría-Peña, DrPH, MPHE; Pilgrim Spikes, PhD; and Mikel Walters, PhD, for their contributions to the development and management of the project. Finally, the authors thank Shubha Rao, MBBS, MPH; Janet Heitgerd, PhD; Lisa Belcher, PhD; Linda Koenig, PhD; and Yuko Mizuno, PhD, for providing feedback on an earlier version of the article.

Authors’ Note

The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of CDC/Agency for Toxic Substances and Disease Registry.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Mesfin S. Mulatu, PhD, MPH

Carla A. Galindo, MPH

References

Centers for Disease Control and Prevention. Estimated HIV incidence and prevalence in the United States, 2015-2019. HIV Surveill Suppl Rep. 2021;26(1):1-81. Accessed July 10, 2021. https://www.cdc.gov/hiv/pdf/library/reports/surveillance/cdc-hiv-surveillance-supplemental-report-vol-26-1.pdf

Purcell

Johnson

Lansky

, et al. Estimating the population size of men who have sex with men in the United States to obtain HIV and syphilis rates. Open AIDS J. 2012;6:98-107. doi:10.2174/1874613601206010098

Centers for Disease Control and Prevention. Diagnoses of HIV infection in the United States and dependent areas, 2019. HIV Surveill Rep. 2021;32:1-123. Accessed July 10, 2021. https://www.cdc.gov/hiv/pdf/library/reports/surveillance/cdc-hiv-surveillance-report-2018-updated-vol-32.pdf

Meerwijk

Sevelius

. Transgender population size in the United States: a meta-regression of population-based probability samples. Am J Public Health. 2017;107(2):e1-e8. doi:10.2105/ajph.2016.303578

Becasen

Denard

Mullins

Higa

Sipe

. Estimating the prevalence of HIV and sexual behaviors among the US transgender population: a systematic review and meta-analysis, 2006-2017. Am J Public Health. 2019;109(1):e1-e8. doi:10.2105/ajph.2018.304727

Rosenberg

Purcell

Grey

Hankin-Wei

Hall

Sullivan

. Rates of prevalent and new HIV diagnoses by race and ethnicity among men who have sex with men, U.S. states, 2013-2014. Ann Epidemiol. 2018;28(12):865-873. doi:10.1016/j.annepidem.2018.04.008

Centers for Disease Control and Prevention. US Public Health Service: preexposure prophylaxis for the prevention of HIV infection in the United States—2017 update: a clinical practice guideline. March 2018. Accessed July 10, 2021. https://www.cdc.gov/hiv/pdf/risk/prep/cdc-hiv-prep-guidelines-2017.pdf

Chou

Evans

Hoverman

, et al. Preexposure prophylaxis for the prevention of HIV infection: evidence report and systematic review for the US Preventive Services Task Force. JAMA. 2019;321(22):2214-2230. doi:10.1001/jama.2019.2591

Deutsch

Glidden

Sevelius

, et al. HIV pre-exposure prophylaxis in transgender women: a subgroup analysis of the iPrEx trial. Lancet HIV. 2015;2(12):e512-e519. doi:10.1016/s2352-3018(15)00206-4

10.

White House Office of National AIDS Policy. The National HIV/AIDS Strategy for the United States: Updated to 2020. July 2015. Accessed July 10, 2021. https://files.hiv.gov/s3fs-public/nhas-update.pdf

11.

Fauci

Redfield

Sigounas

Weahkee

Giroir

. Ending the HIV epidemic: a plan for the United States. JAMA. 2019;321(9):844-845. doi:10.1001/jama.2019.1343

12.

Finlayson

Cha

Xia

, et al. Changes in HIV preexposure prophylaxis awareness and use among men who have sex with men—20 urban areas, 2014 and 2017. MMWR Morb Mortal Wkly Rep. 2019;68(27):597-603. doi:10.15585/mmwr.mm6827a1

13.

Kamitani

Johnson

Wichser

Adegbite

Mullins

Sipe

. Growth in proportion and disparities of HIV PrEP use among key populations identified in the United States national goals: systematic review and meta-analysis of published surveys. J Acquir Immune Defic Syndr. 2020;84(4):379-386. doi:10.1097/qai.0000000000002345

14.

Sullivan

Giler

Mouhanna

, et al. Trends in the use of oral emtricitabine/tenofovir disoproxil fumarate for pre-exposure prophylaxis against HIV infection, United States, 2012-2017. Ann Epidemiol. 2018;28(12):833-840. doi:10.1016/j.annepidem.2018.06.009

15.

Eaton

Matthews

Driffin

Bukowski

Wilson

Stall

. A multi-US city assessment of awareness and uptake of pre-exposure prophylaxis (PrEP) for HIV prevention among Black men and transgender women who have sex with men. Prev Sci. 2017;18(5):505-516. doi:10.1007/s11121-017-0756-6

16.

Poteat

Wirtz

Malik

, et al. A gap between willingness and uptake: findings from mixed methods research on HIV prevention among Black and Latina transgender women. J Acquir Immune Defic Syndr. 2019;82(2):131-140. doi:10.1097/qai.0000000000002112

17.

Sevelius

Poteat

Luhur

Reisner

Meyer

. HIV testing and PrEP use in a national probability sample of sexually active transgender people in the United States. J Acquir Immune Defic Syndr. 2020;84(5):437-442. doi:10.1097/qai.0000000000002403

18.

Holder

Perez-Gilbe

Fajardo

Garcia

Cyrus

. Disparities of HIV risk and PrEP use among transgender women of color in South Florida. J Natl Med Assoc. 2019; 111(6):625-632. doi:10.1016/j.jnma.2019.08.001

19.

Kanny

Jeffries

IV Chapin-Bardales

, et al. Racial and ethnic disparities in HIV preexposure prophylaxis among men who have sex with men—23 urban areas, 2017. MMWR Morb Mortal Wkly Rep. 2019;68(37):801-806. doi:10.15585/mmwr.mm6837a2

20.

Kelley

Kahle

Siegler

, et al. Applying a PrEP continuum of care for men who have sex with men in Atlanta, Georgia. Clin Infect Dis. 2015;61(10):1590-1597. doi:10.1093/cid/civ664

21.

Nunn

Brinkley-Rubinstein

Oldenburg

, et al. Defining the HIV pre-exposure prophylaxis care continuum. AIDS. 2017;31(5):731-734. doi:10.1097/qad.0000000000001385

22.

Mayer

Agwu

Malebranche

. Barriers to the wider use of pre-exposure prophylaxis in the United States: a narrative review. Adv Ther. 2020;37(5):1778-1811. doi:10.1007/s12325-020-01295-0

23.

Petroll

Walsh

Owczarzak

McAuliffe

Bogart

Kelly

. PrEP awareness, familiarity, comfort, and prescribing experience among US primary care providers and HIV specialists. AIDS Behav. 2017;21(5):1256-1267. doi:10.1007/s10461-016-1625-1.

24.

Golub

. PrEP stigma: implicit and explicit drivers of disparity. Curr HIV/AIDS Rep. 2018;15(2):190-197. doi:10.1007/s11904-018-0385-0

25.

Dubov

Galbo

Jr Altice

Fraenkel

. Stigma and shame experiences by MSM who take PrEP for HIV prevention: a qualitative study. Am J Mens Health. 2018;12(6):1843-1854. doi:10.1177/1557988318797437

26.

Pinto

Berringer

Melendez

Mmeje

. Improving PrEP implementation through multilevel interventions: a synthesis of the literature. AIDS Behav. 2018;22(11):3681-3691. doi:10.1007/s10461-018-2184-4

27.

Centers for Disease Control and Prevention. Funding opportunity announcement: PS15-1506: health department demonstration projects to reduce HIV infections and improve engagement in HIV medical care among men who have sex with men (MSM) and transgender persons. 2015. Accessed July 10, 2021. https://www.cdc.gov/hiv/funding/announcements/ps15-1506/index.html

28.

Kim

Sefcik

Bradway

. Characteristics of qualitative descriptive studies: a systematic review. Res Nurs Health. 2017;40(1):23-42. doi:10.1002/nur.21768

29.

Sandelowski

. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334-340. doi:10.1002/1098-240x(200008)23:4<334::aid-nur9>3.0.co;2-g

30.

Centers for Disease Control and Prevention. Project PrIDE. Accessed September 30, 2021. https://www.cdc.gov/hiv/research/demonstration/projectpride.html

31.

AIDSVu. 2012-2019 County PrEP Data. Emory University, Rollins School of Public Health; 2021. Accessed July 10, 2021. https://aidsvu.org/resources/#/datasets

32.

Bhatia

Modali

Lowther

, et al. Outcomes of preexposure prophylaxis referrals from public STI clinics and implications for the preexposure prophylaxis continuum. Sex Transm Dis. 2018;45(1):50-55. doi:10.1097/olq.0000000000000690

33.

Clement

Johnston

Eagle

, et al. Advancing the HIV pre-exposure prophylaxis continuum: a collaboration between a public health department and a federally qualified health center in the southern United States. AIDS Patient Care STDS. 2019;33(8):366-371. doi:10.1089/apc.2019.0054

34.

Cahill

Taylor

Elsesser

Mena

Hickson

Mayer

. Stigma, medical mistrust, and perceived racism may affect PrEP awareness and uptake in Black compared to White gay and bisexual men in Jackson, Mississippi and Boston, Massachusetts. AIDS Care. 2017;29(11):1351-1358. doi:10.1080/09540121.2017.1300633

35.

Ezennia

Geter

Smith

. The PrEP care continuum and Black men who have sex with men: a scoping review of published data on awareness, uptake, adherence, and retention in PrEP care. AIDS Behav. 2019;23(10):2654-2673. doi:10.1007/s10461-019-02641-2

36.

Lelutiu-Weinberger

Golub

. Enhancing PrEP access for Black and Latino men who have sex with men. J Acquir Immune Defic Syndr. 2016;73(5):547-555. doi:10.1097/qai.0000000000001140

37.

Sullivan

Mena

Elopre

Siegler

. Implementation strategies to increase PrEP uptake in the South. Curr HIV/AIDS Rep. 2019;16(4):259-269. doi:10.1007/s11904-019-00447-4

38.

Carter

Jr Salabarría-Peña

Fields

Robinson

. Evaluating for health equity among a cluster of health departments implementing PrEP services. Eval Program Plann. Published online July 27, 2021. doi:10.1016/j.evalprogplan.2021.101981

39.

Fields

Long

Silvestri

, et al. #ProjectPresence: highlighting Black LGBTQ persons and communities to reduce stigma: a program evaluation. Eval Program Plann. Published online July 9, 2021. doi:10.1016/j.evalprogplan.2021.101978

40.

Frank

Starzyk

Hoxworth

, et al. HIV PrEP implementation: a multi-level systems approach. Eval Program Plann. Published online May 30, 2021. doi:10.1016/j.evalprogplan.2021.101966

41.

Greene

Reidy

Felt

, et al. Implementation and evaluation of patient navigation in Chicago: insights on addressing the social determinants of health and integrating HIV prevention and care services. Eval Program Plann. Published online July 8, 2021. doi:10.1016/j.evalprogplan.2021.101977

42.

Long

Jennings

Bademosi

, et al. Storytelling to improve healthcare worker understanding, beliefs, and practices related to LGBTQ + patients: a program evaluation. Eval Program Plann. Published online July 8, 2021. doi:10.1016/j.evalprogplan.2021.101979

43.

Marshall

Salabarría-Peña

Douglas

Nakelsky

Pichon

. The utility of evaluation in optimizing implementation and improvement of HIV prevention programming. Eval Program Plann. Published online July 27, 2021. doi:10.1016/j.evalprogplan.2021.101980

44.

Marshall

Salabarría-Peña

Johnson

Moore

. Reaching racial/ethnic and sexual and gender minorities with HIV prevention information via social marketing. Eval Program Plann. Published online July 30, 2021. doi:10.1016/j.evalprogplan.2021.101982

45.

Nakelsky

Moore

Garland

. Using evaluation to enhance a pre-exposure prophylaxis (PrEP) social marketing campaign in real time in Los Angeles County, California. Eval Program Plann. Published online August 13, 2021. doi:10.1016/j.evalprogplan.2021.101988

46.

Parrish

Johnson

Williams

. PrEP navigation continuum among men who have sex with men, trans women, and people with alternative gender identities in three California counties. Eval Program Plann. Published online September 3, 2021. doi:10.1016/j.evalprogplan.2021.101998

47.

Pichon

Teti

Betts

Brantley

. “PrEP”ing Memphis: a qualitative process evaluation of peer navigation support. Eval Program Plann. Published online August 13, 2021. doi:10.1016/j.evalprogplan.2021.101989

48.

Roach

Fritz

. Breaking barriers and building bridges: increasing community engagement in program evaluation. Eval Program Plann. Published online September 3, 2021. doi:10.1016/j.evalprogplan.2021.101997

49.

Salabarría-Peña

Douglas

Brantley

Johnson

. Informing the future of PrEP navigation: findings from a five-site cluster evaluation. Eval Program Plann. Published online September 2, 2021. doi:10.1016/j.evalprogplan.2021.101999

50.

Salabarría-Peña

Robinson

. Going beyond performance measures in HIV-prevention: a funder–recipient expedition. Eval Program Plann. Published online September 2, 2021. doi:10.1016/j.evalprogplan.2021.101996