Cross-site Monitoring and Evaluation of the Care and Prevention in the United States Demonstration Project,2012-2016: Selected Process and Short-term Outcomes

Abstract

Objective:

The Care and Prevention in the United States (CAPUS) Demonstration Project was a 4-year (2012-2016) cross-agency demonstration project that aimed to reduce HIV/AIDS-related morbidity and mortality among racial/ethnic minority groups in 8 states (Georgia, Illinois, Louisiana, Mississippi, Missouri, North Carolina, Tennessee, and Virginia). Its goals were to increase the identification of undiagnosed HIV infections and optimize the linkage to, reengagement with, and retention in care and prevention services for people with HIV (PWH). We present descriptive findings to answer selected cross-site process and short-term outcome monitoring and evaluation questions.

Methods:

We answered a set of monitoring and evaluation questions by using data submitted by grantees. We used a descriptive qualitative method to identify key themes of activities implemented and summarized quantitative data to describe program outputs and outcomes.

Results:

Of 155 343 total HIV tests conducted by all grantees, 558 (0.36%) tests identified people with newly diagnosed HIV infection. Of 4952 PWH who were presumptively not in care, 1811 (36.6%) were confirmed as not in care through Data to Care programs. Navigation and other linkage, retention, and reengagement programs reached 10 382 people and linked to or reengaged with care 5425 of 7017 (77.3%) PWH who were never in care or who had dropped out of care. Programs offered capacity-building trainings to providers to improve cultural competency, developed social marketing and social media campaigns to destigmatize HIV testing and care, and expanded access to support services, such as transitional housing and vocational training.

Conclusions:

CAPUS grantees substantially expanded their capacity to deliver HIV-related services and reach racial/ethnic minority groups at risk for or living with HIV infection. Our findings demonstrate the feasibility of implementing novel and integrated programs that address social and structural barriers to HIV care and prevention.

Keywords

HIV prevention and care racial/ethnic minorities project monitoring and evaluation Care and Prevention in the United States CAPUS demonstration project

Racial/ethnic minority groups in the United States, particularly non-Hispanic black or African American (hereinafter, non-Hispanic black) and Hispanic/Latino (hereinafter, Latino) people, are disproportionately affected by HIV. In 2016, the HIV diagnosis rates per 100 000 population among non-Hispanic black (43.6) and Latino (17.0) populations were 8 and 3 times higher, respectively, than HIV diagnosis rates among non-Hispanic white people (5.2).¹ On average, non-Hispanic black and Latino people with HIV (PWH) are diagnosed 3.3 years after acquiring infection, whereas non-Hispanic white PWH are diagnosed 2.2 years after acquiring infection.² About 70% of non-Hispanic black and Latino PWH received HIV care in 2014 compared with 76% of non-Hispanic white people. However, among PWH diagnosed by year-end 2014 and alive at year-end 2015, non-Hispanic black people were less likely than both Latino and non-Hispanic white people to remain in care and virally suppressed.³ Social and structural factors, such as racism, poverty, and inadequate access to health care, contribute to racial/ethnic disparities in HIV infection and HIV-related outcomes.⁴

State health departments and community-based organizations (CBOs) are well positioned to implement interventions that promote HIV testing, prevent HIV infection, support linkage to and retention in care, and address social and structural factors that contribute to HIV-related racial/ethnic disparities. However, HIV prevention and care efforts are often fragmented and are not coordinated to address the myriad needs of racial/ethnic minority groups. In 2009, the Centers for Disease Control and Prevention (CDC) emphasized the need for cross-collaboration within and across health department units (eg, surveillance, care, and prevention programs)⁵ to meet the objectives of the National HIV/AIDS Strategy, which include reducing new HIV infections, increasing access to care and optimizing health outcomes for PWH, reducing HIV-related health disparities and health inequities, and achieving a more coordinated national response to the HIV epidemic.^6,7 Nevertheless, persistent racial/ethnic disparities in HIV-related outcomes suggest a continuing need for innovative and integrated approaches to prevention and care.

In 2012, CDC funded 8 state health departments (grantees)—Georgia, Illinois, Louisiana, Mississippi, Missouri, North Carolina, Tennessee, and Virginia—to implement the Care and Prevention in the United States (CAPUS) Demonstration Project (hereinafter, CAPUS).⁸ Each grantee received $3.4 million to $7.5 million (for a total of $42.8 million) during the first 3 years of the 4-year project period (September 30, 2012, through September 29, 2016) to increase the proportion of racial/ethnic minority groups with diagnosed HIV infection and to optimize linkage to, reengagement with, and retention in care and prevention services for PWH. Grantees had considerable latitude in how they addressed project requirements, which included (1) enhancing HIV testing and linkage to, reengagement with, and retention in care, prevention, and support services; (2) providing navigation services; (3) using surveillance and other data systems to improve care and prevention—a Data to Care strategy⁹ that uses surveillance, care, prevention, and other data to identify PWH who need HIV medical care or other services, to facilitate their linkage to those services, and to eventually improve viral suppression and other health outcomes; and (4) addressing social and structural factors that directly reduce participation in HIV-related services.⁸

CAPUS was a unique demonstration project. Although past flagship HIV prevention programs (eg, Comprehensive HIV Prevention Programs for Health Departments, 2012-2016¹⁰) and demonstration projects (eg, Enhanced Comprehensive HIV Prevention Planning project, 2010-2013¹¹) supported HIV testing and comprehensive prevention programs for PWH, they did not fund patient navigation services, Data to Care strategies, or activities addressing HIV-related social and structural factors. CAPUS also explicitly required grantees to collaborate across health department units (surveillance, prevention, and care) and support capacity-building and program implementation by local CBOs.

CAPUS was also a complex intervention characterized by multiple interacting program components, several implementing agencies, and multidirectional and emergent outcomes.^12
-14 Evaluations of CAPUS-like projects that integrate multiple HIV prevention and care programs for racial/ethnic minority groups are limited, partly because few such projects have been funded¹⁵ and partly because teasing out the outcomes of complex projects is challenging.^12
-14 Process evaluation can provide valuable information about how activities are implemented, the factors that affect implementation, and the reach of those activities. When coupled with evaluations of short-term outcomes that are closely tied to program activities, process evaluation provides important information about potential mechanisms of effects and how these effects might be replicated in other interventions.¹⁶

We present the first descriptive findings to answer selected cross-site process and short-term outcome monitoring and evaluation questions. Other reports will address long-term outcome and impact evaluation questions.

Methods

CDC developed a monitoring and evaluation framework that incorporated 2 levels: a cross-site level led by CDC with a focus on documenting processes and outcomes across all grantees and a local level led by grantees to meet their programmatic needs.

Cross-site Monitoring and Evaluation Questions

CDC developed a set of process, short-term outcome, and long-term outcome monitoring and evaluation questions with input from internal and external stakeholders. In this article, “short-term outcomes” refer to outcomes that are expected within 1-2 years of program implementation, and “long-term outcomes” refer to outcomes that are expected ≥3 years after implementation. The 3 process and short-term–outcome monitoring and evaluation questions addressed in this article are:

What activities did grantees implement to meet project requirements?

To what extent did grantees reach non-Hispanic black and Latino people who were a priority for the project?

To what extent did grantees achieve the intended short-term outcomes?

Data Requirements and Indicators

CDC collected limited qualitative and quantitative data for cross-site monitoring and evaluation purposes to minimize the data collection and reporting burden on grantees.¹⁷ Qualitative information was reported in grantees’ work plans and progress reports and included descriptions of program objectives, activities, and strategies implemented during the project. Quantitative data requirements included the following aggregated program outputs and outcomes.

HIV testing programs

Aggregate data from grantees’ testing programs included the number of tests conducted and the number of newly and previously identified HIV-positive tests. A newly identified HIV-positive test is a positive test from a person with no history of a previous positive test. A previously identified HIV-positive test is a test from a person who tested positive for HIV on the current test and who has a history of a previous positive test.

Navigation and other linkage, reengagement, and retention (NLRR) programs

Grantees’ NLRR programs varied in the types of activities and how they were implemented and in the types of services they offered. As a result, aggregate data were limited to the number of people enrolled in the programs and the number of people who received the intended services. We broadly defined “enrollment” to include any level of contact or engagement in a CAPUS-supported program intended to provide services, including intake interviews, screening or needs assessments, referral and navigation to services, or a combination of these activities. To reduce the burden of data collection, we did not collect data on the number of enrolled people with unmet needs from grantees.

Outcomes for NLRR programs included how many people were linked to or reengaged with care, enrolled in retention and medication-adherence support services, and linked to and provided prevention, behavioral health, or social services. “Provision” of intended services refers to delivery of assistance for people enrolled in the programs, such as mental health treatment, temporary housing, or transportation vouchers. People with diagnosed HIV infection were considered linked to or reengaged with care if they attended their first HIV medical appointment after diagnosis or after being out of care for at least the previous 6 months, respectively. Linkage or reengagement was also confirmed by a documented viral load test or CD4 count.

Use of surveillance and other data systems

Data to Care is a multistep strategy⁹ that typically begins with using HIV surveillance data to identify PWH who are not in care. The not-in-care list is further cross-checked with other data systems (eg, care or prevention program data) to complete missing data and to verify care status. Health departments may also apply inclusion or exclusion criteria to determine a priority not-in-care list, which is subsequently shared with field staff members to locate and provide linkage and reengagement services to people who truly need to be engaged in care. An integral part of the Data to Care process is the feedback loop in which newly acquired information from field investigations is shared with surveillance programs to improve data quality. Aggregate data that were reported included the number of PWH presumed to be not in care, the number of PWH whose identifying and locating information was forwarded for public health investigation, the number of PWH located and confirmed as truly not in care, the number of PWH receiving linkage services, and the number of PWH linked to or reengaged with medical care. The number of people not in care whose surveillance records were updated with newly acquired information was also reported.

Addressing social and structural factors

Data from programs addressing HIV-related social and structural factors included the number and reach of social marketing and social media campaigns, interventions to increase access to and delivery of services, and capacity-building trainings. Some of the grantees’ programs to address social and structural factors were tied to delivery of HIV testing and NLRR services. For example, a grantee may focus on addressing housing instability as part of its effort to expand linkage to care and prevention services. In this case, aggregate data on the number of people who enrolled in a housing assistance program and who subsequently received housing assistance were captured under data requirements for other NLRR programs.

Targets and other program data

Grantees also provided their own end-of-project, short-term performance targets for the number of HIV tests to be conducted and the number of people to be enrolled in NLRR programs. Grantees set these targets based on their capacity and programmatic experience. Given the limited evidence-based knowledge about Data to Care at the time of the project and the diversity of programs to address social and structural factors, grantees were not required to specify targets for these programs. In addition to targets, grantees reported the number and type of their implementation partners and their annual funding allocation for contracts with CBOs.

Grantees submitted progress reports with aggregate data twice in the first 2 years and once in the last 2 years of the project. This article is based on cumulative data reported at the end of the project. We stratified aggregate data by race/ethnicity, gender, HIV transmission risk categories, and HIV test setting, where applicable (Table 1). CDC determined that data collection for this project constituted a routine program monitoring activity; therefore, institutional review board approval was not required.

Table 1.

Components of the cross-site monitoring and evaluation of the Care and Prevention in the United States (CAPUS) Demonstration Project, 8 US states, 2012-2016^a

Question	Type of Data Collected	Indicators
What activities and strategies did grantees implement to:
Enhance HIV testing? Provide NLRR services? Use surveillance and other data systems to improve health outcomes? Address HIV-related social and structural factors? Support CBO capacity to implement programs?	Descriptions of activities or strategies implemented and resource allocation for planning, implementing, and evaluating each program requirement	Key themes of activities and strategies used to plan and implement required programs and expand services to priority populations Number of CBOs contracted to work on CAPUS-supported programs Percentage of total funds allocated to support CBO-contracted programs
To what extent did grantees reach the priority populations through their:
HIV testing programs?	Targeted and actual number of tests conducted Site of testing, demographic and behavioral risk characteristics of people tested^b	Number of tests conducted, stratified by site, and characteristics of people tested Percentage of target test numbers that was achieved at the end of the project Percentage of all tests that were conducted among non-Hispanic black or Latino people
NLRR programs?	Targeted number of PWH to be enrolled^c by programs Number of PWH who enrolled in programs Demographic and behavioral risk characteristics of enrollees	Number of PWH who enrolled in these programs, stratified by enrollee characteristics Percentage of targeted PWH who enrolled in these programs Percentage of program enrollees who were non-Hispanic black or Latino
Programs to use surveillance and other data systems to improve health outcomes?	Number of PWH who were identified as presumptively not in care and investigated Number of PWH investigated who were contacted	Number (%) of PWH not in care who were investigated among those identified as not in care Number (%) of investigated PWH not in care who were contacted
Programs to address HIV-related social and structural factors?	Number of PWH who enrolled in these programs Number of people trained to address HIV-related social and structural factors Number of people reached by social media and social marketing campaigns	Number of PWH who enrolled in programs designed to address HIV-related social and structural factors Number of people trained to address HIV-related social and structural factors Number of people reached by social media and social marketing campaigns
To what extent did grantees achieve the intended short-term outcomes for their:
HIV testing programs	Number of tests identifying people with newly diagnosed HIV infection Site of testing Demographic and behavioral risk characteristics of diagnosed people	Number (%) of tests identifying people with newly diagnosed HIV infection among all tests conducted, stratified by characteristics of people tested Percentage of people newly diagnosed with HIV infection who were non-Hispanic black or Latino
NLRR programs?	Number of PWH who were enrolled in NLRR programs and were linked to or provided the intended services Demographic and behavioral risk characteristics of program enrollees Type of services	Number (%) of program enrollees who were linked to or reengaged^d with care, stratified by enrollee characteristics Number (%) of program enrollees who were provided other services, stratified by enrollee characteristics Percentage of non-Hispanic black or Latino people among program enrollees who were linked to, reengaged with, or provided services
Programs to use surveillance and other data systems to improve health outcomes?	Number of contacted PWH not in care who were provided linkage or reengagement services and subsequently linked to or reengaged with HIV care Number of surveillance records updated because of information acquired through the Data to Care⁹ process	Number (%) of PWH not in care who were provided linkage or reengagement services and were subsequently linked to or reengaged with HIV care Number (%) of PWH not in care whose surveillance records were updated with information newly acquired through the Data to Care process
Programs to address HIV-related social and structural factors?	Number of PWH who were enrolled in programs designed to address HIV-related social and structural factors Number of enrollees who were provided the intended services Type of services	Number (%) of enrollees in programs designed to address HIV-related social and structural factors who were provided intended services, stratified by type of service

Abbreviations: CBO, community-based organization; NLRR, navigation and other linkage, reengagement, and retention; PWH, people with HIV.

^aCAPUS was a 4-year cross-agency demonstration project led by the Centers for Disease Control and Prevention with the aim of reducing HIV/AIDS-related morbidity and mortality among racial/ethnic minority groups in the United States. Eight states (Georgia, Illinois, Louisiana, Mississippi, Missouri, North Carolina, Tennessee, and Virginia) received funding to implement the project.⁸

^bRacial/ethnic categories included Hispanic/Latino, non-Hispanic black, non-Hispanic white, and non-Hispanic other racial/ethnic groups. Gender categories included male, female, transgender, or other. Behavioral risk categories included gay, bisexual, and other men who have sex with men (MSM); people who inject drugs (PWID); MSM/PWID; high-risk heterosexual; and other. High-risk heterosexual included people who reported heterosexual contact and 1 or more sexual risk behaviors, such as condomless sex, sex with PWH, sex while high or intoxicated, exchange of sex for drugs or money, sex with anonymous partners, or sex with multiple partners.

^cEnrollment in a CAPUS-supported program refers to any level of targeted people’s contact or engagement in an activity designed to facilitate the provision of care, prevention, or support services. Contact or engagement includes intake interviews, screening or needs assessments, referral, and linkage to service providers. Provision of intended services refers to delivery of assistance for people enrolled in the programs, such as mental health treatment, temporary housing, or transportation vouchers.

^dPeople with diagnosed HIV are considered linked to or reengaged with care if they had attended their first HIV medical appointment ever or after being out of care for at least the past 6 months, respectively. Linkage or reengagement could also be confirmed by a documented viral load test or CD4 count.

Data Analysis

We used a simplified version of the qualitative description method^18,19 to analyze qualitative information. This method is suitable when the goal of evaluation is a straight description of a phenomenon and most appropriate to answer questions focused on discovering the “who,” “what,” or “where” of events or experiences.^18,19 The authors independently reviewed sections of progress reports in pairs, identified key activities implemented for each program component, and jointly summarized the findings. We streamlined our analysis to no more than 2 iterations and minimal interpretation of findings. We compiled aggregate quantitative data, checked the data for quality, and summarized the findings. Where available, we compared grantees’ initial targets with their end-of-project performance. In addition, for some program outcomes, we tested differences in proportions among racial/ethnic minority groups and other groups by using Z tests, with α set at 0.05.

Results

HIV Testing Programs

All 8 grantees implemented activities to improve HIV testing capacity, and 7 grantees (Georgia, Illinois, Louisiana, Mississippi, Missouri, Tennessee, and Virginia) expanded testing services to locations serving racial/ethnic minority groups. Testing strategies included routine opt-out testing in health care settings (eg, emergency departments, pharmacies, and correctional health clinics). Three grantees (Missouri, Tennessee, and Virginia) implemented social network testing in the community. Seven grantees (Georgia, Louisiana, Mississippi, Missouri, North Carolina, Tennessee, and Virginia) launched social media and social marketing campaigns to raise awareness about, and destigmatize, HIV testing. Five grantees (Georgia, Illinois, Louisiana, North Carolina, and Tennessee) acquired or used the latest HIV testing technology (eg, fourth-generation testing) to increase the rate of early HIV diagnoses and to provide timely linkage to care.²⁰

Grantees conducted a total of 155 343 HIV tests—16% more than their initial target of 133 950 tests. Most tests were conducted among non-Hispanic black and Latino people (n = 133 742, 66.8%), among women (n = 85 955, 55.3%), and in health care settings (n = 125 224, 80.6%; Table 2). Project-wide, 558 tests identified newly diagnosed HIV infection, for a newly diagnosed HIV positivity rate of 0.36%. This rate was significantly higher among tests from non-Hispanic black people (0.57%) than among tests from non-Hispanic white people (0.14%, Z = 142.8, P < .001). In general, the percentage of tests identifying people with newly diagnosed HIV was higher in non–health care settings (0.54%) than in health care settings (0.32%; Z = 7.1, P < .001), particularly among tests conducted in communities by CBOs (2.26%). Tennessee reported a 3.2% newly diagnosed HIV positivity rate among young (aged 15-35) black gay, bisexual, and other men who have sex with men (collectively referred to as MSM) who were recruited through a social network strategy.

Table 2.

Number of HIV tests conducted and number of tests identifying people newly and previously diagnosed with HIV, by demographic characteristics and test setting, in 7 Care and Prevention in the United States (CAPUS) Demonstration Project sites, 2012-2016^a

	HIV Tests Conducted (n = 155 343)	Newly Identified HIV-Positive Tests^c (n = 558)		Previously Identified HIV-Positive Tests^c (n = 430)
Demographic Characteristics and Test Setting^b	No. (%)	No. (%)	Positivity Rate, %^d (P Value)^e	No. (%)	Positivity Rate, %^d (P Value)^e
Race/ethnicity
Latino	26 271 (16.9)	38 (6.8)	0.14 (.72)	20 (4.7)	0.08 (<.001)
Non-Hispanic black	77 471 (49.9)	438 (78.5)	0.57 (<.001)	379 (88.1)	0.49 (<.001)
Non-Hispanic white	43 625 (28.1)	63 (11.3)	0.14 (Ref.)	24 (5.6)	0.06 (Ref.)
Other or unknown	7976 (5.1)	19 (3.4)	0.24 (<.001)	7 (1.6)	0.09 (<.001)
Gender
Male	68 721 (44.2)	439 (78.7)	0.64 (<.001)	331 (77.0)	0.48 (<.001)
Female	85 955 (55.3)	111 (19.9)	0.13 (Ref.)	96 (22.3)	0.11 (Ref.)
Transgender	198 (0.1)	7 (1.3)	3.54 (<.001)	3 (0.7)	1.52 (<.001)
Other/unknown	469 (0.3)	1 (0.2)	0.21 (<.001)	0	0 (<.001)
HIV test setting
Health care settings	125 224 (80.6)	395 (70.8)	0.32	280 (65.1)	0.22
Community health centers	58 111 (37.4)	210 (37.6)	0.36 (Ref.)	110 (25.6)	0.19 (Ref.)
Emergency departments	30 635 (19.7)	98 (17.6)	0.32 (<.001)	119 (27.7)	0.39 (<.001)
Correctional facility clinics	13 244 (8.5)	8 (1.4)	0.06 (<.001)	1 (0.2)	0.01 (<.001)
Substance abuse treatment facilities	10 070 (6.5)	3 (0.5)	0.03 (<.001)	1 (0.2)	0.01 (<.001)
Inpatient units	4957 (3.2)	45 (8.1)	0.91 (<.001)	38 (8.8)	0.77 (<.001)
Pharmacy-based clinics	3700 (2.4)	25 (4.5)	0.68 (<.001)	8 (1.9)	0.22 (<.001)
Other health care settings^f	4507 (2.9)	6 (1.1)	0.13 (<.001)	3 (0.7)	0.07 (<.001)
Non–health care settings	30 119 (19.4)	163 (29.2)	0.54	150 (34.9)	0.50
CBOs/other service organizations	4592 (3.0)	104 (18.6)	2.26 (<.001)	111 (25.8)	2.42 (<.001)
Other	25 527 (16.4)	59 (10.6)	0.23 (<.001)	39 (9.1)	0.15 (<.001)
Total	155 343 (100.0)	558 (100.0)	0.36	430 (100.0)	0.28

Abbreviations: CBO, community-based organization; ref, reference group.

^bBehavioral risk characteristics of people tested are excluded because risk information was not required from people with HIV-negative test results in health care settings and was missing for most people tested in non–health care settings.

^cA newly identified HIV-positive test is a test from a person who tested positive but had no history of a previous positive test. A previously identified HIV-positive test is a test from a person who tested positive on the current test and had a history of a previous positive test.

^dHIV positivity rate was calculated as the number of positive tests (separately for newly or previously identified HIV-positive tests) divided by the number of all tests conducted and multiplied by 100. These rates were calculated separately for each racial/ethnic, gender, and test-setting characteristic.

^eWithin-group comparisons on HIV positivity rates were conducted using a Z test for proportions.

^fOther health care settings include urgent care clinics, dental care clinics, other primary care clinics, sexually transmitted disease clinics, tuberculosis clinics, and other public health clinics.

Navigation and Other Linkage, Reengagement, and Retention Programs

All grantees developed navigation programs to facilitate linkage to care, prevention, and other services for PWH; 6 grantees (Georgia, Illinois, Louisiana, Mississippi, North Carolina, and Tennessee) also implemented other linkage, reengagement, and retention programs that were distinct from their navigation programs (eg, use of incentives to increase retention in HIV care). Building these programs required grantees to integrate NLRR services into existing care systems. Grantees’ program models varied by the type of staff members deployed (eg, peers, licensed clinical social workers, nurses), populations served (eg, people newly diagnosed with HIV or PWH who were out of care), and service-delivery strategy or setting (eg, provision of incentives in primary care clinics, use of “safe spaces” where marginalized people come together to communicate with each other and receive services without fear of emotional or physical harm). Additional services offered in these programs included enrollment, screening, referral, linkage to, or provision of other essential support services (eg, medication adherence support, transportation assistance).

During the project period, 10 382 PWH were enrolled in NLRR programs across all grantee sites—which was 71% higher than grantees’ initial target of enrolling 6058 PWH in such programs. Nearly half (n = 4734, 45.6%) of enrollees were previously in care and had dropped out of care, 2283 (22.0%) were never in care, 2821 (27.2%) were in care, and 544 (5.2%) had unknown care status at enrollment. These programs reengaged with care 3649 of 4734 (77.1%) PWH who had dropped out of care and linked to care 1776 of 2283 (77.8%) PWH who were never in care (Table 3). Reengagement rates were higher among non-Hispanic black people (78.7%) than among non-Hispanic white people (73.6%, Z = 3.12, P = .002); however, linkage rates were similar across racial/ethnic groups.

Table 3.

Enrollment in and outcomes of navigation and other linkage, reengagement, and retention (NLRR) programs for people with HIV, by enrollee characteristics, at 8 Care and Prevention in the United States (CAPUS) Demonstration Project sites, 2012-2016^a

Demographic and Behavioral Risk Characteristics	Total Enrollees in NLRR Programs,^b No. (%)	Enrollees Who Were Never in Care		Enrollees Who Were Previously in Care		Enrollees Who Entered or Reentered Care
Demographic and Behavioral Risk Characteristics	Total Enrollees in NLRR Programs,^b No. (%)	Enrolled in Linkage Activity,^c No. (% of Total)^d	Linked to Care, No. (% of Enrolled in Linkage Activity)^e [P Value]^f	Enrolled in Reengagement Activity,^g No. (% of Total)^d	Reengaged With Care, No. (% of Enrolled in Reengagement Activity)^e [P Value]^f	Enrolled in Retention- in-Care Support Activity,^h No. (% of Total)^d	Enrolled in Medication Adherence Support Activity,^h No. (% of Total)^d
Race/ethnicity
Latino	527 (5.1)	138 (26.2)	97 (70.3) [.15]	168 (31.9)	119 (70.8) [.46]	266 (50.5)	192 (36.4)
Non-Hispanic black	7656 (73.7)	1620 (21.2)	1291 (79.7) [.13]	3661 (47.8)	2883 (78.7) [.002]	3205 (41.9)	3210 (41.9)
Non-Hispanic white	1886 (18.2)	461 (24.4)	352 (76.4) [Ref.]	795 (42.2)	585 (73.6) [Ref.]	1041 (55.2)	868 (46.0)
Non-Hispanic other/unknown	313 (3.0)	64 (20.4)	36 (56.3) [.001]	110 (35.1)	62 (56.4) [<.001]	113 (36.1)	126 (40.3)
Gender
Male	7641 (73.6)	1698 (22.2)	1385 (81.6) [<.001]	3636 (47.6)	2810 (77.3) [.84]	3351 (43.9)	3319 (43.4)
Female	2494 (24.0)	516 (20.7)	345 (66.9) [Ref.]	1017 (40.8)	789 (77.6) [Ref.]	1176 (47.2)	1022 (41.0)
Transgender	213 (2.1)	65 (30.5)	44 (67.7) [.90]	75 (35.2)	50 (66.7) [.03]	94 (44.1)	52 (24.4)
Other/unknown	34 (0.3)	4 (11.8)	2 (50.0) [.001]	6 (17.6)	0 [<.001]	4 (11.8)	3 (8.8)
HIV risk category
MSM	4829 (46.5)	1194 (24.7)	1072 (89.8) [<.001]	1988 (41.2)	1603 (80.6) [<.001]	2279 (47.2)	1780 (36.9)
PWID	416 (4.0)	55 (13.2)	42 (76.4) [.92]	172 (41.3)	130 (75.6) [.41]	197 (47.4)	148 (35.6)
High-risk heterosexualⁱ	3023 (29.1)	627 (20.7)	483 (77.0) [Ref.]	1475 (48.8)	1035 (70.2) [Ref.]	1213 (40.1)	1337 (44.2)
MSM/PWID	239 (2.3)	23 (9.6)	21 (91.3) [.11]	75 (31.4)	58 (77.3) [.19]	163 (68.2)	84 (35.1)
Other/unknown	1875 (18.1)	384 (20.5)	158 (41.1) [<.001]	1024 (54.6)	823 (80.4) [<.001]	773 (41.2)	1047 (55.8)
Total	10 382 (100.0)	2283 (22.0)	1776 (77.8)	4734 (45.6)	3649 (77.1)	4625 (44.5)	4396 (42.3)

Abbreviations: MSM, men who have sex with men; PWID, people who inject drugs; ref, reference group.

^bEnrollment is broadly defined to include any level of contact or engagement in a CAPUS-supported program intended to provide services, including intake interviews, screening or needs assessments, referral and navigation services, or a combination of these activities. Total enrollees in NLRR programs include the number of unique people who were contacted or engaged through intake assessment, screening or needs assessment, referral, linkage or navigation to services, or a combination of similar activities. Follow-up data on medication adherence and retention in care were not collected because of concerns about data burden.

^cLinkage-to-care program enrollees include only those with HIV who were never in care.

^dThe denominator for percentage of those enrolled in activities designed to facilitate linkage to or reengagement with care, retention in care, or medication adherence was the total number of NLRR enrollees.

^eThe denominators for the percentages of those linked to and reengaged with care were the number of those who were enrolled in a linkage or reengagement activity, respectively. People with diagnosed HIV are considered linked to or reengaged with care if they had attended their first HIV medical appointment ever or after being out of care for at least the past 6 months, respectively. Linkage or reengagement could also be confirmed by a documented viral load test or CD4 count.

^fWithin-group comparisons on linkage and reengagement rates were conducted using a Z test for proportions. Non-Hispanic white was the reference group for race/ethnicity, female was the reference group for gender, and high-risk heterosexual was the reference group for HIV transmission risk.

^gReengagement with care program enrollees include only those who had fallen out of care.

^hRetention in care and medication adherence support activity enrollees include those who were entering or reentering care through NLRR programs.

ⁱHigh-risk heterosexual includes people who report heterosexual contact and 1 or more sexual risk behaviors, such as condomless sex, sex with people with HIV, sex while high or intoxicated, exchange of sex for drugs or money, sex with anonymous partners, or sex with multiple partners.

Nearly half of the 10 382 NLRR program enrollees participated in activities intended to support retention in care (n = 4625, 44.5%) and medication adherence (n = 4396, 42.3%). However, data on how many of these enrollees needed these services and how many enrollees were subsequently retained in care or adhered to their medication regimens were not collected (Table 3). In addition, 18% to 30% of enrollees of NLRR programs participated in activities intended to facilitate linkage to 1 or more prevention, behavioral health, or social services. Among those enrolled, the most commonly provided services were risk-reduction interventions (1924 of 2516; 76.5%), partner services (1387 of 2050; 67.7%), and transportation services (1548 of 2636; 58.7%; Table 4).

Table 4.

Enrollment and outcomes of navigation and other linkage, reengagement, and retention (NLRR) programs intended to link people with HIV to prevention, behavioral health, and social support services in 8 Care and Prevention in the United States (CAPUS) Demonstration Project sites, 2012-2016^a

Type of Service	Enrollees^b in Activities Intended to Provide Each Service, No. (% of Total NLRR Enrollees; n = 10 382)^c	Enrollees Provided Service/Assistance, No. (% of Those Enrolled in Activity Intended to Provide the Service)^d
Prevention
Partner services^e	2050 (19.7)	1387 (67.7)
Risk-reduction interventions^f	2516 (24.2)	1924 (76.5)
Behavioral health
Mental health treatment services^g	2197 (21.2)	527 (24.0)
Substance abuse treatment services^g	1904 (18.3)	150 (7.9)
Social
Housing services^h	2576 (24.8)	1004 (39.0)
Transportation services^h	2636 (25.4)	1548 (58.7)
Employment services^h	2085 (20.1)	310 (14.9)
Other social servicesⁱ	3153 (30.4)	1650 (52.3)

^bOverall enrollment in NLRR programs is broadly defined to include any level of contact or engagement in a CAPUS-supported program intended to provide services, including intake interviews, screening or needs assessments, referral, and navigation to services, or a combination of these activities. HIV-diagnosed people may additionally enroll in activities intended to facilitate the provision of 1 or more prevention, behavioral health, or social services to meet their specific needs. However, grantees did not uniformly offer the services listed as part of their NLRR programs. Thus, not all 10 382 enrollees of NLRR programs had access to all services.

^cThe number of people enrolled in an activity designed to facilitate linkage to and provision of the service. The denominator for the percentage is the total number of NLRR programs (n = 10 382). For example, 2050 of the 10 382 (19.7%) NLRR program enrollees participated in an activity designed to facilitate provision of partner services.

^dThe number of those enrolled in an activity who were provided needed services. For example, 1387 of 2050 (67.7%) enrollees in a partner services-related activity were provided partner services (ie, were interviewed for partner services and received relevant support).

^ePartner services are a broad array of services designed to notify sex and needle-sharing partners of HIV-infected people of their possible exposure to HIV so they can be offered HIV testing and learn their status or, if already infected, prevent transmission to others.

^fRisk-reduction interventions are individual- or group-level approaches designed to promote changes in sexual or drug-use behaviors that will result in reduced risk for HIV infection or transmission.

^gMental health and substance use treatment services include approaches to screen for unmet mental health or substance use-related health needs, refer and link clients to appropriate behavioral health service providers, and provide treatment to address the unmet needs (eg, provision of psychotherapy, pharmacotherapy).

^hHousing, transportation, and employment services include activities intended to screen for an unmet service need (eg, unstable housing, lack of transportation to attend medical appointment, lack of a job), refer and link those with needs to appropriate social service providers, and provide them with assistance to address the unmet needs (eg, temporary housing, transportation voucher, job training).

ⁱOther social services include referral for or provision of assistance with food, health insurance, disability benefits, financial counseling, legal aid, and other similar services.

Use of Surveillance Data and Other Data Systems to Improve HIV Care and Prevention

All grantees used HIV surveillance and other data systems (eg, Ryan White HIV/AIDS Program services, commercial people-locating databases) to describe HIV prevalence, target project activities to communities or areas most affected by HIV, and monitor improvements in the HIV care continuum at the state, local, or provider levels. Key activities to improve the use of surveillance and other data systems included (1) upgrading laboratory and program data reporting systems (eg, changing from paper to electronic reporting); (2) integrating surveillance, care, and prevention data; (3) developing clinical alerts or data-sharing systems to facilitate linkage to care; and (4) implementing policies and procedures to promote the use of surveillance data for program follow-up.

Using surveillance and other data systems, 7 grantees (Illinois, Louisiana, Mississippi, Missouri, North Carolina, Tennessee, and Virginia) identified 18 806 PWH who were presumed to be not in care and initiated Data to Care processes to confirm who among them were truly not in care and to link to or reengage them with care (Figure). Four findings were noteworthy:

Data to Care programs contacted 4952 of 12 252 (40.4%) PWH who were investigated to verify their care status. The remainder were not contacted, either because the follow-up process was not complete or the people presumed to be not in care were dead, incarcerated, or out of the health department’s jurisdiction.

Only 1811 of 4952 (36.6%) PWH who were contacted were truly not in care; the remaining were either already in care or their care status was not yet verified.

Most PWH who were provided linkage or reengagement services through NLRR programs (851 of 1033, 82.4%) were subsequently linked to or reengaged with HIV care, whereas others were either not linked to or reengaged with care or their linkage status was not yet determined.

Surveillance and other program records for 2913 PWH were updated with newly acquired information (eg, current linkage status) during the Data to Care process. The processes and outcomes of these programs from 4 grantees are detailed elsewhere in this issue.²¹

Figure.

Processes and outcomes of Data to Care programs to link to or reengage in care those people diagnosed with HIV who were presumed to not be in care at 7 Care and Prevention in the United States (CAPUS) Demonstration Project sites, 2012-2016. The 7 sites were Illinois, Louisiana, Mississippi, Missouri, North Carolina, Tennessee, and Virginia. By the end of the project, Georgia had implemented a new policy to allow the use of surveillance data for program follow-up, improve the completeness of its surveillance data, and develop a system to generate the presumed not-in-care list. However, it did not fully implement the Data to Care process. Data to Care is a public health strategy that uses surveillance, care, prevention, and other data to identify people with HIV (PWH) who need HIV medical care or other services, to facilitate their linkage to those services, and to eventually improve viral suppression and other health outcomes.⁹ This figure presents the Data to Care process that typically begins with using HIV surveillance data to identify PWH who are presumed to be not in care. The not-in-care list is further cross-checked with other data systems (eg, care or prevention program data) to complete missing data and to verify care status. Health departments may also apply inclusion or exclusion criteria to determine a priority not-in-care list, which is subsequently shared with field staff members to locate PWH who are presumed to be not in care, confirm their care status, and provide them with linkage and reengagement services if they are confirmed as not in care. Information for some PWH presumed to be not in care was not forwarded for follow-up because of various reasons, including the person being deceased, out of jurisdiction, not in a CAPUS implementation area within the jurisdiction, or in care after checks in other data systems. PWH presumed to be not in care who received follow-up and were not contacted included primarily those who were deceased, out of jurisdiction, or incarcerated. Linkage to or reengagement with care included confirmed PWH who were previously not in care and had attended their first HIV medical appointment. Linkage or reengagement was also confirmed by documented viral load test or CD4 count. Data to Care program process and outcomes reflect data as of September 29, 2016, which was the final data reporting date for the project.

Programs to Address HIV-Related Social and Structural Factors

Grantees implemented activities designed to address 1 or more social and structural factors that can lead to low participation of racial/ethnic minority groups in HIV-related programs. Commonly targeted social and structural factors were social biases (eg, HIV-related stigma, homophobia, transphobia), cultural competence among providers, and the availability or accessibility of comprehensive HIV-related services. Programs were designed to reach underserved populations (eg, racial/ethnic minority groups with HIV; young people in sexual and gender minority groups), service providers, faith leaders, and family members.

Three categories of activities were implemented across grantees. First, capacity-building and practice-improvement activities sought to address cultural competency and social biases among health department staff members or care, prevention, and social services providers to improve the delivery of HIV-related services. In addition, activities to increase awareness about and compliance with HIV-related recommended practices (eg, routine HIV testing) were implemented. Six grantees (Georgia, Illinois, Louisiana, Missouri, North Carolina, and Tennessee) trained 1334 health department and partner staff members through these programs. Second, social marketing and social media campaigns were designed to raise community awareness about HIV testing and linkage to care; increase information about available HIV prevention, care, and support services; reduce HIV-related stigma; and mobilize faith leaders and family members to support PWH. Although precisely assessing the reach of these efforts was not possible, 5 grantees (Georgia, Louisiana, Mississippi, Missouri, and Virginia) reported a range of 700 000–11 million social marketing and social media exposures (broadly defined to include measures that approximate media reach, such as count of website visits, number of brochures distributed to target populations, and estimated number of radio listeners or television viewers exposed to marketing messages). Small-scale HIV awareness events (eg, brochures, flyers, and website advertisements) reached 200 people in North Carolina and 13 000 people in Illinois. Third, 2 grantees (Illinois and Virginia) expanded accessibility and provision of services by co-locating or integrating care, prevention, and support services and directly offering comprehensive services that were previously unavailable to priority populations. These 2 programs provided 169 PWH or people at risk for HIV infection with housing assistance, vocational training, and a combination of care, prevention, and support services. Grantees also sought to address social and structural factors as part of their other program components (eg, addressing transportation needs through navigation programs). The processes or outcomes of these interventions are detailed elsewhere.^22
-24

Strengthening Community-Based Partnerships

Grantees were required to allocate at least 25% of their total awards to collaborating with and supporting capacity-building and program implementation by CBOs and other local organizations. Grantees collaborated with 117 organizations, including CBOs (55.5%), local health departments (20.5%), other nonprofit institutions (20.5%), and businesses (3.5%) to improve the capacity to deliver HIV-related services. Grantees provided capacity-building assistance, HIV-related information, and material and financial support to their partners. Overall, 36.7% of CAPUS funds ($15.7 million of $42.8 million) was allocated for contracts with CBOs, which was higher than the required 25% minimum. In turn, these CBOs created opportunities to more effectively reach underserved populations, share knowledge about community needs and resources, and engage in the planning, implementation, and evaluation of project activities.

Discussion

Addressing the long-standing challenges to implementing programs that reduce racial/ethnic disparities in HIV-related outcomes requires innovative approaches that involve collaborative programming and integrated service delivery. CAPUS was a unique demonstration project that supported grantees and their community partners to implement previously overlooked or new HIV prevention strategies (eg, NLRR services, Data to Care) while implementing activities to address the effects of social and structural factors that may contribute to HIV-related disparities. Our findings suggest that health departments and their partners can implement integrated HIV prevention approaches, demonstrate measurable improvements in access to services, and potentially contribute to long-term impacts on HIV-related health outcomes for racial/ethnic minority groups. CAPUS provides a model that may be useful for other state and local health departments that aim to address HIV-related racial/ethnic disparities in their jurisdictions. Although our cross-site analysis did not permit us to determine the impact on the incidence of HIV or viral suppression, the short-term outcomes and additional findings reported by grantees^25,26 and observed in other similar projects^27
-29 suggest that CAPUS could potentially have these desired long-term effects, particularly among racial/ethnic minority groups.

A Diversity of Grantee Activities

CAPUS allowed grantees to implement diverse approaches that were designed to meet the unique HIV prevention and care needs of their target populations. For HIV testing, grantees focused on expanding testing sites in underserved areas, raising awareness about the importance of testing and, for some, adopting new testing technologies. For NLRR programs, grantees focused on identifying unmet needs that prevented PWH from accessing and staying in care, integrating NLRR activities with existing care systems (eg, Ryan White HIV/AIDS Program services), and facilitating and sustaining delivery of comprehensive HIV-related services. Implementing Data to Care programs required grantees to improve data quality; integrate surveillance, care, and prevention data; and modify policies and procedures for using and sharing data.

Activities to mitigate the negative effects of social and structural factors on the use of HIV services included building capacity to improve service delivery, implementing community-level interventions to reduce stigma and improve awareness, and expanding access to and provision of comprehensive services for racial/ethnic minority groups. The diversity of approaches reflected health departments’ and their partners’ awareness of their target populations’ HIV prevention and care needs and the strategies to meet those needs. Taken together, the diversity of activities and the flexibility allowed by the project laid the foundation for grantee programs’ ability to meet the goals of CAPUS and for program models that other health departments might adopt to support the objectives of the National HIV/AIDS Strategy.^6,7

Reaching Priority Populations

CAPUS grantees conducted more than 155 000 HIV tests, reached more than 10 000 PWH through NLRR programs, and successfully prioritized racial/ethnic minority groups. Grantees exceeded their own targets, suggesting that their performance on these programs was better than expected, perhaps because of the synergy created by the integrated and collaborative delivery of services. HIV tests from non-Hispanic black and Latino people accounted for 66.8% of all tests, and 77.8% of all people who participated in NLRR programs were non-Hispanic black or Latino. Through Data to Care programs, nearly 5000 PWH presumed to be not in care were contacted for linkage to or reengagement with HIV care. Social and structural interventions, including provider training, social marketing and social media campaigns, and efforts directed at improving service delivery reached large numbers of racial/ethnic minority groups, service providers, and communities with messages and strategies that promoted HIV prevention and care. Grantees’ success in reaching priority populations may be attributed to their use of strategies that are known to be effective, including data-driven program planning, community engagement, health system navigation, and social marketing and social media campaigns tailored to specific populations.³⁰

Short-term Program Outcomes

Outcomes of grantees’ HIV testing programs appeared to be influenced by the type of test setting and recruiting strategies used to reach priority populations. Overall, newly identified HIV positivity rates were 0.32% for tests in health care settings and 0.54% for tests in non–health care settings. These results were similar to HIV positivity rates among CDC-funded tests conducted in similar settings nationally.³¹ However, improved targeted testing strategies may continue to be warranted in CAPUS jurisdictions where racial/ethnic minority groups are disproportionately affected by HIV. The high rate of newly identified HIV positivity (3.2%) found by 1 grantee’s community-based testing program—which used a social network strategy with young non-Hispanic black MSM—was consistent with the literature.^32,33 This finding suggests that the success of community-based testing depends in part on the appropriateness of the recruitment strategy and on its capacity to address social and structural barriers to testing for priority populations.

Linkage to or reengagement of PWH with HIV care was the primary outcome of NLRR programs. Among people who were never in care or were lost to care, the linkage or reengagement rate after NLRR services was 77.3%. Although this rate was lower than the 84.3% linkage-to-care rate for people newly diagnosed with HIV nationally in 2015,³ it is reasonably high, considering that NLRR programs targeted underserved PWH with complex health and social challenges who must overcome multiple barriers to accessing HIV care. Also noteworthy is the finding that linkage and reengagement rates for racial/ethnic minority groups, particularly non-Hispanic black people, were similar to or better than linkage and reengagement rates for non-Hispanic white people. This finding suggests that targeted NLRR programs, as found in other studies,^27
-29 may reduce HIV-related disparities through their focus on providing comprehensive services and addressing multiple social and structural barriers to HIV care.

Outcomes of CAPUS-supported Data to Care activities highlight the challenges and opportunities associated with implementing the Data to Care strategy. Consistent with other reports,^34
-36 one-third (36.6%) of investigated PWH who were presumed to be not in care were truly not in care. The fact that most PWH (61.4%) were already in care indicates that surveillance and other data systems queried did not capture data to accurately determine care status. Incomplete or delayed reporting of laboratory data to surveillance and challenges in integrating surveillance, care, and prevention data have been identified as the major challenges to Data to Care programs.^34
-36 This finding suggests that continued improvements are needed in data completeness, data integration, and data sharing among surveillance, care, and prevention programs to enhance the efficiency of Data to Care efforts. It was promising, however, that a high percentage (82.4%) of people confirmed to be not in care were linked to or reengaged with HIV care after a Data to Care–initiated linkage or reengagement service. Consistent with previous reports,^34,35 CAPUS grantees were able to strengthen their surveillance and other data systems once they identified the gaps in their data (eg, incomplete demographic, locating, or care information) and were able to integrate updated information collected during follow-up investigations.

Although cross-site monitoring and evaluation data on the outcomes of grantees’ initiatives to address HIV-related social and structural factors were not collected, some grantee-specific local evaluations^25,26 suggest that improved availability, accessibility, or acceptability of HIV prevention and care services among racial/ethnic minority groups can be attributed, in part, to efforts to address these factors during the project. CAPUS allowed the delivery of comprehensive services along the HIV care continuum in a manner similar to HIV medical homes, which are designed to address the challenges of fragmented care systems and the complexity and social contexts of HIV.³⁷

Limitations

Our monitoring and evaluation approach had several limitations. First, the qualitative analysis focused solely on identifying key implemented activities. A more detailed qualitative analysis could potentially have identified more themes than those reported here. Second, the aggregate quantitative data requirements for CAPUS cross-site monitoring and evaluation were limited because of concerns about grantee data-reporting burden, marked variability in grantee programs, and limited evaluation experience with the new program components. For example, data on unmet needs; the behavioral risk of all people tested; barriers to providing prevention, care, and support services; and clinical outcomes (eg, retention in care, viral suppression) would have improved our evaluation of the project’s outcomes. As a result, the nuances of various CAPUS programs across grantees were not captured adequately through the limited aggregate data collected.

Third, the cross-site monitoring and evaluation approach was not designed to distinguish outcomes for multiple interconnected activities or to compare program effectiveness across grantees. Evaluation designs, such as those that involve before and after or CAPUS site and non–CAPUS site comparisons, could have yielded useful information about program effectiveness. Analysis of grantee-specific data will improve our understanding of the processes and outcomes of implemented programs. In addition, triangulating³⁸ surveillance, care, and prevention program data from across grantees is expected to answer additional outcome evaluation questions about the impact of CAPUS on HIV prevalence, health outcomes for PWH, and the sustainability of effective programs. Lastly, our cross-site monitoring and evaluation design did not enable us to conduct cost analyses of CAPUS programs. Although such analyses are important, the heterogeneous nature of the implemented programs and the potential burden to collect expenditure data across grantees made such analyses unfeasible.

Conclusions

The cross-site monitoring and evaluation findings show that CAPUS support allowed grantees to substantially expand their capacity to deliver HIV-related services and reach predominantly racial/ethnic minority groups at risk for, or living with, HIV infection. Furthermore, grantees demonstrated the feasibility of implementing new HIV prevention strategies, including navigation, Data to Care, and programs to address HIV-related social and structural factors. Assessment of short-term outcomes indicated that grantees successfully identified previously undiagnosed HIV infections, linked or reengaged PWH who were not in care, intervened on several social and structural barriers, and facilitated access to a range of prevention and support services. Further analysis is warranted to identify and disseminate best practices and models of HIV prevention and care and assess the full contributions of CAPUS to meeting the objectives of the National HIV/AIDS Strategy.

Footnotes

Authors’ Note

The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of CDC.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Care and Prevention in the United States (CAPUS) Demonstration Project was supported by the US Department of Health and Human Services Secretary’s Minority AIDS Initiative Fund and led by the Centers for Disease Control and Prevention (CDC) (PS 12-1210).

References

Centers for Disease Control and Prevention. Diagnoses of HIV infection in the United States and dependent areas, 2016. HIV Surveill Suppl Rep. 2017;28:1–125.

Dailey

Hoots

Hall

. Vital signs: human immunodeficiency virus testing and diagnosis delays—United States. MMWR Morb Mortal Wkly Rep. 2017;66(47):1300–1306.

Centers for Disease Control and Prevention. Monitoring selected national HIV prevention and care objectives by using HIV surveillance data—United States and 6 dependent areas, 2015. HIV Surveill Suppl Rep. 2017;22(2):1–63.

Buot

Docena

Ratemo

. Beyond race and place: distal sociological determinants of HIV disparities. PLoS One. 2014;9(4):e91711.

Centers for Disease Control and Prevention. Program collaboration and service integration: enhancing the prevention and control of HIV/AIDS, viral hepatitis, sexually transmitted diseases, and tuberculosis in the United States. Published 2009. https://www.cdc.gov/nchhstp/programintegration/docs/207181-c_nchhstp_pcsi-whitepaper-508c.pdf. Accessed September 22, 2018.

White House Office of National AIDS Policy. National HIV/AIDS strategy for the United States. July 2010. https://www.hiv.gov/sites/default/files/nhas.pdf. Accessed September 22, 2018.

White House Office of National AIDS Policy. National HIV/AIDS strategy for the United States: updated to 2020. 2015. https://www.hiv.gov/sites/default/files/nhas-update.pdf. Accessed September 22, 2018.

Centers for Disease Control and Prevention. The Care and Prevention in the United States (CAPUS) Demonstration Project. https://www.cdc.gov/hiv/research/demonstration/capus/index.html. Accessed September 22, 2018.

Centers for Disease Control and Prevention. Data to Care: using HIV surveillance data to support the HIV care continuum. 2013. https://effectiveinterventions.cdc.gov/docs/default-source/data-to-care-d2c/pdf-of-important-considerations.pdf2013. Accessed September 22, 2018.

10.

Centers for Disease Control and Prevention. Funding opportunity announcement (FOA) PS12-1201: comprehensive human immunodeficiency virus (HIV) prevention programs for health departments. https://www.cdc.gov/hiv/funding/announcements/ps12-1201/index.html. Accessed September 22, 2018.

11.

Centers for Disease Control and Prevention. Enhanced comprehensive HIV prevention planning and implementation for metropolitan statistical areas most affected by HIV/AIDS. https://www.cdc.gov/hiv/research/demonstration/echpp/2010. 2017. Accessed September 22, 2018.

12.

Craig

Dieppe

Macintyre

Michie

Nazareth

Petticrew

. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.

13.

Ling

. Evaluating complex and unfolding interventions in real time. Evaluation. 2012;18(1):79–91.

14.

Rogers

. Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation. 2008;14(1):29–48.

15.

Fenton

Aquino

Dean

. Program collaboration and service integration in the prevention and control of HIV infection, viral hepatitis, STDs, and tuberculosis in the U.S.: lessons learned from the field. Public Health Rep. 2014;129(suppl 1):1–4.

16.

Moore

Audrey

Barker

. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.

17.

National Alliance of State and Territorial AIDS Directors. Reduction of reporting burden for health departments: HIV/AIDS, viral hepatitis and related programs. Published 2014. https://www.nastad.org/resource/reduction-reporting-burden-health-departments-hivaids-viral-hepatitis-and-related-programs. Accessed September 22, 2018.

18.

Kim

Sefcik

Bradway

. Characteristics of qualitative descriptive studies: a systematic review. Res Nurs Health. 2017;40(1):23–42.

19.

Sandelowski

. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–340.

20.

Owen

. Testing for acute HIV infection: implications for treatment as prevention. Curr Opin HIV AIDS. 2012;7(2):125–130.

21.

Sweeney

Hoyte

Mulatu

. Implementing a Data to Care strategy to improve health outcomes for people with HIV: a report from the Care and Prevention in the United States Demonstration Project. Public Health Rep. 2018;133(suppl 2):60S–74S.

22.

North Carolina Department of Health and Human Services. Health equity and HIV in North Carolina, 2016: gay, bisexual, and other men who have sex with men. Published 2017. https://epi.publichealth.nc.gov/cd/stds/figures/factsheet_healthequityHIV_MSM_2016_rev2.pdf. Accessed September 22, 2018.

23.

Virginia Department of Health. CAPUS Demonstration Project. 2015. http://www.vdh.virginia.gov/content/uploads/sites/10/2017/02/CAPUS-Fact-Sheet-2.pdf. Accessed September 22, 2018.

24.

Gruber

. Addressing institutional racism, transphobia, and homophobia in our HIV/AIDS work. 2016. https://targethiv.org/sites/default/files/supporting-files/PlenaryWedGruber.pdf. Accessed September 22, 2018.

25.

Bickham

. SMAIF in action: Louisiana Links program finds “missing” people living with HIV to keep them and others healthy. https://www.hiv.gov/blog/smaif-action-louisiana-links-program-finds-missing-people-living-hiv-keep-them-and-others. August 7, 2017. Accessed September 22, 2018.

26.

Brantley

Burgess

Bickham

Wendell

Gruber

. Using financial incentives to improve rates of viral suppression and engagement in care of patients receiving HIV care at 3 health clinics in Louisiana: the Health Models program, 2013-2016. Public Health Rep. 2018;133(suppl 2):75S–86S.

27.

Bradford

Coleman

Cunningham

. HIV system navigation: an emerging model to improve HIV care access. AIDS Patient Care STDS. 2007;21(suppl 1):S49–S58.

28.

Shacham

Lopez

Brown

Tippit

Ritz

. Enhancing adherence to care in the HIV care continuum: the Barrier Elimination and Care Navigation (BEACON) project evaluation. AIDS Behav. 2018;22(1):258–264.

29.

Sullivan

Schultz

Ramaiya

Berger

Parnell

Quinlivan

. Experiences of women of color with a nurse patient navigation program for linkage and engagement in HIV care. AIDS Patient Care STDS. 2015;29(suppl 1):S49–S54.

30.

Health Resources and Services Administration, HIV/AIDS Bureau. Innovative Approaches to Engaging Hard-to-Reach Populations Living With HIV/AIDS Into Care: Training Manual. Rockville, MD: Health Resources and Services Administration, HIV/AIDS Bureau; 2013.

31.

Centers for Disease Control and Prevention. CDC-funded HIV testing: United States, Puerto Rico and U.S. Virgin Islands, 2014. https://www.cdc.gov/hiv/pdf/library/reports/cdc-hiv-funded-testing-us-puerto-rico-2014.pdf. Published 2016. Accessed September 22, 2018.

32.

Halkitis

Kupprat

McCree

. Evaluation of the relative effectiveness of three HIV testing strategies targeting African American men who have sex with men (MSM) in New York City. Ann Behav Med. 2011;42(3):361–369.

33.

McCree

Millett

Baytop

. Lessons learned from use of social network strategy in HIV testing programs targeting African American men who have sex with men. Am J Public Health. 2013;103(10):1851–1856.

34.

Dombrowski

Bove

Roscoe

. “Out of care” HIV case investigations: a collaborative analysis across 6 states in the northwest US. J Acquir Immune Defic Syndr. 2017;74(suppl 2):S81–S87.

35.

Hague

John

Goldman

. Using HIV surveillance laboratory data to identify out-of-care patients [published online March 6, 2017]. AIDS Behav. doi:10.1007/s10461-017-1742-5.

36.

Udeagu

Webster

Bocour

Michel

Shepard

. Lost or just not following up: public health effort to re-engage HIV-infected persons lost to follow-up into HIV medical care. AIDS. 2013;27(14):2271–2279.

37.

Pappas

Yujiang

Seiler

. Perspectives on the role of patient-centered medical homes in HIV care. Am J Public Health. 2014;104(7):e49–e53.

38.

Rutherford

McFarland

Spindler

. Public health triangulation: approach and application to synthesizing data to understand national and local HIV epidemics. BMC Public Health. 2010;10:447.