Abstract
Objective:
Research synthesis, through qualitative or quantitative systematic reviews, allows for integrating results of primary research to improve public health. We examined more than 2 decades of work in HIV prevention by the Centers for Disease Control and Prevention’s (CDC’s) HIV/AIDS Prevention Research Synthesis (PRS) Project. We describe the context and contributions of research synthesis, including systematic reviews and meta-analyses, through the experience of the PRS Project.
Methods:
We reviewed PRS Project publications and products and summarized PRS contributions from 1996 to July 2020 in 4 areas: synthesis of interventions and epidemiologic studies, synthesis methods, prevention programs, and prevention policy.
Results:
PRS Project publications summarized risk behaviors and effects of prevention interventions (eg, changing one’s perception of risk, teaching condom negotiation skills) across populations at risk for HIV infection and intervention approaches (eg, one-on-one or group meetings) as the HIV/AIDS epidemic and science evolved. We used the PRS Project cumulative database and intervention efficacy reviews to contribute to prevention programs and policies through identification of evidence-based interventions and development of program guidance. Subject matter experts and scientific evidence informed PRS Project products and contributions, which were implemented through strategic programmatic partnerships.
Conclusions:
The contributions of the PRS Project to HIV prevention and public health efforts in the United States can be credited to CDC’s long-standing support of the project and its context within a federal prevention agency, where HIV programs and policies were developed and implemented. The effect of the PRS Project was likely facilitated by opportunities to directly influence program and policy because of connections with other research translation activities and program and policy decision making within CDC.
The Centers for Disease Control and Prevention (CDC) uses public health principles and methods such as research synthesis to understand and control infectious diseases, such as HIV, and to evaluate its work. 1 Research synthesis allows users to integrate results of primary research, via rigorous qualitative or quantitative (meta-analytic) systematic reviews, and to provide evidence for public health action. 2
In 1996, CDC’s Division of HIV/AIDS Prevention (DHAP) initiated the HIV/AIDS Prevention Research Synthesis (PRS) Project. 3 The PRS Project aims to capture findings from the growing HIV prevention research literature, synthesize characteristics and effects of behavioral interventions, identify evidence-based interventions (EBIs) and best practices, and provide data to support policy-related decisions.3,4 We reviewed nearly a quarter-century (1996-2020) of PRS Project synthesis of HIV prevention research from 1988 to 2020.
In summarizing PRS Project contributions to evidence-based HIV prevention research, synthesis, programs, policy, and synthesis methodologies, we aimed to illustrate how the PRS Project contributed to the overall public health, HIV prevention knowledge, and action, as well as to reflect on lessons learned during the past 30 years. Such a summary is important to most effectively implement many federal HIV efforts and for the discipline and science of synthesis.
Methods
We conducted a narrative review of PRS Project publications and products from 1996 to July 2020 that we categorized into 1 of 4 areas: (1) synthesis of behavioral interventions and epidemiologic studies, focused on qualitative or quantitative systematic reviews; (2) HIV prevention programs, focused on identifying and disseminating EBIs and best practices for selection and implementation by CDC grantees, particularly health departments and community-based organizations; (3) research synthesis methods, focused on search methodology, reporting of data in primary research studies, and analytical or statistical approaches; and (4) prevention policy, focused on contributing to public health guidelines and national strategies. Our data include PRS Project publications and products, which we tabulated for each focal area, with a highlight on several key PRS Project products in each focal area and their contributions to HIV prevention. Our objective was to show how the application and impact of a research synthesis project operating within a federal agency contributed to improving prevention and addressing public health needs.
Results
Synthesis of Behavioral Interventions and Epidemiologic Studies
The mission of PRS—to advance HIV prevention through research synthesis—continues to guide the daily operations and strategic directions of the PRS Project. The cumulative up-to-date PRS Project database supports synthesis of HIV intervention research. The database, which is compiled through annual electronic and manual searches for interventions for HIV and other sexually transmitted diseases, contains more than 100 000 citations (January 1988–June 2020), facilitates comprehensiveness of the systematic reviews, and limits bias in study identification, thus enhancing the validity of results.
A total of 50 synthesis publications that focused on HIV prevention across populations at high risk for HIV infection or transmission and examined both behavioral interventions for risk reduction and HIV care outcomes and other epidemiologic studies were published from January 1988 through June 2020. These articles include 36 publications led by the PRS Project (Table 1)5 -40 and 14 publications supported by the PRS Project (ie, publications that were produced by the PRS Project in collaboration with colleagues, used the PRS Project database as a literature source, or used PRS Project methodology) (Table 2).41 -54 Early PRS Project–led publications (n = 18) focused on synthesizing behavioral interventions to reduce risky behaviors (eg, sexual and drug-use behavior) primarily among HIV-negative people at high risk of HIV infection, including men who have sex with men (MSM), adolescents, heterosexual adults, drug users, and partners of members of these populations at risk for HIV infection (Table 1).5 -22 A few reviews also included people with HIV (n = 3).23 -25 Later, as the science of HIV prevention and treatment evolved, the focus of the articles expanded to interventions that promote outcomes along the HIV care continuum (n = 9), including interventions to improve linkage to or retention in HIV care, medication adherence, and the relationship between retention in HIV care and mental health.26 -34
PRS Project–led synthesis publications (systematic reviews or meta-analyses) of behavioral interventions for risk reduction or HIV care outcomes and epidemiologic or other types of studies, by publication date (n = 36), 2002-2020 a
Abbreviation: PRS, Prevention Research Synthesis.
The PRS Project aims to capture findings from the HIV prevention research literature, synthesize characteristics and effects of behavioral interventions, identify evidence-based interventions and best practices, and provide data to support policy-related decisions. 3
PRS Project–supported synthesis publications (systematic reviews or meta-analyses) of behavioral interventions for HIV risk reduction and epidemiological studies (n = 14), by publication date, 2003-2018 a
Abbreviation: PRS, Prevention Research Synthesis.
The PRS Project aims to capture findings from the HIV prevention research literature, synthesize characteristics and effects of behavioral interventions, identify evidence-based interventions and best practices, and provide data to support policy-related decisions. 3
Most recently, the PRS Project led reviews on epidemiologic and other studies (n = 6) that focused on identifying the estimate of HIV prevalence and risk behaviors for transmen and transwomen 37 and the use of preexposure prophylaxis (PrEP) among indicated populations (Table 1).36,40 The PRS Project also synthesized studies on PrEP, MSM, and structural interventions.35,38,39 The syntheses of structural interventions produced the first comprehensive taxonomy in the HIV literature to categorize structural interventions, that is, interventions that alter the environment to facilitate or promote safe behaviors or reduce risk in ways that do not rely on individual change. 35 PRS Project–supported publications followed similar patterns, with 7 publications that synthesized HIV risk-reduction interventions, 1 publication that focused on HIV care outcomes, and 7 publications that synthesized epidemiologic studies (Table 2).41 -54
Although varied in focus, PRS Project publications mirrored the evolving HIV surveillance and epidemiologic data, research priorities, and prevention and treatment science. Specifically, these systematic reviews and meta-analyses summarized results of primary studies, explored relationships between interventions and behavioral and health outcomes, and combined individual study results to estimate pooled intervention effects for various types of interventions or population estimates. Key findings included the efficacy of behavioral interventions for reducing sexual risk for MSM,41 -43 annual HIV testing for MSM,46,55 the efficacy of multisession psychosocial interventions for reducing sexual risk for people who use drugs, 45 and the benefits of electronic health (eHealth) interventions (eg, health interventions delivered by cell phone, internet, web, app, or other electronic media) that improve provider–patient communication to improve care outcomes. 47 The epidemiologic systematic reviews aimed to estimate denominators for groups at risk for HIV,50,52,53 better understand HIV transmission risk,49,51,54 and highlight HIV epidemiology for Asian/Pacific Islander people. 48 Although these PRS Project–supported publications varied in focus, they mirrored evolving HIV surveillance and epidemiologic data and research priorities.
Prevention Programs
Since 1996, the PRS Project has been a critical part of CDC DHAP’s research translation effort in moving science to practice. Toward this end, the PRS Project conducts annual efficacy reviews of intervention studies to identify EBIs and best practices with demonstrated scientific evidence of efficacy for both prevention and care outcomes for populations at risk for HIV acquisition or transmission. The PRS Project disseminates information on EBIs through the continuously updated PRS Compendium of Evidence-Based Interventions and Best Practices for HIV Prevention (hereinafter, the Compendium) 56 (Table 3).4,57 -63
PRS Project publications of prevention programs (n = 8), by publication date, 1999-2012 a
Abbreviation: PRS, Prevention Research Synthesis.
The PRS Project aims to capture findings from the HIV prevention research literature, synthesize characteristics and effects of behavioral interventions, identify evidence-based interventions and best practices, and provide data to support policy-related decisions. 3
The PRS Project process for identifying EBIs uses comprehensive literature searching, rigorous screening methods, and a priori, scientifically determined criteria to assess the quality of study design, implementation, analysis, and strength of intervention effects for HIV-related behavioral and biological outcomes.61,64 -67 The efficacy criteria, refined and finalized in consultation with methodologists and subject matter experts, accommodate the complexity and variations in study designs, evaluation methods, analytic approaches, and data reporting. Coders, who conduct annual screening and data abstraction for the PRS database, receive annual booster trainings and undergo periodic quality assurance checks of their work. Coders screen abstracts and full reports to determine eligibility for reviews and abstract data for qualitative and quantitative analyses. PRS’s first Compendium was initiated in 1996 and reviewed intervention studies published during 1988-1996. 57 Subsequent updates included interventions published through 2000 59 and through 2004. 62 Since 2007, the Compendium has been updated annually and is published online. 56
As of June 30, 2020, the Compendium described 197 EBIs and best practices in 5 chapters of prevention interventions (sequentially initiated in line with evolving HIV surveillance and epidemiologic data and prevention advances). These chapters include 108 EBIs on HIV sex- and drug-risk reduction, 19 EBIs on adherence to HIV medications, and 31 best practices (EBIs and evidence-informed strategies) for linkage to, retention in, and reengagement in HIV care. 56 The PRS Project added a chapter on structural interventions 67 (36 best practices) in 2018 and on PrEP (3 best practices) in 2020. 68
DHAP uses the Compendium as a key source when prioritizing further research translation activities focusing on dissemination, promoting adoption, and supporting the uptake and implementation of EBIs and best practices. Several DHAP funding announcements included information on the Compendium to assist DHAP grantees in implementing interventions that reflected the best available science69,70 as part of a comprehensive approach to HIV prevention. The Compendium serves as a resource for state and local health departments, community-based organizations, and other prevention providers when selecting EBIs that fit best within state or local HIV prevention plans.
To support the implementation of EBIs and best practices identified in the Compendium, DHAP’s Replicating Effective Programs63,71 (1996-2013) focused on translating science into materials for prevention agencies. DHAP funded primary researchers and community partners to translate scientific protocols into user-friendly materials to facilitate adoption and implementation of EBIs within prevention partners’ jurisdictions. 58 These intervention implementation materials appeared in various mediums that facilitated program implementation, including science-based curricula, videos, and online information. DHAP’s efforts to enable widespread uptake and implementation of EBIs with fidelity, through the Diffusion of Evidence-Based Interventions 72 program, consisted of national training, technical assistance, and capacity-building programs tailored to HIV prevention partners across the United States, including state and local health departments and directly funded community-based organizations.72,73 This effort, among the PRS Project, DHAP’s Replicating Effective Programs, and the Diffusion of Evidence-Based Interventions program, together represented the most comprehensive research translation effort for behavioral HIV prevention interventions in the United States during the mid-1990s through mid-2010s.
DHAP’s research translation efforts have evolved to continue to prioritize high-impact prevention interventions and strategies most directly related to DHAP’s updated strategic plan and priorities.74 -77 To reflect the high-impact prevention approach and priorities, the PRS Project has limited the identification of risk-reduction interventions to people with HIV and people most affected by HIV, such as MSM, transgender people, and people who inject drugs, and has added a new chapter on prevention through PrEP. Consistent with a treatment-as-prevention approach, 78 the PRS Project continues to identify EBIs and best practices for HIV care outcomes, including medication adherence, viral suppression, and engagement in HIV care. DHAP’s target audiences have expanded to include HIV care organizations and primary care providers, and DHAP’s interventions have thus expanded to include structural interventions and Data to Care prevention strategies.56,79 Although DHAP continues to provide training, technical assistance, and capacity building to HIV prevention and care partners on a subset of high-impact prevention interventions identified by the PRS Project, 80 the range of available prevention options has expanded, requiring new ways to prioritize interventions for translation, dissemination, and recommendation to partners for use in programs that combine multiple prevention approaches.
PRS Project systematic reviews differ from efficacy reviews by synthesizing information from across the pool of intervention studies to produce generalizable findings. PRS Project systematic reviews supplement efficacy reviews by identifying which kinds of interventions or prevention approaches work or which characteristics help interventions work best. For example, systematic reviews have demonstrated evidence for condom distribution interventions, 18 structural interventions, 35 eHealth interventions, 47 and HIV patient navigation. 31 PRS Project systematic reviews and meta-analyses have identified which approaches or methods may work best among certain risk groups or populations (eg, MSM,5,10,13,41 -43 people with HIV,6,25,27,29 African American people)16,17,19 or how best to implement types of interventions (eg, retention/reengagement), 30 providing evidence for structural interventions 18 or identifying new technological approaches to supplement biomedical and individual-level interventions. 47
Research Synthesis Methods
The PRS Project was one of the first synthesis projects in HIV prevention, and assistance from external (non-CDC) experts in the HIV prevention field allowed it to contribute quickly and rigorously to the knowledge on effective interventions, including synthesis of the first generation of evaluation studies on risk-reduction intervention. 81 The PRS Project developed and implemented procedures for all aspects of systematic reviews, based on well-established research synthesis methods,2,82 ranging from systematic searches, to screening for eligibility, to data abstraction, to analyses. PRS Project publications on search strategies highlighted the importance of comprehensive systematic searches for all available and relevant information and the utility of complete and transparent reporting of search strategies.83,84 Relying on methodological consultants and existing quantitative analytic methods, the PRS Project established and implemented data analytical methods and procedures, including meta-analyses, to combine effect sizes across pooled studies within systematic reviews.2,82,85
The initial embrace of research synthesis by the fields of education, psychology, and medicine, followed by public health,86,87 allowed the PRS Project to grow rapidly as it used techniques from these fields. The more than 40-year discipline of research synthesis88 -90 enabled the use of a common language among various domains, thereby facilitating the PRS Project’s interdisciplinary synthesis work and its consultations and collaborations with federal and academic partners. 91 Publications on PRS Project methods (Table 4) demonstrate application of several principles, including substantive (HIV prevention interventions), conceptual (eg, internal, external, and construct validity), and statistical (eg, quality of interventions, combining results, and heterogeneity) principles. The 13 publications on methods cover several domains: search methods,83,84,92,93 reporting standards for primary studies,94 -97 and analytical or statistical methods3,81,85,98,99 (Table 4). These publications included and contributed to HIV synthesis methodology for HIV prevention interventions and reflect advances in synthesis methods and techniques.87,90 The conduct of high-quality systematic reviews depends on the reporting quality of primary studies. To improve the quality of reporting, the PRS Project developed a 22-item reporting checklist, the Transparent Reporting of Evaluations with Non-randomized Designs (TREND), 94 after a 2-day meeting in 2003 with editors and representatives of 18 HIV-focused journals. 94 Fuller et al 100 reported that, during 2004-2013, the TREND publication 94 was cited 412 times and the use of TREND was associated with more comprehensive reporting and higher quality ratings than publications that cited but did not use TREND, illustrating the value of using standard reporting guidelines. 101
PRS Project–led publications on research synthesis methods (n = 13), by publication date, 2002-2014 a
Abbreviation: PRS, Prevention Research Synthesis.
The PRS Project aims to capture findings from the HIV prevention research literature, synthesize characteristics and effects of behavioral interventions, identify evidence-based interventions and best practices, and provide data to support policy-related decisions. 3
Another contribution was developing a systematic and principle-based set of decision rules guiding analytical procedures to address variations in the types of effect sizes for different outcomes (eg, behavioral or biological outcomes) and outcome measures (eg, number of occasions of condomless sex, percentage of times that condoms were used) as reported across studies included in systematic reviews. To address this variation in reporting, the PRS Project developed for its first syntheses of sexual risk-reduction interventions, published in 2002, 98 a hierarchical outcome construct of risk-reduction behaviors. The construct reflected the association of various behaviors with probabilities of risk of HIV infection and abided by principles of construct validity, internal validity, and external validity. Specifically, outcomes measuring condomless sex, condom use, and number of partners were selected in this hierarchical order. 85 This application was also important for statistical reasons, specifically, to ensure that results of a particular meta-analysis included, as required statistically, only independent data from multi-outcome primary studies. 85 Furthermore, the construct provided several advantages (validated by sensitivity analyses for the 3 outcome behaviors). 8 The construct allowed the PRS Project to conduct meta-analyses (rather than qualitative systematic reviews only), assess the effects of risk-reduction interventions for several populations at risk for HIV,5,7,8,85,98 compare synthesis results across various reviews (eg, reviews examining multiple populations at risk for HIV), 81 address HIV prevention needs, and quantify the effect of US-based HIV intervention studies on psychosocial risk reduction. 102 The PRS Project was the first to develop this hierarchical outcome construct for synthesis of behavioral interventions on HIV risk reduction, which has been adopted by both PRS Project6 -8,10 and non–PRS Project investigators.45,103,104
Furthermore, the PRS Project used statistical formulas designed for the meta-analysis of cross-sectional screening and diagnostic tests 105 to permit conversion of effect sizes between outcomes with continuous formats (ie, standardized mean differences) and outcomes with dichotomous formats (ie, log odds ratios). 85 This approach allowed integration of 2 types of outcome metrics, thereby increasing the number of eligible intervention studies in a single meta-analysis. 85 In addition, the PRS Project was the first synthesis project to use additional analytical methods 85 followed by other researchers who adjusted for baseline differences 18 or intraclass correlation.106,107 In the dissemination phase, PRS Project publications adhered to reporting guidelines, such as those of Preferred Reporting Items for Systematic Reviews and Meta-Analyses 108 ; use of complex statistical methods, such as those of meta-regression 109 ; and application of advanced digital and software applications for data management and analysis.110 -112
PRS Project meta-analyses provided overall robust estimates for the effectiveness of intervention categories by combining estimates from relevant publications identified in systematic reviews. Meta-analyses result in estimates with greater precision and accuracy than estimates reported in any individual study because of combining data across studies. Systematic reviews and meta-analyses also provide prevalence and risk estimates for mathematical modeling and allow for estimating the overall and relative effect of various types of interventions.113 -116 One of the first cost-threshold estimates of risk-reduction interventions for people who use drugs 117 provided results consistent with effectiveness findings reported in an article synthesizing risk-reduction interventions for this population.8,45
Prevention Policy
Evidence-based policy and policy-focused products reflect a culmination of decades of science-based clinical and public health interventions. 118 Many decision makers include results of research synthesis in their development of guidelines, policies, and prevention strategies. 119 The PRS Project published early on this relationship between synthesis and guidelines, policies, and strategies 102 and has supported DHAP in producing policy-related documents and collaborating with federal partners and national professional organizations on the development and publication of HIV prevention guidelines and recommendations (Table 5).55,77,102,120 -123 For example, the PRS Project cumulative database provided the literature source for the International Association of Providers of AIDS Care guidelines for optimizing clinical care outcomes, 122 improving entry into and retention in HIV care, 120 and HIV home and self-testing 123 and provided the basis for the adherence and risk-reduction chapters of the multiagency national guidelines on HIV prevention for adults and adolescents living with HIV. 121 In addition, collaborations with federal partners such as the National Institute of Mental Health and the Health Resources and Services Administration to identify PRS criteria for determining evidence-based medication adherence interventions 65 had wide impact. For example, the National Institute of Mental Health subsequently incorporated these same criteria for strong evidence (ie, longer required length of follow-up and positive outcomes on both behavioral and biomedical outcomes) into requirements for its next round of funded interventions on medication adherence. 124
Publications of synthesis and policy of various origins, including PRS Project–led publications, PRS Project–supported publications, and partnerships and collaborations (n = 7), by publication date, 2002-2019 a
Abbreviation: PRS, Prevention Research Synthesis.
The PRS Project aims to capture findings from the HIV prevention research literature, synthesize characteristics and effects of behavioral interventions, identify evidence-based interventions and best practices, and provide data to support policy-related decisions. 3
Furthermore, several PRS Project systematic reviews have been used to support recommendations for practice by DHAP60,77 and the Community Preventive Services Task Force.13,14 The PRS Project also contributed to the systematic review of various prevention strategies and their estimates of effectiveness in reducing HIV risk, 125 and these estimates are used by DHAP to communicate up-to-date prevention information through an interactive, online educational tool for consumers. 126
Future Role of the PRS Project
The PRS Project is continually evolving. Despite the availability of new and effective prevention strategies, the number of new HIV infections in the United States has been stable in recent years. 127 Although stable HIV incidence despite increasing numbers of people living with HIV indicates a drop in transmission rates overall, 128 increased HIV incidence in selected populations indicates that not all people are benefiting equally from new options for prevention. In light of increasing interest in implementation research, to increase reach and uptake of effective prevention, the PRS Project now includes implementation-related findings on the information sheets for EBIs and evidence-informed strategies posted on the Compendium. 56 In addition, current work includes synthesizing behavioral aspects of biomedical interventions (eg, PrEP interventions), interventions that address social and structural barriers (eg, racism and stigma) that contribute to HIV disparities,129,130 and studies evaluating interventions that use surveillance data to better target prevention services for people not in HIV care or not attaining HIV viral suppression (eg, Data to Care). With the fast pace of new developments in testing and prevention, increasing support for systematic reviews will be needed as CDC DHAP updates its clinical guidance.
Advances in HIV prevention also provide the opportunity to consider the limitations of the PRS Project and realign the project to meet future needs. To improve the accessibility of Compendium findings, an upgraded search feature on the Compendium website was recently launched that allows users to easily search for effective interventions based on strategy, outcome, or population. 131 The upgraded PRS database—currently located on internal servers—is not accessible to outside researchers, thereby limiting its broader use. However, CDC has been working with federal partners to disseminate results of reviews and methodologies and will continue to explore outlets to disseminate the results of PRS Project work.
Conclusion
As we near a quarter-century of the PRS Project, we reflected on its evolution, contributions, and place in advancing HIV prevention through research synthesis and translation. HIV surveillance and epidemiologic data, scientific advances, and program needs guided PRS Project topics. Accordingly, the PRS Project produced work that influenced primary research, synthesis methods, reporting standards, prioritization of prevention interventions, and prevention policies. During the course of the project, PRS incorporated advances in digital applications, implemented rigorous synthesis methods, and adhered to transparent reporting standards.
By using research synthesis as a scientific tool to help advance public health, PRS Project publications and products documented the growth and scope of more than 3 decades of HIV prevention interventions, reflecting national prevention priorities as they changed during 1980-1990 on behavioral, social, and structural interventions; the 2000s on treatment as prevention; and during 2010-2020 on integrated behavioral and biomedical prevention strategies and structural intervention strategies.35,132 The PRS Project will continue to evolve to support synthesis activities that make science more accessible to programs in line with DHAP priorities and principles of high-impact HIV prevention, but also with a new focus on ending the HIV epidemic. 133
Footnotes
Acknowledgements
The authors thank all PRS Project staff members and colleagues who worked on the PRS Project during 1996-2018, as well as Adebukola H. Adegbite, Nicole Crepaz, Steve Flores, Wayne Johnson, Emiko Kamitani, Christina McDonald, Mary Neumann, and Cherie Rooks-Peck for comments on earlier versions of this article.
Authors’ Note
The findings and conclusions of this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention (CDC).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for the HIV/AIDS Prevention Research Synthesis Project, including this article, is provided by the Division of HIV/AIDS Prevention at CDC.
