Abstract
Community event-based surveillance aims to enhance the early detection of emerging public health threats and thus build health security. The Ministry of Health of Vietnam launched a community event-based surveillance pilot program in 6 provinces to improve the early warning functions of the existing surveillance system. An evaluation of the pilot program took place in 2017 and 2018. Data from this evaluation were analyzed to determine which factors were associated with increased detection and reporting. Results show that a number of small, local events were detected and reported through community event-based surveillance, supporting the notion that it would also facilitate the rapid detection and reporting of potentially larger events or outbreaks. The study showed the value of supportive supervision and monitoring to sustain community health worker reporting and the importance of conducting evaluations for community event-based surveillance programs to identify barriers to effective implementation.
C
Signals are deliberately designed to be broad, as the goal is to have a detection system of high sensitivity; signals can be adjusted to optimize sensitivity and specificity in specific settings.8,9 Community-level signals are designed to be simple so that village health workers and community members may participate in community event-based surveillance. Upon notification of signals by the community to the public health authorities, the signals must be verified and triaged by trained public health workers to determine if they truly represent a public health threat before any response is initiated. In Vietnam, once a signal has been verified, it is called an “event”; an event then requires a response from the public health authorities.8,9 Triage and verification of incoming signals are crucial steps to reduce the background noise from non-events and subsequent over-burdening of the response system; it requires a dialogue between commune health stations (CHSs) and district levels.8-10 In addition, the district shares information and updates on reported events with the respective CHS, which in turn shares this information with community health workers in routine meetings.
The Ministry of Health of Vietnam has recognized the need to improve the sensitivity of outbreak detection systems and accelerate progress toward a national capacity to prevent, detect, and respond earlier to events and outbreaks, so as to meet the country's obligations under the International Health Regulations (IHR 2005) and strengthen capacities in the Global Health Security Agenda (GHSA) framework. 11 Accordingly, the ministry of health developed an event-based surveillance program to complement the existing routine indicator-based surveillance system. 10 Signals were developed by a technical working group, which also developed guidelines, training materials, and advocacy tools. The implementation of community event-based surveillance was launched as a Phase 1 pilot program in 2016 in 43 districts in 4 provinces in Vietnam: Quang Ninh and Nam Dinh in the north region, and An Giang and Ba Ria Vung Tau in the south region. 10 Building on the experience of the Phase 1 pilot in the 4 provinces, the ministry of health expanded the pilot program in August 2017 to include 2 additional provinces: 1 from the central highlands region (Dak Nong), and the other from the south central coast region (Binh Thuan) (Phase 2). The process of implementation and evaluation are described in detail in a previously published article. 10
Phase 1 of the pilot program was evaluated in 2017, while Phase 2 was evaluated in 2018, each about 6 months after implementation. The current article presents a detailed analysis of data collected during the evaluation and describes the factors that were associated with increased signal reporting and improved event-to-signal ratio at the community level. The article also describes lessons learned from the pilot implementation; the ministry of health is currently using these lessons in the national scale-up of the community event-based surveillance.
Methods
Surveillance Structure
Vietnam has 4 administrative health regions, each with a regional public health institute (RI) that is responsible for the overall supervision of outbreak surveillance and response. Within each region, provincial preventive medicine centers (PPMCs) lead public health surveillance and response activities within their respective provinces, involving the regional institute for larger events. The provincial preventive medicine centers are supported by 2 lower administrative levels, the district health centers (DHCs) and commune health stations. Commune health stations are generally staffed by a physician, a nurse, and a variable number of village health workers who work largely on a voluntary basis. All outbreaks are ultimately reported upward through each administrative level; however, the urgency with which reporting occurs and the level at which response occurs depends on several characteristics, such as the size, severity, and progression of the event. The district health center and commune health station conduct regular meetings with village health workers to ensure a feedback loop is completed. The provinces and districts for this pilot were selected by the ministry of health in collaboration with the regional institutes based on criteria that included provinces with districts along international borders, mobile populations, accessibility and connectivity, and the presence of willing leadership at provincial preventive medicine centers and district health centers.
Data Collection
Population data were extracted from the 2009 population census and the General Statistics Office of Vietnam. 12 Data collection tools for the evaluation included: (1) a form for data extraction from district-level logbooks and monthly community event-based surveillance summary reports that provided the number, type of signals, and events reported, as well as the time of detection and reporting by district; (2) a simple table sent electronically to pilot provinces to collect the type and number of training sessions and the number of people trained; and (3) an online survey tool with questions aimed at understanding the acceptability and sustainability of the pilot program and to obtain feedback on the training sessions. The online survey also included questions on demographics (eg, age, sex, education level of implementers), barriers faced in implementing community event-based surveillance, and types of key informants detecting and reporting signals. The survey was sent to all village health workers, commune health stations, district health centers, and provincial preventive medicine centers participating in the pilot project. Most online survey questions were based on a Likert scale model. The online survey was used only in Phase 1 provinces.
Data Analyses
The incidence rate for signal reporting was calculated as the number of signals detected from each province and district per 100,000 population for the number of days engaged in signal reporting. In addition, the ratio of events to signals in all provinces (Phase 1 and Phase 2) over time was calculated as events detected per month divided by signals detected per month. Event-to-signal ratio was compared between Phase 1 and Phase 2 provinces for the first 5 months of implementation in each group to understand the effect of introducing a modified signal list during Phase 2 implementation. The median number of signals reported per commune was also calculated to assess the burden of the program on commune health stations.
Data on factors that may have influenced implementation in individual districts were only available from Phase 1 provinces, as these data were collected primarily through the online survey. Individual responses from the survey were aggregated at the district level as a percentage for each variable. The unit of analysis was the district for both univariate and multivariate analyses. Univariate analyses used district demographic data (total population, population density, district type [urban or rural], village health worker density per 1,000 population) and data from the survey tool (type of training, information sources in the community). For the regression analysis, variables included total population, population density, district type (urban or rural), village health worker density per 1,000 population, and the proportion of communities within districts that had community members and teachers as active reporters. Pearson correlation tests were conducted between the continuous explanatory and outcome variables to identify possible associations. For categorical variables,
Results
The pilot implementation of community event-based surveillance was done in 2 phases (Figure 1). Signals that were used in Phase 1 were revised after field evaluation 10 and included in Phase 2 implementation (Table 1). From September 2016 to December 2017, 4,854 signals and 370 events were reported by all provinces participating in both Phase 1 and Phase 2 of the pilot program. Data on the type of event were available for 253 events and included a variety of endemic diseases, vaccine-preventable diseases, and zoonoses (Table 2). The signal incidence rate had a slight downward linear trend over the 16-month period of Phase 1; however, the ratio of events to signals slightly increased during that period, including marked increases at several specific points during that timeframe (Figure 2). These points roughly correspond to 3 specific interactions with implementers: in February 2017, a team from the ministry of health and the US Centers for Disease Control and Prevention (CDC) conducted supportive monitoring of all 4 provinces that included site visits, logbook review, and interviews with select districts and communes. In June 2017, an evaluation of the community event-based surveillance was conducted by a ministry of health/CDC team, which included many discussions with provincial preventive medicine center leaders on community event-based surveillance and ways to improve event-to-signal ratio. Finally, in October 2017, the Phase 1 provinces were asked to begin using the revised signal list for community event-based surveillance implementation in their jurisdiction and some refresher training was done.

Provinces that implemented the event-based surveillance pilot program (Phase 1, black starts, and Phase 2, red stars), Vietnam, September 2016-December 2017.

Signal incidence (solid line, left axis) and event-to-signal ratio (dashed line, right axis) over time. Linear trends are represented by dotted lines and over time show a slight decreasing signal incidence and a slight increasing event-to-signal ratio. Arrows a, b, and c represent the interactions between ministry of health leadership and event-based surveillance implementers, namely the midterm progress check, evaluation, and signal revision, respectively. These data were collected from Phase 1 from September 2016 to December 2017.
List of Signals from Phase 1 and Phase 2 Implementation
This category was reorganized and split into 2 categories for Phase 2.
A signal to detect potential cases of measles was added.
Increased specificity of the signals to decrease spurious reports.
Signal case definition was simplified.
The signal was more clearly elucidated.
List of Events Detected from Phase 1 and Phase 2 Provinces from September 2016 to December 2017
The event-to-signal ratio was found to be significantly higher in the Phase 2 provinces, after the revision of signals, when compared to the first 5 months of community event-based surveillance implementation in Phase 1 provinces (mean 0.12 vs 0.06, respectively;
The median number of signals reported per commune per month for all provinces was 0.3 and ranged from 0.03 to 2.8 signals.
Phase 1 provinces included 31 rural and 12 urban districts. The population sizes of the 43 participating districts varied considerably, ranging from a total population of 4,985 in Co To district (Quang Ninh province) to 345,200 in Cho Moi district (An Giang province). Population densities ranged from 32.7 per km2 in Ba Che (Quang Ninh province) to 5,286 per km2 in Nam Dinh city (Nam Dinh province). Village health worker density ranged from 0.39 per 1,000 population in Long Xuyen (An Giang province) to 4.4 per 1,000 population in Hai Ha (Quang Ninh province) (see Supplemental Material, https://www.liebertpub.com/suppl/doi/10.1089/hs.2018.0066).
A total of 1,633 (22%) of the 7,160 village health workers, 428/475 commune health stations (90%), 39/43 district health centers (91%), and all 4 Phase 1 provincial preventive medicine centers completed the online survey by June 30, 2017, from which other characteristics were derived. Survey participation varied markedly, from 52% of the village health worker respondents in Nam Dinh province to 8% in Ba Ria-Vung Tau (BRVT) province. Responses from commune health station and district health center respondents paralleled that of village health worker respondents in each province. Based on the survey results, 67% of village health workers were over the age of 40 years (range: 26%-100% by district), and 72% (range: 0%-100% by district) were female. More than half (61%, range: 0%-100% by district) of village health workers had education beyond primary school (grade 6).
The 2 groups of key informants most commonly identified by commune health station focal points were community members (76%, 40%-100% by district) and school teachers (59%, 0%-100% by district) (Table 3). Five characteristics were associated with differences in district-level signal incidence calculated in the univariate analysis. Three had a positive association: village health worker density (
Potential Factors Associated with Signal Incidence from Phase 1 Provinces from September 2016 to June 2017
VHW = village health worker.
Factors that are VHW attributes are reported as the percentage of survey respondents.
Bivariate Analysis of Factors and Signal Incidence Showing a Significant Association with Signal Incidence in Rural Districts a
Data are from Phase 1 provinces from September 2016 to June 2017.
VHW = village health worker.
Total population, population density, village health worker density, and the percentage of communities in the districts with community members and teachers as active informants were incorporated into a multivariate linear regression model. Of these, only 2 variables, village health worker density (β = 0.199,
Results of Linear Regression Analysis Showing Variables Associated with Signal Incidence and Interactions a
Data are from Phase 1 provinces from September 2016 to June 2017.
VHW = village health worker.
Statistical significance at the 95% confidence level (
The survey asked questions to better understand implementers' opinions regarding the program, training, and support. Ninety-one percent of village health workers and commune health station focal points and 87% of district focal points for community event-based surveillance reported that, as a result of training, they understood how to recognize, triage, and verify community event-based surveillance signals, and that they knew how to report community event-based surveillance data to the public health unit above them (Figure 3). A majority of respondents indicated that they received enough support from the levels above them: 94% from the commune health station, 92% from the district level, and 84% from the province level. Notably, a large majority of workers (89% of village health workers and commune health station focal points) and a similar number of district focal points (90%) reported a willingness to continue conducting community event-based surveillance, stating that it was useful for detecting outbreaks. Most workers at the commune level felt that additional training and guidelines were needed and listed insufficient training as a barrier for successful community event-based surveillance (77% of village health workers and commune health stations, and 59% for district health centers).

A selection of survey questions and responses related to attitudes and beliefs from the acceptability survey administered to village health workers (VHW), commune health station (CHS), and district focal points during Vietnam's community event-based surveillance pilot in 2017.
Discussion
Direct community involvement in the detection and reporting of potentially important public health events can overcome many limitations of routine indicator-based surveillance in detecting outbreaks while they are small and localized.13-17 In the past, most community-based surveillance programs have focused on reporting single diseases, such as a specific vaccine-preventable disease, or one targeted for elimination (eg, guinea worm).4,5,13,18 The current community event-based surveillance study took a broad approach, targeting the pathogens of highest concern to the ministry of health, and included signals that may detect emerging and reemerging threats as part of the development of an early warning and response system. Additionally, the community event-based surveillance project was launched on a large scale: This pilot involved 9% of the total population of the country, more than 8 million people.
During the evaluation of the pilot, we identified several factors that we believe were responsible for the success of the project. The number of village health workers per population was associated with a higher reporting of signals, and teachers were identified as a major source of those signals. The important role of teachers in community event-based surveillance was further reflected in the observation that a large portion of the detected events were vaccine-preventable diseases among children. In our study, differences in age, sex, and education level of village health workers did not appear to influence the level of signal reporting. However, reporting was highest when village health workers had been given instruction as part of a small group rather than being taught in a large classroom (data not shown). These factors seemed relevant primarily for rural settings.
Importantly, while the number of signals per unit of population trended slightly downward over the course of the 16 months of Phase 1 of the pilot, the quality of the reporting—that is, the proportion of signals that were judged to be events—trended upwards and also appeared to improve sharply after specific times when the village health workers, commune health stations, district health centers, and provincial preventive medicine centers were engaged either in supportive monitoring or refresher training.
Our study found that commune health stations received less than 1 signal per month on average, and the busiest commune health station received fewer than 3 signals per month. Village health workers and other implementers did not report the work of community event-based surveillance as an undue burden during face-to-face interviews. In fact, a small and continuous stream of signal reporting may serve as an important opportunity for public health authorities to encourage and sustain reporting behavior, as well as to maintain open channels of communication with the community.
Community event-based surveillance served as an effective One Health surveillance system, detecting signals associated with 3 of the 5 priority zoonoses in Vietnam: avian influenza, rabies, and infections with
More than half the events reported during Phase 1 were dengue and hand, foot, and mouth disease, both of which are endemic seasonal diseases in Vietnam. Because they occur annually and over a broad area rather than in discrete outbreaks, these specific diseases are probably better suited for monitoring in a routine reporting system, such as one in a healthcare facility. Signals for these 2 diseases were removed from the signal list in Phase 2 of the pilot. In addition to this, the signals were revised to improve the clarity for lay workers, rearranged and regrouped for simplicity, and the specificity of certain signals was increased to decrease spurious reports. These modifications to the signal definitions, along with ongoing supportive supervision, resulted in overall improvements in the quality of reporting as reflected in the event-to-signal ratio.
The study had a number of limitations. First, although the online surveys were sent to all of the workforce in the 4 pilot provinces, only a relatively small proportion of village health worker respondents completed it, which limited the representativeness of some of the findings and may have skewed some of the analyses. Without knowing the demographic profiles of nonrespondents, there may have been undetected bias in survey respondents. In addition, there may be specific factors such as community composition and cultural issues that play a role in the effectiveness of village health workers that were not captured by our study. Finally, it was not possible to include all district health centers, commune health stations, and communes participating in the pilot in the site visits and some qualitative data collection, and the degree to which the included districts represent the whole is uncertain.
The community event-based surveillance pilot has demonstrated several issues that need further study: (1) the need to develop better tools to measure the impact of community event-based surveillance including sensitivity and positive predictive value when the total number of events that has occurred is unknowable; (2) development of effective training strategies to efficiently train large numbers of widely dispersed health workers (and the potential utility of electronic training platforms); (3) signal definitions that give the best balance of sensitivity and specificity and effectively signal for an unknown, novel pathogen; (4) the utility of electronic data collection and signal reporting methods; and (5) methods to motivate village and community health workers in a sustainable way.
We found that the use of sensitized key informants from within the community who are likely to have knowledge of events occurring around them is an effective way to quickly establish “eyes and ears” on the ground and also likely helped to limit the amount of signal noise that might occur if all members of a community were invited to report. It may be that over a longer period of time, districts with a lower number of village health workers may catch up with those that have higher numbers through the accumulation of informant networks over time.
The finding that the number of village health workers and other reporters was most important in rural districts deserves further study. It may reflect less community cohesion in urban areas, resulting in less connectedness and fewer social networks. Village health workers from rural areas commonly and spontaneously reported that they were well respected and recognized by their communities, while interviewers were less likely to hear this sentiment expressed among village health workers working in urban areas.
In summary, this pilot project has demonstrated that community event-based surveillance can be an important complement to other types of surveillance and may be useful for the early detection of outbreaks, potentially before the pathogen is even recognized. As such, it should be considered a key component of global health security. 20
This pilot implementation experience helped the ministry of health to meet its requirements under the IHR and strengthen capacities within the GHSA framework. More importantly, it created a framework that improves the system's ability to detect and respond to outbreaks at their source as they emerge. As a result of the success of the pilot, the vice minister of health of Vietnam issued a mandate in March 2018, Decision No. 2018/QD-BYT
Footnotes
Acknowledgments
The pilot project was funded by Global Health Security Agenda funds provided through a cooperative agreement between the US Centers for Disease Control and Prevention and the Vietnam Ministry of Health's General Department of Preventive Medicine.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
