Sage Journals: Discover world-class research

Abstract

Objectives

Evidence-based decision making (EBDM) allows public health practitioners to implement effective programs and policies fitting the preferences of their communities. To engage in EBDM, practitioners must have skills themselves, their agencies must engage in administrative evidence-based practices (A-EBPs), and leaders must encourage the use of EBDM. We conducted this longitudinal study to quantify perceptions of individual EBDM skills and A-EBPs, as well as the longitudinal associations between the 2.

Methods

An online survey completed among US state health department practitioners in 2016 and 2018 assessed perceptions of respondents’ skills in EBDM and A-EBPs. We used χ² tests, t tests, and linear regressions to quantify changes over time, differences by demographic characteristics, and longitudinal associations between individual skills and A-EBPs among respondents who completed both surveys (N = 336).

Results

Means of most individual EBDM skills and A-EBPs did not change significantly from 2016 to 2018. We found significant positive associations between changes in A-EBPs and changes in EBDM skill gaps: for example, a 1-point increase in the relationships and partnerships score was associated with a narrowing of the EBDM skill gap (β estimate = 0.38; 95% CI, 0.15-0.61). At both time points, perceived skills and A-EBPs related to financial practices were low.

Conclusions

Findings from this study can guide the development and dissemination of initiatives designed to simultaneously improve individual and organizational capacity for EBDM in public health settings. Future studies should focus on types of strategies most effective to build capacity in particular types of agencies and practitioners, to ultimately improve public health practice.

Keywords

evidence-based decision making capacity building public health organization chronic disease

Evidence-based public health is a systematic approach to public health practice that can assist practitioners in effectively implementing evidence-based programs and policies that fit the communities they serve.¹ In light of the high prevalence of chronic disease in the United States, this approach has been highlighted by authoritative bodies as important for achieving the ultimate goal of improving population-wide health and longevity.^2,3 Despite knowledge of evidence-based programs and policies, challenges to implementing interventions include the capacity of public health agencies and the individual skills of practitioners to implement, evaluate, and adapt programs and policies.^4

-7

The use of evidence-based decision making (EBDM) in public health agencies is central to evidence-based public health.^8

-13 EBDM is the process of integrating the best available research evidence, practitioner expertise, and the characteristics, needs, and preferences of the community.^12,14 Using EBDM, practitioners can better practice evidence-based public health and tailor their approach to their communities.¹² EBDM allows for the efficient use of limited financial and personnel resources in an agency, and it is ethically justified.^15,16 The use of EBDM is associated with improved use of evidence-based programs and public health agency performance, that is, delivery of essential public health services such as evaluating the effectiveness of population-based health services.^17

-22

The extent to which a health department uses EBDM is driven by the individual skills of the practitioners in a health department and the organizational supports (ie, administrative practices) in an agency.²³ Skills needed for practitioners to engage in EBDM include prioritizing program and policy options, adapting interventions, using economic evaluations, and communicating research to stakeholders.¹⁴ Administrative evidence-based practices (A-EBPs) are modifiable structures and activities in an organization that are positively associated with performance measures (ie, carrying out evidence-based interventions²⁴ and capacity for EBDM).²⁵ A-EBPs are grouped into 5 domains: workforce development, organizational climate and culture, leadership, relationships and partnerships, and financial processes.¹³ Evidence-based processes rely on people and organizations engaging in a reciprocally beneficial relationship (Figure).²³ As people shape organizations and organizations support individual skill development, overall capacity for evidence-based public health can improve.

Figure

Relationships between people and organizations in public health decision making. Reprinted with permission from the Annual Review of Public Health.²³

To date, research is limited about changes in organizational-level A-EBPs and individual-level EBDM skills, as well as the relationships between these 2 over time. Understanding the relationships between the people working in health departments and their perceptions of the organizational practices in their work unit is important for identifying which components to prioritize within capacity-building efforts. Thus, the objective of this study was to quantify the longitudinal associations between the perceptions of A-EBPs and EBDM skills in a sample of state health department practitioners in the United States.

Methods

This longitudinal study used data from an online survey completed by state health department practitioners in the United States. We conducted the survey in 2016 and 2018 to assess the current use of evidence-based processes and supports in state health departments, in partnership with the National Association of Chronic Disease Directors, a professional organization dedicated to improving “the health of the public by strengthening state-based leadership and expertise for chronic disease prevention and control in states and at the national level” in the United States.²⁶ The institutional review board of Washington University in St. Louis reviewed and approved all study protocols.

Participant Recruitment

Eligible practitioners were state health department employees who were members of the National Association of Chronic Disease Directors (ie, state health department practitioners working in chronic disease or associated risk factors). Of the 2771 members, we invited a random sample of 943 eligible practitioners to complete an initial survey in January 2016. Of those invited, 571 participated in the survey in 2016 (61% response). We distributed follow-up surveys in January 2018 to those who completed the survey in 2016 and an additional random sample of 241 members working in state health departments who were not in the initial invited sample. Of the 479 respondents in 2018 (70% response), 336 had completed the survey in 2016 and were included in this analysis.

Data Collection

We collected data using Qualtrics online survey software (Qualtrics). We sent an email to eligible respondents that included an invitation to participate, study information, and the survey link. Those who had not completed the survey received up to 3 reminder emails and 2 telephone calls during an 8-week period. Practitioners who completed the survey could select a charity to donate $15 for participating.

Measures

The survey assessed participant demographic characteristics, their perceptions of the importance and availability of individual skills related to EBDM, and A-EBPs. Survey items were from previously administered surveys developed by the study team and practitioners using literature reviews and cognitive response testing among state health department practitioners.^{7,17,18,27
-29} Details on the psychometric properties of these constructs (ie, test–retest reliability and Cronbach α) are available elsewhere.³⁰

Demographic characteristics were tenure (years working in current position, years working in current organization, years working in public health), type of position (program manager or coordinator; specialist; director, deputy director, or division head; other), educational degrees (master’s or doctoral degree in any field, including public health; master’s or doctoral degree in public health), sex (male, female), age (20-29, 30-39, 40-49, 50-59, ≥60), race (White, Black/African American, Asian or Pacific Islander, American Indian/Alaska Native, other, prefer not to respond/don’t know), Hispanic ethnicity (yes, no), program area (cancer prevention and control; tobacco; obesity prevention, physical activity, and diet/nutrition; maternal and child health; cardiovascular health; diabetes; other), and geographic region (West, Midwest, South, Northeast).

Participants rated the importance of 13 skills related to EBDM (“importance of each of the skills”) and then assessed the availability of each competency (“how available you feel each skill is to you when you need it [either in your own skill set or among others in your work unit]”). Response options for these items were an 11-point ordered scale from 1 (unimportant/not available) to 11 (very important/very available). These items are important skills for public health practitioners, are reflected in Core Competencies for Public Health Professionals and the Public Health Accreditation Board standards,^31,32 and are used extensively in research among public health practitioners.^7,28,29

The survey assessed A-EBPs using 34 items in 5 domains: workforce development (4 items), leadership support (11 items), organizational culture and climate (10 items), relationships and partnerships (9 items), and financial practices (4 items). Responses for the workforce development items were 1 = yes, 2 = no, and 3 = I don’t know. The survey used a 7-point Likert scale (1 = strongly disagree; 2 = disagree; 3 = somewhat disagree; 4 = neither agree nor disagree; 5 = somewhat agree; 6 = agree; 7 = strongly agree) for items in the other 4 domains.

Statistical Analysis

We grouped states into the 4 US Census Bureau categories (West, Midwest, South, Northeast).³³ We calculated gap scores in the 13 EBDM skill scores by subtracting the score in perceived availability from the score in perceived importance for each respondent and for each skill. Gap scores can range from −10 to +10, with higher gap scores indicating larger gaps. For example, a skill rated high in importance but low in availability would have a large gap score. We calculated a summary skill-gap score for each respondent by averaging the skill-gap scores for the 13 EBDM skills, with a potential range from −10 to +10. We calculated changes in skill gaps by subtracting the Time 2 score (2018) from the Time 1 score (2016), with a potential range from −20 to +20, where a positive change indicates a reduction in the gap or an increase in the availability of skills. We grouped respondents according to tertiles of change in an overall skill-gap score (calculated as the change in all skill gaps). The range of changes in EBDM skill gaps for tertile 1 (n = 113) was −5.62 to −0.46; for tertile 2 (n = 105) was −0.38 to 0.46; and for tertile 3 (n = 113) was 0.54 to 5.92.

We created a summary score for the workforce development A-EBP domain by recoding individual practices as 0 for no/don’t know and 1 for yes; we then calculated the sum of the 4 items. Scores could range from 0 to 4. We calculated summary scores for the other 4 domains of A-EBPs by averaging the responses for the individual items in a domain, with a potential range from 1 to 7. We calculated changes in A-EBPs by subtracting the Time 2 score (2018) from the Time 1 score (2016), such that positive changes indicated increases in perceived organizational supports. These changes could range from −6 to +6.

Inferential Statistics

We quantified changes in perceived individual skill gaps and organizational A-EBPs over time by using paired Student t tests and a significance level of .05. We conducted Pearson χ² tests to examine associations between the characteristics of the state health department and practitioners.

We fit linear regression models to quantify the association between the change in A-EBPs (continuous independent variable) and skill-gap change (continuous dependent variable). We calculated β estimates and 95% CIs. We identified several characteristics as potential confounders: type of position, program area, educational degrees, geographic region, years working in public health, and years working in current position. None of these covariates changed the point estimates of the association between the A-EBPs and skill-gap change or were associated with skill-gap change in multivariate models. Thus, we tabulated data for the unadjusted models.

Results

Most participants were program managers or coordinators (43.9%) or specialists (39.5%) (Table 1). Nearly three-quarters (72.1%) of respondents had a master’s or doctoral degree in any field, but only one-third (32.9%) of respondents had a master’s or doctoral degree in public health. The most common program areas of work were cancer prevention and control (23.5%), tobacco prevention and control (16.9%), and obesity prevention, physical activity, and diet/nutrition (14.4%). The greatest proportion of respondents were from the West (31.8%) and the smallest from the Northeast (18.1%).

Table 1

Baseline state health department respondent characteristics among respondents (N = 336) who completed 2 surveys on evidence-based decision making and administrative evidence-based practices, United States, 2016

Characteristic	Value^a
Years working in current position, mean (SD)	6.3 (5.8)
Years working in current organization, mean (SD)	11.9 (8.6)
Years working in public health, mean (SD)	15.3 (9.5)
Type of position
Program manager or coordinator	147 (43.9)
Specialist^b	47 (39.5)
Director, deputy director, or division head	133 (13.9)
Other	9 (2.7)
Educational degrees^c
Master’s or doctoral degree in any field, including public health	242 (72.0)
Master’s or doctoral degree in public health	111 (33.0)
Sex
Female	277 (82.4)
Male	59 (17.6)
Age, y^d
20-29	22 (6.6)
30-39	64 (19.1)
40-49	96 (28.7)
50-59	110 (32.8)
≥60	43 (12.8)
Race^b ^, ^d
White	276 (82.4)
Black/African American	29 (8.7)
Asian or Pacific Islander	15 (4.5)
American Indian/Alaska Native	8 (2.4)
Other	5 (1.5)
Prefer not to respond/don’t know	13 (3.9)
Hispanic ethnicity^d	15 (4.5)
Program area
Cancer prevention and control	79 (23.5)
Tobacco	57 (17.0)
Obesity prevention, physical activity, diet/nutrition	48 (14.3)
Maternal and child health	33 (9.8)
Cardiovascular health	21 (6.2)
Diabetes	18 (5.4)
Other	80 (23.8)
Geographic region
West	107 (31.8)
Midwest	93 (27.6)
South	74 (22.0)
Northeast	62 (18.5)

^aAll values are number (percentage) unless otherwise indicated.

^bSpecialists include health educators, epidemiologists, statisticians, program evaluators, community health nurses, social workers, dietitians, or nutritionists.

^cResponses for these items are not mutually exclusive, and the education variables are not exhaustive.

^dNot all respondents answered all questions, so the number for each characteristic may not add to 336. One respondent omitted a response for age, 1 for race, and 2 for Hispanic ethnicity.

Changes in Individual EBDM Skill Gaps

The mean (SD) score for the 13 EBDM skill gaps decreased from Time 1 (1.9 [1.6]) to Time 2 (1.8 [1.5]), but the decrease was not significant (P = .30) (Table 2). The changes in importance ratings from Time 1 to Time 2 were less than 0.1, indicating that decreases in skill gaps were due to an increase in availability of skills. We found no differences in changes in EBDM skill gaps by demographic characteristics.

Table 2

Changes in state health department practitioners’ individual evidence-based decision making (EBDM) skill gaps and perceived organizational administrative evidence-based practices (A-EBPs) in the United States, 2016 and 2018

Measure	No. of survey items	No. of respondents	Score at Time 1, mean (SE)	Score at Time 2, mean (SE)	Difference in score, mean (SE)	Paired Student t test between score at Time 1 and score at Time 2
Measure	No. of survey items	No. of respondents	Score at Time 1, mean (SE)	Score at Time 2, mean (SE)	Difference in score, mean (SE)	Df	t value	P value^a
EBDM skill gap^a ^,b	—	331	1.9 (1.6)	1.8 (1.5)	0.1 (1.6)	330	1.04	.30
Administrative evidence-based practices^c
Workforce development	4	331	2.7 (1.3)	2.8 (1.2)	0.2 (1.5)	330	–2.01	.045
Leadership support	11	336	4.9 (1.0)	5.0 (1.0)	0.1 (0.9)	335	–1.55	.12
Organizational culture and climate	10	335	5.4 (1.0)	5.4 (1.0)	0 (0.9)	334	–0.22	.83
Relationships and partnerships	9	333	5.6 (0.8)	5.6 (0.7)	0 (0.8)	332	–0.47	.64
Financial practices	4	335	4.8 (1.2)	4.7 (1.2)	−0.1 (1.1)	334	–2.25	.02

^aPaired Student t tests were used to examine differences in scores at Time 1 and scores at Time 2. Significant at P < .05.

^bImportance and availability of 13 skills were measured on an 11-point ordered scale. For each skill, a skill gap was calculated by subtracting the availability rating from the importance rating. A mean skill-gap score was created as an average of the 13 individual skill gaps. Changes in individual EBDM skill gaps were calculated by subtracting the Time 2 skill gap from the Time 1 skill gap, such that a positive change indicates a reduction in the gap or an increase in the availability of skills; we did not observe a reduction in the perceived importance of a skill. The possible range was –10 to +10 for Time 1 and Time 2 skill-gap scores and –20 to +20 for the difference score.

^cAll A-EBPs were measured on a 7-point Likert scale, except items in the workforce development domain, which were measured as yes, no, and don’t know. These items were recoded as 0 = no or don’t know, 1 = yes. A summary score was created for workforce development as a sum of the 4 items, and an average was calculated for the remaining domains. Changes in A-EBPs were calculated by subtracting Time 1 from Time 2, such that a positive change indicates an increase in perceived organizational supports. The possible range was 0 to 4 for the workforce development Time 1, Time 2, and difference; 1 to 7 for the other Time 1 and Time 2 A-EBP domains; and –6 to +6 for the difference in domains other than workforce development.

For 10 of the 13 skills assessed, the skill gap decreased, although these decreases were not significant. The skill gaps with the greatest decreases were financial management, using surveillance data, and quantitative evaluation, whereas the skill gaps that stayed the same were economic evaluation, written communication, and leadership/management with diverse staff members. Availability ratings remained low from Time 1 to Time 2 for economic evaluation, communication with policy makers, leadership, and management with diverse staff members.

Changes in A-EBPs

Participants rated the A-EBPs from 2.7 to 5.6 at Time 1 and from 2.8 to 5.6 at Time 2 (Table 2). At both points, the mean ratings for organizational culture and climate (5.4 at Time 1 and Time 2) and relationships and partnerships (5.6 at Time 1 and Time 2) were higher than for leadership support (4.9 at Time 1 and 5.0 at Time 2) and financial practices (4.8 at Time 1 and 4.7 at Time 2). The increase in the mean (SD) number of workforce development practices (from 2.7 [1.3] at Time 1 to 2.8 [1.2] at Time 2; P = .045) and the decrease in financial practices (from 4.8 [1.2] at Time 1 to 4.7 [1.2] at Time 2; P = .02) were significant, but we found no other significant changes in A-EBPs.

Relationships Between Perceptions of Individual Skill-Gap Changes and A-EBP Changes

We found significant positive associations between changes in A-EBPs and changes in EBDM skill gaps (Table 3). For example, when organizational supports for EBDM increased, so too did individual practitioners’ skills in EBDM. Beta estimates from linear models quantifying these associations ranged from 0.04 (95% CI, −0.08 to 0.16) to 0.59 (95% CI, 0.41 to 0.78). For example, a 1-point increase in the relationships and partnerships score was associated with a decrease of 0.38 (95% CI, 0.15 to 0.61) in the EBDM skill gap. These associations all were significant except workforce development.

Table 3

Associations between changes in administrative evidence-based practices (A-EBPs) and changes in state health department practitioners’ individual evidence-based decision making (EBDM) skill gap in the United States, 2016 and 2018^a

A-EBP	Change in EBDM skill gap from 2016 to 2018, β (95% CI) [P value]^b
Workforce development	0.04 (–0.08 to 0.16) [.52]
Leadership support	0.41 (0.22 to 0.60) [<.001]
Organizational culture and climate	0.59 (0.41 to 0.78) [<.001]
Relationships and partnerships	0.38 (0.15 to 0.61) [.001]
Financial practices	0.18 (0.01 to 0.34) [.03]

^aAs measured in surveys administered to state health department practitioners in 2016 and 2018; 336 practitioners completed both surveys. Linear regressions were fit to quantify associations between changes in the A-EBPs (independent variables) and individual EBDM skill gap (dependent variable). For example, leadership support and skill-gap changes were positively associated, such that a 1-point increase in leadership support was associated with a 0.41 decrease in skill gap (positive changes in skill gaps indicate narrowing of the skill gap).

^b P values represent tests of the null hypothesis that the β estimate was 0; P < .05 was considered significant.

Discussion

To our knowledge, our study is the first observational study to examine temporal changes in the individual and organizational capacity for EBDM among state health department practitioners across the United States. Findings from this study can be used to guide the development of initiatives focused on EBDM in public health settings. The measures used can help practitioners understand which EBDM competencies and organizational supports should be prioritized for improvement.

Mean changes in perceptions of EBDM-related individual skills and A-EBPs did not improve greatly over time, and the mean individual EBDM skills in our study are consistent with ranges found in other studies.^17,28,29 In other studies, similar to our study, the mean value for the skill gap in economic evaluation typically is the largest and the mean value for the skill gap in quantitative evaluation is the smallest.^17,28,29 Two skill gaps, communication with policy makers and understanding evaluation designs, were smaller (ie, more favorable) in our sample than in previous studies with state health department practitioners.^17,28,29

Our study was not designed to examine why changes in constructs did or did not occur, nor was the study tied to a specific capacity-building initiative, so we did not determine why we did not observe changes. The lack of change may indicate a lack of sustained change in individual skills or organizational supports. Capacity-building efforts may ebb and flow as practitioners and agencies participate in capacity-building efforts, but on the whole, long-term changes may not be sustained. It is also possible that the measures used in our study were not sensitive to changes in the population examined. Future mixed-methods research is needed to triangulate how and why changes in these constructs are observed or not.

Our finding that improvements in individual and organizational capacity were minimal, despite increases in workforce development scores and efforts by researchers and practitioners,^{23,34

-40} indicates that continued work is needed to improve evidence-based public health in public health practice at the state level. Innovative or tailored dissemination strategies may be needed to overcome commonly cited barriers to engaging in EBDM, particularly lack of incentives, time, appropriate funding, managerial support, and staff turnover.^21,28,41
-43 In addition, training must be supported at the agency level, for example, by requiring continuing education of employees or providing on-site training. Strategies to improve individual skills and organizational supports for EBDM may also need to focus on enhancing the thinking of practitioners about evaluation, that is, “critical thinking applied in the context of evaluation, motivated by an attitude of inquisitiveness and a belief in the value of evidence.”⁴⁴ Such thinking about evaluation has been identified as an important component to developing the evaluative capacity of people and organizations,^44

-47 which can then be applied to EBDM processes. Thus, strategies to address evaluation at a deeper level may be required for efforts to be successful in improving EBDM in public health agencies.

Consistent with previous research, we found that perceptions of individual skills and organizational supports for economic evaluation were rated low at both time points.²⁸ Practitioners may perceive that economic evaluation skills are more difficult or less important to obtain or that they lack access to training. Having the skills to conduct economic evaluation and working within an organization that prioritizes economic evaluation are critical for public health practitioners to demonstrate the value of their work to key stakeholders and to identify which programs and policies are effective, scalable, and sustainable.^48
-50 Policy makers, one group of these stakeholders, often look for cost data but have difficulty finding them.⁵¹ In light of budget constraints in public health departments,⁵² it is more important than ever for public health practitioners to apply skills in economic evaluation.

Economic evaluation remains a competency gap despite a call to action from national bodies for the use of economic evidence⁵³ and training opportunities available for practitioners.^54,55 Future research should evaluate the effectiveness of this training and identify other strategies for improving economic evaluation in public health practice. In addition, research should include cost data about programs, policies, and the strategies used to support the implementation of evidence-based programs and policies, so cost data are available when needed.⁵⁶ Practice-focused work should focus on expanding these training opportunities and possibly adding economic evaluation–related competencies to the standards of the Council on Education for Public Health and the Public Health Accreditation Board.

Another notable finding from our study was that the changes in perceived leadership support and organizational climate and culture showed the largest associations with changes in individual skill gaps. This finding is consistent with the findings of previous literature highlighting the importance of leadership in promoting the use of EBDM.^24,40,57
-59 Many opportunities for leaders to improve EBDM within their organizations exist, including directly stating expectations for EBDM and assessing employee performance according to EBDM skills.⁶⁰ Practitioners’ perceptions that EBDM is an agency priority and that documenting evidence-based practices is expected are associated with use of EBDM.^41,61 A reciprocal relationship exists between people (eg, leaders) and aspects of the organizational climate related to implementation of evidence-based programs and policies.^23,62

-66 Capacity-building efforts designed to improve leadership and managerial skills supportive of EBDM may improve other A-EBPs and, in turn, individual skills in EBDM.^67

-70

Our study did not examine the temporal sequence of changes (ie, whether individual skills changed first or A-EBPs changed first), and it cannot address causality. A bidirectional relationship likely exists between the 2 constructs.²³ Individual-level capacity may be easier to change than organizational-level capacity,¹⁷ because changing organizational-level factors is difficult amid factors such as staff turnover, competing priorities, and a lack of incentive to implement changes in EBDM.⁷¹ Intervening at the individual and organizational levels in tandem may work synergistically to improve the EBDM capacity of an agency and its employees, although changing organizational capacity will require a more substantial, longer-term commitment from public health agency leaders than will improving individual skills.⁷² Previous research identified effective training and technical assistance strategies to support short-term improvements in individual and organizational EBDM capacity.^7,17 Future research should extend that research to identify strategies that are effective for a wide variety of agencies.

Limitations

Our study had several limitations. First, data were self-reported, and such data may be subject to social desirability bias. Second, ratings of the perceived importance and availability of EBDM competencies could differ from objectively measured competencies, and evidence of the validity of the measures we used was limited. Third, reports of perceived organizational supports may not be representative of all state health department practitioners working in the various facets of chronic disease (eg, nutrition, physical activity, obesity). Fourth, our results may be subject to selection bias: analyses included only respondents who completed both surveys. However, responders and nonresponders to the follow-up survey differed only in that the proportion of White people was significantly higher among respondents than among nonrespondents, and the proportion of Black people was significantly lower among respondents than among nonrespondents. Despite these limitations, our study was strengthened by the use of a longitudinal design and recruitment of a sample that was diverse in program areas and geographical distribution, increasing the external validity of our findings.

Conclusions

Our study adds to the understanding of perceptions of individual skills in EBDM and A-EBPs in public health agencies, how they change over time, and the relationships between the 2. Future research using robust mixed-methods designs is needed to understand the changes in individual skills and organizational supports and the individual- and agency-level factors that explain these changes. The complex dynamics of people and organizations should be taken into account when developing and disseminating capacity-building efforts to improve EBDM. Making lasting changes to people’s skills and organizational capacity may be difficult and time consuming, but making such changes is necessary to ensure public health agencies reach their highest potential to improve the health of the populations they serve.

Footnotes

Acknowledgments

The authors thank the state public health practitioners for their participation in our surveys and acknowledge the administrative support of Linda Dix and Mary Adams at the Prevention Research Center in St. Louis, Brown School at Washington University in St. Louis.

Authors’ Note

The following materials are available from the corresponding author upon request: (1) the full text of the survey assessing EBDM supports, professional development activities, and individual skills; (2) information on the possible and observed ranges of the EBDM skill and A-EBP items and summary scores; (3) mean skill-gap scores for Time 1 (2016) and Time 2 (2018) surveys; and (4) box and whisker plot of EBDM skills and A-EBPs.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported in part by the National Association of Chronic Disease Directors agreement number 1612017. Additional support was provided by the National Institute of Diabetes and Digestive and Kidney Diseases (award number P30DK020579) and the National Cancer Institute (award number P50CA244431) of the National Institutes of Health and the Centers for Disease Control and Prevention award number U48DP006395. The findings and conclusions in this article are those of the authors and do not necessarily represent the official positions of the National Institutes of Health or the Centers for Disease Control and Prevention.

ORCID iD

Stephanie Mazzucca, PhD

References

Kohatsu

ND.

Robinson

JG.

Torner

. Evidence-based public health: an evolving concept. Am J Prev Med. 2004;27(5):417-421.doi:10.1016/j.amepre.2004.07.019

15556743

Institute of Medicine . Primary Care and Public Health: Exploring Integration to Improve Population Health. National Academies Press; 2012.

Public Health Accreditation Board . Guide to national public health department initial accreditation. 2015. Accessed January 16, 2018. https://phaboard.org/wp-content/uploads/2018/11/Guide-to-Accreditation-final_LR2.pdf

Briss

PA.

Brownson

RC.

Fielding

JE.

Zaza

. Developing and using the Guide to Community Preventive Services: lessons learned about evidence-based public health. Annu Rev Public Health. 2004;25:281-302.doi:10.1146/annurev.publhealth.25.050503.153933

15015921

Brownson

RC.

Ballew

Dieffenderfer

et al. Evidence-based interventions to promote physical activity: what contributes to dissemination by state health departments. Am J Prev Med. 2007;33(1 Suppl):S66-S73.doi:10.1016/j.amepre.2007.03.011

17584593

Lawrence

. Diffusion of the U.S. Preventive Services Task Force recommendations into practice. J Gen Intern Med. 1990;5(5 Suppl):S99-S103.doi:10.1007/BF02600852

2231074

Jacobs

JA.

Duggan

Erwin

et al. Capacity building for evidence-based decision making in local health departments: scaling up an effective training approach. Implement Sci. 2014;9:124.doi:10.1186/s13012-014-0124-x

25253081

Brownson

RC.

Fielding

JE.

Maylahn

. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health. 2009;30:175-201.doi:10.1146/annurev.publhealth.031308.100134

19296775

Fielding

JE.

Briss

. Promoting evidence-based public health policy: can we have better evidence and more action? Health Aff (Millwood). 2006;25(4):969-978.doi:10.1377/hlthaff.25.4.969

16835176

10.

Glasziou

Longbottom

. Evidence-based public health practice. Aust N Z J Public Health. 1999;23(4):436-440.doi:10.1111/j.1467-842X.1999.tb01291.x

10462873

11.

Jenicek

. Epidemiology, evidenced-based medicine, and evidence-based public health. J Epidemiol. 1997;7(4):187-197.doi:10.2188/jea.7.187

9465542

12.

Brownson

RC.

Baker

EA.

Deshpande

AD.

Gillespie

. Evidence-based Public Health. Oxford University Press; 2017.

13.

Brownson

RC.

Allen

Duggan

Stamatakis

KA.

Erwin

. Fostering more-effective public health by identifying administrative evidence-based practices: a review of the literature. Am J Prev Med. 2012;43(3):309-319.doi:10.1016/j.amepre.2012.06.006

22898125

14.

Brownson

RC.

Gurney

JG.

Land

. Evidence-based decision making in public health. J Public Health Manag Pract. 1999;5(5):86-97.doi:10.1097/00124784-199909000-00012

10558389

15.

Macintyre

Petticrew

. Good intentions and received wisdom are not enough. J Epidemiol Community Health. 2000;54(11):802-803.doi:10.1136/jech.54.11.802

11027191

16.

Tannahill

. Beyond evidence—to ethics: a decision-making framework for health promotion, public health and health improvement. Health Promot Int. 2008;23(4):380-390.doi:10.1093/heapro/dan032

18971394

17.

Brownson

RC.

Allen

Jacob

et al. Controlling chronic diseases through evidence-based decision making: a group-randomized trial. Prev Chronic Dis. 2017;14:E121. doi:10.5888/pcd14.170326

29191262

18.

Jacobs

JA.

Dodson

EA.

Baker

EA.

Deshpande

AD.

Brownson

. Barriers to evidence-based decision making in public health: a national survey of chronic disease practitioners. Public Health Rep. 2010;125(5):736-742.doi:10.1177/003335491012500516

20873290

19.

Jacobs

JA.

Jones

Gabella

BA.

Spring

Brownson

. Tools for implementing an evidence-based approach in public health practice. Prev Chronic Dis. 2012;9:E116. doi:10.5888/pcd9.110324

22721501

20.

Maylahn

Fleming

Birkhead

. Health departments in a brave new world. Prev Chronic Dis. 2013;10:E41. doi:10.5888/pcd10.130003

23517584

21.

Dodson

EA.

Baker

EA.

Brownson

. Use of evidence-based interventions in state health departments: a qualitative assessment of barriers and solutions. J Public Health Manag Pract. 2010;16(6):E9-E15.doi:10.1097/PHH.0b013e3181d1f1e2

20885175

22.

Centers for Disease Control and Prevention . The public health system & the 10 essential public health services. 2020. Accessed May 31, 2020. https://www.cdc.gov/publichealthgateway/publichealthservices/essentialhealthservices.html

23.

Brownson

RC.

Fielding

JE.

Green

. Building capacity for evidence-based public health: reconciling the pulls of practice and the push of research. Annu Rev Public Health. 2018;39:27-53.doi:10.1146/annurev-publhealth-040617-014746

29166243

24.

Brownson

RC.

Reis

RS.

Allen

et al. Understanding administrative evidence-based practices: findings from a survey of local health department leaders. Am J Prev Med. 2014;46(1):49-57.doi:10.1016/j.amepre.2013.08.013

24355671

25.

Eyler

AA.

Valko

Ramadas

Macchi

Fershteyn

Brownson

. Administrative evidence-based practices in state chronic disease practitioners. Am J Prev Med. 2018;54(2):275-283.doi:10.1016/j.amepre.2017.09.006

29162297

26.

National Association of Chronic Disease Directors . Our mission, about NACDD. Accessed December 7, 2019. www.chronicdisease.org/page/About_NACDD

27.

Allen

Sequeira

Jacob

et al. Promoting state health department evidence-based cancer and chronic disease prevention: a multi-phase dissemination study with a cluster randomized trial component. Implement Sci. 2013;8:141.doi:10.1186/1748-5908-8-141

24330729

28.

Jacob

RR.

Baker

EA.

Allen

et al. Training needs and supports for evidence-based decision making among the public health workforce in the United States. BMC Health Serv Res. 2014;14:564. doi:10.1186/s12913-014-0564-7

25398652

29.

Jacobs

JA.

Clayton

PF.

Dove

et al. A survey tool for measuring evidence-based decision making capacity in public health agencies. BMC Health Serv Res. 2012;12:57. doi:10.1186/1472-6963-12-57

22405439

30.

Reis

RS.

Duggan

Allen

Stamatakis

KA.

Erwin

PC.

Brownson

. Developing a tool to assess administrative evidence-based practices in local health departments. Am J Public Health. 2014;104(12):e43. doi:10.2105/AJPH.2014.10412e43.2

31.

Council on Linkages Between Academia and Public Health Practice Public Health Foundation . Core competencies for public health professionals. 2014. Accessed August 25, 2018. http://www.phf.org/resourcestools/Documents/Core_Competencies_for_Public_Health_Professionals_2014June.pdf

32.

Public Health Accreditation Board . PHAB standards and measures version 1.5. 2013. Accessed August 25, 2018. https://phaboard.org/wp-content/uploads/PHABSM_WEB_LR1-1.pdf

33.

US Department of Commerce . Appendix A: census 2000 geographic terms and concepts. 2003. Accessed August 25, 2018. https://www2.census.gov/geo/pdfs/reference/glossry2.pdf

34.

Armstrong

Waters

Dobbins

et al. Knowledge translation strategies to improve the use of evidence in public health decision making in local government: intervention design and implementation plan. Implement Sci. 2013;8:121.doi:10.1186/1748-5908-8-121

24107358

35.

Armstrong

Waters

Moore

et al. Understanding evidence: a statewide survey to explore evidence-informed public health decision-making in a local government setting. Implement Sci. 2014;9:188.doi:10.1186/s13012-014-0188-7

25496505

36.

Serrano

Diem

Grabauskas

et al. Building the capacity—examining the impact of evidence-based public health trainings in Europe: a mixed methods approach. Glob Health Promot. 2020;27(2):45-53.doi:10.1177/1757975918811102

30943109

37.

Waters

Armstrong

Swinburn

et al. An exploratory cluster randomised controlled trial of knowledge translation strategies to support evidence-informed decision-making in local governments (The KT4LG study). BMC Public Health. 2011;11:34. doi:10.1186/1471-2458-11-34

21226966

38.

Pettman

TL.

Armstrong

Jones

Waters

Doyle

. Cochrane update: building capacity in evidence-informed decision-making to improve public health. J Public Health (Oxf). 2013;35(4):624-627.doi:10.1093/pubmed/fdt119

24293453

39.

Diem

Brownson

RC.

Grabauskas

Shatchkute

Stachenko

. Prevention and control of noncommunicable diseases through evidence-based public health: implementing the NCD 2020 action plan. Glob Health Promot. 2016;23(3):5-13.doi:10.1177/1757975914567513

25758171

40.

Yousefi Nooraie

Lohfeld

Marin

Hanneman

Dobbins

. Informing the implementation of evidence-informed decision making interventions using a social network analysis perspective; a mixed-methods study. BMC Health Serv Res. 2017;17(1):122. doi:10.1186/s12913-017-2067-9

28178958

41.

Dobbins

Cockerill

Barnsley

Ciliska

. Factors of the innovation, organization, environment, and individual that predict the influence five systematic reviews had on public health decisions. Int J Technol Assess Health Care. 2001;17(4):467-478.doi:10.1017/S0266462301107026

11758291

42.

Dreisinger

Leet

TL.

Baker

EA.

Gillespie

KN.

Haas

Brownson

. Improving the public health workforce: evaluation of a training course to enhance evidence-based decision making. J Public Health Manag Pract. 2008;14(2):138-143.doi:10.1097/01.PHH.0000311891.73078.50

18287919

43.

Mainor

Leeman

Sommers

et al. A systematic approach to evaluating public health training: the Obesity Prevention in Public Health course. J Public Health Manag Pract. 2014;20(6):647-653.doi:10.1097/PHH.0000000000000046

24402433

44.

Buckley

Archibald

Hargraves

Trochim

. Defining and teaching evaluative thinking: insights from research on critical thinking. Am J Eval. 2015;36(3):375-388.doi:10.1177/1098214015581706

45.

Adams

Dickinson

. Evaluation training to build capability in the community and public health workforce. Am J Eval. 2010;31(3):421-433.doi:10.1177/1098214010366586

46.

Preskill

. Evaluation’s second act: a spotlight on learning. Am J Eval. 2008;29(2):127-138.doi:10.1177/1098214008316896

47.

Preskill

Boyle

. A multidisciplinary model of evaluation capacity building. Am J Eval. 2008;29(4):443-459.doi:10.1177/1098214008324182

48.

Rabarison

KM.

Bish

CL.

Massoudi

MS.

Giles

. Economic evaluation enhances public health decision making. Front Public Health. 2015;3:164.doi:10.3389/fpubh.2015.00164

26157792

49.

Frieden

. Government’s role in protecting health and safety. N Engl J Med. 2013;368(20):1857-1859.doi:10.1056/NEJMp1303819

23593978

50.

Mays

GP.

Smith

. Evidence links increases in public health spending to declines in preventable deaths. Health Aff (Millwood). 2011;30(8):1585-1593.doi:10.1377/hlthaff.2011.0196

21778174

51.

Dodson

EA.

Geary

NA.

Brownson

. State legislators’ sources and use of information: bridging the gap between research and policy. Health Educ Res. 2015;30(6):840-848.doi:10.1093/her/cyv044

52.

Association of State and Territorial Health Officials . Budget cuts continue to affect the health of Americans. 2014. Accessed January 5, 2020. https://www.astho.org/budget-cuts-Sept-2014

53.

Institute of Medicine . The Future of Public Health. National Academies Press; 1988.

54.

Harvard University Prevention Research Center on Nutrition and Physical Activity . CHOICES Learning Collaborative Partnership. Accessed January 9, 2020. https://choicesproject.org/learning-collaborative-partnership

55.

Centers for Disease Control and Prevention . CDC Steven M. Teutsch Prevention Effectiveness (PE) Fellowship. 2019. Accessed January 9, 2020. https://www.cdc.gov/pef/index.html

56.

Reeves

Edmunds

Searles

Wiggers

. Economic evaluations of public health implementation-interventions: a systematic review and guideline for practice. Public Health. 2019;169:101-113.doi:10.1016/j.puhe.2019.01.012

30877961

57.

Bekemeier

Grembowski

Yang

Herting

. Leadership matters: local health department clinician leaders and their relationship to decreasing health disparities. J Public Health Manag Pract. 2012;18(2):E1-E10.doi:10.1097/PHH.0b013e318242d4fc

22286291

58.

Peirson

Ciliska

Dobbins

Mowat

. Building capacity for evidence informed decision making in public health: a case study of organizational change. BMC Public Health. 2012;12:137. doi:10.1186/1471-2458-12-137

22348688

59.

Allen

Yan

Reis

RS.

Jacob

RR.

Brownson

. Organizational supports for research evidence use in state public health agencies: a latent class analysis. J Public Health Manag Pract. 2019;25(4):373-381.doi:10.1097/PHH.0000000000000821

31136511

60.

Institute of Medicine . The Future of the Public’s Health in the 21st Century . National Academies Press; 2003.

61.

Brownson

RC.

Eyler

AA.

Harris

JK.

Moore

JB.

Tabak

. Getting the word out: new approaches for disseminating public health science. J Public Health Manag Pract. 2018;24(2):102-111.doi:10.1097/PHH.0000000000000673

62.

Birken

Clary

Tabriz

et al. Middle managers’ role in implementing evidence-based practices in healthcare: a systematic review. Implement Sci. 2018;13(1):149.doi:10.1186/s13012-018-0843-5

30541596

63.

Kislov

Waterman

Harvey

Boaden

. Rethinking capacity building for knowledge mobilisation: developing multilevel capabilities in healthcare organisations. Implement Sci. 2014;9:166.doi:10.1186/s13012-014-0166-0

25398428

64.

Meyer

A-M.

Davis

Mays

. Defining organizational capacity for public health services and systems research. J Public Health Manag Pract. 2012;18(6):535-544.doi:10.1097/PHH.0b013e31825ce928

23023278

65.

Robinson

Farmer

Riley

Elliott

Eyles

CHHDP Investigative Team, Strategic, and Research Advisory Groups . Realistic expectations: investing in organizational capacity building for chronic disease prevention. Am J Health Promot. 2007;21(5):430-438.doi:10.4278/0890-1171-21.5.430

17515008

66.

Schneider

Ehrhart

MG.

Macey

. Organizational climate and culture. Annu Rev Psychol. 2013;64:361-388.doi:10.1146/annurev-psych-113011-143809

22856467

67.

Association of State and Territorial Health Officials . ASTHO Leadership Institute. 2020. Accessed February 14, 2020. https://www.astho.org/Member-Services/ASTHO-Leadership-Institute

68.

Center for Health Leadership & Practice . National Leadership Academy for the Public’s Health. 2020. Accessed February 14, 2020. https://www.healthleadership.org/program_nlaph

69.

National Association of Chronic Disease Directors . About GEAR groups. 2020. Accessed February 14, 2020. https://www.chronicdisease.org/page/GEAR_Groups

70.

National Association of Chronic Disease Directors . Chronic Disease Directors leadership development. 2020. Accessed February 14, 2020. https://chronicdisease.org/page/leadershipdevelopment/nacdd-action-on-leadership-development

71.

Aarons

GA.

Ehrhart

MG.

Farahnak

LR.

Sklar

. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. 2014;35:255-274.doi:10.1146/annurev-publhealth-032013-182447

24641560

72.

Ward

Dobbins

Peirson

World Health Organization . Lessons learnt from implementing an organizational strategy for evidence-informed decision making. Public Health Panoram. 2016;2(3):327-332.

The Relationships Between State Health Department Practitioners’ Perceptions of Organizational Supports and Evidence-Based Decision-Making Skills

Abstract

Objectives

Methods

Results

Conclusions

Keywords

Methods

Participant Recruitment

Data Collection

Measures

Statistical Analysis

Inferential Statistics

Results

Changes in Individual EBDM Skill Gaps

Changes in A-EBPs

Relationships Between Perceptions of Individual Skill-Gap Changes and A-EBP Changes

Discussion

Limitations

Conclusions

Footnotes

Acknowledgments

Authors’ Note

Declaration of Conflicting Interests

Funding

ORCID iD

References