Evaluating Effective Leadership and Governance in a Midwestern Agricultural Safety and Health Coalition

Abstract

Agricultural Centers are a coalition of organizations and individual members with a common purpose: to improve the health and safety of the agricultural community. Successful leadership and governance are essential in accomplishing these goals. This article examined the effectiveness of a midwestern Agricultural Health and Safety Center (Ag Center) leadership and governance structure. The Internal Coalition Outcomes Hierarchy (ICOH) framework and the Internal Coalition Effectiveness (ICE^©) instrument were used, with field visit interviews conducted to gain further insight. Combined comparative findings from both research methods showed that scores in each of the categories increased. Adjustments led to stronger collaborative leadership, vital to successful population health improvement programs. This study showcases coalition qualities in a broader environment, capturing a clearer depiction of leadership and member interaction. Field visit interviews confirmed that this midwestern Ag Center continued to have strong levels of effectiveness in each of the conceptual constructs of a coalition.

Keywords

program planning and evaluation occupational health and safety programs leadership agriculture public health systems and nursing

Exposure hazards in the agricultural workforce have long been of significant concern (U.S. Bureau of Labor Statistics [BLS], 2015). The National Institute for Occupational Safety and Health (NIOSH; Centers for Disease Control and Prevention, 2016) established the Centers for Agricultural Disease and Injury Research, Education, and Prevention (Ag Centers) to address this alarming trend. Ag Centers are a nation-wide coalition of interested academic institutions, community-based individuals, and agriculture-based organizations dedicated to developing strategies to reduce agriculture-related illnesses and injuries. Currently, 11 NIOSH Ag Centers are funded across the nation.

Nurses are major constituents of Ag Centers and they function in diverse roles (e.g., center director, deputy director, clinical director, evaluation director, educator, researcher, scientist, occupational, agricultural and rural health nurse). The occupational health nurse authors of this article comprise the evaluation core for their midwestern Ag Center. The evaluation team’s previous study (Cramer & Wendl, 2015) showed that effectively-led coalitions foster learning environments which are highly valued by members and promote satisfaction within the network.

The global community agrees that addressing modern public health challenges requires building the capacity of coalitions, establishments, and systems (Alexander et al., 2016; Batras, Duff, & Smith, 2014). Substantial effort in planning and ongoing coordination is required when forming a coalition; thus, evaluating success is vital to ensuring quality improvement (Zakocs & Edwards, 2006). Health and safety programs must make a difference; accomplishing this goal requires effective leadership and governance, collaborating with a host of community individuals and organizations. To ensure the efficacy of a coalition, it is vital to study its administrative structure to determine strengths and weaknesses. Organizational adjustments can then support Ag Centers in achieving their intended outcomes. Moreover, with greater competition for government and private funding, it is imperative that coalitions demonstrate their collaborative success to positively influence financiers (Mueller, 2007). This study is an example of how occupational health nurses and other professionals can evaluate their coalitions or networks.

The focus on coalitions and partnerships among various sectors has become more common (El Ansari, Phillips, & Zwi, 2004). Coalitions require leaders who provide vision and direction. Despite the fact that Ag Centers have been in existence for more than 20 years, little attention has been paid to evaluating their leadership and governance, or how they function as organizations (McDonald & McDivitt, 1998). The study literature review found a noticeable gap in coalition evaluation studies focused on occupational health nurses in the past 10 years. This article is a case study evaluation for a midwestern Ag Center. The evaluation used a mixed method explanatory sequential design to assess the coalition’s organizational effectiveness. Results were used to build on identified leadership strengths and address areas of governance improvement.

Background

This study involved an Ag Center serving a midwestern seven-state region with a relatively large concentration of farms (n = 437,042) that are mostly corn and soybean operations and cattle and hog production facilities (U.S. Department of Agriculture Census of Agriculture, 2012). Each farm type exhibits exposure risks to hazardous noise and dust. Personal protective equipment (PPE; i.e., facial masks or respirators, ear plugs, or ear muffs) has been available for many years and, when used properly, can protect workers from developing respiratory diseases or hearing impairments (OSHA Fact Sheet, U.S. Department of Labor, 2006). However, little evidence suggests that agricultural workers routinely wear PPE to reduce their exposures (Schenker, Orenstein, & Samuels, 2002). The Ag Center goals included promoting PPE, and accomplished its work through the four core areas of administration, research, prevention/intervention, and education.

Organizational Structure

Depicted in Figure 1, the Ag Center has a clear organizational structure, which facilitates well-defined routes of communication, shared governance, and collaboration among projects and programs, including evaluation. The Administrative Core provides the process that connects all cores, programs, and advisory committees into a cohesive effective leadership structure. Many of the members and leaders are university faculty and leaders of agriculture-related organizations (Table 1).

Figure 1.

Midwest agricultural center organizational chart.

Table 1.

Agricultural Center Members/Leaders Participation

Measure/method	Members^a	Leaders^b	Both members and leaders
ICE^© Survey (quantitative)
Year 1	12/19 = 63%	10/13 = 77%	22/32 = 68%
Year 3	17/27 = 63%	11/11 = 100%	28/38 = 74%
Field Visit (qualitative)
Year 1	3/3	2/2	5/5 = 100%
Year 3	3/3	3/3	6/6 = 100%

Note. ICE = Internal Coalition Effectiveness.

Members are defined as those who participate in essential work on a regular or intermittent basis; includes community participants, committee/project/team members, advisory/consultant/participating personnel, or others deemed essential to the work of the project.

Leaders are defined as those in positions of accountability; includes principal and co-investigators; chairs/co-chairs of committees, teams, project, boards; participating agency directors.

The Internal Advisory Committee is comprised of core and program directors and administrative staff as shown in the organizational chart (Figure 1). This group meets monthly to assist the Center Director with scientific and administrative decisions about Center operations.

The External Advisory Committee is composed of multidisciplinary and geographically diverse agricultural experts who meet annually to review evidence regarding the Ag Center’s outcomes and impacts. The yearly meeting provides an open exchange of ideas, information, approaches, and knowledge, particularly regarding translating research to practice, linking researchers and stakeholders, and identifying effective outreach channels.

Evaluation

The evaluation team is internal to the Ag Center but operates as an independent entity that reports to the Ag Center leadership and both advisory committees to address overall program improvement based on findings. Three aims were developed to achieve Center goals including process evaluation monitored by the project leader’s database entries, outcomes and end results measurement using surveys and other tools, and a leadership and governance focus. Meeting schedule is shown in Table 2.

Table 2.

Ag Center Meeting Schedule With Evaluation Touchpoints

Type of meeting	Purpose	Who attends
Monthly Member	Mechanism for all member communication	All leaders and members are invited in person or by distance modalities.
Monthly Internal Advisory	Assists the Center Director with scientific and administrative operational decisions	Center Director, Project Coordinator, Deputy Director, Administrator
Annual EAB	Obtain feedback from Ag Community	12 members of EAB and all project members/leaders
Regular Evaluation	Monitor milestones and metrics	PI (the primary researcher with responsibility for completing the goals of their particular project) and Evaluation Coordinator
Regular Leadership and PI	Stay abreast of projects	Five-year-funded PIs, Center Director and Coordinator
Periodic Evaluation Coordinator and Center Coordinator	Review findings	Evaluation and Center Coordinator

Note. PI = Project Investigator; EAB = External Advisory Committee.

The specific aims of this study were to answer three key evaluation questions: (a) What are the organizational strengths of the Ag Center? (b) What are areas for improvement in leadership and governance? and (c) Did recommendations implemented from baseline evaluation result in improvements at Year 3? The evaluation study was conducted at two points in time. Year 1 provided a baseline of information that generated specific recommendations for the Ag Center’s governance, and a second evaluation at Year 3 provided a measure of effectiveness regarding the recommendations. The study was approved by the organization’s Institutional Review Board.

Method

The evaluation team conducted a mixed method explanatory sequential study (Figure 2); the study first used quantitative methods and then employed qualitative techniques to explain the quantitative results in more depth (Creswell, 2015). Quantitative data were collected using the Internal Coalition Effectiveness (ICE^©) instrument (Cramer, Atwood, & Stoner, 2006a). The results were reviewed, and a qualitative survey questionnaire was developed to further describe the areas needing additional discernment. The questionnaire responses provided greater understanding through personal examples, and provided data for developing recommendations that could strengthen and improve programs (McDonald & McDivitt, 1998; National Institutes of Health, 2011; Pincus, Abedin, Blank, & Mazmanian, 2013). O’Brien’s Standards guided the qualitative data collection process (field interviews) and reporting (O’Brien, Harris, Beckman, Reed, & Cook, 2014). Both methods were employed in Year 1 (baseline) and Year 3 (second evaluation) of the project 5-year funding cycle.

Figure 2.

Mixed methods explanatory sequential design.

Figure 3.

Internal Coalition Outcomes Hierarchy (ICOH).

The sample for the ICE^© instrument included all Ag Center members and leaders at baseline (n = 32) and second evaluation (n = 38). In accordance with the ICE^©, members and leaders each served a particular role. Members were defined as those who participated in the work of the Ag Center on a regular or intermittent basis (i.e., community participants, advisory committee members, project team members, and consultant personnel). Nineteen members participated at baseline and 27 members participated in the second evaluation (Table 1). Leaders were individuals in designated roles of accountability, and, for purposes of this evaluation, the leaders were defined as the Center Director, Deputy Director, Coordinator, and Core and Project Directors (baseline n = 13, second evaluation n = 11).

The sample for the baseline field interviews was pilot project investigators (18-month funding) and fully funded investigators (5-year funding) who were located external to the University of Nebraska Medical Center (UNMC) campus. During the second data collection phase, the sample included pilot project investigators and core directors who had not been previously interviewed and were purposively selected as rich data sources. At baseline, five in-person field interviews were conducted (n = 3 members, n = 2 leaders) at site locations across the seven-state region served by the Ag Center. Three new pilot investigators and two project directors were interviewed at their individual locations in Fargo, ND; Fremont, NE; Minneapolis, MN; and Boone, Ames, and Urbandale, IA. During the second evaluation, six in-person field interviews were conducted (n = 3 members, n = 3 leaders) at site locations across the region. The locations were Omaha, NE (n = 4) and Spencer and Iowa City, IA.

Measures/Instruments

Quantitative

The ICE^© instrument corresponds to the Internal Coalition Outcomes Hierarchy (ICOH) framework (Cramer, Atwood, & Stoner, 2006b). The instrument is valid and reliable for measuring the organizational effectiveness of coalitions. It has 30 total items across six theoretical constructs of the ICOH model (Cronbach’s α ≥ 0.70 for six of seven subscales with items for Resources, α = −.30 to 0.70 excluded from the instrument; Cramer et al., 2006a). The six constructs of the ICOH and ICE^© include Social Vision, Efficient Practices, Knowledge & Training, Relationships, Participation, and Activities. The 30 items are separated into two sections: Section 1 includes 13 items to rate member contributions to the coalition and Section 2 includes 17 items to rate Leader contributions to the coalition. Members and leaders rate their own and each other’s contributions to coalition success. Mean scores for members versus leaders are compared overall (Figure 4) and at each construct level to determine congruency or divergence of perspectives.

Figure 4.

ICE^© instrument combined member/leader responses (baseline and Year 3).

Qualitative

Following results from the baseline ICE^© survey, a focused interview guide was developed with leadership input to provide greater insights and personal perspectives on constructs of the ICOH model. The same interview guide was used during the Year 3 evaluation. The guide relied on open-ended questions that included the following:

Social Vision: familiarity and perceptions of the Ag Center’s organizational structures, mission, and goals

Relationships and Participation: satisfaction with collaborations within the Ag Center

Efficient Practices, Knowledge, and Training: satisfaction with new knowledge, training, and practices gained from participation in the Ag Center

Resources: satisfaction with Ag Center’s operational support and resources

Activities: progress on projects; participants were encouraged to offer specific examples and suggestions for improvement in each of the areas.

Procedures

Quantitative

The ICE^© instrument was formatted in SurveyMonkey and sent via email both years. The Dillman-tailored design (Dillman, Smyth, & Christian, 2009) was used to maximize response rates. The first emails included a cover letter and a link to the survey. A few days later, reminder emails were sent. Approximately 1 week later, the final email requests for participation were sent to all nonrespondents.

Qualitative

Interviews were conducted in person by a single evaluator for approximately 1 hour. The same questions were asked of all participants. The evaluator sent the interview guide to the participants in advance so they could consider their responses. The interviews were audio taped, transcribed, and coded to analyze themes. Raw data were again analyzed using an integrated qualitative analysis that incorporated both inductive development of codes as well as a deductive organizing framework (Bradley, Curry, & Devers, 2007). Leader data were analyzed separately to identify general agreement or disagreement with members’ findings.

Analytic Strategy

Quantitative

The ICE^© instrument data were analyzed using Excel for descriptive statistics by category responses (i.e., member responses, leader responses, and combined). All p values < .05 were considered significant.

Qualitative

The interview themes were studied using an integrated qualitative analysis incorporating conceptual codes and subcodes (Bradley et al., 2007). The qualitative interview responses were then used to clarify the quantitative results, the objective of the explanatory sequential research design (Creswell, 2015).

Results

Integrated Baseline Findings

The survey response rate was 68% (n = 22/32), with 63% (n = 12/19) of member responding and 77% (n = 10/13) of leaders responding (Table 1). Integrated analysis from the survey and interviews showed that the most commonly identified strength was commitment to the Ag Center’s vision and goals (p >.05). However, the interviews also revealed a general lack of familiarity with the Ag Center’s organizational structure. Respondents were not aware of the lines of communication or functions of the Advisory Committees. Most seemed comfortable just knowing how to contact the Project Coordinator, who would put them in touch with others in the Ag Center.

Overall growth opportunities for the Center included respondents noting Knowledge as somewhat lower in ranking; the second lowest ranking was Activities; and respondents seemed less clear about Ag Center advancement on goals and aims related to the total project.

Members rated Practices, Participation, and Knowledge slightly higher than did leaders.

Leaders rated Social Vision, Relationship, Activities, and Overall Effectiveness somewhat higher than members (Figure 4).

Qualitative themes

The field visit response rate was 100% (n = 5/5; 3 members and 2 leaders; Table 1). Interview responses are summarized in the following section with a sampling of interviewee quotes by ICOH construct (Table 3).

Social Vision: Respondents noted an adequate understanding of the Ag Center Mission and Goals, but lack of familiarity with its Organizational Structure. All respondents said they generally understood the Ag Center’s mission and goals but most were not familiar with the organizational structure of the Ag Center’s leadership and governance.

Relationship and Efficient Practices: Respondents reported high levels of satisfaction with the Ag Center partnership and saw the Ag Center as a good partner. The valued resources mentioned included leadership personnel, assistance with proposal writing, and funding support. All respondents were satisfied with the level of communication from the Ag Center. Three respondents noted they were looking forward to future research collaborations with the Ag Center using larger external funding mechanisms.

Participation: Opportunities to strengthen the Ag Center included concerns about distance as a barrier to participation in Ag Center activities. One respondent said the most significant barrier was working with a large organization.

Relationships and Practices: Valued scientific collaboration was noted as the Ag Center was viewed as a scientific partner by respondents. They valued the opportunity to work and learn from the Ag Center’s senior scientists and mentioned the excellent training and support they received in learning about requests for proposals, conferences, and grant preparations.

Efficient Practices and Activities: Respondents valued the experience and planned to use current funding to launch additional research projects.

Table 3.

Ag Center Field Visit Interview Results Categorized by ICOH Construct

ICOH construct	Year 1 participant quotes	Year 3 participant quotes
Social Vision	Cannot recite but we are in a parallel line, assume most Ag Centers are about the same	Very familiar I have the org chart Does the best job of all the centers in outreach A recent example is with the bird flu flyer . . .
Relationship and Efficient Practices	Yes, a positive experience Glad to see they are receptive to partner [with] those outside the state university, It is a new center and figuring out its role . . . kind of exciting The PI and Coordinator are wonderful, excellent people all the way [They] are always communicating with me—inclusiveness is fabulous	Most part an open communication, meet on a monthly basis and discuss issues, Appreciate each other’s work and try to communicate.
Participation	Distance is the toughest Distance, and we deal well, but it is still a barrier—can’t go into someone’s office for a bit of advice Multiple levels, moves slower, and multiple layers due to large organization. Need to use more Adobe Connect, Skype for communication) We need to work with rural farmers better and develop satellite locations. Would like to see more social marketing	. . . these mtgs. brought insurance industry professionals to us and giving me a better understanding of their role in Ag safety education and practices I would typically not have said I would gain as much from more research people but I found good interaction with them and they helped me think about things differently
Relationship	Helped me with funding to do the pilot which I could not do on my own Good to work with in any capacity	. . . functions admirably well and essentially fit into the field in traditional manner. Within context of being a diverse center (heavily weighted toward injury and less about health and disease) . . . there are communication challenges).
Efficient Practices and Activities	They can use the respiratory data and resources so a sharing of outcomes evolve so many people pulling in the same direction AgrAbility applications maintain lifestyle and community contributions	Absolutely on target with project It has launched us to something else. We are going to put in distinctly new project Example of invoice for expenses and agreed to take minutes for the mtg. tonight and stand in for RR Nothing but great experiences with actual money management from JP’s side of things; no negative issues whatsoever.

Note. ICOH = Internal Coalition Outcomes Hierarchy.

Organizational Action Taken

Baseline survey findings and field visit interview results were shared with the leadership group, resulting in multiple organizational action steps. The communication venue was mainly through the Ag Center meeting schedule (Table 2). In response to the finding that Knowledge was regarded somewhat lower in ranking, project investigators were asked to present their projects, soliciting feedback from the team to enhance outcomes, and an organizational chart was posted on the website.

The second lowest ranking was Activities; respondents seemed less clear about the Ag Center advancement on goals and aims related to the total project. The Evaluation Core addressed this by providing more formative feedback/reports (Table 2). Quarterly reminders were sent to members for data entry into the Ag Center database, and periodic updates were presented on how data were used. The Center coordinator sent emails to update the team on new media releases, awards, and resources related to Ag Center projects to keep all partners informed. The entire Ag Center team was invited to the annual external advisory committee meeting to present their projects and discuss outreach suggestions. Leadership provided informal opportunities for leaders/members to know other investigators and form collaborations. Assistance and orientation were provided to new grantees via “field interviews” so they could better understand “the system,” including regular budget updates. The Evaluation team provided member education on the Ag Center Logic Model evaluation tool and on the value of “success stories” as a result of their research. Distance was a reported factor in coalition functioning during each round of field visit interviews; this finding was consistent with other studies noting geographic distance as a major challenge for community partners (Chen, Roberts, Xu, Jacobson, & Palm, 2012). In addition to actions described above to decrease the effects of this factor, leadership continued to offer and improve communication access via distance conference technology.

Integrated Year 3 Findings

The survey response rate in Year 3 was 74% (n = 28/38), 63% of members (n = 17/27) and 100% of leaders (n = 11/11). No significant differences in results between Year 1 and Year 3 surveys were found (Figure 4). The Center continues to have strong levels of effectiveness in each of the conceptual constructs of a coalition (Figure 2). In particular, a strong sense of shared Social Vision and Mission and highly collaborative Relationships among Center members/leaders was found.

Members comparison: Ratings in each of the conceptual constructs increased in the Year 3 survey and were high overall. Members rated Activities lower than other categories indicating that Ag Center members may be less certain about the degree to which aims/goals and the work plan are implemented in their entirety and on a timely basis (Figure 5).

Leaders comparison: Leaders continued to rate Participation at the same high point, with Relationship continuing as the highest construct (Figure 6).

Figure 5.

ICE^© instrument member responses (Year 1 and Year 3).

Figure 6.

ICE^© instrument leader responses (Year 1 and Year 3)

Qualitative themes

The field visit response rate for those who were contacted for interviews was 100% (n = 6/6) with three interviews of members and three interviews of leaders (Table 1). Interview responses are summarized in the following section with a sampling of interviewee quotes per ICOH constructs shown in Table 3.

Social Vision: Familiarity with Organizational Structure, Mission, and Goals: All respondents reported a strong familiarity with the organizational chart. This group not only strongly agreed that the Ag Center was achieving its mission, but also gave examples.

Relationship and Efficient Practices: As in the baseline field visit responses, participants were eager to continue the affiliation. Higher levels of involvement were identified in responses.

Participation: Center members reported improved understanding of agricultural health through their association with the Ag Center. Other ideas were to establish a cohort of patients over time; consider other stakeholders such as insurance companies, Farm Bureau, and Extension offices; continue encouraging other Ag Centers to make National Agricultural Safety Database (NASD) contributions in the new funding cycle; and consider options to overcome geographical barriers, as all are essential to the others.

Relationship: This category also showed more informed responses from baseline.

Efficient Practices and Activities: All respondents were working to keep the project “on target.” The evaluator attended an Ag Center pilot project meeting of Certified Safe Farms at the International Society of Agricultural Safety and Health (ISASH) annual meeting and participated in a bimonthly telephone conference with national evaluators, where NASD project updates were discussed.

All project investigators have plans for requesting additional funding and subsequently publishing findings.

All participants reported efficient practices with the leadership which improved since baseline. Several participants reported improvement in monthly member meetings due to sharing project updates.

Improvement Following Action

The outcome of this study revealed improved familiarity with the Center organization and its mission and accomplishments (i.e., response to bird flu epidemic, increasing Ag event participation, new partnership possibilities such as introduction to insurance companies). Several participants reported improvements in monthly member meetings, including shared project updates and focused agendas. All respondents reported a strong familiarity with the organizational chart as opposed to a fair level of understanding at baseline. Findings from the field visits showed that investigators were highly satisfied with the Center’s support and resources for their research and all requested ongoing affiliation through list serves or newsletters. As the grant years passed, members/leaders were more familiar with the organizational structure showing growth in Knowledge and Relationships.

Limitations

Study limitations included a small number of interviewees, a purposive sample, a different group of interviewees from baseline to Year 3, narrow ICE^© scoring latitude, and the fact that baseline interviews were completed when the Center program was new. Baseline field interviews were conducted with fully funded project investigators; however, they were not core leaders.

Discussion

Effective leadership and governance are crucial to the success of programs and projects (Janosky et al., 2013). Coalitions are complex and require significant skill to initiate, lead, and evaluate (Clark-McMullen, 2010). According to Butterfoss and Francisco (2004), “Community coalitions and partnerships can be powerful agents for social change and for solving complex public health challenges (p. 113).” The results of this study reveal the continuing successful leadership and governance of the Ag Center. In particular, the organization demonstrates strength in their shared Social Vision and Mission, and highly collaborative Relationships among Center members and leaders.

Opportunities for organizational growth were identified in the two lower scoring ICOH constructs of Knowledge and Activities. Leadership modifications designed to address these themes included reorganizing the monthly Ag Center meeting to include presentations on projects; posting an organizational chart on the Ag Center website; increasing emails to the team regarding all project updates on new media releases, awards, and resources; inviting all team members to the annual external advisory committee meeting; hosting regular meetings with Project Investigators; providing informal opportunities for leaders and members to know other investigators; providing assistance and orientation for new grantees; and presenting education on the Ag Center Logic Model evaluation tool and the value of “success stories” as a result of research studies.

The outcome of this study revealed improved familiarity with Center organization and accomplishments. Several participants reported improvements in monthly Central States Center for Agricultural Safety and Health (CS-CASH) member meetings including shared project updates and focused agendas. All respondents reported familiarity with the organizational chart as opposed to only fair understanding at baseline. Findings from the field visits showed that investigators were highly satisfied with the Center’s support and resources for their research and all have requested ongoing affiliation through collaboration on research funding, list serves, and newsletters. During the Year 3 interviews, members were able to give examples of goal attainment such as outreach participation in events and development of relevant resources for the community and building substantial inroads in Nebraska and the seven-state region such that the Center’s profile and reputation have been growing over the past 4 years, all evidence of how CS-CASH is achieving its mission.

In conclusion, strong collaborative leadership is vital to successful population health improvement programs. The ICOH constructs provide elements that may be basic for a functioning team but can be challenging with diverse work groups (Bennett & Gadlin, 2012). A mixed methods evaluation approach highlighted coalition qualities in a broader environment, capturing a clearer depiction of leadership and member interactions. Findings were shared with the coalition to identify strengths and areas for quality improvement, thereby promoting coalition sustainability (Cramer & Wendl, 2015). Liaisons are being developed by the Center, and investigators are appreciative of the resources and support that the Center brings to them. Longitudinal studies will continue to evaluate progress and provide data for continued quality improvement.

Implications for Practice

Work settings for occupational health nurses can include corporate offices as well as various community locations. Many occupational health nurses are employed in agricultural health, involving a multitude of worksites for which this study has practical applications. This article also has implications for occupational health nurses who frequently guide and evaluate partnerships and community-based coalitions for efficiency and economy (Clark-McMullen, 2010). The base competencies for occupational health nursing certification include management and evaluation of programs similar to the example in this article. These findings can be applied to corporate and community occupational health nursing situations to improve the effectiveness of leadership and demonstrate a successful program to funders. The occupational health nursing literature review showed a gap in relevant articles about coalition leadership evaluation methods. Evaluation is an essential occupational health nursing task, and this study provides an evaluation approach for program leaders, including nurses, who often serve as program or project evaluators.

Applying Research to Practice

Ag Centers are coalitions, which broaden individual efforts to reduce agricultural illnesses, injuries, and fatalities. Agricultural health and safety programs are often most successful when coalitions are formed with strong evaluation plans.

The ICE^© instrument is a validated tool that can assist in evaluating coalitions. A mixed methods approach includes focused interviews and provides deeper insight from evaluation data.

The occupational/agricultural health nursing role often requires interprofessional practice and multiple stakeholder coalitions. Evaluation methods for these coalition infrastructures can assist nurses in fulfilling their varied job functions.

The methods in this study can be used in other community health and safety programs to document effective leadership and governance, and aid in coalition sustainability.

Footnotes

Conflict of Interest

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this project was provided by Central States Center for Agricultural Safety and Health (CS-CASH) NIOSH (U54 OH010162), Project Director, Risto Rautiainen, PhD, and by the National Institute of General Medical Sciences, 1U54GM115458-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health (NIH).

Author Biographies

Mary J. Wendl is the evaluation coordinator for the Central States Center for Agricultural Safety and Health (CS-CASH) at UNMC. She is nationally certified in occupational health nursing.

Mary E. Cramer is a professor at UNMC College of Nursing. She teaches health policy and her research expertise is community-based participatory research and evaluation of community-based programs and coalitions.

References

Alexander

B. H.

Bender

J. B.

Kampa

Keifer

M. C.

Liebman

A. K.

Peterson

. . . Sheridan

(2016). Growing agricultural education: Embracing health and safety. Journal of Agromedicine, 21, 298-300. doi:10.1080/1059924X.2016.1179240

Batras

Duff

Smith

B. J.

(2016). Organizational change theory: Implications for health promotion practice. Health Promotion International, 31, 231-241. doi:10.1093/heapro/dau098

Bennett

L. M.

Gadlin

(2012.). Collaboration and team science: From theory to practice. Journal of Investigative Medicine, 60, 768-775. doi:10.2310/JIM.0b013e318250871d

Bradley

E. H.

Curry

L. A.

Devers

K. J.

(2007). Qualitative data analysis for health services research: Developing taxonomy, themes, and theory. Health Services Research, 42, 1758-1772. doi:10.1111/j.1475-6773.2006.00684.x

Butterfoss

F. D.

Francisco

V. T.

(2004). Evaluating community partnerships and coalitions with practitioners in mind. Health Promotion Practice, 5, 108-114. doi:10.1177/1524839903260844

Centers for Disease Control and Prevention. (2016). The National Institute of Occupational Safety and Health (NIOSH). Retrieved from https://www.cdc.gov/niosh/oep/agctrhom.html

Chen

L. W.

Roberts

Jacobson

Palm

(2012). Effectiveness and challenges of regional public health partnerships in Nebraska. Journal of Public Health Management and Practice, 18, 148-155. doi:10.1097/PHH.0b013e318239918f

Clark-McMullen

D. M.

(2010). Evaluation of a successful fetal alcohol spectrum disorder coalition in Ontario, Canada. Public Health Nursing, 27, 240-247. doi:10.1111/j.1525-1446.2010.00849.x

Cramer

M. E.

Atwood

J. R.

Stoner

J. A.

(2006a). A conceptual model for understanding effective coalitions involved in health promotion programming. Public Health Nursing, 23, 67-73. doi:10.1111/j.0737-1209.2006.230110.x

10.

Cramer

M. E.

Atwood

J. R.

Stoner

(2006b). Measuring community coalition effectiveness using the ICE instrument. Public Health Nursing, 23, 74-87. doi:10.1111/j.0737-1209.2006.230111.x

11.

Cramer

M. E.

Wendl

M. J.

(2015). Children’s agricultural safety network: Evaluating organizational effectiveness and impacts. Journal of Agromedicine, 20, 105-115. doi:10.1080/1059924X.2015.1010067

12.

Creswell

J. W.

(2015). A concise introduction to mixed methods research ( Knight

, Ed.). Thousand Oaks, CA: SAGE.

13.

Dillman

Smyth

Christian

(2009). Internet, mail and mixed-mode surveys: The tailored design method (3rd ed.). Hoboken, NJ: John Wiley.

14.

El Ansari

Phillips

C. J.

Zwi

A. B

. (2004). Public health nurses’ perspectives on collaborative partnerships in South Africa. Public Health Nursing, 21, 277-286. doi:10.1111/j.0737-1209.2004.021310.x

15.

Janosky

J. E.

Armoutliev

E. M.

Benipal

Kingsbury

Teller

J. L.

Snyder

K. L.

Riley

(2013). Coalitions for impacting the health of a community: The Summit County, Ohio, experience. Population Health Management, 16, 246-254. doi:10.1089/pop.2012.0083

16.

McDonald

McDivitt

J. A.

(1998). Evaluation challenges in agricultural health and safety centers. Journal of Agricultural Safety and Health, 4(Suppl. 5), 139-147.

17.

Mueller

(2007). “When doing good is just the start to being good”: A possible tool to improve the organizational effectiveness of non-profit health care organizations. Journal of Hospital Marketing & Public Relations, 17(2), 45-60. doi:10.1300/J375v17n02_05

18.

National Institutes of Health. (2011). Principles of community engagement. Washington, DC: CTSA Community Engagement Key Function Committee Task Force on the Principles of Community Engagement. Retrieved from http://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf

19.

O’Brien

B. C.

Harris

I. B.

Beckman

T. J.

Reed

D. A.

Cook

D. A.

(2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine: Journal of the Association of American Medical Colleges, 89, 1245-1251.doi:10.1097/ACM.0000000000000388

20.

Pincus

H. A.

Abedin

Blank

A. E.

Mazmanian

P. E.

(2013). Evaluation and the NIH clinical and translational science awards: A “top ten” list. Evaluation & the Health Professions, 36, 411-431. doi:10.1177/0163278713507061

21.

Schenker

M. B.

Orenstein

M. R.

Samuels

S. J.

(2002). Use of protective equipment among California farmers. American Journal of Industrial Medicine, 42, 455-464. doi:10.1002/ajim.10134

22.

U.S. Department of Labor. (2006). OSHA Fact Sheet. Retrieved from https://www.osha.gov/OshDoc/data_General_Facts/ppe-factsheet.pdf

23.

U.S. Bureau of Labor Statistics. (2015). Current Population Survey, Census of Fatal Occupational Injuries. Retrieved from https://www.bls.gov/iif/oshcfoi1.htm

24.

U.S. Department of Agriculture Census of Agriculture. (2012). State summary highlights: 2012. Retrieved from http://www.agcensus.usda.gov/Publications/2012/Full_Report/Volume_1,_Chapter_2_US_State_Level/st99_2_001_001.pdf

25.

Zakocs

R. C.

Edwards

E. M.

(2006). What explains community coalition effectiveness? A review of the literature. American Journal of Preventive Medicine, 30, 351-361. doi:10.1016/j.amepre.2005.12.004