Abstract
Background: Assessment of workplace safety climate is a recognized approach to assess safe culture in an organization. We developed, implemented, and evaluated a 12-module safety leadership and management e-learning training program tailored for front-line dairy farm supervisors and hypothesized that such a training program would have a positive effect on occupational safety climate. Methods: We enrolled 103 dairy farm supervisors from 35 U.S. farms to participate in a 12-module dairy leadership training program. We assessed safety climate change among supervisors and their workers using a pre-post training intervention methodology. A 24-item, 8-factor safety climate survey was developed and completed pre- and post-training by 65 supervisors and 313 (pre) and 238 (post) workers under their supervision. Findings: Significant improvements were seen on the 14-item supervisor safety climate scale, the 10-item group safety climate scale, and subscales regarding safety priority, empowerment, and dealing with conflicts. No significant improvements were seen for the factor regarding leaders encouraging workers to report safety incidents, as these were already rated highly. Conclusions: Findings suggest safety leadership e-learning training among dairy farm supervisors can result in positive safety climate change among supervisors and subordinate workers. Application to Practice: This study provides a framework for researchers, safety professionals, and training developers for the successful implementation of a safety leadership training program, and its effect on safety climate. Additional research is needed on the effectiveness and sustainability of safety leadership training in high-risk industrial sectors such as agriculture.
Background
As dairy farms in the United States (U.S.) and other countries continue to expand their herd sizes to capitalize on economies of scale and maximize profits and sustainability, farm owners seek effective supervisory leadership to oversee expanding workforces (Barkema et al., 2015). Large dairy farms involve complex systems with inherent occupational safety, health and well-being (OSH) hazards, which increase the risk for injuries, illnesses, and fatalities (Driscoll et al., 2022; Panikkar & Barrett, 2021). Communicating these risks is challenging, as dairy farm workers often have low-literacy and proficiency in the region’s primary language. In light of the inherent (OSH) hazards on dairy farms, dairy workers remain a vastly understudied population, and limited research has addressed gaps in safety intervention efforts leading to the reduction of unintentional injuries in this vulnerable working group (Coman et al., 2020).
Farmworkers, and in particular immigrant workers in Canada and the U.S., often feel that they have little control over their own OSH and rarely complain, in part due to job insecurity, coercion, and fear of deportation (Arcury et al., 2012; Caxaj & Cohen, 2019). Farmworkers’ beliefs in their abilities to affect safety, an internal safety locus of control, has shown to be correlated with safety climate (Cigularov et al., 2009). Coupled with this are the farmers’ and supervisors’ own focus on production goals and risk tolerance. Faced with both an insecure working climate, high risk tolerance and a focus on milk production, immigrant workers on farms can resort to a fatalistic approach to work (Arcury et al., 2015).
In this project, we developed, delivered, and evaluated a safety leadership and management e-learning training program tailored for front-line dairy supervisors. This paper is the second of a three-part series which evaluates the training effects of the training program for dairy farm supervisors. This paper addresses the fourth level of Kirkpatrick’s four-level model of training effectiveness, with a focus on evaluating the impact of the training program on organizational results (Kirkpatrick & Kirkpatrick, 2016). This paper supplements two accompanying papers dealing with level 1 evaluation—reaction to the training program—of the Kirkpatrick model (Douphrate, Rodriguez, Kines, Javid, et al., 2024), and levels 2 and 3—learning and behavior change respectively (Douphrate, Rodriguez, Kines, & Javid, in press). This study will fill a research gap in terms of evaluating the effects on safety climate of a safety leadership and management training program for front-line supervisors on large-herd U.S. dairy farms.
In keeping with the evidence of what is effective in preventing safety incidents at work (Dyreborg et al., 2022), we hypothesized that a multifaceted safety leadership and management training program for front-line dairy farm supervisors will have a positive effect on safety outcomes including safety climate. Workplace safety climate is a well-established concept using survey tools that provide a snap-shot of the safety culture (and sub-climates and sub-cultures) at various levels in an organization (e.g., leaders and workgroups) regarding safety practices and priorities as perceived by leaders and workers (Mearns et al., 2003). Safety climate has been shown to be a reliable predictor of safety outcomes (e.g., safety behavior, injuries, accidents) in many sectors (Zohar, 2010), and therefore often used in intervention studies where a more rigorous evaluation of the effects on injuries and fatalities is not efficient for short-term research designs (Driscoll et al., 2022).
There has been some research on safety climate in the agricultural (i.e., livestock) sector, particularly in the U.S. (Arcury et al., 2012, 2014, 2015, 2020; Kearney et al., 2015; Schwatka et al., 2010; Swanberg et al., 2017; Whalley et al., 2009), but also in Australia (Pollock et al., 2018), France (Grimbuhler & Viel, 2019), Italy (Fargnoli & Lombardi, 2020, 2021), Norway (Kjestveit et al., 2021), as well as in a multinational study in Europe (Leppälä et al., 2021). Much of this latter research has been done within the last few years, running parallel with this current study. Although some studies included dairy farms in larger agricultural sector studies, they have not reported on the dairy farm specific results (Cigularov et al., 2009; Fargnoli & Lombardi, 2020)
Applying Research to Occupational Health Practice
This study provides a framework to develop and deliver safety leadership training content using e-learning technologies. When time and budgetary constraints limit the assessment of a training program’s effect on safety outcomes such as injuries or fatalities, other safety outcomes can be considered. Prior research has demonstrated that safety climate is a reliable predictor of safety outcomes. As a result, safety climate change can be regarded as a safety outcome in a Kirkpatrick Level 4 training effectiveness evaluation (Kirkpatrick & Kirkpatrick, 2016). We developed, implemented, and evaluated a safety leadership training program among supervisors in an industry recognized as high-risk for injuries and fatalities (i.e., agriculture). We demonstrated the positive training effects on safety climate among supervisors and workers. The methods used in this study can be applied in other high-risk industries such as construction, healthcare, forestry, or manufacturing to assess safety leadership training effect on safety climate.
Methods
Training Development
Development of the safety leadership training intervention was based on a training needs assessment conducted among dairy farm owners and supervisors in seven western U.S. states. The farm owners and supervisors informed the training format, duration, content, and farm relevance, and were provided the opportunity of training review and final endorsement prior to commencement of the training program.
Training Content
The training program was designed to build supervisory skills, which ultimately would lead to improved worker safety performance (Petersen, 2001). The training took into account the relationships-oriented culture that is characteristic of Hispanic working populations (Conejo, 2001). Twelve e-learning modules and a 13’th summary module were developed and made available in both English and Spanish, with each module taking up to a half hour to complete. The training topics dealt with: common causes of injuries and fatalities, safety hazards, hazard recognition and prevention, supervisory and worker roles in safety, modeling safety, effective safety communication (delivered over two modules), safety discipline, safety meetings, workplace conflicts, and safety culture.
Mechanism of Training
The training modules were delivered to the supervisors over a 16 to 35 week period on their on-line or off-line mobile devices (smartphones or tablets), and included interactive opportunities that were developed using Articulate® Storyline 2 training software. At the end of each module, supervisors were encouraged to regularly carry out and to later report on their newly learned safety leadership behaviors.
Farm and Participant Recruitment
We soliced the help of dairy extension specialists in intending to recruit 120 supervisors between 2017 and 2022. This was however, hampered by COVID-19 outbreaks between 2020 and 2022. A total of 37 large-herd dairy farm owners across seven U.S. states were called, visited and subsequently invited to participate in the study. We ceased recruitment after enrolling a convenience sample of 103 supervisors due to continuing COVID-19 related health and travel restrictions.
Inclusion criteria for the supervisors included having a supervisory position for at least five workers, for at least half a year. The University of Texas Health Science Center at Houston Committee for the Protection of Human Subjects (HSC-SPH-16-0559) approved all study procedures, and written informed consent was obtained from all the supervisors.
Study Design
We employed a pre-post intervention longitudinal study methodology, which is a novel approach in OSH research in the agricultural sector. We developed and evaluated the effects of the multifaceted safety leadership training program on supervisor and worker safety climate between 2017 and 2022 on large-herd dairy farms in four western U.S. states. We hypothesized that the intervention would result in statistically significant increases on eight safety climate factors, as outlined in the following paragraph.
Data Collection
A 24-item, eight-factor safety climate survey was developed with 14 items related to respondents’ perceptions of supervisor’s behavior, and 10 related to work-crew behavior (Table 2). The items were adapted from the Empowering Leadership Questionnaire (ELQ; Arnold et al., 2000) and the Nordic Occupational Safety Climate Questionnaire (NOSACQ-50; Kines et al., 2011). A 6-point Likert scale from 1 to 6 was used: “Never (0%)” (score 1—poor safety climate), “Rarely (under 20%),” “Occasionally (20%–40%),” “Often (41%–60%),” “Almost always (81%–99%),” and “Always (100%)” (score 6—good safety climate). A “Not relevant” option was included.
The same 24 items were given to both supervisors and workers, with the wording modified in the supervisor survey to reflect self-evaluation for the 14 supervisor behavior items, for example, “I encourage my crew to do their job safely, even when we are behind schedule” (supervisor version), versus “My supervisor encourages my crew to do their job safely, even when we are behind schedule” (worker version). The 14 supervisor-focused items addressed four themes of safety priority (five items), safety empowerment (five items), encouraging the reporting of incidents (two items) and dealing with conflicts (two items).
The 10 work-crew -focused items referred to the work crew (group level) with each item beginning with “Workers in my crew . . .,” as opposed to an individual level approach. This reflects on the safety culture approach, whereby “culture” is a reflection of “sharedness” within a group. The items addressed two themes—work crew priority of safety (eight items) and reporting of injuries and near misses (two items).
The worker survey was made available in English and Spanish and administered in person by bilingual research personnel. Worker responses were recorded electronically on mobile devices to minimize expenses and maximize efficiency of data collection. Workers under the supervision of the dairy supervisors enrolled in the intervention group completed the survey before and after the supervisor training intervention. The survey instrument was subject to field-testing and exploratory factor analysis using Stata/SE 17.0 software (StataCorp, 2021).
Data Management and Analysis
QualtricsXM Offline Surveys application (Qualtrics, Provo, UT) centrally housed study surveys administered on portable tablet devices. Encrypted institutional QualtricsXM software stored all deidentified data collected from supervisors throughout their training. Only research personnel had access to electronically stored data. All data was stored on a secure institutional server with password access by research team leadership. Data was downloaded periodically, prepared, and analyzed for preliminary finds and annual funder reports.
Pre and post-intervention survey data for each of the 24 items and eight factors were analyzed for data reliability (Cronbach Alpha, α), and significant pre to post changes (t-test) using SPSS version 29 (IBM Corp, 2021). Alpha values over .7 were considered acceptable (Taber, 2018), and p-values less the .05 in the t-tests were considered statistically significant.
Results
Participants
A total of 103 supervisors enrolled in the training program from 35 farms in six U.S. states, and 73 completed all the training modules. For this analysis a total of 65 supervisors from 28 farms completed all training modules and both pre- and post-safety climate surveys (Colorado: 17 of the 27 completed both; Kansas: 18/25; Nebraska: 0/2; New Mexico: 0/9; Oklahoma: 3/3; and Texas: 27/37; Table 1). A total of 38 supervisors did not complete all modules and pre- and post-surveys. Dropout was due to competing organizational priorities at the time of the study, natural employment turnover, farms closing, career changes, migration, workplace conflicts, medical procedures, and sickness absence (e.g., due to the COVID-19 pandemic). The demographics of those who completed the training program and dropouts of the course were quite similar, with both groups being predominantly Hispanic males with over 10 years of work experience on a dairy farm.
Demographics of Supervisors Enrolled in a Leadership Training Program on U.S. Large-Herd Dairy Farms, and Who Completed Both Pre-Post Safety Climate Surveys
The 65 supervisors represented 28 farms across four western U.S. states, with the number of participating supervisors from each farm ranging from 1 to 6. The number of workers working under the supervision of the 65 enrolled supervisors varied over time, with 373 and 278 workers completing the climate survey in the pre- and post-training respectively (Table 2). We collected an average of 4.5 pre-training and 4.7 post-training worker climate surveys per participating supervisor. Selected demographics were collected from workers, showing that many had limited experience working on a farm, with an average of 3 to 4 years’ experience (range = 0–40 years).
Safety Climate
Safety climate survey data were highly reliable, for both supervisors (α = .91) and workers (α = .95) on the 14-item supervisor safety climate scale and the 10-item group climate scale (α = .90 and .93, respectively). Both supervisors and workers had increases in safety climate scores from pre- to post-intervention on all 24 items (Table 2). There were significant increases on the 14-item supervisor safety climate scale and the 10-item group safety climate scale, as well as subscales dealing with safety priority, empowerment, and dealing with conflicts. However, no significant improvements were seen for the factor regarding leaders encouraging workers to report incidents (injury and near-miss), or worker perceptions of colleague’s reporting incidents, as both were already scored relatively highly at baseline, for example, the supervisor’s item “I encourage my crew to report injuries” had pre-post scores of 5.7 and 5.7, respectively, on a scale of 1 to 6.
Safety Climate Survey Results of Supervisors Completing a Supervisor Leadership and Management Training Program on U.S. Large-Herd Dairy Farms. Scale 1 (Poor) to Good (6)
Average increases on the 14 leader items and 10 workgroup items were 0.52 and 0.55 scale points respectively among supervisors, whereas the increases were smaller from the worker’s perspectives, with 0.29 and 0.26 increases, respectively. The greatest increases among workers’ perspectives were on the items regarding: “My supervisor encourages my crew to express their ideas about safety at work” (4.9–5.3, p = .001) and “My supervisor solves conflicts between crew members” (4.6–5.1, p = .003).
The item with the lowest baseline score among supervisors’ scores was also the item with the greatest scale point increase at follow-up: “Workers in my crew try to find solutions to health and safety problems” (3.7–4.7, p = .001). Similarly, another factor with a low baseline score with significant increase at follow-up was: “I spend time discussing with my crew the safest way to do things at work” (3.9–4.5, p = .026). Although these items showed significant improvement, they still reveal opportunity for further improvement, as most of the 24 scores at follow-up were above 5.0 on the 6-point scale.
In terms of improving safety and health communication the supervisors also had significant and relatively large improvements on items such as: “Workers in my crew discuss health and safety with me” (4.1–5.0, p = .001), and “I listen to crew members’ concerns about safety on the job” (4.9–5.7, p = .001).
Discussion
The results of the safety climate survey and interviews are unique, in that this is the first study to look at the effects of a multifaceted safety intervention on safety climate on dairy farms from the viewpoint of both front-line supervisors and workers. Previous studies have looked at safety climate in the agricultural sector in general (Arcury et al., 2012, 2014, 2015, 2020; Fargnoli & Lombardi, 2020, 2021; Grimbuhler & Viel, 2019; Kearney et al., 2015; Kjestveit et al., 2021; Leppälä et al., 2021; Pollock et al., 2018; Schwatka et al., 2010; Swanberg et al., 2017; Whalley et al., 2009), or have included dairy farms in their general results for the agricultural sector (Cigularov et al., 2009; Fargnoli & Lombardi, 2020), but have not had a specific focus on dairy farms, which continue to grow in both herd and workforce size.
There is strong evidence in the literature for a correlation between workers’ safety climate scores and safety outcomes, whereas there is less evidence of a correlation between leader, manager, and/or supervisor safety climate scores and safety outcomes (Zohar, 2010). The results in this current study showed that workers had lower safety climate scores than their supervisors. Leaders (e.g., supervisors) across most industrial sectors are known to provide higher safety climate scores than their workers (Zohar, 2010), and there is emerging evidence of this in the agricultural sector (Fargnoli & Lombardi, 2020, 2021). The current study also shows that the pre-post changes in safety climate scores are smaller from the worker’s perspective, than from the leader’s perspective.
Improvements were seen on all safety climate measures in the current study, although empowerment, and helping dairy farm work crews to “find solutions to health and safety problems,” should receive more focus in future studies. In addition, other safety climate studies of farm workers in the U.S. have found associations not just with safety (injuries and illness) and health outcomes (e.g., musculoskeletal discomfort; Arcury et al., 2012; Swanberg et al., 2017), but also with workers’ internal/external locus of control (Cigularov et al., 2009), and working while sick or injured (Arcury et al., 2012). Future studies may also consider the possibility of tailoring interventions on farms by taking many of the factors into consideration, as well age and gender (Fargnoli & Lombardi, 2020; Pollock et al., 2018).
Limitations
A review of safety interventions for the prevention of incidents at work showed that greater effects are found with the coordination of multifaceted and multi-level initiatives that focus on the upper levels of the public health hierarchy of controls (i.e., elimination, substitution, technical solutions, organizational initiatives, personal protective equipment; Dyreborg et al., 2022). The training materials in the current study included the principles from the hierarchy of controls, but training was directed solely at the supervisor, as the change agent (Lilley et al., 2009). Thus it was a limitation that owners and workers were not more directly included in the multifaceted intervention, as it was not a multi-level intervention. The study was limited to western U.S. states due to the proximity to the research team based in Texas. Caution should be given in generalizing the results to other U.S. states and internationally. Care should also be taken in adapting the program to other languages, cultures, and safety cultures. Additionally, a limited sample size, which was affected by the COVID-19, limits generalizability to other dairy farms.
The formal or informal working conditions of either the supervisors or workers were not taken into account in the current study. Previous research has shown that U.S. (North Carolina) migrant farm workers with an H-2A visa for temporary agricultural work perceive a better safety climate than those without the visa (e.g., undocumented immigrants; Whalley et al., 2009). Another study from the same U.S. state with experienced, male, migrant farm workers with H-2A visas found a positive association between safety climate perceptions and the number of working hours, yet a negative association with precarious employment and family-related stress (Arcury et al., 2015). These aspects could also be taken into consideration in future studies, along with testing the training program on a larger sample.
Finally, another limitation of the current study was that there was no control group setup with the safety climate survey; therefore, interpreting the pre-post results should be conducted with caution, as similar changes may have occurred in the sector or across sectors due to national or global influences (e.g., legislation, COVID-19 pandemic).
A safety climate study on small fruit, vegetable, and dairy farms in Italy found that farmers’ COVID-19 concerns increased attention to OSH issues, and were correlated with workers’ safety climate scores—particularly in regards to managers and worker’s commitment to safety (Fargnoli & Lombardi, 2021). Similar to the results of the current study was that reporting of safety incidents (including near-misses) did not increase; however, in the current study the reporting levels were already high.
Conclusions
The multifaceted safety leadership and management training program for front-line supervisors on U.S. large-herd dairy farms had a significant, positive effect on the safety climate perceptions of both the supervisors and their workers. Improvements were seen in the perceptions of the “softer” sides of safety leadership and behavior with safety communication, safety empowerment, and safety priority. Greater effects were seen for supervisors than workers, and both groups perceive to a high degree that supervisors already encourage the reporting of injuries, and that workers report injuries. Future studies could test the intervention on a larger sample in a randomized control design.
Implications for Occupational Health Practice
Assessment of workplace safety climate is a well-recognized approach to assess safe culture in an organization. Safety climate has been shown to be a reliable predictor of safety outcomes (e.g., safety behavior, injuries, accidents) in many industries, and often used in intervention studies as an intervention’s effects on injuries and fatalities is not feasible due to time or funding constraints. We developed, implemented, and evaluated a safety leadership and management training program tailored for front-line dairy farm supervisors, and hypothesized that such a training program would have a positive effect on safety climate. Our findings suggest a leadership and management training program for front-line supervisors on dairy farms can have a significant, positive effect on the safety climate perceptions among both supervisors and workers. Occupational practitioners or researchers in the agriculture sector can benefit from the methods and findings as presented in this paper by gaining an understanding of supervisor training determinants of a positive safety climate among agricultural workers, as well as methods used to develop and evaluate safety leadership training curriculum.
Footnotes
Acknowledgements
We would like to express our appreciation to Dr. Lisa Pompeii in training development needs assessment and Dr. David Gimeno for study development and methods; Dr. Robert Hagevoort, Dr. Noa Roman-Muniz, and Dr. Luís G. D. Mendonça for their contributions to training development and dairy farm recruitment. The authors would like to express their appreciation to the dairy owners and workers who were willing to participate in this study. Additionally, the primary author would like to express his appreciation to the late Dr. Stephen J. Reynolds, former HICAHS Director, for his mentorship, friendship, and dedication to the betterment of agricultural worker health and safety.
Author Contributions
Conceptualization, methodology, validation, investigation, formal analysis, writing original draft preparation, review, and editing, D.D., A.R, P.K., and A.H.; data curation, D.D. and A.R.; funding acquisition, resources, and project administration, D.D. All authors have read and agreed to the published version of the manuscript.
Conflict of Interest
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research was funded by the Centers for Disease Control and Prevention (CDC)/National Institute for Occupational Safety and Health (NIOSH) through the High Plains and Intermountain Center for Agricultural Health and Safety (HICAHS), grant number U54OH008085. The contents of this manuscript are solely the responsibility of the authors and do not necessarily represent the official views of the CDC/NIOSH.
Ethical Approval
Approval for the study (HSC-SPH-16-0559) was obtained on July 31, 2018 from the Committee for the Protection of Human Subjects, which is the Institutional Review Board for the University of Texas Health Science Center at Houston.
