Sage Journals: Discover world-class research

Abstract

The measurement properties of indices about workplace aggression initiated by the public, referred to as external workplace aggression, are under-researched. This study addressed how exposure to external workplace aggression is best measured and modeled in three types of emergency responders. The study inspected the factor structure and explored the addition of severity to an existing measure of frequency of exposure to workplace aggression, which addresses forms of physical aggression, threats, and verbal and nonverbal/nonphysical aggression (gestures) by people outside the organization directed toward employees. Self-reported data from 1,499 emergency responders, including emergency medical workers, firefighters, and police officers in the Netherlands, were analyzed using factor analyses in Mplus. In addition, the relationships between workplace aggression indices and a measure of the situational risk for violence were tested. Results show that the frequency index measured exposure to external workplace aggression better than the index combining the frequency and severity, and that factor structures of indices differed, regarding number and content of factors, between the three groups of emergency responders. An important implication is that researchers and policy employees can use a relatively simple measure to examine exposure to aggression in organizations.

Keywords

workplace aggression victimization frequency severity

Introduction

Over the years, policy and research attention for experiencing aggression at work has increased (Barrick, Hickman, & Strom, 2014; Eurofound, 2013; Oliver & Levine, 2015), as being exposed to aggression at work can have a negative impact on workers and organizations (Hershcovis & Barling, 2010; Nielsen & Einarsen, 2012). Various authors have discussed the differences in definitions and measurement of workplace aggression in academic research, which result in difficulties to compare results (Barling, 1996; Barling, Dupré, & Kelloway, 2009; Hershcovis, 2011; Hershcovis et al., 2007). However, few have focused on measurement properties of external workplace aggression indices. External workplace aggression is aggression caused by people outside the organization, such as clients, patients, or citizens (the public) in general (e.g., Chappell & Di Martino, 2006), whether they are known to the emergency responders or not. It does not include aggression by colleagues, supervisors, or subordinates (which is “internal aggression”). It has previously been argued that the type of offender should be explicitly addressed in measures, and that consequences of exposure to workplace aggression may depend on the type of offenders (Hershcovis & Barling, 2010). Therefore, it is important to study the measurement properties of indices about experienced external workplace aggression to discover the most optimal way of measuring external workplace aggression. This study will focus on emergency responders, as they are frequently confronted with external aggression (e.g., Health and Safety Executive, 2017). Results will contribute in improving comparability between study results, which will increase effective knowledge building, and better prevention of workplace aggression and its consequences, which is especially important in this high-risk population.

In the present article, the measurement properties of an index about the exposure to external workplace aggression will be explored that has been developed by Dupré, Dawe, and Barling (2014). This index will be used because it seems particularly appropriate for measuring the exposure to external workplace aggression, as it addresses a wide range of aggression, from nonverbal and nonphysical (such as gestures someone makes), verbal to physical aggression, directed toward the employee (cf. Barling et al., 2009) in a workplace context, and was developed from previously used measures (i.e., Greenberg & Barling, 1999; Rogers & Kelloway, 1997). The index of Dupré et al. consists of 24 items about the number of times employees encountered citizens enacting several types of aggressive behavior in the past 6 months. The workplace context includes locations where workers need to go because of their work, such as citizens’ homes or public space.

This article has three aims. First, the aim is to explore the contribution of adding the perceived severity of workplace aggression to a measure of workplace aggression. Thus far, studies have mainly focused on the frequency of aggression experiences. However, the concept of exposure to aggression at work may not only entail the frequency of exposure to a variety of behaviors, but also the severity. This idea is supported by the notion that the perceived severity of workplace aggression might be related to the (severity of) psychological and physical consequences of workplace aggression (Barling, 1996). Therefore, it might be better to measure exposure to external workplace aggression by a combination of the frequency and severity. The first aim will thus be to inspect and compare external workplace aggression indices based on the frequency and a combination of the frequency and severity.

Second, Hershcovis and Reich (2013) have discussed limitations of common indices of workplace aggression, which include weighting each aggressive incident equally. In fact, it is likely that not all types of aggression will contribute equally to the concept of exposure to aggression. Therefore, not all items may load equally on the latent construct exposure to external workplace aggression. In addition to unequal factor loadings, a construct may have a uni- or multidimensional structure. If different dimensions of exposure to workplace aggression exist, these dimensions may have different indicators and consequences (e.g., Van Reemst, Fischer, & Zwirs, 2015). This may result in dimension-specific results and policies. Therefore, the second aim of this article is to inspect the factor structure of indices of exposure to external workplace aggression.

Third, the aim is to explore to what extent the index measures the same concept across occupational groups. In other words, to what extent is measurement invariance present when studying workplace aggression among occupational groups? This study will focus on emergency responders: emergency medical workers, firefighters, and police officers. These occupational groups have several characteristics in common, such as protecting public safety, having frequent contact with the public in various locations, and working in periods of routine interrupted by periods of activity and stress. However, work situations, organizations, and composition of employees may also differ. Therefore, the concept of exposure to workplace aggression could differ between the three occupational groups, and this will be explored.

By fulfilling these three aims, this study addresses the following research question: How is exposure to external workplace aggression best measured and modeled in three types of emergency responders? The following paragraphs will discuss two main themes: (a) measurement of external workplace aggression: the frequency and severity, and (b) differences between occupational groups in measuring workplace aggression, and will present the current study.

Measuring External Workplace Aggression

Frequency

In general, quantitative studies measure external workplace aggression by the frequency of exposure to aggression, often self-reported. Content of the items, response options, and how those are used in analyses vary greatly. Items can include specific forms of aggression, such as items about being yelled at and being hit (e.g., Dupré et al., 2014), or item(s) about aggression in general (e.g., Estryn-Behar et al., 2008). In addition, items often incorporate a certain period over which a respondent reports (Hershcovis & Reich, 2013), such as aggression experienced in the past 6 months, 12 months, or ever (in their career). In analyses, the frequency item (in categories or not) in itself is used or the frequency items are combined to obtain a total score of workplace aggression frequency (e.g., Rogers & Kelloway, 1997), with or without testing the coherence between items’ scores.

An important assumption for testing the coherence between items is that aggression items are related to each other. Although one may think multiple experiences of aggression occur independent of each other, this assumption of relatedness can be motivated by theory and empirical data. Victimological theories suggest that some people are more prone to be confronted with aggression than others, for example, due to characteristics related to regular contact with people who are motivated to offend, lacking guardianship, or being a suitable or “attractive” target for some reason (Cohen & Felson, 1979; Hindelang, Gottfredson, & Garofalo, 1978; Sparks, 1981; Wolfgang, 1958). (Repeat) victimization literature indicates that some individuals are indeed more often confronted than others (Hope & Norris, 2013; Kuijpers, van der Knaap, & Lodewijks, 2011; Tseloni & Pease, 2003), which was also found in studies about workplace aggression against emergency responders (Van der Velden, Bosmans, & Van der Meulen, 2015).

Although workplace aggression is often considered to contain one dimension, some scholars have suggested that workplace aggression contains multiple dimensions. Theoretically, physical and psychological forms of workplace aggression (Barling, 1996; Barling et al., 2009), or more covert (subtle), and overt (more observable to others) forms have been distinguished (Baron, Neuman, & Geddes, 1999). Sometimes, threats are perceived to be a separate dimension of workplace aggression (Barling et al., 2009). Studies that have measured dimensions separately suggest that exposure to psychological and physical workplace aggression are moderately to strongly related to each other (r or Cramer’s V > .35), but correlates of workplace aggression may differ between dimensions (Van Reemst & Fischer, 2016; Aquino & Bradfield, 2000; Schat & Kelloway, 2003). The scale in the present study has previously been used as if it contains one dimension (Dupré et al., 2014).

Most literature on workplace aggression and, more particularly, on the measurement of exposure to workplace aggression has focused on internal workplace aggression (for an overview of indices, Jex & Bayne, 2017). For example, the Negative Acts Questionnaire is a questionnaire about the frequency of workplace aggression that is frequently used and validated (Einarsen, Hoel, & Notelaers, 2009).

Severity

It is important to explore the addition of severity in the measurement of workplace aggression, for the reason that the severity of acts of aggression is considered to vary. For example, physical aggression is generally considered to be more severe than psychological aggression (Schat & Kelloway, 2005). Varying severity of workplace aggression experiences was assumed to be related to its consequences: Being exposed to more severe aggression might be related to more harm afterward (Barling, 1996). Even though frequency measures often are acceptable in reliability, at least regarding internal aggression (Jex & Bayne, 2017), being exposed to workplace aggression may theoretically include both the severity and the frequency of exposure to workplace aggression. Therefore, an index based on this combination might perform better than an index based on solely the frequency of exposure to workplace aggression. However, few studies have addressed the severity of workplace aggression (for internal workplace aggression: Escartín, Rodríguez-Carballeira, Zapf, Porrúa, & Martin-Pena, 2009; Meglich, Faley, & DuBois, 2012 and general violence: Osman, Pupic, & Baigent, 2017), let alone incorporated it in workplace aggression indices (cf. Hershcovis & Reich, 2013).

The study of Weiss et al. (2010) about trauma experienced by police officers (including, but broader than exposure to external workplace aggression) has previously tested whether indices based on frequency differ in reliability and validity from indices including the severity. The conclusion that was drawn was that none of the measured indices were preferred, comparing internal reliability, convergent, and discriminant validity. However, the study did not focus on workplace aggression specifically and only focused on police officers. Therefore, the role of severity in exposure to external workplace aggression, specifically, in these three types of emergency responders remains unclear and will be explored.

Differences Between Occupational Groups

Emergency responders all respond to emergencies, and therefore have many similarities. However, differences between occupational groups exist as well. Important differences for interaction with the public are that they have different tasks in case of an emergency, varying from law enforcing (police officers) to providing medical help (emergency medical workers), that police officers can legitimately use physical force and are most often outside the office (police station), and that firefighters often work with more employees at a time and have least contact with citizens, in general. These differences may result in varying experiences of workplace aggression. The study of Van Reemst and Fischer (2019) among emergency responders shows that, in general, police officers experience most and firefighters experience least workplace aggression. In addition, it shows that work and social demographic correlates of exposure to workplace aggression differ between occupational groups. Still, previous workplace aggression studies have often studied multiple occupational groups together (e.g., Gettman & Gelfand, 2007). Because of potential differences, it is possible that the same workplace aggression index also measures a different latent construct between occupational groups, also known as measurement non-invariance (Vandenberg & Lance, 2000). For example, if an occupational group experiences more victimization, such as police officers, it could be possible that all types of aggression are more similar to each other, and therefore more likely to be one dimension. As a competing hypothesis, it is possible that it is easier to distinguish between dimensions of workplace aggression, and therefore more likely to contain multiple dimensions. Other than differences in number of factors, measurement non-variance could also include different factor–indicator relationships, factor loadings, intercepts, or residual variance (Van de Schoot, Lugtig, & Hox, 2012). Measurement invariance will be explored in measuring exposure to workplace aggression in these three occupational groups.

The Present Study

In this article, we will explore the contribution of each workplace aggression item (factor loadings) and dimensions of the measures of external workplace aggression among emergency responders, by conducting a factor analysis. In addition, we will explore an index combining the frequency and severity (“combination index”), and compare it with a frequency index of exposure to workplace aggression (as is the original scale of Dupré et al., 2014). Although a combination index could contain essential information, no explicit hypothesis will be formulated about this comparison, as only one study is known to compare frequency and combination indices, regarding a broader concept (general trauma). This study does not show a preference for one of the indices (Weiss et al., 2010). To compare indices, the relationship between each index and employees’ situational risk to experience workplace aggression (LeBlanc & Kelloway, 2002) will be tested. LeBlanc and Kelloway’s (2002) measure captures this situational risk by addressing “job characteristics” (p. 449), more specifically work contexts of employees, such as work task, time, location, and type of citizens people work with. The situational risk for workplace aggression is considered a predictor of exposure to workplace aggression, and previously found to be cross-sectionally related to aggression initiated by the public (LeBlanc & Kelloway, 2002). As we believe no objective measure of aggression would exist in this context, the situational risk for workplace aggression would be appropriate to relate to the aggression indices. Therefore, the relationship between the situational risk for violence and each index will be compared, and the strongest relationship will be considered the index that measures workplace aggression best.

Data and Methods

This study was part of a longitudinal research project about workplace violence against emergency medical workers, firefighters, and police officers in the Netherlands. All regional organizations involved in ambulance care and firefighting (N = 25 each) were asked to participate in the study, of which 13 ambulance regions and seven fire department regions agreed to participate. The National Police of the Netherlands provided permission to send the questionnaire to a random selection of 2,250 police officers in three (out of 10) regional units, including a more urban and more rural region, and regions in which ambulance and fire departments also participated.

Emergency medical workers, firefighters, and police officers were invited to fill in the survey by email or through a message on the regional organizations’ intranet. The inclusion criteria for respondents were that they were (a) a paramedic or driver (emergency medical technician), as, in the Netherlands, the paramedic and driver work as a team and both have frequent contact with citizens; (b) at least for 50% a firefighter (crew), supervisor in the field (crew commander), or officer on duty,¹ including those who work on voluntarily basis; or (c) a police officer working in primary policing (see Government of the Netherlands, 2016). This way, the selection of respondents had face-to-face contact with citizens in their work at the organizations. Out of 1,916 people who opened the questionnaire, 1,620 respondents (84.6%) reported to comply with the inclusion criteria, consisting of 462 emergency medical workers, 465 firefighters, and 693 police officers. Respondents did not receive incentives for their participation in the study and the survey took about 25 min to complete.

The mean age of respondents was 44.4 years (SD = 8.2) for emergency medical workers, 44.2 (SD = 9.2) for firefighters, and 39.0 (SD = 11.2) for police officers. Most respondents were male (72.2%, 91.3%, and 78.4%, respectively). Among respondents, 66.5% of emergency medical workers, 28% of firefighters, and 35.2% of police officers completed a higher education level than secondary vocational education. The mode of contact with citizens was 1 to 10 times a work day for emergency medical workers (82.3%), between 1 and 4 times per five work days for firefighters (42.8%), and 11 to 50 times a work day for police officers (47.8%). To test representability of the final sample, public (Ambulancezorg Nederland, 2017; Brandweer Nederland, 2017) and requested data (2017 Human Resource data of the National Police) were consulted on gender, age, and job position of the three populations. Chi-square tests showed that the sample only weakly differed or did not differ in the distribution of these characteristics from the populations. The strongest difference was found for ranks of police officers, χ²(4) = 48.91, Cramer’s V = .21, p < .001, but this was an inconsistent difference: The sample contained less employees not only in lowest ranks but also in one of the highest ranks than the population of police officers.

Measures

A frequency, severity, and combination measure were used to explore the exposure of workplace aggression. Items about the frequency of exposure to workplace aggression were derived from Dupré et al. (2014; based on Greenberg & Barling, 1999; Rogers & Kelloway, 1997). This measure consists of items about the number of times employees encountered citizens enacting several types of behavior in the past 6 months, including forms of physical aggression, threats, and verbal and nonverbal/nonphysical aggression (gestures and looks), from now on “(non-)verbal” aggression. These types of behavior were described in terms of specific behavior, which limits underreporting due to lack of recognition of behavior as “violence,” for example. The period over which a respondent reported is therefore relatively short, which limits retrospective bias (e.g., due to the tendency to remember serious forms of aggression more than minor forms) (Hershcovis & Reich, 2013), although this bias could exist (Rubin & Wenzel, 1996). The accompanying seven answer categories were used: never (0), once (1), twice (2), 3 to 5 times (3), 6 to 10 times (4), 11 to 20 times (5), and more than 20 times (6). Examples of items are “swore or cursed at you,” “hit or tried to hit you,” and “threatened to kill you.” One item of the original scale was not included in the survey (“destroyed something in presence of you”) because it was not considered behavior “directed to” employees, resulting in 23 items. The Cronbach’s alpha of the frequency measure in this study was .91, .89, and .94, which is good to excellent, for emergency medical workers, firefighters, and police officers, respectively (the Cronbach’s alpha was not reported in Dupré et al., 2014).

In addition, respondents were asked to rate the severity of possible exposure to the workplace aggression types in the previous paragraph, on a scale of 1 (“not severe at all”) to 5 (“extremely severe”). This resulted in scores indicating the normative view of employees who are experts by experience on severity of the workplace aggression items (cf. Weiss et al., 2010). The Cronbach’s alpha of the severity measure was .91, .95, and .94, and thus excellent, for emergency medical workers, firefighters, and police officers, respectively. The combined score multiplied people’s own severity rating for each type of aggression by the reported frequency on the same type of aggression (cf. Weiss et al., 2010). This only resulted in a score higher than zero if a person had experienced workplace aggression in the previous 6 months.²

The situational risk for violence, which will be used to test the relationship between indices and situational risk, was measured by the original items of the Risk for Violence Measure of LeBlanc and Kelloway (2002, see their supplemental appendix). These items address job characteristics identified as increasing employee risk of workplace aggression. Two adjustments in items were made, for the measure to fit the job of emergency responders. First, items that would not result in variation within an occupational group were excluded. For example, this resulted in deleting items about handling or selling items or goods, as this is not part of emergency responders’ job. Second, response options were adjusted from relative (such as “often”) to concise (“1 to 4 times per duty”). Using relative response options, it would be unclear to whom one compares themselves with. For example, a respondent could compare themselves with other emergency responders in the same or also other occupational groups and regions. Concise response options do not need comparison, and were therefore preferred and used in this study. The scale contains 18 items, for example, “In your job, how often do you take emotional care of others” and “In your job, how often do you go to peoples’ home.” Respondents were asked to rate items on a scale of 1 (“not at all”) to 6 (“5 times or more per duty”). Response options of four items ranged from 1 to 5 (“every duty”), because it was not possible to experience these events multiple times per duty, such as “working at night.” These were transformed to a 6-point scale (1 = 1; 2 = 2.25; 3 = 3.5; 4 = 4.75; 5 = 6), so each item could theoretically contribute the same to the factor.³ The Cronbach’s alpha of the situational risk for violence measure was .82, .81, and .87, and thus good, for emergency medical workers, firefighters, and police officers, respectively.

Data Analysis

First, the data were inspected and described. Differences between occupational groups in frequency and severity of exposure to external workplace aggression were tested using bootstrap one-way analysis of variance (ANOVA) (1,000 resamples) and Bonferroni post hoc pairwise comparisons. Bootstrap was used so that no assumptions had to be made about normality of the exposure to workplace aggression data. Second, the factor structure of the frequency of exposure to aggression index was inspected in exploratory factor analysis, with Promax rotation, and confirmatory factor analysis (CFA) in Mplus version 7.11, using centered variables (which does not influence results, only sets the intercepts to 0). Both analyses were used to explore the data, but with CFA, we allowed items to load on one of the factors instead of on multiple. Full Information Maximum Likelihood (FIML) was used,⁴ including auxiliary variables, to correct for possible selective missingness. Out of the employees who complied to the inclusion criteria, 74.5% (N_total = 1,200, N_{emergencymedicalworkers} = 354, N_firefighters = 312, N_{policeofficers} = 534) completed all exposure to external workplace aggression items used in this study. This means that there were missing values on variables. FIML uses all information available in the data, but in addition, we included auxiliary variables to use information from the data that is not part of the analytical model (Collins, Schafer, & Kam, 2001; Graham, 2009). Selected auxiliary variables were continuous correlates with values being missing in the variable exposure to workplace aggression.⁵ This way, more respondents could be included in final factor models (N_e = 443, N_f = 463, N_p = 593).

A good model fit is considered to have a nonsignificant chi-square (Kline, 2005), comparative fit index (CFI) and Tucker–Lewis index (TLI) of at least .90 (Hu & Bentler, 1999), and root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR) values of at most .08 (Browne & Cudeck, 1989). As each goodness-of-fit index has its limitation, a set was reported and the combination of outcomes shows the goodness of the fit. To optimize models of the indices, modification suggestions by Mplus regarding the addition of covariates were inspected, and only applied if a good fit was not yet reached and if indicating covariance within factors (not between factors) for all occupational groups.

Third, information on the frequency factor structure was used in CFA for a combined index, based on the frequency and severity of exposure to workplace aggression. As information about the severity was used to construct a combination index, also the severity index was presented, which was also based on the frequency factor structure. Fourth, as the scale to measure situational risk for violence was adjusted for these populations, the model fit indices of this (one factor) measure was described. Finally, linear regression models predicting the situational risk for violence were used to compare (using Betas) the relationship between situational risk and the frequency and combination indices. The severity index was taken into account in this model to control for the main effect of severity.

Results

To address the first aim of this article, which is the addition of severity information to frequency measures, first the frequency and severity of exposure to external workplace aggression will be described. Table 1 shows the descriptive statistics of the frequency and severity of various external workplace aggression types among emergency responders. The table suggests that, overall, police officers were most often and firefighters least often exposed to workplace aggression in the past 6 months, although in almost half of the types of aggression (10/23) frequencies did not differ between firefighters and emergency medical workers. For example, on average police officers were insulted more than 2 times (M = 2.20, SD = 1.85), emergency medical workers once (M = 1.00, SD = 1.29), and firefighters less than once (M = 0.42, SD = 0.79) in the past 6 months. Only the frequency of exposure to choking was similar among all occupational groups, and choking occurred only rarely (M = 0.02, SD = 0.18). Table 1 also indicates that most severity scores differed between at least two of the occupational groups. In general, emergency medical workers considered exposure to aggression most severe and police officers least severe. However, severity scores between police officers and firefighters did not differ significantly in about half of the items (12/23). Severity scores were only similar across all occupational groups for being insulted, which was on average considered little to quite severe (M = 2.63, SD = 1.13). The inter-item correlations for the frequency scores are presented in the Supplemental appendix.

Table 1.

Frequency and Severity of External Workplace Aggression Reported by All Respondents (Total), and EMW, Firefighters, and PO Separately.

Items	Frequency (0-6)								Severity (1-5)
	Total (N = 1,200)		EMW (N = 354)		Firefighters (N = 312)		PO (N = 534)		Total (N = 1,192)		EMW (N = 352)		Firefighters (N = 309)		PO (N = 531)
	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD
1 Said to spite	2.20^a	1.91	1.47	1.50	0.93	1.23	3.41	1.75	1.92^a	0.83	2.08^b	0.83	1.98^b	0.89	1.77	0.78
2 Was rude	2.58^a	1.91	2.07	1.54	1.15	1.27	3.74	1.73	2.05^a	0.88	2.24	0.89	2.02^b	0.88	1.95^b	0.86
3 Transmitted damaging information	0.46^a	1.03	0.36^b	0.89	0.30^b	0.77	0.62	1.21	3.83^a	1.03	3.99^b	1.00	3.67^c	1.03	3.82^{b c}	1.03
4 False accusation	0.69^a	1.25	0.39^b	0.82	0.33^b	0.77	1.10	1.56	4.07^a	1.03	4.29	0.90	4.04^b	1.00	3.95^b	1.11
5 Yelled	2.14^a	1.96	1.56	1.52	0.70	1.08	3.35	1.86	2.13^a	1.02	2.39^b	1.06	2.25^b	1.09	1.88	0.89
6 Swore	1.43^a	1.85	0.71	1.18	0.27	0.69	2.58	2.00	2.62^a	1.17	2.85^b	1.17	2.75^b	1.31	2.40	1.05
7 Insulted	1.39^a	1.66	1.00	1.29	0.42	0.79	2.20	1.85	2.63	1.12	2.71	1.14	2.58	1.12	2.60	1.10
8 Name-calling^d	1.48^a	1.77	0.95	1.30	0.38	0.87	2.47	1.91	2.50^a	1.13	2.66^b	1.17	2.48^{b c}	1.16	2.41^c	1.08
9 Glared or dirty looks	2.27^a	2.07	1.51	1.47	0.72	1.12	3.66	1.92	1.86^a	0.94	2.15^b	1.02	1.99^b	1.00	1.59	0.76
10 Rude gestures	1.46^a	1.77	0.99	1.34	0.50	0.94	2.33	1.96	2.14^a	0.98	2.32	1.01	2.09^b	1.01	2.05^b	0.94
11 Threatened to hit	0.69^a	1.15	0.43	0.80	0.06	0.34	1.22	1.39	3.31^a	1.11	3.64	1.10	3.33	1.19	3.08	1.01
12 . . . to throw	0.46^a	0.99	0.22^b	0.60	0.13^b	0.46	0.81	1.28	3.22^a	1.11	3.57	1.11	3.20	1.19	3.00	1.01
13 . . . to kill	0.24^a	0.74	0.08^b	0.32	0.03^b	0.22	0.47	1.02	4.24^a	1.05	4.40	1.00	4.18^b	1.20	4.17^b	0.98
14 . . . with a weapon	0.16^a	0.54	0.05^b	0.27	0.02^b	0.16	0.30	0.74	4.63^a	0.74	4.80	0.56	4.55^b	0.89	4.56^b	0.74
15 Tried to hit	0.57^a	1.02	0.42	0.79	0.03	0.19	0.99	1.25	3.85^a	1.04	4.14	0.92	3.86	1.12	3.64	1.03
16 Bumped	0.39^a	0.93	0.17^b	0.58	0.07^b	0.35	0.72	1.20	3.21^a	1.05	3.58	1.01	3.24	1.12	2.96	0.95
17 Pushed	0.45^a	0.94	0.25^b	0.63	0.10^b	0.41	0.78	1.18	3.16^a	1.09	3.51	1.10	3.09^b	1.14	2.96^b	0.99
18 Spit	0.14^a	0.53	0.17^b	0.53	0.02	0.15	0.18^b	0.65	4.25^a	0.92	4.44^b	0.80	3.91	1.06	4.32^b	0.86
19 Threw	0.22^a	0.68	0.07^b	0.32	0.09^b	0.39	0.40	0.90	3.77^a	1.04	4.13	0.97	3.67^b	1.10	3.59^b	0.99
20 Grabbed	0.33^a	0.80	0.26	0.64	0.04	0.21	0.54	1.02	3.95^a	1.04	4.34	0.88	3.94	1.08	3.70	1.03
21 Hit	0.14^a	0.52	0.08^b	0.34	0.01^b	0.08	0.26	0.71	4.30^a	0.89	4.60	0.68	4.26^b	0.97	4.13^b	0.91
22 Kicked	0.12^a	0.47	0.05^b	0.24	0.01^b	0.11	0.23	0.66	4.30^a	0.90	4.62	0.67	4.22^b	0.98	4.13^b	0.93
23 Choked	0.02	0.18	0.08	0.08	0.01	0.13	0.02	0.24	4.72^a	0.68	4.89^b	0.43	4.60^b	0.81	4.68^b	0.70

Note. EMW = Emergency Medical Workers; PO = Police Officers.

Bootstrap ANOVA showed a significant difference between occupational groups (p < .01).

Bonferroni post hoc test did not show a difference (p ≥ .05) between occupational groups with the same annotation.

To accommodate for cultural differences, the Dutch translation resembled “name-calling” rather than “verbal abuse,” as used by Dupré, Dawe, and Barling (2014).

The second and third aim of this article were the inspection of the factor structure and inspection of differences between occupational groups. Addressing these aims, exploratory and CFAs showed that it was not possible to fit one-factor structure for the frequency of exposure to external workplace aggression of emergency medical workers, firefighters, and police officers—for example, the model fit of exploratory factor analysis for one factor for frequency for all occupational groups: chi-square = 7,536.43, df = 230, p < .001; CFI = .69; TLI = .66; RMSEA = .16; SRMR = .12.⁶ Fitting the same factor was also not possible if variation of factor loadings and intercepts was allowed between factor structures of occupational groups. This indicates that the factor structures of exposure to workplace aggression entail a different set of items among occupational groups. Therefore, subsequent analyses addressed each occupational group individually. Figures 1, 2, and 3 show the final frequency factor structures, using FIML. For emergency medical workers (Figure 1) and firefighters (Figure 2), final models of exposure to external workplace aggression consists of one factor. For firefighters, only items about (non-)verbal aggression remain to form a good factor model. For police officers (Figure 3), exposure to external aggression consists of two factors. One factor addresses exposure to (non-)verbal aggression, whereas the other factor addresses threats and physical aggression. For emergency medical workers and police officers, the same items were deleted to provide the best fit. Deleted items consist of transmitted damaging information, false accusations, and having been choked. These items seem to have been perceived as more severe (M = 4.29, SD = .90, M = 3.99, SD = 1.00, M = 4.89, SD = .43, respectively, for emergency medical workers; M = 3.82, SD = 1.03; M = 3.95, SD = 1.11, M = 4.68, SD = .70 for police officers) and to have occurred less often (M = .36, SD = .89, M = .39, SD = .82, M = .08, SD = .08 for emergency medical workers; M = .62, SD = 1.21, M = 1.10, SD = 1.56, M = .02, SD = 24 for police officers), compared to other items, see Table 1. However, for firefighters, transmitted damaging information and false accusations could still be included in the (non-)verbal aggression factor.

Figure 1.

Standardized factor loadings of exposure to external workplace aggression among emergency medical workers.

Figure 2.

Standardized factor loadings of exposure to external aggression among firefighters.

Figure 3.

Standardized factor loadings of exposure to external workplace aggression among police officers.

Table 2 shows good model fit according to most model fit indices for frequency indices of all occupational groups. Only the chi-square test was significant for all frequency indices. The same items were used in the severity and combination indices, which resulted in good fit according to the same goodness-of-fit indices. The severity index of emergency medical workers showed good fit in three out of five goodness-of-fit indices, as the SRMR was .09, not .08 or lower. In addition, Table 2 shows good model fit according to four out of five model fit indices for the situational risk for violence indices (resulting in one factor) of all occupational groups. Again, only the chi-square test was significant for all situational risk for violence indices.

Table 2.

Model Fit Indices of Confirmatory Factor Models of Frequency (Freq), Severity and Combination (Combi) Workplace Aggression Indices, and SRV of Emergency Medical Workers, Firefighters, and Police Officers.

Model Fit Indices	Emergency Medical Workers				Firefighters				Police Officers
Model Fit Indices	Freq	Severity	Combi	SRV	Freq	Severity	Combi	SRV	Freq	Severity	Combi	SRV
Chi-square	439.85	500.55	285.60	281.47	70.69	92.64	75.88	297.83	529.05	693.47	463.00	448.01
df chi-square	144	145	121	125	29	32	31	126	151	161	150	118
p-value chi-square	<.001	<.001	<.001	<.001	<.001	<.001	<.001	<.001	<.001	<.001	<.001	<.001
CFI	.93	.95	.94	.92	.98	.97	.95	.92	.96	.94	.93	.93
TLI	.91	.93	.91	.91	.97	.96	.92	.90	.95	.93	.91	.91
RMSEA	.07	.07	.06	.06	.06	.06	.06	.05	.07	.08	.06	.06
SRMR	.08	.09	.05	.05	.04	.04	.05	.06	.06	.06	.06	.05
N	443	443	443	372	463	463	463	463	593	593	593	693

Note. SRV = Situational Risk for Violence; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual.

Pearson’s correlations showed strongest correlations between situational risk for violence and the frequency indices (r = .35, p < .001 for emergency medical workers; r = .26, p < .001 for firefighters; r = .51, p < .001 for the first factor, and r = .49, p < .001 for the second factor for police officers), in addition to a strong correlation between factors for police officers (r = .69, p < .001 for frequency, r = 61, p < .001 for severity, and r = .38, p < .001 for the combination index). Table 3 shows the results of the regression analyses for the relationships between the situational risk for violence and the frequency index, the severity index and the combination index. The Variation Inflation Factors (VIF) indicated that multicollinearity was not a problem, with VIF varying from 1.00 to 2.01. Comparison of the betas indicates that for all occupational groups, the frequency indices are stronger predictors of the situational risk for violence (β = .36, p < .001 for emergency medical workers; β = .26, p < .001 for firefighters; β = .34, p < .001 for Factor 1, and β = .26, p < .001 for Factor 2 of police officers) than the combination indices. The combination index does not explain the situational risk for violence (nor the severity index, but this was not part of the research’s focus nor expected), except for the first, (non-)verbal, aggression factor of the combination index among police officers (β = .07, ns for emergency medical workers; β = .03, ns for firefighters; β = −.08, p < .05 for Factor 1, and β = .02, ns for Factor 2 of police officers⁷). However, hierarchical regression analysis indicates that the combination index does not explain additional variance on top of the frequency and severity measure for police officers, not shown in table, ΔR² = .01, ΔF(2, 586) = 2.18, p = .11.

Table 3.

Linear Regression Analyses of the Frequency, Severity and Combination Measures Predicting the Situation Risk for Violence of Emergency Medical Workers, Firefighters, and Police Officers.

	Emergency Medical Workers		Firefighters		Police Officers
	B (SE)	β	B (SE)	β	B (SE)	β
Frequency Factor 1	.38(.05)	.36***	.27(.05)	.26***	.33(.05)	.34***
Frequency Factor 2					.26(.05)	.26***
SeverityFactor 1	.07(.05)	.07	−.03(.05)	−.03	.01(.05)	.01
SeverityFactor 2					.03(.04)	.03
Combination Factor 1	.08(.06)	.07	.04(.05)	.03	−.08(.04)	−.08*
Combination Factor 2					.02(.04)	.02
Constant	.01(.04)		.00(.04)		.07(.03)
F value	18.56***		11.38***		43.02***
df regression, df error	3, 368		3, 459		6, 586
R ²	.13		.07		.31

Note. The frequency, severity, and combination factors consist of different items, thus direct (statistical) comparison between occupational groups on these statistical results is not possible. B = B (regression) coefficient.

p < .05. ***p < .001.

Discussion

This study addressed the question how exposure to external workplace aggression was best measured and modeled in three types of emergency responders. The study was based on questionnaires filled in by emergency medical workers, firefighters, and police officers (N > 300 for each occupational group). The study had three aims: (a) inspecting and comparing a frequency index (based on Dupré et al., 2014) with an index combining frequency and severity information of exposure to external workplace aggression, (b) inspecting the factor structure of these indices, and (c) inspecting the measurement invariance of indices between occupational groups. Results showed that the frequency index measured exposure to external workplace aggression best compared with indices combining the frequency and severity of workplace aggression. In addition, it showed that the factor structures of indices differed between emergency medical workers, firefighters, and police officers. The factor structures of separate occupational groups were presented. This paragraph will first discuss differences in factor structures between occupational groups, and then the preference of the frequency index over the combination index.

First, the results regarding differences between emergency responders indicated that workplace aggression was measured best by only (non-)verbal aggression items for firefighters. For emergency medical workers and police officers, the indices also included threats and physical aggression items. Another variation was the amount of dimensions: for emergency medical workers and firefighters, indices consisted of one dimension, whereas for police officers, the indices consisted of two dimensions. The fact that models differed between occupational groups can potentially be explained by variations in the frequency of exposure to aggression. For example, firefighters were less often exposed to threats and physical aggression, and those types of aggression may therefore not be part of the concept of workplace aggression in this occupational group. The question is whether not incorporating these items in the measure of firefighters limits the external validity of the workplace aggression measure, as more severe forms of aggression are not represented in the index. However, the results suggest that these types of aggression are less relevant to measure as these seem to be rarely experienced by firefighters. It could be more relevant to measure threats and physical workplace aggression in larger samples of firefighters.

It is notable that the two statistical dimensions of workplace aggression of police officers do not seem to reflect physical versus psychological forms of workplace aggression, as has been distinguished theoretically by previous literature (Barling, 1996; Barling et al., 2009), but rather verbal aggression versus threats and physical aggression. As can be seen in Table 1, the items about threats and physical aggression seem more similar to each other in frequency than the items about (non)verbal aggression and threats.

Second, in all frequency and combination indices, four out of five goodness-of-fit indices indicated that models were good. Only the chi-square test was significant in all indices. Therefore, we consider the overall fit to be adequate. The results of independent relationships with situational risk for violence showed that the frequency index was more strongly related than the index combining frequency and severity information. The result that the combination index was not preferred over a frequency index is in correspondence with the study of Weiss et al. (2010), addressing general trauma, instead of workplace aggression, among police officers. The present study was able to show that the frequency measure was even preferred in this study, by simultaneously regressing the situational risk for violence measure on all indices, which was not done in Weiss’ study.

Methodological issues of the study should be addressed to correctly interpret the finding that the frequency measure was considered to be preferred. First, the frequency index may have performed best as it is possibly the most reliable and valid measure. Compared to the more objective frequency estimate, severity is a hypothetical and subjective concept. Therefore, the severity can be a less reliable measure and more strongly influenced by psychological characteristics. In addition, the severity and combination indices were not used before (although based on Weiss et al., 2010), which means that reliability and validity was not previously tested. However, the indices showed good fit indices, and the measurement error was taken into account by conducting factor analyses.

Second, the situational risk for violence measure (based on LeBlanc & Kelloway, 2002) could be considered to resemble the frequency of aggression, as it seems a measure of the likelihood of being exposed to aggression. Therefore, it could be more strongly related to the frequency of exposure to aggression. However, the situational risk for violence also contains information about how severe aggression could be, such as how often emergency responders deal with potential offenders of more severe aggression (e.g., people with psychiatric illnesses). Therefore, the combination index could still have been more strongly related to the situational risk for violence measure, but it was not. The combination index multiplies the severity with the frequency of exposure, which may better reflect a possible cumulative effect of multiple aggression incidents. Therefore, it might be more strongly related to target consequences of aggression such as mental health issues (Nielsen & Einarsen, 2012), lower physical well-being (Hershcovis & Barling, 2010), or turnover rates, than a variable such as the situational risk for violence, which is considered a predictor of exposure to aggression (LeBlanc & Kelloway, 2002). This could be studied in the future.

A general limitation is that we had to rely on self-report data, as registrations of workplace aggression against emergency responders in the Netherlands are incomplete and selective. As the study does not mean to claim causality, self-reported data in one measurement occasion were considered the best option for this study. The study did rely on memory, as exposure to workplace aggression was measured retrospectively, over the past 6 months. A future study could attempt to use a diary study, to limit retrospective bias more. Another general limitation is the samples, which depended on permission of organizations to cooperate. Therefore, the aggression in the region (either high or low) could influence the permission, and thus the descriptive statistics in this study. However, overall, organizations and units in both urban and rural regions participated and differences between occupational groups in frequencies of exposure to workplace aggression were similar to another sample of emergency responders (Van Reemst & Fischer, 2019).

The first implication of this study is that future research and policies should be directed toward the frequency of exposure to workplace aggression. This means workplace aggression can be measured in a simple way, rather than having to use multiple scales and combining them. Second, occupational groups should be addressed separately, as results indicate that the concept of exposure to workplace aggression was different between occupational groups. The present study implies that occupational groups should be studied separately, which could potentially result in separate policy measures.

To strengthen the results about the indices to measure exposure to external workplace aggression best, studies are needed that test the indices and its measurement properties for more populations experiencing external workplace aggression. The results of the current study indicate that, even though the factor structure and used items may differ between populations, the frequency measure of exposure to external workplace of Dupré et al. (2014) is useful for multiple populations that are at (high) risk of exposure to external workplace aggression. By gaining more information on best ways to measure external workplace aggression and using these indices, the extent of the risk of exposure can be identified and results can be compared. Thereby, measures to decrease external workplace aggression in the future could be taken, improved upon, and become more evidence-based.

Supplemental Material

Appendix – Supplemental material for Measuring and Modeling Exposure to External Workplace Aggression in Three Types of Emergency Responders

Supplemental material, Appendix for Measuring and Modeling Exposure to External Workplace Aggression in Three Types of Emergency Responders by Lisa van Reemst and Joran Jongerling in Journal of Interpersonal Violence

Footnotes

Acknowledgements

The authors would like to thank Tamar Fischer, PhD, and Professor Frank Weerman for their valuable feedback on an earlier version of this text.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Lisa van Reemst

Notes

Author Biographies

Lisa van Reemst, MSc, is completing her PhD research at the Department of Criminology of Erasmus School of Law, Erasmus University Rotterdam. Her research focuses on workplace aggression among emergency responders and other employees, the relationship with psychological characteristics of the target, differences between occupational groups, and employee–citizen interaction in general.

Joran Jongerling, PhD, is an assistant professor at the Department of Pedagogical and Educational Sciences, Erasmus School of Social and Behavioural Sciences, Erasmus University Rotterdam. He earned a PhD in Statistics and specializes in structural equation modeling, multilevel analysis, time series analysis, and bayesian statistics.

References

Ambulancezorg Nederland. (2017). Sector compass. Retrieved from https://www.ambulancezorg.nl/themas/sectorkompas-ambulancezorg/toelichting-sectorkompas

Aquino

Bradfield

(2000). Perceived victimization in the workplace: The role of situational factors and victim characteristics. Organization Science, 11, 525-537. doi:10.1287/orsc.11.5.525.15205

Barling

(1996). The prediction, experience, and consequences of workplace violence. In Bernard

C. I.

(Ed.), The function of the executives (pp. 29-49). Cambridge, MA: Harvard University.

Barling

Dupré

K. E.

Kelloway

E. K.

(2009). Predicting workplace aggression and violence. Annual Review of Psychology, 60, 671-692. doi:10.1146/annurev.psych.60.110707.163629

Baron

R. A.

Neuman

J. H.

Geddes

(1999). Social and personal determinants of workplace aggression: Evidence for the impact of perceived injustice and the type A behavior pattern. Aggressive Behavior, 25, 281-296.

Barrick

Hickman

M. J.

Strom

K. J.

(2014). Representative policing and violence towards the police. Policing, 8, 193-204. doi:10.1093/police/pau013

Brandweer Nederland. (2017). Nulmeting BPV functies brandweer (baseline measurement personell in safety regions of the firefighter sector). Retrieved from http://docplayer.nl/29542430-Nulmeting-bpv-functies-brandweer.html

Browne

M. W.

Cudeck

(1989). Single sample cross-validation indices for covariance structures. Multivariate Behavioral Research, 24, 445-455.

Chappell

Di Martino

(2006). Violence at work. Geneva, Switzerland: International Labour Organization.

10.

Cohen

L. E.

Felson

(1979). Social change and crime rate trends: A routine activity approach. American Sociological Review, 44, 588-608. doi:10.2307/2094589

11.

Collins

L. M.

Schafer

J. L.

Kam

(2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6, 330-351.

12.

Davis

M. H.

(1980). A multidimensional approach to individual differences in empathy. Catalog of Selected Documents in Psychology, 10, 85-100.

13.

Dupré

K. E.

Dawe

K. A.

Barling

(2014). Harm to those who serve: Effects of direct and vicarious customer-initiated workplace aggression. Journal of Interpersonal Violence, 29(13), 1-23. doi:10.1177/0886260513518841

14.

Einarsen

Hoel

Notelaers

(2009). Measuring exposure to bullying and harassment at work: Validity, factor structure and psychometric properties of the negative acts questionnaire-revised. Work & Stress, 23, 24-44.

15.

Escartín

Rodríguez-Carballeira

Zapf

Porrúa

Martin-Pena

(2009). Perceived severity of various bullying behaviours at work and the relevance of exposure to bullying. Work & Stress, 23, 191-205.

16.

Estryn-Behar

van der Heijden

Camerino

Fry

Le Nezet

Conway

P. M.

Hasselhorn

(2008). Violence risks in nursing—Results from the European “NEXT” study. Occupational Medicine, 58, 107-114. doi:10.1093/occmed/kqm142

17.

Eurofound. (2013). Physical and psychological violence at the workplace. Luxembourg City, Luxembourg: Publications office of the European Union.

18.

Gettman

H. J.

Gelfand

M. J.

(2007). When the customer shouldn’t be king: Antecedents and consequences of sexual harassment by clients and customers. Journal of Applied Psychology, 92, 757-770. doi:10.1037/0021-9010.92.3.757

19.

Government of the Netherlands. (2016). Tasks of the Dutch police. Retrieved from https://www.government.nl/topics/police/contents/tasks-of-the-dutch-police

20.

Graham

J. W.

(2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549-576. doi:10.1146/annurev.psych.58.110405.085530

21.

Greenberg

Barling

(1999). Predicting employee aggression against coworkers, subordinates and supervisors: The roles of person behaviors and perceived workplace factors. Journal of Organizational Behavior, 20, 897-913. doi:10.1002/(SICI)1099-1379(199911)20:63.0.CO;2-Z

22.

Health and Safety Executive. (2017). Violence at work: Findings from the crime survey for England and Wales and from RIDDOR. Retrieved from http://www.hse.gov.uk/Statistics/causinj/violence/index.htm

23.

Hershcovis

M. S.

(2011). “Incivility, social undermining, bullying . . . oh my!”: A call to reconcile constructs within workplace aggression research. Journal of Organizational Behavior, 32, 499-519. doi:10.1002/job.689

24.

Hershcovis

M. S.

Barling

(2010). Towards a multi-foci approach to workplace aggression: A meta-analytic review of outcomes from different perpetrators. Journal of Organizational Behavior, 31, 24-44. doi:10.1002/job.621

25.

Hershcovis

M. S.

Reich

T. C.

(2013). Integrating workplace aggression research: Relational, contextual, and method considerations. Journal of Organizational Behavior, 34(Suppl. 1), S26-S42. doi:10.1002/job.1886

26.

Hershcovis

M. S.

Turner

Barling

Arnold

K. A.

Dupré

K. E.

Inness

. . . Sivanathan

(2007). Predicting workplace aggression: A meta-analysis. Journal of Applied Psychology, 92, 228-238. doi:10.1037/0021-9010.92.1.228

27.

Hindelang

M. J.

Gottfredson

M. R.

Garofalo

(1978). Victims of personal crime: An empirical foundation for a theory of personal victimization. Cambridge, MA: Ballinger Publishing.

28.

Hope

Norris

P. A.

(2013). Heterogeneity in the frequency distribution of crime victimization. Journal of Quantitative Criminology, 29, 543-578.

29.

Bentler

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.

30.

Jex

Bayne

(2017). Measurement of workplace aggression. In Bowling

Hershcovis

(Eds.), Research and theory on workplace aggression (pp. 9-33). New York, NY: Cambridge University Press.

31.

Kline

(2005). Principles and practice of structural equation modeling. New York, NY: Guilford.

32.

Kuijpers

K. F.

van der Knaap

L. M.

Lodewijks

I. A. J.

(2011). Victims’ influence on intimate partner violence revictimization: A systematic review of prospective evidence. Trauma, Violence, & Abuse, 12, 198-219. doi:10.1177/1524838011416378

33.

LeBlanc

Kelloway

(2002). Predictors and outcomes of workplace violence and aggression. Journal of Applied Psychology, 87, 444-453. doi:10.1037//0021-9010.87.3.444

34.

Lobbestael

Cima

Arntz

(2013). The relationship between adult reactive and proactive aggression, hostile interpretation bias, and antisocial personality disorder. Journal of Personality Disorders, 27, 53-66.

35.

Meglich

P. A.

Faley

R. H.

DuBois

C. L.

(2012). The influence of actions and actors on the perceived severity of workplace bullying. Journal of Management Policy & Practice, 13(1), 11-25.

36.

Nielsen

M. B.

Einarsen

(2012). Outcomes of exposure to workplace bullying: A meta-analytic review. Work & Stress, 26, 309-332.

37.

Oliver

Levine

(2015). Workplace violence: A survey of nationally registered emergency medical services professionals. Epidemiology Research International, 2015, Article 137246.

38.

Osman

Pupic

Baigent

(2017). How many slaps is equivalent to one punch? New approaches to assessing the relative severity of violent acts. Psychology of Violence, 7, 69-81.

39.

Rogers

Kelloway

E. K.

(1997). Violence at work: Personal and organizational outcomes. Journal of Occupational Health Psychology, 2, 63-71.

40.

Rubin

D. C.

Wenzel

A. E.

(1996). One hundred years of forgetting: A quantitative description of retention. Psychological Review, 103, 734-760.

41.

Schat

A. C.

Kelloway

E. K.

(2003). Reducing the adverse consequences of workplace aggression and violence: The buffering effects of organizational support. Journal of Occupational Health Psychology, 8, 110-122. doi:10.1037/1076-8998.8.2.110

42.

Schat

A. C.

Kelloway

E. K.

(2005). Workplace aggression. In Barling

Kelloway

E. K.

Frone

M. R.

(Eds.), Handbook of work stress (pp. 189-218). Thousand Oaks, CA: SAGE.

43.

Sparks

R. F.

(1981). Multiple victimization: Evidence, theory, and future-research. Journal of Criminal Law and Criminology, 72, 762-778. doi:10.2307/1143014

44.

Tseloni

Pease

(2003). Repeat personal victimization: “Boosts” or “flags”? The British Journal of Criminology, 43, 196-212. doi:10.1093/bjc/43.1.196

45.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-70.

46.

Van der Velden

Bosmans

Van der Meulen

. (2015). Predictors of workplace violence among ambulance personnel: A longitudinal study. Nursing Open, 3, 90-98. doi:10.1002/nop2.38

47.

Van Reemst

Fischer

T. F. C.

Zwirs

B. W. C

. (2015). Response decision, emotions and victimization of police officers. European Journal of Criminology, 12, 635-657. doi:10.1177/1477370815587767

48.

Van Reemst

Fischer

T. F. C

. (2019). Experiencing external workplace violence: Differences in indicators between three types of emergency responders. Journal of Interpersonal Violence, 34(9), 1864-1889. doi:10.1177/0886260516657913

49.

Van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9, 486-492.

50.

Weiss

D. S.

Brunet

Best

S. R.

Metzler

T. J.

Liberman

Pole

Marmar

C. R.

(2010). Frequency and severity approaches to indexing exposure to trauma: The critical incident history questionnaire for police officers. Journal of Traumatic Stress, 23, 734-743.

51.

Wolfgang

M. E.

(1958). Patterns in criminal homicide. Oxford, UK: University of Pennsylvania Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.38 MB