Abstract
The Effort Reward Imbalance tool is a measure of psychosocial work characteristics used to identify work-related stress. The purpose of this study was to assess the reliability and validity of the Korean version of the effort-reward imbalance–short form (ERI-SF) in industrial workers. The Korean version of ERI-SF was tested among 250 blue-collar workers. The reliability, content validity, and construct validity with confirmatory factor analysis, as well as interaction terms were analyzed. Reliability assessed by Cronbach’s alpha coefficients were satisfactory for all three subscales: effort 0.75, reward 0.74, and over-commitment 0.72. Confirmatory factor analysis showed an acceptable model fit with the three component theoretical structure (root mean square error of approximation = 0.07, comparative fit index = 0.84). Content validity was assessed with respect to a measure of perceived health. In addition, a significant synergistic interaction of ERI and over-commitment on job strain was found. In conclusion, the ERI-SF demonstrated good psychometric properties with Korean industrial workers. The ERI-SF is ideal for examining work-related stress in the workplace by occupational health professionals. This shorter version makes it easier to administer and score in the occupational health setting.
Introduction
The reported prevalence of cardiovascular disease (CVD) among Korean blue-collar workers has been estimated at 32% within a study population and 20% nationally (D. R. Kang, Ha, & Hwang, 2013; Won, Hong, & Hwang, 2013). Previous studies report associations between psychosocial work environment and CVD risk, as well as musculoskeletal disorders, especially when measured in terms of job stress such as Effort-Reward Imbalance (ERI). The ERI consists of high-demand and low-control measures work which may create an imbalance between high effort spent and low rewards received by workers (Siegrist et al., 2004). The components of the model include effort, reward, and over-commitment, which have been associated with CVD, depression, and self-reported health status (Li, Loerbroks, Shang, et al., 2012; Nielsen et al., 2013). Support for the interaction between the effort-reward ratio (ER ratio) and over-commitment on psychological factors such as job strain and perceived health status is inconsistent (Siegrist, 2010). Understanding the association between the physical and psychological status of blue-collar workers, their adoption and reduction of ERI, and the effects of intervention programs for work-related stress could be informative to occupational health practitioners and researchers who work with this population.
Imbalance between effort and reward at work has been evaluated using the ERI questionnaire (Peter, Siegrist, Hallqvist, Reuterwall, & Theorell, 2002). Findings suggested that job stress experiences are explained by a lack of reciprocity between effort spent doing work and rewards received like monetary, promotion expectation, and job security (Siegrist, 1996). Several studies have reported that an imbalance between effort and reward were related to increased health problems in both male and female workers (Fiorentino et al., 2016; Smith et al., 2016). However, the effect of ERI on job strain has not been fully studied (Fauvel et al., 2003; Ferris, Sinclair, & Kline, 2005).
The validated published measure of the ERI model is based on a 23-item scale, which has demonstrated satisfactory internal consistency in terms of Cronbach’s alpha coefficient(> 0.70) for its three scales of effort, reward, and over-commitment (Siegrist et al., 2004). However, the scale is long and an abbreviated, shorter form would be more conducive for researchers and occupational health practitioners to use in Korean workers. The purposes of this study were to test the reliability and validity of the short form of the ERI (ERI-SF) and to identify its relationship with psychosocial factors, such as job strain, perceived health status, as well as interaction of the ER ratio and over-commitment in Korean workers.
Method
Study Design, Setting, and Sample
This secondary analysis is based on cross-sectional data from industrial workers in South Korea. The study protocol was approved by the Ethical Committee of the Yonsei University Medical Center (YUMC). All participants completed the survey and health assessment from July to August 2010. More detailed information on the recruitment of the study participants was presented elsewhere (Hwang, Hong, & Kim, 2012).
In brief, industrial workers were recruited from eight small companies during the occupational health center’s annual health assessment screening. These worksites did not have their own in-house occupational health professional to provide annual physical examinations and occupational injury prevention programs. Workers were recruited by convenience sampling if they met the inclusion criteria of being older than 18 years of age, with no history of myocardial infarction (MI), percutaneous transluminal coronary angioplasty, or cognitive impairment.
Confirmatory factor analyses (CFA) model permutations per Monte Carlo data simulation techniques were employed to evaluate sample size requirements (Wolf et al., 2013). Because “models with stronger factor loadings required dramatically smaller samples relative to models with weaker factor loadings” (Wolf et al., 2013, p. 21), the needed sample size was estimated to be 215 participants to construct the model fit with three factors and three to four indicators, which is a moderate number for loading condition (0.65) based on a prior study (Li, Loerbroks, Shang, et al., 2012).
Measures
ERI-SF
This study used the ERI-SF developed by Siegrist et al. (2010) to measure the work-related stress. The English version 14-item ERI-SF (10-item effort and reward; 4-item over-commitment) was translated to Korean by a bilingual researcher, and independent back-translation was done by a second bilingual translator. In addition, the adequacy of translation was checked by two bilingual professionals who rated the linguistic and cultural equivalence by using a 4-point Likert-type scale (1 = strongly disagree, 4 = strongly agree), and they consistently rated the quality of the translation to be high.
The ERI-SF is composed of three scales, “effort (3 items),” “reward (7 items),” and “over-commitment (4 items).” Item responses were scored on a 4-point Likert-type scale (1 = strongly disagree, 4 = strongly agree). Each subscale scores were computed by summing the item values. High scores indicate high effort and high reward. The ERI-SF has been tested with working populations in Germany and China, and demonstrated satisfactory internal consistency with Cronbach’s coefficients that were equal to or higher than 0.80 (effort = 0.80, reward = 0.84, over-commitment = 0.85) (Li, Loerbroks, Shang, et al., 2012; Siegrist, Wege, Puhlhofer, & Wahrendorf, 2009). The original ERI measure also showed a satisfactory level of internal consistency as effort = 0.71 to 0.81, reward = 0.86 to 0.88, and over-commitment = 0.63 to 0.75 (Eum et al., 2007; Hwang et al., 2012).
For the over-commitment scale the total sum of 4 items was calculated (range of total score: 4-16). Higher score indicates more over-commitment. A personal characteristics of coping with high demands indicated by an excessive work-related commitment and a high need for approval has been measured by the over-commitment scale, with 4-point Likert-type responses (1 = strongly disagree, 4 = strongly agree; Li, Loerbroks, Shang, et al., 2012).
In addition, the ratio of effort to reward includes the perceived ERI at work and was calculated based on the formulation of effort-reward ratio, which is E (effort score) divided by R (reward score) × C (correction factor), as indicated by Siegrist and colleagues (Siegrist et al., 2004). The original ERI measure with a value close to zero means relatively low effort and high reward. Anyone with a value over 1.0 is considered to have high job stress, and anyone with a value of 1.0 and lower is considered to have low job stress. As part of the proposed work, we expanded this and determined the cut-point for the ERI-SF which has been identified as a gap in a prior study (Kurioka, Inoue, & Tsutsumi, 2014).
Job strain
Job strain was measured using the Job Content Questionnaire (JCQ) for convergent validity. The JCQ is composed of the 5-item psychosocial demand scale (excessive work, conflicting demands, insufficient time to do work, fast work pace, and hard work) and job control (Karasek et al., 1998). The responses were measured on a 4-point Likert scale (1 = strongly disagree, 4 = strongly agree). Job strain was calculated by placing the score for psychological demands in the numerator and that for job control in the denominator. Those in the top quartile of job stress scores were defined as being exposed to high job stress. The reported internal consistency ranged from .61 to .72 for the psychosocial job demand and control scale (Karasek et al., 1998). The Korean version of the JCG have also shown a satisfactory level of internal consistency at .87 (M. G. Kang et al., 2005) and .74 (Eum et al., 2007). Cronbach’s alpha for the Korean JCQ version with workers in the current study was .76.
Perceived health status
Perceived health status was assessed by a single question (“How is your health, in general?”) with a 5-point Likert scale (1 = excellent, 2 = very good, 3 = good, 4 = fair, and 5 = poor), which was dichotomized into good (excellent-good) and poor (fair-poor) and reverse coded. This single self-rated health question has been found to be a significant predictor of later health outcomes (Burstrom & Fredlund, 2001).
Study demographics
Workers provided demographic and work-related characteristics such as age, marital status, education level, employer (company type), type of employment (full-time/part-time), and number of hours of work per week.
Statistical Analysis
Data analysis was conducted using SAS 9.2. All the significance of reported p values was set at .05. Cronbach’s alpha coefficients were calculated to assess the internal consistency of each scale, and CFA was performed to assess construct validity. The three factor model with “effort,” “reward,” and “over-commitment” was assumed. Goodness of fit was assessed by chi-square, as well as a series of fit indices: The root mean square error approximation (RMSEA, a measure based on the analysis of residuals whose values below 0.08 indicate acceptable fit), and the comparative fit index (CFI, a measure of comparative fit with values exceeding 0.90 suggesting a good fit). Parsimonious model fit was measured by the Akaike information criterion (AIC), with lower values indicating more appropriate model specification (Hulley, Cummings, Browner, Grady, & Newman, 2007). Third, the associations between the scales of the ERI-SF and perceived health status and job strain were examined using logistic regression analyses with adjustment for age, gender, marital status, company type, and full- or part-time employment. We hypothesized that people with a high level of ERI, and high over-commitment would report high job strain. In addition, we also tested the interaction between ERI and over-commitment per the synergy index (SI), which indicate synergistic interaction (SI > 1), additive interaction (SI = 1), or antagonistic interaction (SI < 1) (Andersson, Alfredsson, Kallberg, Zdravkovic, & Ahlbom, 2005).
Results
Characteristics of Study Participants
The participants had a mean age of 36.9 (SD = 8.7) years, the majority (81.7%) finished high school, and over 18% had some college or higher. Most participants (93%) were employed as full-time workers. More than 30% worked shift work, and about 30% worked more than 60 hours per week. Most participants (> 82%) indicated that their health status was “good.”
Reliability
The means and SDs of ERI-SF of effort, reward, and over-commitment were 5.49 ± 0.18, 9.49 ± 0.37, and 8.79 ± 0.16, respectively. The internal consistency of the ERI-SF was satisfactory (Cronbach’s α coefficient > 0.72), while the Cronbach’s alpha coefficients for ERI-SF subscales of effort, reward, and over-commitment were 0.75, 0.74, and 0.72, respectively (Table 1). Mean values of effort, reward, and over-commitment were significantly different by gender and employment status. Effort scores were significantly higher in males relative to females, while part-time employment showed significantly lower reward than full-time employment (Table 2).
Mean and Cronbach’s alpha Coefficients of the ERI-SF Among Industrial Blue-Collar Workers in Korea
Note. ERI-SF = effort-reward imbalance–short form; CI = confidence interval; OC = over-commitment.
Scores and Cronbach’s Alpha Coefficients of the Scales of the ERI-SF by Gender and Employment Position Among Industrial Blue-Collar Workers in Korea (N = 250)
Note. ERI-SF = effort-reward imbalance–short form; CI = confidence interval.
Construct Validity
The construct validity was assessed with the CFA with the three-factor model (effort, reward, and over-commitment). The model had an acceptable fit (RMSEA = 0.07, CFI = 0.84, and AIC = 265.55). The AIC indicated that the model had a parsimonious fit. The three-factor model was a superior fit relative to the null or two-factor model (Figure 1).
The model of confirmatory factor analysis testing the theoretical construct underlying the ERI-SF.
Criterion Validity: The Relationship of ERI to Perceived Health Status and Job Strain, and Test of Interaction Terms
The ERI-SF was associated with poor perceived health status at the cut-point score of 1.4, which had a higher odds ratio (OR: 1.85) that reached statistical significance (95% confidence interval [CI] [1.01, 3.45]; Table 3). These findings suggest that the ERI-SF achieved criterion validity with perceived health (poor) with the higher level cut-point of 1.4 compared to cut-point 1.0 in the original ERI model.
Association (OR and 95% CI) of the ER-Ratio in ERI-SF With Perceived Health Status Among Industrial Blue-Collar Workers in Korea (n = 245)
Note. OR = odds ratio; CI = confidence interval; ER = effort-reward; ERI-SF = effort-reward imbalance–short form; AIC = Akaike information criterion.
Adjustment: age, gender, education, type of employment, type of work.
Table 4 summarizes the ORs for association between effort, reward, ER ratio, and over-commitment with job strain. ORs for these scales ranged from 4.26 to 4.64. The odds of job strain was significantly higher if workers reported an elevated ER ratio and over-commitment (OR = 6.33, 95% CI [2.05, 19.61]). The interaction terms were also significant in the analyses, indicating that the synergistic effects of effort and reward on job strain was dependent on the over-commitment level.
Associations (OR and 95% CI) Between The Subscale of ERI-SF With Job Strain Among Industrial Blue-Collar Workers in Korea (n = 245)
Note. Synergy Index > 1. Job strain = Karasak’s Job contents questionnaire. OR = odds ratio; CI = confidence interval; ERI-SF = effort-reward imbalance–short form; OC = over-commitment.
Adjustment = age, gender, education, type of employment, type of work.
Discussion
This research was performed to identify two related aspects of recent research on work-related stress in terms of the ERI model, psychometrics test of ERI-SF and its association with job strain. We also explored associations between ERI-SF and perceived health status. The study demonstrated good reliability and validity of the ERI-SF, and we observed a synergistic interaction of ERI-SF and over-commitment on job strain.
Using the 14 item questionnaire of ERI-SF, this study replicated the three components (effort, reward, and over-commitment) of the model, and CFA documented satisfactory fit. The study results showed psychometric properties such as internal consistency, construct validity, and content validity of the ERI-SF were at satisfactory levels, which were comparable to those reported in earlier studies (Leineweber et al., 2010; Siegrist et al., 2004; Siegrist et al., 2009).
The factorial structure of the ERI-SF scales was consistently confirmed across gender and type of employment, with an adjusted goodness of fit index > 0.90. In earlier studies (Siegrist et al., 2004; Siegrist et al., 2009), the fit of the data to the theoretical structure of the ERI model was addressed, with RMSEA values ranging between 0.07 and 0.11. This study lends some support to the cut-off point of 1.4 in the ERI-SF measurement compared to the cut-off point of 1.0 in the original ERI questionnaire. The findings support the cut-off value of 1.4 for the Japanese ERI-SF questionnaire because it eliminated the ERI group most compatible with the original version (Kurioka et al., 2014).
The validity of ERI-SF was assessed by testing the association between the subscale of ERI-SF and perceived health status and job strain. With regard to poor health status, there is convincing evidence of adverse health effects on ERI from previous studies (Li, Loerbroks, Shang, et al., 2012; Li, Shang, Wang, & Siegrist, 2010). The current study found significant associations between ERI and job strain, consistent with an earlier study (Tsutsumi, Kayaba, Theorell, & Siegrist, 2001). The ERI model is widely used and validated for assessing psychosocial stress at work (Siegrist et al., 2009). There are several similarities between the ERI and JCQ, and job strain measures. The ERI and job strain measures represent general models of job stress with a selective analytical perspective which means we can add over-commitment or not. Both measures are standardized self-reported questionnaires representing general models of work-related stress (Tsutsumi et al., 2001). A strong conceptual and methodological duplication between ERI and Job Strain with respect to the demand component exists. Roughly 9 of 14 concepts employed in the ERI that measure extrinsic effort and low reward include job strain (Siegrist et al., 2004).
There are clear conceptual and methodological differences in some of the metrics that are worth noting. Job strain has been used as a concept that is restricted to the structural aspects of the psychosocial work environment, whereas the ERI includes both structural and personal characteristics. Second, components of the ERI (salaries, career opportunities, and job security) are linked to more distant macroeconomic labor market status, while the major focus of job strain is on workplace characteristics (Hwang & Hong, 2012; Siegrist et al., 2004). Finally, the two work-related stress measures have different theoretical terms (control vs. reward, demand vs. effort) and have different implications for work-related stress and health outcomes such as demand and control in job strain and effort and reward in the ERI. This study showed that ERI in Korean blue-collar workers was associated with job strain. The relationship between ERI and job strain has not been extensively examined. Several studies have reported that job strain (Choi, Schnall, & Dobson, 2016; Whitfield et al., 2016) and ERI (Smith et al., 2016) are related to CVD risk in workers. Furthermore, job stress has been shown to be associated with coronary heart disease in a meta-analysis performed in western countries (Kivimaki et al., 2006) and is emerging as an effective measurement to assess the risk of CVD. This provides rationale for the use as a job strain measure since we demonstrated convergent validity. Perceived health status was included for testing convergent validity, which was also used in other studies (Li, Loerbroks, Shang, et al., 2012)
The scientific contributions of our study’s findings add to the generalizability of ERI by demonstrating its significant relationship not only with effort, reward, ER ratio, over-commitment, and job strain, but by also extending the outcome that was controlled for by age, sex, education, type of employment, and type of work. In theoretical terms, the observed synergistic effect of the ER ratio and over-commitment on job strain is of interest. Recent studies showed significant main effects of ERI and over-commitment (Kudielka et al., 2005; Li, Loerbroks, Jarczok, et al., 2012) on physical and mental health functioning, and quality of life (Watanabe, Tanaka, Aratake, Kato, & Sakata, 2008), without investigating a synergistic interaction. In this study, this synergistic interaction indicated that the ER ratio was related to over-commitment. For example, blue-collar workers with high ERI and high over-commitment were more likely to complain about job strain than those with high ERI and low over-commitment. Thus, researchers and practitioners should also consider over-commitment in psychosocial work factors.
There are several limitations in this study. First, this is a cross-sectional study that only described the reliability and validity at one time point. We were not able to analyze several aspects of psychometric properties, such as the predictive validity (Leineweber et al., 2010) to sensitivity of changes over time that can be assessed in the longitudinal studies.
Second, we need to acknowledge the possible bias of common method variance due to the self-report survey we used to measure the model of ERI. Some techniques have been recommended to detect and correct bias due to common method variance, such as correlation marker, CFA, and unmeasured latent method construct. However, identifying an ideal marker which is not theoretically related to the scales of interest for use in real data was shown to be quite difficult, with questionable practical relevance (Ginsburg et al., 2009). The last limitation is related to the generalization of findings as the sample of study was limited to predominantly blue-collar workers in small-scale industries. More studies with larger samples of diverse worker populations such as hospital nurses and staff is desirable. Thus, further applications of this approach in other countries and other settings are needed, in addition to longitudinal studies.
In conclusion, the Korean version ERI-SF is a valid and reliable instrument for this study population in Korea. This parsimonious measure can be used for future research and practice. Furthermore, a synergistic effect of the model’s components (ERI and over-commitment) on job strain indicates that we should consider all components of the model, given its implications for comprehensive approaches toward developing appropriate occupational nursing intervention for workers with psychosocial work stress.
Implications for Practice
Findings from this study foster the use of the short version of the ERI survey in the worksite by occupational health nurses. Given the short nature of the survey, this can be easily implemented and scored by the occupational health nurse for purposes of measuring ERIs among blue-collar workers.
Applying Research to Practice
The Effort Reward Imbalance–Short Form (ERI-SF) demonstrated good psychometric properties with Korean blue-collar workers. This supports further use of the instrument in future research and practice for working populations. This form could be easily adapted by the occupational health nurse for purposes of assessing stress among their workers.
Footnotes
Acknowledgements
We also acknowledge Dr. Jian Lee for her valuable comments during the preparation of this manuscript.
Authors’ Note
W. J. H. and D. R. K. participated in data collection and analysis considering that this study is related to international and global health. O. S. H. directed the study and writing the manuscript. Data management was done by W. J. H.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Science, ICT & Future Planning (No. 2017R1A2B4008496).
Author Biographies
Won Ju Hwang is currently a professor at the College of Nursing Science, Kyung Hee University, Seoul, South Korea. Dr. Hwang earned a PhD degree from University of California San Francisco before joining the university as a professor. Dr. Hwang has extensive experiences in occupational health and participated global health project in Ethiopia. Dr. Hwang has interests in the research areas of cardiovascular health, job stress, health promotion for vulnerable population, and global health nursing.
Oi Saeng Hong is currently a professor at the University of California San Francisco and director of center of environmental and occupational health.
Dae Ryong Kang is a director, Center of Biomedical Data Sciences and a professor of Wonju College of Medicine Yonsei University.