Sage Journals: Discover world-class research

Abstract

This research investigated the construct and criterion validity of the English version of the Dutch Work Addiction Scale (DUWAS) within the South African financial services context (n = 345). A quantitative research approach with a cross-sectional design was utilized to determine the psychometric properties of the Dutch Work Addiction Scale (DUWAS-10). Confirmatory factor analysis was used with a latent variable modeling approach for the construct validity and a structural path model for criterion validity. One of the working compulsively items did not perform well and was omitted to establish a 9-item scale. Additionally, due to problematic discriminant validity between the working compulsively and working excessively factors, the results did not corroborate the expected two-factor structure. However, a revised second-order structure of the DUWAS showed an acceptable fit to the data indicating that an overall workaholism score is appropriate in this context. Furthermore, the results showed that workaholism had positive relationships with work overload, work hours, and burnout but negatively related to work engagement and organizational commitment. All in all, evidence suggested that an adapted 9-item version of the DUWAS performed well.

Keywords

Workaholism overwork overload validation structural equation modeling

Introduction

Society has different perceptions regarding the value and consequences of working hard (Van Beek et al., 2011). Some consider time to be money and are determined to use it well, ultimately managing, investing, and using time in a productive manner (Gini, 1998). Furthermore, individuals tend to devote a significant amount of time to work activities, yet some allocate more time to work than others (Van Wijhe et al., 2014). There are several reasons why people work hard, but in some cases, individuals are motivated by a compulsive inner drive to work excessively, which is referred to as workaholism (Van Wijhe et al., 2014). Workaholism is evident when employees over-commit their resources (i.e., time and energy) to their job (Snir & Harpaz, 2006). Thus, workaholics are overly involved in their work and struggle to detach themselves from work due to their internal compulsion to work extremely hard (Bakker et al., 2013; Kanai & Wakabayashi, 2001) found a strong positive relationship between workaholism and job demands, such as work overload. Workaholics are aware of their excessive workload, yet unable to control their compulsion to work due to the anxiety experienced when away from work (Scott et al., 1997). These findings are supported by Schaufeli et al. (2006), who indicate that workaholism is positively linked to numerous indicators of overload; more specifically, they work longer hours than their colleagues, continue to work at home and even work on their holiday and during weekends. Different typologies of “workaholics” also exist, for example Scott et al. (1997) referred to the: (1) compulsive-dependent workaholic, (2) perfectionist workaholic, or (3) achievement-oriented workaholic.

Due to the excessive amount of time and energy spent on work activities, workaholics have insufficient opportunities to recover from their expended efforts, ultimately resulting in distress, both emotionally and cognitively (Balducci et al., 2018; Shimazu & Schaufeli, 2009). Hence, workaholism has severe implications for employee health and well-being as it drains employee resources to the point of exhaustion (Van Wijhe et al., 2014), which is the core component of burnout (Schaufeli & Taris, 2005).

Workaholism is a relevant and applicable phenomenon in modern society, yet there is a lack of scientific knowledge concerning workaholism, especially within the South African context (Mostert et al., 2011). The DUWAS relies on Taris and Schaufeli’s (2003) conceptualization of workaholism, where workaholism refers to an irresistible inner drive to work extremely hard and comprises: working excessively and working compulsively. “Working excessively” encapsulates a behavioral dimension of workaholism, whereas “working compulsively” refers to the cognitive dimension (Schaufeli et al., 2009). The items of the DUWAS-10 scale were adapted from two general workaholism scales: Work Addiction Risk Test (WART; Robinson, 1999) and the Workaholism Battery (WorkBat; Spence & Robbins, 1992). Schaufeli et al., (2009) presented a 10-item measure with the most promising psychometric features (Del Líbano et al., 2010). Most past studies have found evidence for a two-factor structure in their contexts: Japan, The Netherlands, Italy, Bulgaria, Hungary, Portugal, and Spain (Butucescu & Uscătescu, 2019; Repaczki et al., 2021; Schaufeli et al., 2009). However, a second-order model with first-order factors has also been tested in the Netherlands and Finland (Rantanen et al., 2015). Rantanen and colleagues also state that the two proposed components of workaholism correlate highly. This points to the possibility that a unidimensional structure might also be possible, but the main recommendation is that the components be assessed separately as they indicate different aspects of workaholism.

Other studies have also published on different language versions of the DUWAS-10: Hebrew (Littman-Ovadia et al., 2014), French (Sandrin & Gillet, 2016), Norwegian (Schou Andreassen et al., 2013), Finnish (Rantanen et al., 2015), Spanish (Del Líbano et al., 2010), Japanese (Schaufeli et al., 2009), and Italian (Balducci et al., 2017). Nevertheless, no validation studies have been conducted within the South African context.

Despite its popularity, workaholism remains a controversial topic, and as such, has led to diverse—even conflicting—opinions about whether the condition is desirable or not. For example, Baruch (2011) argues by way of the “chocoholism” metaphor that being addicted to chocolate is not necessarily inherently bad; certain types such as dark chocolate can have health benefits, and just as someone will not just consume chocolate the whole day, workaholics do engage in other activities besides work as well. However, Baruch acknowledges that future studies should consider the distinction between workaholism and work engagement. Other studies, of course, indicate workaholism as a negative phenomenon as it is a stressor that has consequences over time (for an overview, see Clark et al., 2014; McMillan et al., 2004; Schou Andreassen et al., 2013).

For the current study, relationships between workaholism and variables that have been established were considered for criterion validity purposes. Many empirical studies reinforce the notion that workaholism is mainly related to adverse outcomes for both the individual and the organization (Balducci et al., 2018; Burke, 2004; Douglas & Morris, 2006; McMillan & O’Driscoll, 2004; Mudrack, 2004; Van Beek et al., 2012, 2014;Van Wijhe et al., 2011). For instance, workaholism is negatively related to work engagement (Schaufeli et al.,2008), organizational commitment (Burke et al., 2004; Douglas & Morris, 2006; Liang & Chu, 2009), performance (Shimazu et al., 2012, 2015) and positively related with burnout (McMillan & O’Driscoll, 2004) and mental distress (Balducci et al., 2018).

Furthermore, even though the significant implications of workaholism is apparent, it is frequently considered by some employers as an “acceptable addiction” and is often mistakenly regarded as model behavior (Fry et al., 2006). Therefore, this addiction is appreciated and encouraged by some elements of society because these employees appear productive and produce results (Gini, 1998). However, organizations overlook the potentially fatal “karoshi” (Kanai, 2009). The concept karoshi, more commonly known as “death from overwork,” is a popular term within Japanese culture, which refers to individuals’ experiencing health problems due to long working hours, which can lead to permanent disability or even death (Herbig & Palumbo, 1994; Iwasaki et al., 2006). Therefore, it is not unknown that long working hours can indeed deteriorate employee health (Kanai, 2009). For instance, recently, Balducci et al. (2018) found that workaholism is associated with high systolic blood pressure, a well-known risk factor for cardiovascular disease. This is in line with the health impairment process of the job demands-resources (JD-R) model, which states that work overload results in an erosion of employee energetic capacity, which ultimately leads to poor health (both psychological and physical) and impairment of employee performance (Bakker et al., 2014).

In the context of the effort-recovery model (Van Hooff et al., 2007), workaholics become so immersed in their work due to their inner drive to work excessively that they struggle to detach themselves from it, ultimately neglecting their need for recovery (Bakker et al., 2013). Effort expenditure at work is related to acute load reactions, consisting of short-term physiological and psychological costs (Meijman & Mulder, 1998). Under optimal circumstances, these costs are manageable if the individuals decrease their effort expenditure after work hours and over weekends to facilitate recovery before the next working day/week (Taris et al., 2010). However, workaholics do not engage in these opportunities to recover due to the excessive nature of working, causing an accumulation of adverse load reactions, which increases the probability of long-lasting and irreversible conditions (Taris et al., 2005).

The financial services sector is a competitive, demanding environment (Williams, 2012) and contributes about 20% to South Africa’s gross domestic product (StatsSA, 2020). The sector is mature and effectively regulated. However, a survey of financial services chief executives showed concerns around skills shortages which hinder the ability to innovate at the desired rate (PwC, 2019). A large amount of time is spent training financial services professionals (Rautenbach et al., 2020). Given this competitive environment and the associate skills shortages, it would be safe to assume that some employees work harder and longer hours. Some employees may therefore be more susceptible to workaholic behavior due to these conditions, or conversely, such an environment may attract people who can be classified as “workaholics.” Consequently, given the importance of the financial services industry to the economy, it is crucial to investigate workaholism within this context. However, to identify workaholism, it is important to measure the phenomenon accurately and consider its impact on employee and organizational outcomes.

Therefore, it is considered necessary and timely to investigate a workaholism scale, more specifically the Dutch Work Addiction Scale (DUWAS-10), within the South African financial services context to evaluate the functioning and the relationships between workaholism and other outcomes.

For this study, the following hypotheses are listed:

H₁: Workaholism is a two-factor structure.

H₂: There is a positive relationship between workaholism and work overload.

H₃: There is a positive relationship between workaholism and work hours.

H₄: Workaholism has a negative path relationship to work engagement.

H₅: Workaholism has a negative path relationship to organizational commitment.

H₆: Workaholism has a positive path relationship to burnout.

Method

A quantitative research approach was utilized for purposes of the study, to explain phenomena by means of collecting data to answer research questions with the assistance of statistical techniques (De Vos et al., 2011). Specifically, a cross-sectional design was employed to describe the differences of a group of individuals at a specific point in time (De Vos et al., 2011). This study is based, in part, on a reworked chapter of the master’s dissertation of the second author (Horn, 2015).

Research Participants

Purposeful sampling was used to collect data for a sample, which included different South African employees from a large financial services institution that provides different financial services throughout many business units (n = 345). The financial services industry was an applicable target population because of the nature of the workplace environment (i.e., long hours, fast-paced and high pressure, competitive, work environment); workaholic behavior was therefore expected to be prevalent among financial services employees (Devi, 2012).

The mean age of the sample was 35.17 years (SD = 9.79). The majority (n = 210; 60.90%) of the sample comprised female employees and the overall sample consisted mostly of black (n = 151; 43.80%) and white (n = 93; 27.00%) employees. Furthermore, participants that were single (25.20%) and married or living with a partner (52.80%) were the main contributors of the sample, and many of the respondents either spoke English (34.50%) or Afrikaans (20.60%). Most of the participants were employed with the organization for 1-5 years (n = 163; 47.20%) and 5 to 10 years (n = 72; 20.90%).

Measuring Instruments

Workaholism: The Dutch Work Addiction Scale (DUWAS-10; English version) was used to measure workaholism (Schaufeli et al., 2009). The scale included a total of 10 items with two 5-item subscales measuring working excessively (e.g., “I spend more time working than on socializing with friends, on hobbies, or leisure activities”) and working compulsively (e.g., “I feel obliged to work hard, even when it is not enjoyable”). The DUWAS-10 is scored on a four-point Likert-type scale, ranging from 1 (Almost never) to 4 (Almost always).

Work engagement: The 9-item Utrecht Work Engagement Scale (UWES-9) was utilized to measure the participants’ levels of engagement (Schaufeli et al., 2006). The UWES is scored on a seven-point frequency scale, ranging from 0 (Never) to 6 (Every day). Work engagement was constituted as a single latent variable (Fong & Ng, 2012), measured by nine items from its three components: Vigor (3 items, e.g., “When I get up in the morning, I feel like going to work”), Dedication (3 items, e.g., “I find the work that I do full of meaning and purpose”) and Absorption (3 items, e.g., “I am immersed in my work”).

Organizational commitment: Items from the scale of Allen and Meyer (1990) was used to measure the participants’ affective commitment levels. Specifically, seven items (e.g., “I would be very happy to spend the rest of my career with this organization”) were used. The OCS is scored on a seven-point Likert-type scale, ranging from 1 (Strongly disagree) to 7 (Strongly agree).

Burnout: The Maslach Burnout Inventory (MBI; Schaufeli et al., 1996) was used to measure the participants’ levels of burnout. Items from the core components of burnout were measured for this study: Emotional exhaustion (5 items, e.g., “I feel used up at the end of the workday”) and Cynicism/Depersonalisation (5 items, e.g., “I have become more cynical about whether my work contributes anything.”) as a one-factor structure (De Beer & Bianchi, 2019). The MBI is scored on a seven-point frequency scale, ranging from 0 (Never) to 6 (Always).

Work overload: Measured with 3 items from the scale developed by Sverke et al., (1999). An example item was: “My work contains elements that are too demanding”. The items were scored on a five-point Likert-type scale, ranging from 1 (Strongly disagree) to 5 (Strongly agree), where a high score represents a higher workload.

Total work hours: This variable was continuous indicated by 2 items to assess the total number of working hours per week. The items that were included: “How many hours do you work during a standard working week?” and “How many hours do you spend working at home (beyond normal working hours and flexitime) during a standard week?” were added together as the final continuous variable.

Statistical Analysis

Structural equation modeling (SEM) methods were implemented in Mplus 8.6 (Muthén & Muthén, 2021). The measurement model was formed using confirmatory factor analysis (CFA) (Brown, 2015). The weighted least squares (mean- and variance adjusted) estimation algorithm (WLSMV; Muthén et al., 1997) was used to estimate the parameters of the SEM models. WLSMV is the most appropriate and indicated technique as the study variables were of ordered categorical nature, including four-point scales (e.g., workaholism—the primary scale of interest to this study), and research has shown that WLSMV provides more accurate results compared to maximum likelihood (ML) implementations which consider the variables to be continuous—underestimating the proper relationships between indicators (cf. Flora & Curran, 2004; Liang & Yang, 2014). WLSMV computes polychoric correlations between the categorically ordered indicators and performs more efficiently compared to even Bayesian methods in terms of robustness and assumptions of normality of data on samples of 200 participants and above (Holgado–Tello et al., 2010; Liang & Yang, 2014). The following fit indices were considered: The comparative fit index (CFI), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA), and standardized root mean residual (SRMR). For the CFI and TLI, values of .90 and above are considered adequate, and the RMSEA values of 0.08 and below (Van de Schoot et al., 2012). Furthermore, Cronbach’s alpha and omega coefficients were calculated for all the study variables in the final measurement model as the indicator of reliability (internal consistency).

Effect sizes for correlations were considered medium (r = .30–.49) and large effects (r = .50–.84) (Cohen, 1992)—this assisted in answering the convergent validity of the workaholism scale. Excessive correlations (r = .85 and above) would necessitate further investigations for discriminant validity (Brown, 2015). Discriminant validity was assessed by the average variance extracted (AVE) of the two individual latent variables being compared should be larger than the shared variance between those two variables.

For criterion validity, past relationships were specified to be related to the literature—specifically for validity purposes. The significance, direction, and sizes of the standardized beta coefficients were considered.

Results

Measurement Models

It is important to note that the latent variable(s) did not explain a statistically significant amount of variance in item three for working compulsively (“I feel that there is something inside me that drives me to work hard”; p = .66)—Engelbrecht et al. (2020) also found this same item to be problematic within the South African engineering sector. Therefore, this item was removed from all further models tested (see Table 2 below for the factor loadings for workaholism in the final measurement model).

Specifically, four different models were assessed: model 1: The expected two-factor model consisting of working compulsively and working excessively; model 2: A unidimensional model; model 4: a second-order model consisting of the working compulsively and working excessively first-order factors; and model 3: a two-factor second-order model working compulsively (indicated by “obsessive work drive” and “unease if not working”) and working excessively (indicated by “working frantically” and “working long hours”) as per Rantanen et al. (2015). The models were specified, and all indicators were set as categorical data except for the total work hours, which remained a continuous covariate in the model. Results of the CFA revealed that the two-factor model (model 1) was an acceptable fit to the data (CFI = .921; TLI = .914; RMSEA = .063 90% CI[.059, .067]; SRMR = 0.067). However, the correlation between working excessively and working compulsively was .86, indicating problematic discriminant validity. Farrell’s test for discriminant validity showed that the shared variance between the two constructs was larger than the AVE extracted. This result necessitated that the two-factor model be rejected (H₁ rejected)—as discriminant validity was not evident.

The unidimensional model (model 2) was also an acceptable fit to the data: (CFI = .917; TLI = .910; RMSEA = .064 90% CI[.060, .068]; SRMR = 0.069). However, Rantanen et al. (2015) state that even though a unidimensional construct might be possible due to high correlations between the two workaholism components, they should be assessed separately as they conceptually refer to two different aspects of workaholism. To this end, two second-order models were tested: model 3 (in line with Rantanen and colleagues) and model 4 (a normal second-order model). The results showed that model 4 could not be accurately estimated due to a non-positive definite matrix and was therefore discarded from further consideration. However, model 4 was an acceptable fit to the data (CFI = .918; TLI = .911; RMSEA = .064 90% CI[.060, .068]; SRMR = .069)

As can be seen for the remaining models, models 2 and 3, the fit statistics were all similar with the RMSEA values show overlapping estimates—indicating that these models are not statistically distinguishable from each other. However, model 3 is considered superior to model 2 as it provides an overall workaholism score and considers the separate components. Therefore, substantively, the second-order model was selected as the model of choice for the remainder of the study. This result is also consistent with Engelbrecht et al. (2020).

Table 1 below presents the factor loadings for the remaining DUWAS items.

Table 1.

Standardized Loadings for the Workaholism Second-Order and First-Order Factors.

DUWAS-10 item text	Item	Loading	S.E.	p
Working compulsively factor	workc	0.92	.02	.001
It is important to me to work hard even when I don’t enjoy what I’m doing	compulse1	0.70	.05	.001
I feel obliged to work hard, even when it’s not enjoyable	compulse2	0.98	.04	.001
I feel guilty when I take time off	compulse4	0.38	.06	.001
It is hard for me to relax when I’m not working	compulse5	0.31	.06	.001
Working excessively factor	worke	0.93	0.02	.001
I seem to be in a hurry and racing against the clock	excess1	0.74	.04	.001
I find myself continuing to work after my co-workers have called it quits	excess2	0.50	.05	.001
I spend more time working than on socializing with friends, on hobbies, or on leisure activities	excess3	0.63	.05	.001
I stay busy and keep many irons in the fire	excess4	0.60	.04	.001
I find myself doing two or three things at one time such as eating and writing a memo while talking on the telephone	excess5	0.56	.05	.001

Note. workc and worke = first order components of the second-order model, compulse = working compulsively, excess = working excessively, S.E. = standard error; all p-values < .001.

All the items loaded significantly on the first-order workaholism factors (p < .001) similarly for the higher-order factor loadings. The highest factor loading for working compulsively was item compulse2 (λ = .92, S.E. = .04; “I feel obliged to work hard, even when it’s not enjoyable”), and for working excessively excess1 (λ = .74, SE = .04; “I seem to be in a hurry and racing against the clock”). Furthermore, all the standard errors were relatively small, indicating the accuracy of the estimation process.

Reliability Coefficients and Correlation Matrix for the Study Variables

Cronbach’s alpha and omega reliability coefficients were calculated as indicators of internal consistency for the variables and are given on the diagonal of the correlation matrix in brackets below.

Table 2 below presents the correlation matrix for the study variables.

Table 2.

Reliabilities and Correlation Matrix for the Latent Variables.

Variables	1	2	3	4	5	6	7
1. Work compulsively	(.64/.61)
2. Work excessively	0.85*^b	(.72/.73)
3. Workaholism	0.92*^b	0.93*^b	(.78/.78)
4. Work overload	0.52*^b	0.52*^b	0.57*^b	(.70/.72)
5. Burnout	0.51*^b	0.51*^b	0.55*^b	0.66*^b	(.88/.89)
6. Commitment	−0.28*^a	−0.28	−0.31*^a	−0.48*^a	−0.65*^b	(.78/.78)
7. Work engagement	−0.17	−0.17	−0.18	−0.38*^a	−0.67*^b	0.69*^b	(.84/.85)
8. Total workhours	0.31*^a	0.31*^a	0.34*^a	0.18	0.13	−0.03	0.09

Note. Cronbach’s reliability/Omega reliability coefficients in brackets on the diagonal, *correlation statistically significant p < 0.01.

medium effect.

large effect.

The Cronbach’s and omega reliability coefficients were all above the cut-off threshold (α ≥ .70), indicating acceptable internal consistency, except for the working excessively as a first-order factor (α = .64; ω = .61) which can still be considered acceptable (Nunnally & Bernstein, 1994). Moreover, the overall workaholism score was acceptable (α = .78; ω = .78). The correlation matrix showed that workaholism was positively correlated with work overload (r = .57 large effect; supporting H₂) and total work hours (r = .34; medium effect supporting H₃)—providing evidence for convergent validity. Furthermore, workaholism was statistically significantly negatively correlated to work engagement (r = −.18) and commitment (r = −.31; medium effect) but correlated positively with burnout with large effect size (r = .55). Therefore, workaholism was negatively correlated with positive outcomes and positively correlated with negative outcomes, as expected. The correlations between all the variables were below the .85 cut-off suggested concern regarding the discriminant validity of variables (Brown, 2015), except, of course, for the higher-order model components.

Criterion Validity

Per the research hypotheses (H₃–H₆), regression paths were added to the final measurement model. The structural model was considered an acceptable fit to the data (CFI = 0.90; TLI = 0.90; RMSEA = 0.07). The results of the regressions are given in Table 3 below.

Table 3.

Criterion Validity of the Workaholism Scale.

Structural path	β	S.E.	p	Result
Workaholism → Work overload	.57	0.05	.001	Supports H₃
Workaholism → Work engagement	−.18	0.06	.001	Supports H₄
Workaholism → Organizational commitment	−.31	0.05	.001	Supports H₅
Workaholism → Burnout	.55	0.04	.001	Supports H₆

Notes: β = Standardized beta coefficient; S.E. = Standard error; p = Two-tailed statistical significance; p < .001.

Workaholism had a positive structural path to work overload (β = .57, S.E. = 0.05, p < .001; supporting H₃) and negative structural path to work engagement in the sample (β = −.18, S.E. = 0.06, p < .001; supporting H₄). Workaholism also had a negative structural path to organizational commitment (β = −.31, S.E. = 0.05, p < .001; supporting H₅). Finally, workaholism had a positive structural path to burnout (β = .71, S.E. = .04, p < .001; supporting H₆). Therefore, collectively, all these criterion-related validity hypotheses (H₅–H₈) were supported. Figure 1 presents the structural path relationships between workaholism and the organizational outcomes (i.e., work overload, work engagement, organizational commitment and burnout).

Figure 1 below presents a visual structural model with regression results.

Figure 1.

Structural regression paths for the criterion-related hypotheses.

Discussion

The study aimed to validate a workaholism scale (the English version of the DUWAS-10) by investigating its construct validity with the following variables: workaholism, work overload, work hours and several other outcomes (i.e., work engagement, organizational commitment and burnout). Firstly, the study investigated if the workaholism scale comprised a two-factor structure. The results of CFA revealed that the proposed two-factor measurement model was an acceptable fit for the data. However, upon closer inspection, the correlation between working excessively and working compulsively showed problematic discriminant validity (Brown, 2015). Specifically, the shared variance between the two proposed components was larger than the average variance extracted by each component, indicating problematic discriminant validity (Farrell, 2010).

Therefore, workaholism was operationalized as a second-order structure, which adequately fitted the data—isolating and integrating the explained variance of the working excessively and working compulsively variables into a higher-order latent variable, in contrast to most past studies. Balducci et al. (2017) also found that an adapted solution (correlation of errors) would best fit their data—even though this was not implemented. We believe the second-order model to be the model of choice, given that it allows for the first-order components (working compulsively and excessively) to remain available for future study comparison, but also provides an overall score which is “best of both worlds” when considering only a unidimensional model. This is also in line with the suggestion by Rantanen et al. (2015) that a unidimensional model should be avoided as workaholism should be considered by its conceptualized components.

Reasons why the two-factor model was not the model of choice remain speculative but includes that participant interpretation of the item text does not clearly distinguish between the behavioral and cognitive aspects of the phenomenon as it does in other contexts. The removal of item three from the working compulsively component (“I feel that there is something inside of me that drives me to work hard”) might also indicate variation in participants’ interpretation of the literal interpretation of the item—although this is also speculation. The removal of this item created an adapted 9-item version of the DUWAS-10, a so-called “DUWAS-9” for further analyses.

In terms of reliability, the results indicated that Cronbach’s alpha and the omega showed acceptable values. Specifically, the workaholism construct attained a satisfactory value (α = .78; ω = .78)—indicating acceptable internal consistency.

Then, convergent validity was established, as work overload and work hours were found to have positive and significant relationships with workaholism. This is in line with Shimazu and Schaufeli (2009) that found workaholism, mainly working excessively, related to increased levels of job demands such as work overload. This can be explained by workaholics’ tendencies to work compulsively and excessively. For example, Taris et al. (2005) indicated that workaholics are inclined to increase their workload themselves by making simple tasks much more complicated, and as a result, they perceive to have a higher workload, ultimately increasing work hours.

Indeed, workaholism was also positively, significantly related to work hours (total per week), indicating that as workaholism increases, so does total work hours. This finding is consistent with past research, which also found a positive relation between workaholism and working beyond regular working hours (e.g., Bonebright et al., 2000; Shimazu & Schaufeli, 2009). Similarly, Taris et al. (2005) found that workaholics spend more time on work than their colleagues, suggesting that their total working hours are somewhat inflated in comparison.

In terms of discriminant validity, it was imperative for the established workaholism construct to constitute a variable that was not measuring the same phenomenon as other variables in the study (e.g., work overload). Therefore, it was postulated that workaholism would be related to work overload, burnout, organizational commitment, and work engagement—but that workaholism would be statistically distinguishable from these variables and not correlated (or overlapping) too highly. The results indicated that the correlations between all the variables were well below .85, suggesting sufficient and acceptable discriminant validity for the workaholism construct with the other constructs (Brown, 2015).

Workaholism showed a negative relationship with work engagement. Shimazu et al., 2012) differentiated two types of working hard: workaholism (i.e., undesirable type) and work engagement (i.e., desirable type). The underlying motivations can clarify these types for working hard: workaholics are driven by their compulsive behavior, whereas engaged individuals are encouraged by their intrinsic motivation (Shimazu et al., 2015). Also, both work engagement and workaholism are characterized by high effort (e.g., time and energy), but workaholism is more related to destructive (negative) affect and work engagement to constructive (positive) affect (Shimazu & Schaufeli, 2009). Thus, workaholism and work engagement are two entirely different concepts (Shimazu & Schaufeli, 2009). This result is also in line with Schaufeli et al.’s (2008) evidence of a negative relationship between workaholism and engagement.

Workaholism also had a negative relationship with organizational commitment. This is supported by Killinger (2006), who revealed that workaholic employees work hard because they feel they should; feelings of guilt and anxiety are evoked if they do not work, so their work devotion has little to do with commitment toward their work or organization. Negative affect is stimulated through feelings of nervousness and anxiety, which are characteristic of workaholics (Killinger, 2006), which can explain workaholics having lower organizational commitment. Douglas and Morris (2006) also showed that workaholics experience lower job satisfaction, insufficient growth opportunities, high work pressure and a stronger intention to leave the organization. The latter points to poor organizational commitment. Van Beek et al. (2012) also found that workaholic employees are more dissatisfied with their careers, less committed to their organization, and actively seek other jobs.

Lastly, workaholism was found to have a positive relationship with burnout. As mentioned above, workaholics are not motivated to work hard because they derive satisfaction from work or their high achievement orientation—instead, they are driven by their perfectionistic nature and overly strict standards (Bakker et al., 2013). Workaholics set unreasonably high standards for themselves and only allow a small margin for errors—therefore, little is ever completed well enough (Taris et al., 2010). Consequently, after finalizing work, they are known to re-check and redo work, initiating another cycle of excessive work activity (Stoeber & Damian, 2016). Thus, they continuously invest energy and effort toward work-related activities, ultimately draining that energy, resulting in burnout (Molino et al., 2015). This argument is supported by the job-demands resources (JD-R) model that explains how excessive work (overt job demands) develop into job stressors (Bakker & Demerouti, 2007) and the effort-recovery model that explains how individuals are then unable to recover adequately (Taris et al., 2006). Furthermore, workaholics continuously increase their work efforts to achieve their personal goals, which expend the individual’s energy, ultimately leading to burnout (Schaufeli et al., 2009). Therefore, these path results indicate that workaholism is an undesirable phenomenon within the South African financial services sector and should not be considered model behavior as it will result in individual and organizational costs.

Limitations and Recommendations for Future Research

The first limitation of the study is that it only included employees within the financial services sector, presenting a somewhat limited view of the phenomenon. Therefore, future researchers should consider collecting samples from other industries and sectors to broaden our understanding of workaholism’s impact.

Secondly, the study utilized a cross-sectional research approach for data collection, meaning that data was only collected at a single point in time. This design is a valuable technique to understand how individuals feel about and view their occupation, but this design prevents the study from definitively presenting evidence for causal conclusions regarding the predictive hypotheses (Spector, 1994). Therefore, future researchers should also employ a longitudinal design to understand causal processes regarding the variables of interest (Taris & Kompier, 2003).

Lastly, the study used a self-report survey and people are inclined to answer personally sensitive questions in a socially desirable manner (Spector, 1994). Employees could have attempted to portray themselves more positively, tainting the overall picture of workaholism in this industry. Future researchers should utilize a mixed-method design by including interviews, structured, unstructured or semi-structured, depending on the responses they want to elicit—and perhaps objective data (if available) on, for example, total working hours.

Conclusion

This study aimed to validate a workaholism scale within the South African financial services industry. Investigation of instruments in other contexts is important as specific differences might exist as shown in this study. The adapted Dutch Work Addiction Scale (DUWAS-9; item 3 of working compulsively removed—see Table 1) demonstrated construct validity and presented as a reliable measuring instrument. Specifically, these study results in the South African financial services context have shown that workaholism is best operationalized as an overall score indicated by two first-order components (working compulsively and working excessively). Statistically significant relationships were established between workaholism and work overload, work hours, work engagement, organizational commitment, and burnout. Organizations are encouraged to utilize this scale to identify employees’ potential dysfunctional behavior patterns, which can influence the eventual profitability of an organization.

Footnotes

Authors’ Note

Significance of work: This validation study contributes to the understanding of the measurement of workaholism by showing that a second-order model indicated by an overall workaholism score made up of the individual workaholism component factors (working compulsively and working excessively) is the model of choice. This study enables researchers and professionals to measure workaholism more accurately on the African continent. It further adds to the debate that workaholism is undesirable and leads to negative consequences within the organizational context.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Leon T. de Beer

References

Allen

N. J.

Meyer

J. P.

(1990). The measurement and antecedents of affective, continuance, and normative commitment to the organisation. Journal of Occupational Psychology, 63(1), 1–18. https://doi.org/10.1111/j.2044-8325.1990.tb00506.x

Balducci

Avanzi

Consiglio

Fraccaroli

Schaufeli

(2017). A cross-national study on the psychometric quality of the Italian version of the Dutch Work Addiction Scale (DUWAS). European Journal of Psychological Assessment, 33(6), 422–428. https://doi.org/10.1027/1015-5759/a000300.

Balducci

Avanzi

Fraccaroli

(2018). The individual “costs” of workaholism: An analysis based on multisource and prospective data. Journal of Management, 44(7), 2961–2986. https://doi.org/10.1177/0149206316658348.

Bakker

A. B.

Demerouti

(2007). The job demands-resources model: State of the art. Journal of Managerial Psychology, 22(3), 309–328. https://doi.org/10.1108/02683940710733115

Bakker

A. B.

Demerouti

Oerlemans

Sonnentag

(2013). Workaholism and daily recovery: A day reconstruction study of leisure activities. Journal of Organizational Behavior, 34(1), 87–107. https://doi.org/10.1002/job.1796

Bakker

A. B.

Demerouti

Sanz-Vergel

A. I.

(2014). Burnout and work engagement: The JD–R approach. Annual Reviews in Organizational Psychology and Organizational Behavior, 1, 389–411. https://doi.org/10.1146/annurev-orgpsych-031413-091235

Baruch

(2011). The positive well-being aspects of workaholism in cross cultural perspective: The chocoholism metaphor. Career Development International, 16(6), 572–591. https://doi.org/10.1108/13620431111178335

Bonebright

C. A.

Clay

D. L.

Ankenmann

R. D.

(2000). The relationship of workaholism with work-life conflict, life satisfaction and purpose in life. Journal of Counseling Psychology, 47(4), 469–477. https://doi.org/10.1037/0022-0167.47.4.469

Brown

T. A.

(2015). Confirmatory factor analysis for applied research. The Guilford Press.http://www.kharazmi-statistics.ir/Uploads/Public/book/Methodology%20in%20the%20Social%20Sciences.pdf

10.

Burke

R. J.

(2004). Workaholism, self-esteem, and motives for money. Psychological Reports, 94(2), 457–463. https://doi.org/10.2466/pr0.94.2.457-463

11.

Burke

R. J.

Richardsen

A. M.

Mortinussen

(2004). Workaholism among Norwegian managers: Work and well-being outcomes. Journal of Organizational Change Management, 17(5), 459–470. https://doi.org/10.1108/09534810410554489

12.

Butucescu

Uscătescu

L. C.

(2019). What does the Romanian workaholic look like? A first glimpse into the links between workaholism and employee characteristics; A validation attempt of DUWAS scale. Psihologia Resurselor Umane, 11(1), 17–32.

13.

Clark

M. A.

Michel

J. S.

Zhdanova

Pui

S. Y.

Baltes

B. B.

(2014). All work and no play? A meta-analytic examination of the correlates and outcomes of workaholism. Journal of Management, 42(7), 1–35. https://doi.org/10.1177/0149206314522301

14.

Cohen

(1992). Quantitative methods in psychology: A power primer. Psychological Bulletin, 112(1), 155–159.

15.

De Beer

L. T.

Bianchi

. (2019). Confirmatory factor analysis of the Maslach Burnout Inventory: A Bayesian structural equation modeling approach. European Journal of Psychological Assessment, 35(2), 217–224. https://doi.org/10.1027/1015-5759/a000392

16.

Del Líbano

M. D.

Llorens

Salanova

Schaufeli

. (2010). Validity of a brief workaholism scale. Psicothema, 22(1), 143–150.

17.

De Vos

A. S.

Strydom

Fouché

C. B.

Delport

C. S. L

. (2011). Research at grass roots for the social science and human service professions (4th ed.). Van Schaik.

18.

Devi

(2012). Coping strategies adopted by bank employees to manage role stress: A study. Journal of Management Research, 11(4), 21–36.

19.

Douglas

E. J.

Morris

R. J.

(2006). Workaholic, or just a hard worker? Career Development International, 11(5), 394–417. https://doi.org/10.1108/13620430610683043

20.

Engelbrecht

G. J.

De Beer

L. T.

Schaufeli

W. B.

(2020). The relationships between work intensity, workaholism, burnout and self-reported musculoskeletal complaints. Human Factors and Ergonomics in Manufacturing & Service Industries, 30(1), 59–70. https://doi.org/10.1002/hfm.20821

21.

Farrell

A. M.

(2010). Insufficient discriminant validity: A comment on Bove, Pervan, Beatty, and Shiu (2009). Journal of Business Research, 63(3), 324–327. https://doi.org/10.1016/j.jbusres.2009.05.003

22.

Flora

D. B.

Curran

P. J.

(2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466-491. https://doi.org/10.1037/1082-989X.9.4.466

23.

Fong

T. C.

S. M.

(2012). Measuring engagement at work: Validation of the Chinese version of the Utrecht Work Engagement Scale. International Journal of Behavioural Medicine, 19(3), 391–397. https://doi.org/10.1007/s12529-011-9173-6

24.

Fry

L. W.

Matherly

L. L.

Vitucci

(2006). Research companion to working time and work addiction. Edward Elgar Publishing.

25.

Gini

(1998). Working ourselves to death: Workaholism, stress, and fatigue. Business and Society Review, 100(1), 45–56. https://doi.org/10.1111/0045-3609.00012

26.

Herbig

P. A.

Palumbo

F. A

. (1994). Karoshi: Salaryman sudden death syndrome. Journal of Managerial Psychology, 9(7), 11–16. https://doi.org/org/10.1108/02683949410075831

27.

Holgado–Tello

F. P.

Chacón–Moscoso

Barbero–García

Vila–Abad

(2010). Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Quality & Quantity, 44(1), 153–166. https://doi.org/10.1007/s11135-008-9190-y

28.

Horn

(2015). The validation of a workaholism scale within the South African banking industry [Unpublished dissertation]. North-West University. http://hdl.handle.net/10394/21296

29.

Iwasaki

Takahashi

Nakata

(2006). Health problems due to long working hours in Japan: Working hours, workers’ compensation (karoshi), and preventative measures. Industrial Health, 44(4), 537–540. https://doi.org/10.2486/indhealth.44.537

30.

Kanai

(2009). Karoshi (work to death) in Japan. Journal of Business Ethics, 84(2), 209–216. https://doi.org/10.1007/s10551-008-9701-8

31.

Kanai

Wakabayashi

(2001). Workaholism among Japanese blue-collar employees. International Journal of Stress Management, 8(2), 129–145. https://doi.org/10.1023/A:1009529314121

32.

Killinger

(2006). The workaholic breakdown syndrome. In Burke

(Ed.), Research companion to working time and work addiction (pp. 61–88). Edward Elgar.

33.

Liang

Chu

(2009). Personality traits and personal and organisational inducements: Antecedents of workaholism. Social Behavior and Personality, 37(5), 645–660. https://doi.org/10.2224/sbp.2009.37.5.645

34.

Liang

Yang

(2014). An evaluation of WLSMV and Bayesian methods for confirmatory factor analysis with categorical indicators. International Journal of Quantitative Research in Education, 2(1), 17–38. https://doi.org/10.1504/IJQRE.2014.060972

35.

Littman-Ovadia

Balducci

Ben-Moshe

(2014). Psychometric properties of the Hebrew version of the Dutch Work Addiction Scale (DUWAS-10). The Journal of Psychology, 148(3), 327–346. https://doi.org/10.1080/00223980.2013.801334

36.

McMillan

L. H. W.

O’Driscoll

M. P.

(2004). Workaholism and health: Implications for organisations. Journal of Organizational Change Management, 17(5), 509–519. https://doi.org/10.1108/09534810410554515

37.

McMillan

L. H. W.

O’Driscoll

M. P.

Brady

E. C.

(2004). The impact of workaholism on personal relationships. British Journal of Guidance & Counselling, 32(2), 171–186. https://doi.org/10.1080/03069880410001697729

38.

Meijman

T. F.

Mulder

(1998). Psychological aspects of workload. In Drenth

P. J.

Thierry

de Wolff

C. J.

(Eds.), Handbook of work and organisational psychology (2nd ed., pp. 5–33). Psychology Press.

39.

Molino

Bakker

A. B.

Ghislieri

(2015). The role of workaholism in the job-demands resources model. Anxiety, stress and coping, 29(4), 400–414. https://doi.org/10.1080/10615806.2015.1070833

40.

Mostert

Peeters

Rost

(2011). Work-home interference and the relationship with job characteristics and well-being: A South African study among employees in the construction industry. Stress and Health, 27(3), 238–251. https://doi.org/10.1002/smi.1374

41.

Mudrack

P. E.

(2004). Job involvement, obsessive-compulsive personality traits, and workaholic behavioral tendencies. Journal of Organizational Change Management, 17(5), 490–508. https://doi.org/10.1108/09534810410554506

42.

Muthén

du Toit

S. H. C.

Spisic

(1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes [Unpublished technical report]. https://www.statmodel.com/download/Article_075.pdf

43.

Muthén

L. K.

Muthén

B. O.

(2021). Mplus User’s Guide (8th ed.). Muthén & Muthén.

44.

Nunnally

J. C.

Bernstein

I. R.

(1994). Psychometric theory, (3rd ed.) McGraw-Hill.

45.

PwC. (2019). Financial services: The competitive landscape is rapidly changing and increasing in complexity across the financial services industry. https://www.pwc.co.za/en/industries/financial-services-industry.html

46.

Rantanen

Feldt

Hakanen

J. J.

Kokko

Huhtala

Pulkkinen

Schaufeli

(2015). Cross-national and longitudinal investigation of a short measure of workaholism. Industrial Health, 53(2), 113–123. https://doi.org/10.2486/indhealth.2014-0129

47.

Rautenbach

Els

De Beer

L. T.

(2020). The influence of thinking styles on the sales performance of South African financial advisors. Management Dynamics, 29(4), 33–46.

48.

Repaczki

Hristova

Krumova

Juhasz

Garvanova

(2021). Workaholism in the conditions of a pandemic: A comparative analysis between Bulgaria and Hungary. Diogenes, 29(1), 31–46.

49.

Robinson

B. E.

(1999). The work addiction risk test: Development of a tentative measure of workaholism. Perceptual and Motor Skills, 88(1), 199–210. https://doi.org/10.2466/pms.1999.88.1.199

50.

Sandrin

Gillet

(2016). Validation d’une version française de la Dutch Work Addiction Scale (DUWAS). Psychologie du Travail et des Organisations, 22(3), 147–159. https://doi.org/10.1016/j.pto.2016.02.007

51.

Schaufeli

W. B.

Bakker

A. B.

Salanova

(2006). The measurement of work engagement with a short questionnaire: A cross-national study. Educational and Psychological Measurement, 66(4), 701–716. https://doi.org/10.1177/0013164405282471.

52.

Schaufeli

W. B.

Bakker

A. B.

Van Rhenen

(2009). How changes in job demands and resources predict burnout, work engagement, and sickness absenteeism. Journal of Organizational Behavior, 30(7), 893–917. https://doi.org/10.1002/job.595

53.

Schaufeli

W. B.

Leiter

M. P.

Maslach

Jackson

S. E.

(1996). MBI-General Survey (MBI-GS). Mindgarden.

54.

Schaufeli

W. B.

Shimazu

Taris

T. W.

(2009). Being driven to work excessively hard: The evaluation of a two-factor measure of workaholism in the Netherlands and Japan. Cross-Cultural Research, 43(4), 320–348. https://doi.org/10.1177/1069397109337239

55.

Schaufeli

W. B.

Taris

T. W.

(2005). The conceptualisation and measurement of burnout: Common ground and worlds apart. Work & Stress, 19(3), 256–262. https://doi.org/10.1080/02678370500385913

56.

Schaufeli

W. B.

Taris

T. W.

Bakker

A. B.

(2006). Dr Jekyll or Mr Hyde? On the differences between work engagement and workaholism. In Burke

(Ed.), Research companion to working time and work addiction (pp. 193–217). Edward Elgar.

57.

Schaufeli

W. B.

Taris

T. W.

Van Rhenen

(2008). Workaholism, burnout, and work engagement: Three of a kind or three different kinds of employee well-being? Applied Psychology, 57(2), 173–203. https://doi.org/10.1111/j.1464-0597.2007.00285.x

58.

Schou Andreassen

Hetland

Pallesen

. (2013). Psychometric assessment of workaholism measures. Journal of Managerial Psychology, 29(1), 7–24. https://doi.org/10.1108/JMP-05-2013-0143

59.

Scott

K. S.

Moore

K. S.

Miceli

M. P.

(1997). An exploration of the meaning and consequences of workaholism. Human Relations, 50(3), 287–314. https://doi.org/10.1177/001872679705000304

60.

Shimazu

Schaufeli

W. B.

(2009). Is workaholism good or bad for employee well-being? The distinctiveness of workaholism and work engagement among Japanese employees. Industrial Health, 47(5), 495–502. https://doi.org/10.2486/indhealth.47.495

61.

Shimazu

Schaufeli

W. B.

Kamiyama

Kawakami

(2015). Workaholism vs. work engagement: The two different predictors of future well-being and performance. International Journal of Behavioral Medicine, 22(1), 18–23. https://doi.org/10.1007/s12529-014-9410-x

62.

Shimazu

W. B.

Schaufeli

W. B.

Kubota

Kawakami

(2012). Do workaholism and work engagement predict employee well-being and performance in opposite directions? Industrial Health, 50(4), 316–321. https://doi.org/10/2486/indhealth.MS1355

63.

Snir

Harpaz

(2006). The workaholism phenomenon: A cross-sectional perspective. Career Development International, 11(5), 374–393. https://doi.org/10.1108/13620430610683034

64.

Spector

P. E.

(1994). Using self-report questionnaires in OB research: A comment on the use of a controversial method. Journal of Organizational Behavior, 15(5), 385–392. https://doi.org/10.1002/job.4030150503

65.

Spence

J. T.

Robbins

A. S.

(1992). Workaholism: Definition, measurement and preliminary results. Journal of Personality Assessment, 58(1), 160–178. https://doi.org/10.1207/s15327752jpa5801_15

66.

StatsSA (2020). GDP rallies as lockdown restrictions ease. http://www.statssa.gov.za/?p=13849

67.

Stoeber

Damian

L. E.

(2016). Perfectionism in employees: Work engagement, workaholism, and burnout. In Sirios

F. M.

Molnar

D. S.

(Eds.), Perfectionism, health, and well-being (pp. 265–283). Springer International Publishing/Springer Nature.

68.

Sverke

Hellgren

Öhrming

(1999). Organisational restructuring and health care work: A quasi-experimental study. In le Blanc

P. M.

Peeters

M. C. W.

Bussing

Schaufeli

W. B.

(Eds.), Organisational psychology and health care: European contributions (pp. 15–32). Rainier Hampp Verlag.

69.

Taris

T. W.

Beckers

D. G.

Verhoeven

L. C.

Geurts

S. A.

Kompier

M. A.

Van der Linden

(2006). Recovery opportunities, work–home interference, and well-being among managers. European Journal of Work and Organizational Psychology, 15(2), 139–157. https://doi.org/10.1080/13594320500513889

70.

Taris

T. W.

Kompier

(2003). Challenges in longitudinal designs in occupational health psychology. Scandinavian Journal of Work, Environment & Health, 29(1), 1–4. https://doi.org/10.5271/sjweh.697

71.

Taris

T. W.

Schaufeli

W. B.

(2003). Work, work, and more work. The conceptualisation, causes and consequences of workaholism. De Pscholoog, 38(10), 506–512.

72.

Taris

T. W.

Schaufeli

W. B.

Verhoeven

L. C.

(2005). Workaholism in the Netherlands: Measurement and implications for job strain and work-nonwork conflict. Applied Psychology: An International Review, 54(1), 37–60. https://doi.org/10.1111/j.1464-0597.2005.00195.x

73.

Taris

T. W.

Van Beek

Schaufeli

W. B.

(2010). Why do perfectionists have higher burnout risk than others? The mediational effect of workaholism. Romanian Journal of Applied Psychology, 12(1), 1–7.

74.

Van Beek

Taris

T. W.

Schaufeli

W. B

. (2011). Workaholic and work engaged employees: Dead ringers or worlds apart? Journal of Occupational Health Psychology, 16(4), 468–482. https://doi.org/10.1037/a0024392

75.

Van Beek

Schaufeli

W. B.

Taris

T. W.

Schreurs

B. H. J

. (2012). For fun, love, or money: What drives workaholic, engaged, and burned-out employees at work? Applied Psychology: An International Review, 61(1), 30–55. https://doi.org/0.1111/j.1464-0597.2011.00454

76.

Van Beek

Taris

T. W.

Schaufeli

W. B.

Brenninkmeier

. (2014). Heavy work investment: Its motivational make-up and outcomes. Journal of Managerial Management, 29(1), 46–62. https://doi.org/10.1108/JMP-06-2013-0166

77.

Van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486–492. https://doi.org/10.1080/17405629.2012.686740

78.

Van Hooff

M. L. M.

Geurts

S. A. E.

Kompier

M. A. J.

Taris

T. W

. (2007). Workdays, in-between workdays and the weekend: A diary study on effort and recovery. International Archives of Occupational and Environmental Health, 80(7), 599–613. https://doi.org/10.1007/s00420-007-0172-5

79.

Van Wijhe

Peeters

Schaufeli

W. B.

Van den Hout

. (2011). Understanding workaholism and work engagement: The rule of mood and stop rules. Career Development International, 16(3), 254–270. https://doi.org/10.1108/13620431111140156

80.

Van Wijhe

C. I.

Peeters

M. C. W.

Schaufeli

W. B

. (2014). Enough is enough: Cognitive antecedents of workaholism and its aftermath. Human Resource Management, 53(1), 157–177. https://doi.org/10.1002/hrm.21573

81.

Williams

(2012). Are you a workaholic? http://business.financialpost.com/executive/careers/are-you-a-workaholic

Construct and Criterion Validity of the Dutch Workaholism Scale (DUWAS) Within the South African Financial Services Context

Abstract

Keywords

Introduction

Method

Research Participants

Measuring Instruments

Statistical Analysis

Results

Measurement Models

Reliability Coefficients and Correlation Matrix for the Study Variables

Criterion Validity

Discussion

Limitations and Recommendations for Future Research

Conclusion

Footnotes

Authors’ Note

Declaration of Conflicting Interests

Funding

ORCID iD

References