‘Same,but different’: A mixed-methods realist evaluation of a cluster-randomized controlled participatory organizational intervention

Abstract

Participatory organizational interventions are a recommended approach to improve the psychosocial work environment. As interventions of this type are shaped by employees and managers, their implementation can vary considerably, making evaluation challenging. This study contributes to our understanding of interventions by focusing on how the intervention mechanisms and the organizational context interact. In a mixed-methods design, we use multi-group structural equation modelling of pre-and post-intervention survey data (N = 204) to test multiple mediational mechanisms in three different contexts. We then analyse interviews (N = 67) and field observations of workshops to identify the role of contextual factors. The findings suggest that participatory organizational interventions do not produce one-size-fits-all results; on the contrary, intervention results are better understood as products of multiple intervention mechanisms interacting with the specific organizational contexts.

Keywords

collective efficacy context structural equation modelling transformational leadership work environment

Introduction

Participatory organizational interventions (POIs), defined as participatory ‘planned, behavioural, theory-based actions that aim to improve employee health and well-being through changing the way work is designed, organized and managed’ (Nielsen, 2013: 1030), have been recommended to improve the psychosocial work environment (EU-OSHA, 2000; ILO, 2001). The advantages of POIs are fivefold: (i) they make use of employees’ expertise of what needs to change, allowing for tailoring of the intervention for a specific context; (ii) they ensure participants’ ownership of the intervention; (iii) they promote a learning process in which employees and managers collectively become empowered to deal with problems in their work environment (Mikkelsen et al., 2000; Nielsen and Randall, 2012); (iv) they address the problems related to working conditions at the source rather than treating symptoms and consequences of a poor work environment alone (Lamontagne et al., 2007); and (v) POIs often include elements targeting the individual, team and manager levels, thus taking into account the interdependence of organizational levels (Nielsen, 2013) and potentially targeting intended areas of change at multiple levels (Lamontagne et al., 2007).

One disadvantage of POIs is that their outcomes are highly situation-specific as the aims of the intervention, the intervention activities and how these activities are implemented is determined by managers and employees during the intervention (Nielsen and Miraglia, 2017; Nielsen et al., 2006). Furthermore, the outcomes of POIs are multifarious, meaning that the outcomes are notoriously difficult to measure (Holman and Axtell, 2016). To address these issues, it has been argued that effect evaluations of POIs should focus on proximal rather than distal outcomes of the intervention process, and that a suitable proximal outcome is awareness of the psychosocial work environment within the organization and the participants’ capability to manage the psychosocial work environment (von Thiele Schwarz et al., 2017). Also, because randomized and quasi-experimental, controlled designs with pre- and post-intervention measurements have proved inadequate for detecting the breadth of effects of participatory interventions, calls have been issued to further explore the links between POI processes and outcomes, for example through application of a realist evaluation framework (Nielsen and Miraglia, 2017).

From a realist evaluation perspective, interventions do not have an effect in and of themselves; rather, the activation of various mechanisms is what makes the intervention work. Mechanisms ‘can be expressed through interpretations, considerations, decisions and behaviours of participants, and outcomes are the result of their actions and interpretations of the intervention’ (Pawson, 2013, cited in Nielsen and Miraglia, 2017: 46). Mechanisms are not only individual factors (such as change readiness) but also collective factors related to managerial behaviour and culture (Marchal et al., 2012). The same intervention might work through a number of different mechanisms at the same time (Holman and Axtell, 2016); the mechanisms can interact with each other, and their relative importance for the overall effect of the intervention can shift over the duration of the intervention (Lacouture et al., 2015). Although it is possible to capture outcomes and mechanisms using quantitative data, it is difficult to determine up front which contextual factors may trigger these mechanisms (Pawson, 2013). Qualitative methods may better capture the complex nature of the context, and thus a mixed-methods design is suitable (Pawson, 2013).

In the present study, we use a mixed-methods design to analyse an organization-level participatory, cluster-randomized, controlled intervention conducted in three industrial production plants. We apply a realist evaluation (Pawson, 2013) methodology to explore how mechanisms bring about the resulting outcomes work in context-specific ways (Pawson, 2013; Pawson and Tilley, 1997). Context is hence viewed as something that neither can nor should be kept stable, as in a controlled experiment, but as a multifaceted factor that impacts whether intervention mechanisms work as intended (Pawson, 2013). Other studies have examined related aspects of POIs. For a POI in the Danish postal service, Abildgaard et al. (2018) used a realist evaluation-based approach to both examine the program theory developed by the participants and subsequently demonstrated an effect on the outcomes targeted by participants. Similarly, von Thiele Schwarz et al. (2017) demonstrated how kaizen work functioned as a mechanism in two studies.

Our study contributes to the existing literature in two ways. First, the study addresses Nielsen’s (2013) call to explore the mechanisms of the participatory process. Since POIs involve discussions between employees and managers about how to improve the work environment, we propose and test two mechanisms that might shape the intervention outcomes: (i) employees need to feel capable of solving the problems identified in the intervention as a group and of making changes to the way work is organized, designed and managed (i.e. a mechanism of collective efficacy); and (ii) managers need to formulate a vision for the intervention and encourage independent decision making and development of innovative solutions to problems (i.e. a mechanism of transformational leadership) (Lundmark et al., 2017; Nielsen, 2013).

Second, the study applies Pawson’s (2013) suggestion of using a mixed-methods realist evaluation approach to POI evaluation in order to understand ‘what works for whom in which circumstances’. To the best of our knowledge, only a few studies have done so previously (Abildgaard et al., 2018; von Thiele Schwarz et al., 2017).

In this study, we explore two mechanisms linked to core aspects in the literature of how interventions could improve participants’ perceived ability to manage the psychosocial work environment: collective efficacy and transformational leadership. The choice of these mechanisms is based on them covering two core aspects of POIs, namely active employee participation and empowerment (collective efficacy) and proactive line manager behaviour (transformational leadership) (Nielsen, 2013). The three organizations in the study are in the same sector (i.e. industrial production plants), but produce different products. Other similarities are that they all employ low-skilled machine operators – although the demand for technical skill is increasing, many tasks are still simple – and that most of the employees are male and work in shifts.

The primary contribution of the study is to further knowledge of how different POI mechanisms are contextual and hence suggest new venues of intervention evaluation research focusing on the how intervention mechanisms work in different contexts.

Collective efficacy as a concept

Collective efficacy can be defined as ‘a group’s shared belief in its conjoint capabilities to organize and execute the courses of action required to produce given levels of attainment’ (Salanova et al., 2003: 45).The concept is based on social cognitive theory (Bandura, 1997), which posits that people’s actions are shaped by beliefs formed over time as a result of their social interactions. This means that in a work context, employees are only likely to attempt changing the way work is designed, organized and managed if they believe that they can cope with the challenges they might face in the process. Collective efficacy is ‘not simply the sum of the efficacy beliefs of individual members’, but ‘an emergent group-level property’ (Salanova et al., 2003: 45). Central to collective efficacy are beliefs that the team can manage challenges, keep on track while implementing plans, and that employees trust the group’s ability to solve problems.

The effects of interventions on collective efficacy

It seems likely that the process of working together in POIs to assess problems and devise and implement solutions may lead to improvements in collective efficacy, which in turn affect the employees’ perceived ability to manage the psychosocial work environment. Previous studies have shown that individual self-efficacy can be improved through workplace interventions (Bresó et al., 2011), and related group properties such as group coherence have been found to improve as a result of POIs more specifically (Arneson and Ekberg, 2005). Employees’ collective efficacy beliefs may increase owing to how POIs facilitate their learning of skills, which they can use to improve their working conditions in the future (Mikkelsen et al., 2000). In addition, POIs may motivate employees to try to influence their working conditions as a group (Nielsen, 2013).

The effects of collective efficacy on the intervention outcome

Because POIs are collective endeavours, the participating employees’ collective efficacy beliefs are likely to influence the participatory process and thus its outcome. For example, the participants’ capability to manage the psychosocial work environment is likely to depend on whether participants persist in the face of adversity, engage in difficult discussions about how to change their working practices and procedures and subsequently work towards implementing these changes. Although, to the best of our knowledge, no interventions studies have focused on whether employees’ collective efficacy shapes the overall outcomes of the intervention, it has been suggested that collective efficacy is related to change efficacy, which is defined as having a strong belief in one’s own ability to handle the challenges in a workplace and is vital for proactive problem-solving (Weiner et al., 2009). While POIs have been said to increase employees’ job control in general (Mikkelsen and Gundersen, 2003; Nielsen 2013), collective efficacy has been found to moderate the stress-buffering effect of job control, so that increased job control has particularly positive effects on well-being when collective efficacy is also high (Schaubroeck et al., 2000). Others have argued that collective efficacy is an important component of both employees’ psychological empowerment and organizational readiness for change in relation to successfully implementing worksite health and safety programs (McQuiston, 2000; Weiner et al., 2009). At a more general level, it has been argued that efficacy belief contributes to positive gain spirals, for example by increasing employees’ work engagement (Llorens et al., 2007; Salanova et al., 2011), and such positive gain spirals may be reflected in the intervention outcome.

Overall, a potential mechanism in POIs may be that the participatory activities both influence the participating employees’ collective efficacy beliefs, and that a larger effect will be seen on the intervention outcome in cases where the participants’ collective efficacy has developed positively. Accordingly, we hypothesize the following:

Hypothesis 1 (H1): The POI will lead to increases in collective efficacy, which in turn leads to increases in the awareness of and capability to manage the psychosocial work environment, i.e. collective efficacy is a mediational mechanism that brings about the intended outcome.

Transformational leadership as a mechanism

Line managers are known to be instrumental in shaping the outcome of interventions (Nielsen, 2013, 2017) as they typically hold the main responsibility for implementing interventions (Kompier et al., 2000; Nielsen, 2013). They are also central to enabling participants to participate in the intervention without becoming overworked or fearing negative consequences if critiques are raised, and they help employees successfully raise suggestions for various aspects of the organization (Mowbray et al., 2015; Nielsen, 2013). Overall, manager support, and in particular a transformational leadership style, has been suggested as a core mechanism of POIs (Coyle-Shapiro, 1999; Dahl-Jørgensen and Saksvik, 2005; Edwards et al., 2008; Nielsen et al., 2009; Randall et al., 2005). The core components of such a transformational leadership style are idealized influence, inspirational motivation, intellectual stimulation and individualized consideration (Bass and Riggio, 2012).

The effects of interventions on transformational leadership

Taking an active role in POIs with an emphasis on the relationship between leaders and followers may influence line managers’ behaviour towards a more transformational leadership style. For example, line managers may exercise idealized influence by focusing on moral commitment (Bass and Riggio, 2012), such as by prioritizing employees’ long-term health over short-term financial gains during discussions of the work environment. Trust and openness are crucial in POIs as employees need to trust there are no hidden motives behind the changes introduced (Aust et al., 2010). Line managers may also engage in inspirational motivation by formulating a clear vision of the POI and engaging followers in how to achieve this vision (Nielsen et al., 2010). Line managers may promote intellectual stimulation in POIs by challenging followers to think of new ways of doing work that can improve employees’ health and well-being. And finally, the line managers may display individualized consideration by taking into account the needs of individual employees (Bass and Riggio, 2012), showing consideration for the employees’ suggestions in POIs. Empirically, some studies have demonstrated that POIs may change the way managers communicate with and are perceived by employees (Bourbonnais et al., 2006), and that when leaders were responsible for action planning, they were seen as more supportive, empowering and fair (Björklund et al., 2007).

The effects of transformational leadership on intervention outcome

By way of how they promote employee engagement and facilitate a collaborative atmosphere between employees and managers, transformational leadership behaviours (Judge and Piccolo, 2004) are in turn likely to increase the positive effects of POIs. Transformational leaders who are successful in creating strong identification within the employee group might influence the employees’ health-related behaviours (Haslam et al., 2008). However, transformational leaders also influence the employees’ work characteristics, and this mediated effect seems to account for an important part of the well-established relationship between transformational leadership and employee well-being (Fernet et al., 2015; Nielsen et al., 2008). In participatory interventions, transformational leadership behaviours have also been shown to predict health-related outcomes through the line managers’ attitudes and actions towards the intervention (Lundmark et al., 2017). Indeed, managers may display a range of facilitatory behaviours that are indicative of a transformational leadership style in participatory settings, such as inviting all employees to take part in discussions and offering support and intellectual stimulation through discussions of how employees’ suggestions can be implemented (Yeung, 2004). Thus, it is plausible that increases in transformational leadership behaviours act as a mechanism by which POIs could improve the way employees manage the psychosocial work environment. We therefore hypothesize the following:

Hypothesis 2 (H2): The POI will lead to increases in transformational leadership, which in turn leads to increases in the awareness of and capability to manage the psychosocial work environment, i.e. transformational leadership is a mediational mechanism that brings about the intended outcomes.

The role of context in shaping intervention outcomes

Most often, the context of POIs is seen as ‘noise’ (Nielsen et al., 2006) and something that should be controlled for (Nielsen et al., 2010). However, according to the principles of realist evaluation, contexts need to be explicitly explored and studied (Pawson, 2013). This is because the context within which POIs are implemented shapes which mechanisms are triggered (Greenhalgh et al., 2015). Specifically, organizational context can be defined as ‘situational opportunities and constraints that affect the occurrence and meaning of organizational behaviour as well as functional relationships between variables’ (Johns, 2006: 386). Furthermore, the context of organizational interventions covers a wide range of phenomena including: . . .the time and place where it is implemented, and the nature of the work that is carried out by participants [. . .], the implementation of new organizational structures concurrent to the intervention (Nielsen et al., 2006), other conflicting change initiatives (Guastello, 1993; Nielsen et al., 2010a), lack of integration of the intervention with corporate strategic planning (Schurman & Israel, 1995) or macro-economic factors, e.g. economic recession and subsequent organizational downsizing. (Nielsen and Abildgaard, 2013: 285–286) In practice, contexts are diverse and are studied in a number of different ways, and, as a result, establishing general patterns of how combinations of organizational interventions and contexts produce (or fail to produce) certain effects is still challenging (Daniels et al., 2017; Havermans et al., 2016). However, as an example of an important contextual factor, Poulsen et al. (2015) reported that the participants in their POI study experienced a pressure to bill time on work projects that deterred employees from engaging in the intervention. In relation to concurrent contextual events, Nielsen et al. (2006) demonstrated how the differences in the context and implementation of a health promotion initiative led to markedly different effects in the targeted workplaces. Important contextual factors included ‘change fatigue’ experienced by some participants owing to a number of changes implemented before the intervention, and also the appointment of a new manager. Albertsen et al. (2014) provided an excellent example of how context may influence an intervention’s outcomes by showing that the same rostering intervention implemented in three different contexts had three levels of effectiveness. The three contexts did not provide an equally suitable fit for the intervention (Randall et al., 2012), for example because of how the new system was counteracted by measures from the management which decreased the employees’ ability to control their working hours in two of the three workplaces. Since this article focuses on collective efficacy and transformational leadership as potential mechanisms in POIs, we will specifically address contextual circumstances which seem likely to have influenced the action of these mechanisms, such as previous experiences with addressing work environment issues as a group or seeking the help of one’s line manager.

A limitation of the above studies is that these did not test for differential activation of the underlying mechanisms. Our study expands on this line of research by examining whether and how three different organizational contexts triggered the mechanisms of the intervention.

We therefore propose a third hypothesis which we will answer using quantitative data, and a research question, which we will answer using qualitative data:

Hypothesis 3 (H3): The impact of the two mediating mechanisms will differ across the three companies.

Research question 1 (RQ1): What factors in the organizational context play a role in the mechanisms of the POI?

Methods

The study was based on a POI using a standardized procedure implemented in three different industrial production plants. The baseline questionnaire (T1) was distributed before the POI activities began, and the follow-up (T2) was sent 1 year later, leaving at least 6 months between last intervention workshop to T2. The data for the study were collected 2014–2015.

The participatory organizational intervention

The POI consisted of a number of interrelated activities conducted at different organizational levels following a five-phase (initiation, screening, action planning and evaluation) model (Nielsen and Noblet, 2018). The core component was a series of three 3-hour workshops with each team and their line managers facilitated by a member of the research group. The aim of the POI workshops was to improve working conditions, with the subsequent long-term goal to improve employee work ability and sustain labour market affiliation. In the mapping workshop, employees assessed the demands and resources associated with their job (screening phase). Graphical facilitation and dialogue tools were used to develop an assessment of the demands and resources of that particular team. In the action planning workshop, the team used the assessment from the mapping workshop to develop a number of action plans targeting their specific context (action planning phase), and in the follow-up workshop they followed up on the action plans and reflected on the process so far (implementation phase). Action plans developed in the project include diverse activities such as expanding the level of detail in customer orders to improve the meaningfulness of work, installing additional video surveillance of the shop floor to help operators assess the state of the production, and increasing dialogue between day and evening shifts to improve collaboration. Afterwards, the companies would continue with the implementation, and evaluate with the researchers at the end of the process (evaluation phase). In total, 73 workshops across the three companies were conducted (24 mapping, 24 action planning and 25 follow-up). In addition, individual talks with managers about employees’ working conditions and work ability were offered, and teams were given the opportunity to call in support from an ergonomic consultant. Visual and hands-on tools and approaches common in lean production were used in the workshops (Ohno, 1988; von Thiele Schwarz et al., 2017), both to ensure a structured process as well as ensuring a fit of the intervention to the practice of the workplace (Randall et al., 2012). The workshops focused on both demands and resources based on the job demands-resources model (Demerouti et al., 2001). Employee participation and direct assessment of improvement opportunities were inspired by a participatory ergonomics approach (Wilson and Haines, 2006). A detailed description of the intervention programme can be found in Gupta et al. (2015), Wåhlin-Jacobsen et al. (2017) and Wåhlin-Jacobsen (2018), and more information can be obtained upon request from the first author.

Population

The population of all three companies consisted mainly of industrial machine operators, with some mechanics, electricians and metal workers. The three companies vary in terms of goods produced and organizational structure. The plant of Company 1 (C1) produces pharmaceutical products subject to strict product safety regulations (N = 294, mean age 45 years, 72% male, mean tenure 11 years). Company 2 (C2) produces plastics, predominantly packaging materials (N = 104, mean age 45 years, 91% male, mean tenure 12 years). Company 3 (C3) is a food production plant, and the participating teams are tasked with packaging and preparing products for packaging (N = 95, mean age 41 years, 46% male, mean tenure 9 years). Employees were cluster randomized at the departmental level to either the intervention group or a comparison group to minimize contamination. The total sample consisted of 493 employees recruited from the three companies. One department in C1 was taken out of the study before randomization by the company as they would not be able to participate if they were randomized to the intervention condition. In total, 204 of the remaining 422 employees completed both the T1 and T2 questionnaires, leading to a response rate of 48% (204/422). For C1, 42 percent completed both rounds (80/190), 55% completed both rounds (61/111) for C2, and 52% completed both rounds (63/121) for C3.

Qualitative data collection

A selection of this sample – 45 employees, 17 line managers and 5 foremen – were interviewed (in C2, teams had foremen who did not have formal managerial authority but were responsible for managing production). All line managers were interviewed, and at least two employees from each team were interviewed. All shifts and job groups were represented. Employee interviewees were selected at random from each team’s personnel list. The interviews were semi-structured (Brinkmann and Kvale, 2015), lasting an average of 35 minutes, and conducted shortly prior to follow-up measurement. They focused on interviewees’ experiences with the POI and concurrent events. Each interviewee was given an anonymized identification number for the purpose of this article.

Additionally, during each workshop, an observer took structured notes about the process, focusing on the quality of social interactions.

Measures

Collective efficacy (four items)

We measured perceived collective efficacy using an adapted version (Salanova et al., 2003) of the General Self-Efficacy scale (Schwarzer and Jerusalem, 2010). We rephrased generic personal self-efficacy items to focus on the perceived work-related efficacy of a team. A sample item includes: ‘I am confident that my team can deal efficiently with unexpected events’. Ratings were provided on a five-point scale from 1 (never) to 5 (very often). Cronbach’s alpha was .89 at T1 and .92 at T2.

Transformational leadership (seven items)

Employees’ perceptions of line managers’ transformational leadership were measured using the Global Transformational Leadership scale (Carless et al., 2000). This scale is a psychometrically valid short form of longer measures of transformational leadership, such as the Multifactor Leadership Questionnaire (MLQ) (Avolio and Bass, 2004), and has proven useful for measuring transformational leadership behaviours in workplace interventions for which questionnaire length is an issue (see, for instance, Lundmark et al., 2017). A sample item includes: ‘My leader communicates a clear and positive vision of the future’. Ratings were provided on a five-point scale from 1 (to a very small extent) to 5 (to a very large extent). Cronbach’s alpha was .95 at T1 and .94 at T2.

Awareness of and capability to manage the psychosocial work environment (ACM-PWE) (five items)

To assess the impact of the intervention on psychosocial risk management and following the example of von Thiele Schwarz et al. (2017), a scale consisting of items assessing generic changes in working conditions, awareness of the psychosocial work environment and improvements in the management of working conditions was developed for this intervention study. This scale includes the following items: ‘During the last year I have had more influence in relation to the implementation of changes’, ‘During the last year the dialogue of physical and psychosocial working conditions, work ability and well-being has improved’, ‘During the last year we have had good opportunity to improve the physical and psychosocial working conditions, work ability and well-being in the team’, ‘During the last year the company has, all in all, become a better workplace’ and ‘During the last year we have had more focus on work environment, work ability and well-being’. Ratings were provided on a five-point scale from 1 (highly disagree) to 5 (highly agree). Cronbach’s alpha was .89 at T2. Given that the ACM-PWE scale is a tailored measurement tool, we further evaluated its validity using Item Response Theory (IRT; van der Linden and Hambleton, 1997) and CFA (see details in Supplemental file A). The overall IRT (chi-squared = 38.7, d.f. = 28, P = .0859) and CFA (chi-squared = 8.6, d.f. = 4, P = .0701, RMSEA = .0076, TLI = .996, CFI = .998) model fit was excellent, indicating that the ACM-PWE scale is a valid measurement tool.

Mixed-methods approach

We employed a parallel, mixed-methods data collection approach (Teddlie and Tashakkori, 2009), collecting questionnaires at the beginning and end of the POI, observations during activities, and interviews in the weeks before the follow-up questionnaire. Using an explanatory mixed-methods evaluation design (Creswell and Plano Clark, 2011), we analysed the qualitative data to explain and nuance the quantitative findings, which implies a heavier reliance on quantitative rather than qualitative aspects in the mixed-methods analysis (a follow-up explanations model with quantitative aspects emphasized; see Creswell and Plano Clark, 2011). The choice of a mixed-methods design was to triangulate results and provide a more comprehensive evaluation of the POI mechanisms (Nastasi et al., 2007). The results are presented sequentially with quantitative analysis preceding the qualitative analysis.

Statistical analysis

Structural equation models based on polychoric correlations were fitted to the data using Mplus (version 7.4). Nested models were compared using the DIFFTEST option to compare differences between chi-squares. Estimates of total, direct and indirect effects are reported, along with their 95% confidence intervals obtained using the delta method. The fitted model is represented by the path diagram in Figure 1 (based on model 1c in Hayes, 2009), the latent variable collective efficacy was measured using four indicators (items) restricted to have the same parameters across time points, and the latent variable transformational leadership was measured using seven indicators (items) restricted to have the same parameters across time points. To evaluate the fit of the SEM, we used the RMSEA, Tucker-Lewis index (TLI) (Tucker and Lewis, 1973) and comparative fit index (CFI) (Bentler, 1990), adhering to the evaluation criteria presented by Hu and Bentler (1999).

Figure 1.

Statistical model.

We used multi-group structural equation models with the three companies as separate groups (Company 1, N = 80; Company 2, N = 61, Company 3, N = 63, Total sample size 204), and (i) restricting item thresholds, factor loadings and structural parameters to be equal, or (ii) restricting item thresholds and factor loadings but allowing structural parameters to vary across workplaces. When restricting all parameters to be equal, the model had a poor fit (RMSEA = .060, p = .063, TLI = .986, CFI = .987). The modification indices in Mplus suggested that a correlation between transformational leadership and collective efficacy at T2 should be included in the model. The correlation between collective efficacy and transformational leadership was deemed theoretically reasonable, and was subsequently introduced in model 1 as well. The fit of the multi-group model improved satisfactorily (RMSEA = .053, p = .331, TLI = .989, CFI = .990). Measurement invariance was tested using a model where factor loadings of specific items were allowed to vary across time points and between groups (RMSEA = .049, p = .527, CFI = .991), and comparing it to the multi-group model with item loading parameters restricted across time and group (RMSEA = .053, p = .331, TLI = .989, CFI = .990). The change in fit ( ΔCFI = .001, ΔRMSEA = .004 ) was deemed satisfactory (Chen, 2007).

Compared across intervention and comparison groups with a chi-squared test, no gender, tenure or age differences were observed in non-response rate at T2, but Company 1 had a statistically significantly higher non-response rate at T2 compared with that for Companies 2 and 3. Furthermore, for both transformational leadership and collective efficacy, chi-squared tests showed no difference in baseline values between the employees responding at both time points and those who did not respond at T2.

Qualitative analysis

Analysis of the interviews and observations was used to assess the character and quality of the contexts for each company. We examined the contextual factors and conditions that may or may not trigger the two mechanisms proposed in the study. The roles of the qualitative analysis are thus to aid in interpretation of the quantitative result, corroborate puzzling results and provide empirically founded explanations for the results.

The qualitative analysis was based on two rounds of coding. First, a round of structural coding focused on identifying elements in the interviews and observer notes containing information on the topic of the POI, especially the influence of the context and its effect. During structural coding, all notes taken by the workshop observer and interviews were screened for statements about how the context affected the intervention. The coded factors were analysed in a second round of evaluation coding (Rallis and Rossman, 2003; Saldaña, 2015) in regard to their relevance to the two mechanisms of interest in the study. Specifically, we coded for factors that likely promoted or hindered the two mechanisms in question, with the goal of providing an answer for the research question of why the mechanisms differed between companies. As suggested by MacQueen et al. (2008), the structural coding served as an initial strategy to identify material for the more focused, theoretical and detailed evaluation coding analysis. The results of these coding processes were written up as case descriptions. NVivo 11 (QSR International, 2015) was used for transcription, coding and analysis of the qualitative data.

Results

Hypothesis testing

Table 1 shows the measures, means, standard deviations and correlations of the variables in the study. Figure 2 shows the results of the path model for all three companies.

Table 1.

Means (standard deviations) and correlations for measures before and after the intervention.

	M (SD)T1	1	2	3	4	5
Transformational leadership T1	3.36 (.89)	1	–	–	–	–
Transformational leadership T2	3.34 (.91)	.35**	1	–	–	–
Collective efficacy T1	4.13 (.67)	.10**	.06	1	–	–
Collective efficacy T2	4.14 (.71)	.11**	.18**	.28**	1	–
Awareness of and capability to manage the psychosocial work environment	3.26 (.86)	.20**	.31**	.06	.20**	1

M = mean, SD = standard deviation, N = 204,

Correlation is significant at the .05 level (2-tailed). ** Correlation is significant at the .01 level (2-tailed). Correlations for each company can be found in supplemental file B.

Figure 2.

Path model of all data.

The model presented in Figure 2 shows that, viewing the intervention as comparable across cases, there is only clear support for the direct path (i.e. the intervention leads to improvements in ACM-PWE. Regarding the mechanisms presented in H1 (improvements via changes in collective efficacy) and H2 (improvements via changes in transformational leadership), only parts of the paths are triggered. Increasing transformational leadership and collective efficacy all lead to improved psychosocial risk management, but in the overall model, participation in the intervention did not increase collective efficacy or transformational leadership.

In Figure 3, we present the same model in a multi-group analysis with estimates for each company. For all three companies, changes in transformational leadership were positively related to ACM-PWE, but even though there are positive tendencies in C1, no significant intervention effects on transformational leadership were found (supporting only part of H1). Changes in collective efficacy were not linked to ACM-PWE in any company.

Figure 3.

Paths models for each company.

In C2 there was a significant direct relation between intervention group status and improved ACM-PWE. The C3 model also showed a significant relation between intervention status and increased collective efficacy, partially supporting H1. Additionally, the correlation between transformational leadership (T2) and collective efficacy (T2) was statistically significant for C3. In Figure 2, both total and direct effects on ACM-PWE were apparent. In the company-based models presented in Figure 3 there were clear differences between companies. Company 1 showed no apparent direct or indirect effect. Company 2 showed a positive direct effect but not a total effect owing to a negative indirect effect. Company 3 showed both a total and direct effect but no indirect effect. Thus, the direction of the indirect effect differed between companies, even though none of the individual mediational paths was statistically significant. The model in Figure 3 also had a statistically significant better fit than the model in Figure 2 (chi-squared = 52.6, d.f. = 20, p = .0001), which suggests that groups differ (Hayes, 2009). This led us to confirm that the modelled paths differ between the three companies, clearly supporting H3.

Context and mechanisms in the three companies (RQ1)

To explain the triggering of intervention mechanisms in C1, we examined in the qualitative data the contextual factors in C1 that influenced both mediating mechanisms: collective efficacy and transformational leadership.

Mediational mechanism via transformational leadership in C1

With regards to transformational leadership, the context seemed supportive. A contextual factor of the workshops was managers’ skill level. According to an observer, ‘[they] are experienced and employees appreciate that they use their personal experiences as a manager in the workshop’ (C1 mapping workshop 1), which would suggest a support for the mechanism of transformational leadership. Furthermore, the fact that line managers seemed to have a good understanding of the issues reported by employees, and avoided directing the meeting but supported the action plans of the team, was positive to the mechanism of transformational leadership. As one observer noted, ‘the team respects the line manager as she expresses confidence towards the team in regard of their professional conduct’ (C1 mapping workshop 11).

In the interviews, one manager explicitly talks about utilizing and developing her managerial skills during the intervention workshops. Leaders’ reflection on their own practice and developing understanding that they need to be supportive rather than directive is another contextual factor relevant for the mechanism of transformational leadership. She presents an appreciative attitude to the employees during the intervention by underscoring that the company’s value is not based on auxiliary functions, but on machine operators:

. . . because it is not in the paperwork that they sit and produce out here [that value is made]. There is of course a researcher, or someone, who has found out that we make [the product] in this way, but when the process has been decided and is running, then it is the employees who are spinning gold out there every day. (Interview with line manager 11)

Overall, the contextual factors related to interactions between teams and their line managers indicate that the potential for the mechanism of transformational leadership was present in C1, even though it did not turn out statistically significant in the statistical model.

Mediational mechanism via collective efficacy in C1

Regarding collective efficacy, a contextual factor in C1 was change fatigue. One observer noted that it seemed as if employees ‘simply do not believe that the project can change anything’ (C1 action planning workshop 6). Perceived low decision latitude was another contextual factor that may have hindered the mechanism of collective efficacy from being triggered. An observer noted that employees felt that only top management could make decisions about anything important. A third contextual factor hindering the mechanism of collective efficacy was that some employees felt that similar concepts had been tried previously with little success and that the POI was not applicable to their company since complex production and documentation demands made action plans difficult to develop and implement. One observer noted that, ‘in comparison to how much in agreement the team has been on what the problems are, there is a clear apathy. There is consensus in the team that it is not possible to solve anything’ (C1 follow-up workshop 10). Together, these contextual factors suggest that the mechanism of collective efficacy was unlikely to be triggered in the context of C1.

Mediational mechanism via transformational leadership in C2

A contextual factor of the C2 workshops likely hindering the transformational leadership mechanism was the line manager’s very directive approach during the POI, which ensured that action plans would be developed but caused conflict and a negative atmosphere. One observer noted that the line manager ‘threatens one employee that [the employee] himself has to do the action plan when [that employee] explains about a problematic aspect of work’ (C2 mapping workshop 1). The line manager continuously demands that employees be more ‘to the point’ and precise, ‘even though the line manager at the beginning noted that the psychosocial working environment was very important. He repeatedly prioritizes technical solutions’ (C2 mapping workshop 3).

Mediational mechanism via collective efficacy in C2

The negative atmosphere in the C2 workshops likely not only have hindered the mechanism of transformational leadership from being triggered but the negative mood and directive approach might have had a negative impact on the collective efficacy of the participants as well.

The line manager had been part of the company for more than two decades, and so some teams seemed to be used to his behaviour and tried to focus on the task at hand. One observer noted that ‘the action plan development is moving forward, [and] there are no long pauses or defeatist mood, but instead a willingness to get to business’ (C2 action planning workshop 1). This suggests that the context might have supported action plan development and implementation, but the mechanisms of transformational leadership and collective efficacy were likely not triggered.

The effect of line manager behaviour on the mechanism of transformational leadership is seen in the interviews with the participating line manager where he explains his view on his role in the POI workshops:

Line manager:

If there wasn’t a line manager who was present as a midwife [for action plans] I think it would be difficult for the employees.

Interviewer:

Then it was good that you participated.

Line manager:

It is a ‘must’, it is crucial. Because we can’t demand . . . People are humble, employees won’t go in and ask for a meeting with the head of production who they might barely know. (Interview with line manager 21)

When asked about collaboration with the line manager during the project, a foreman described the workshop context:

Foreman:

For the most part I think it was quite constructive. And people have the right to express their options, that’s my point of view.

Interviewer:

So it was positive that there was an open . . .

Foreman:

Yes, but I will say that the line manager should have kept his distance. It puts a lid on the discussions of the working conditions. It is not a realistic picture when he is present. (Interview with foreman 22)

The foreman also explained that the line manager was useful for implementing action plans but underscored some issues: ‘there were some workshops where things got loud, and it was almost an argument. It is not productive for the employees’ (Interview with foreman 22). Again, this interpretation of the behaviour and attitudes of the line manager indicates that the mechanisms of transformational leadership and collective efficacy were not triggered. However, with regard to the context of the intervention process as a whole, another foreman noted that the processes of developing and implementing action plans were very positive and transparent, even though several very ambitious plans were infeasible:

Overall it has been positive. There haven’t been substantial obstacles. And employees have been able to see that the plans they developed have been implemented. Or at least they have gotten an answer to why they haven’t been implemented. But they have not been flat out rejected. (Interview with foreman 23)

Analysis of the context in C2 suggests that the primary outcomes of the workshops were action plans and changes in working conditions (which may have had a direct effect on the employees’ perceived ability to manage the psychosocial work environment). However, it seems unlikely that the context would trigger the mechanisms of collective efficacy or transformational leadership.

Mediational mechanism via collective efficacy in C3

In C3, constructive dialogue between the employees and experiences of empowerment may have triggered the mechanism of collective efficacy. Although employees had problematic relationships with the line manager, one observer noted, ‘finally there seems to develop dialogue and engagement in the workshop. Employees start to share knowledge and they start joking with each other’ (C3 action planning workshop 2). In addition, ‘[every time a problem is discussed] an employee says, “Yes, we need to do an action plan for that!” and they feel clearly empowered’ (C3 action planning workshop 3). These contextual factors suggest that the mechanism of collective efficacy was triggered in C3. It was also noted that ‘employees come up with good ideas, focus on dialog and collaboration in the task of improving the relation between day and night shifts’ (C3 action planning workshop 6). In relation to the action plan to develop collaboration between shifts, an employee explained that they experienced

. . . a bit more team cohesion, a bit more cooperation with people talking more to each other. And more information also. [Teams] help each other day and night. [. . . And] help from the day shift has improved, so I’m not put under as much time pressure as I used to be. (Interview with night-shift employee 31)

Mediational mechanism via transformational leadership C3

In C3, a contextual factor that may have hindered the mechanism of transformational leadership was the tense relationship between employees and the line manager. Some participants in C3 mapping workshop 2 explicitly mentioned that they would have liked the mood to have been more positive. In some workshops, differences in opinion were recognized in a positive manner, but in others, collaboration problems made assessment and dialogue difficult. An observer noted that

. . . the discussion stops as the consultant asks about the collaboration with the line manager. The problem with machines not running re-surfaces, and the line manager immediately gets defensive. In her own view she does not ask in an angry fashion why the machines are shut down. (C3 mapping workshop 2)

The contextual factor of the line manager’s listening skills was also noted to be problematic in the action planning phase: ‘[the] line manager is bad at listening to what the employees have to say’ (C3 action planning workshop 2). This suggests that the mechanism of transformational leadership was not triggered.

Analysis of the context of C3 supports the interpretation that the mechanism of collective efficacy (H1) was triggered, but not that of transformational leadership (H2).

Discussion

Using a realist evaluation framework and a mixed-methods approach, this study aimed to explore whether the proposed mechanisms of transformational leadership and collective efficacy were triggered by a participatory organizational intervention in three different organizations. The main finding is that the proposed mechanisms were activated to some extent and led to the increases in awareness of and capability to manage the psychosocial work environment, but not in every organizational context. The direct effect was seen in the overall model in Figure 2 and in the C2/C3 models in Figure 3, proving that it is reasonably robust. The mechanism of transformational leadership was not statistically significant in any company. The modelled path from transformational leadership to ACM-PWE proved to be context-invariantly significant (i.e. it was significant for each company in Figures 2 and 3). The mechanism of improved collective efficacy was not supported in any model, in spite of the contextual factors of C3, such as experiences of empowerment and positive dialogue. Interpreting the results suggests that the direction of mediational effects is different between the companies, and that these effects differ across mechanisms. In C2, the indirect paths, specifically a decline in transformational leadership, though not significant, still detract from the significant direct effect, causing the total effect to be smaller and non-significant. In C3, the total effect is larger than the direct effect, suggesting that the two indirect paths boost the effect of the intervention. An initial interpretation of the results is that the intervention led to, mainly, positive developments in all three workplaces and hence appears to have been successful, at least to some extent. This interpretation supports Semmer’s (2011) suggestion that interventions generally have positive, but not always simple, effects. The results suggest that the employees of each company benefit from the intervention, but how they benefit depends on the context and how the context interacts with the intervention’s mechanisms to produce outcomes.

Collective efficacy

The results of the study regarding collective efficacy failed to support H1. Collective efficacy was positively related to ACM-PWE only in the overall model, and only in C3 was the intervention related to collective efficacy. Each component of the mechanism is triggered in different models, but the whole mechanism is not triggered in one model. This is likely caused in part by the reduced statistical power of the individual company models relative to the overall model. It is not clear why the positive changes in collective efficacy for C3 did not lead to increases in ACM-PWE, but the relatively high prevalence of short-term employees in C3 may play a role in this outcome. In contrast to C1 and C2, some employees of C3 reportedly treated their jobs as merely temporary, suggesting that these employees may have opted to ‘wait out’ problems rather than make an effort to manage the work environment and produce outcomes that they may not benefit from themselves. In any case, the results from C3 suggest that it is possible to increase collective efficacy. Though the intervention may only have led to a minor improvement in work environment in C3, it may have improved the employees’ collective efficacy, which will hopefully allow them to better cope with future challenges they face.

Transformational leadership

The study provides interesting results regarding H2. Though transformational leadership in all models was related to ACM-PWE, no model showed statistically significant intervention effects on transformational leadership, though C1 showed positive but non-significant increases. Though POIs are often assumed to target the work environment directly, they often rely on there being satisfactory employee–manager relationships, especially when the managers are present in the POI activities. When the quantitative model is triangulated with qualitative data, especially from C2 and C3, it is unsurprising that the mechanism of transformational leadership did not mediate the effect as the managers were seemingly neither able to act in a transformational manner during the workshops nor to use the workshops as a venue to develop such a leadership style. In this sense, some degree of positive experience of leadership seems to be a prerequisite for the activation of the mediational mechanism via transformational leadership. That poor preconditions inhibit indirect effects regarding transformational leadership is perhaps owing to those leaders lacking capabilities and being unable to learn and benefit from the intervention, as well as a potentially pre-existing conflicted relationship with their employees inhibiting the mediation via transformational leadership. Previous research has found a link between leadership, especially transformational leadership, and the implementation and outcome of interventions, which this study at least partially supports (Lundmark et al., 2017).

Implications for research

Several recent studies have focused on complex mechanisms underpinning interventions and have improved our understanding of the working mechanisms of such endeavours (notable examples include Busch et al., 2017; Holman and Axtell, 2016; Lundmark et al., 2017; von Thiele Schwarz et al., 2017). With regard to the present study, the novel aspect was the testing of several mechanisms in three different contexts. The qualitative results from the three companies shed light on the contextual factors that play a substantial role in shaping the possibilities for different mechanisms to be activated, as suggested by the realist POI evaluation framework presented by Nielsen and Abildgaard (2013). This aligns with and expands upon the theorized importance of conducting an intervention that fits its specific context (Randall et al., 2012). Regarding the dynamic role of the context, factors in the organizational context often seemed to influence the workshops, which made assessment of the intervention mechanisms quite complex. It is necessary to further develop and refine methods to assess the interplay between context mechanism and outcomes, as suggested by Nielsen and Miraglia (2017).

A clear implication from the study is that it is inadequate to address the question of only ‘what works for whom in which circumstances’ (Pawson, 2013); instead, one needs to ask the even more contextually specific, ‘how does the intervention work under different circumstances?’ We saw that line managers’ managerial style could hinder or enable the mechanism of transformational leadership. In addition, employees’ experiences with regard to the atmosphere of the workshops and the potential to experience empowerment could enable or hinder the mechanism of collective efficacy. The analysis is a demonstration of the crucial interplay between context and mechanisms in creating POI outcomes. It suggests that we need to move away from working with interventions as if they were a bounded mechanism in their own right to seeing them as collections of potential mechanisms and processes that interact with contextual factors. Hence, evaluations of complex participatory interventions should analyse multiple paths and how these are activated differently across multiple contexts. In this regard, we should advance analysis that encompasses the complexity and contextually situated nature of interventions.

It is important to underscore that the mechanisms we examined in the present article are not necessarily the most powerful in the present intervention. Only through future research can we achieve a comprehensive description of the potentially relevant mechanisms and how to examine their relevance for the intervention in the given case.

Implications for practice

From an organizational development perspective, the findings that the mechanisms of transformational leadership and collective efficacy were each only activated in one of the three companies suggests that practitioners should be aware of how partial activation of the mediating mechanisms might be caused by local circumstances, potentially preventing interventions from having their full effects. For example, if the participating managers are not able to espouse the values of the intervention in their own actions, the positive effects of participatory interventions might be reduced. By taking compensatory steps, such as including mainly work groups whose managers who are thought to be able to exhibit transformational behaviours and encouraging the managers to do so, practitioners might increase the likelihood of successful implementation. In short, consultants and other agents conducting POIs should consider which mechanisms they can utilize in specific contexts to bring about the intended outcome.

The study also demonstrated how POI might increase both line managers’ transformational leadership behaviours and the employees’ collective efficacy, although these effects were only seen in one context each. Thus, practitioners should consider whether it is possible to use POIs as a venue for developing managers’ leadership behaviour as well as the way employees cooperate around changing their working conditions. Likewise, it should be considered whether the probability of these effects occurring could be increased, for example by supporting POIs with leadership training.

Additionally, improvements in ACM-PWE through POIs could be beneficial in the participants’ subsequent efforts to improve the work environment outside the POI. For example, participants may learn valuable lessons about how to influence the work environment and may become motivated to try similar approaches. In other words, the participants become more reflexive of the options available to them as a form of ‘double loop learning’ (Argyris, 1991).

Strengths and limitations

This study features three main strengths. First, examining three separate companies with randomized functional departments into intervention and comparison conditions allowed for a strong comparative analysis. Second, the data suggest that the results are valid in the sense that the SEM models provided good fit and matched the qualitative data. Third, the parallel mixed-methods evaluation design allows for methodological triangulation strengthening the results. Specifically, in addition to the two rounds of questionnaire data, the substantial amount of qualitative material provides a methodologically strong basis for mixed-methods analysis, allowing the study to provide details on the specific contexts and how they might have triggered the mechanisms.

Though the study has considerable merit, a number of limitations need to be mentioned. First, it relies on self-reported measures, which is necessary when assessing perceived phenomena such as experiences with psychosocial work environment. Using the same method for all quantitative data collection leads to a risk of common method bias (Podsakoff and Organ, 1986), but the substantial time difference (1 year) between measurement points minimizes this risk (Podsakoff et al., 2012). Additionally, common method bias would, theoretically, have a similar effect on each group. As we assess differences between intervention/control in three different companies the problem of common method bias in the chosen design is limited. Second, the outcome measure of awareness and capability to manage the work environment is a developmental measure focusing on perceived changes (following the example of von Thiele Schwarz et al., 2017), and thus it was not possible to control for baseline levels. The other measures (collective efficacy and transformational leadership) were controlled for baseline scores, which increased the validity of the results (Field, 2013). Third, the relatively small sample size and moderate response rate suggest that testing the same mechanisms in a much larger sample might yield different results (the modest sample size, for instance, meant that bootstrap testing for mediation was infeasible). However, although this method limited the study’s generalizability, the questionnaires were supplemented by interviews and observation data, which provided a comprehensive picture of the intervention. Fourth, all three companies are within the industrial production sector and employ similar types of employee. Although this allows for comparison of the cases, it also limits the generalizability of the results to other sectors and job groups. Fifth, we have taken the clustered nature of the data into account by using multiple groups’ structural equation models. Other methods could have been used, e.g. mixed models as described by Le Blanc et al. (2007), or GEE methods as described by Muthén and Satorra (1995), and implemented in the ‘complex’ command in Mplus. The mixed-model approach was deemed unsuitable because we only have three workplaces, and the GEE approach yielded structural parameters that were similar to those obtained from multiple groups analysis with confidence intervals that contained the values from multiple groups analyses for all coefficients except one. Here, the difference was modest (a 9% change).

Sixth, the chosen mixed-methods approach applies an explanatory approach to the qualitative data, which limits the scope of qualitative analysis to focus on the organizational context and hypothesized mechanisms. More explorative analyses also need to be conducted in the future to more comprehensively illuminate the complexities in the intervention processes. Likewise, one could wish for more alignment between the quantitative and qualitative analyses; for example, that each path in the SEM model was assessed qualitatively. With the nature of qualitative data being diverse and collected at workshops that took part in daily practice, this was not possible; one could consider the use of diary study methods in future studies if alignment between data sources in mixed methods is to be achieved.

Conclusion

This study shed light on the complex nature of POIs and demonstrated how such interventions can be studied using a realist evaluation framework. We focused on how two different intervention mechanisms interacted with different contexts to produce outcomes. That the three companies had markedly different contexts and outcomes leads us to conclude that the outcome of the intervention relied heavily on context, and that this variation can be attributed to the intervention working through several different mechanisms that are activated differently. We urge researchers to more closely assess which mechanisms are likely to be activated from a particular intervention, and examine how the context enables or hinders triggering of these mechanisms.

Supplemental Material

Supplementary_material – Supplemental material for ‘Same, but different’: A mixed-methods realist evaluation of a cluster-randomized controlled participatory organizational intervention

Supplemental material, Supplementary_material for ‘Same, but different’: A mixed-methods realist evaluation of a cluster-randomized controlled participatory organizational intervention by Johan Simonsen Abildgaard, Karina Nielsen, Christian Dyrlund Wåhlin-Jacobsen, Thomas Maltesen, Karl Bang Christensen and Andreas Holtermann in Human Relations

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: The research has been supported by grants from the Danish Government Rate Adjustment Pool (Satspulje, 2012; Nye Veje) and The Danish Work Environment Research Fund (grant numbers 44-2014-03 and 29-2016-09).

ORCID iDs

Johan Simonsen Abildgaard

Karina Nielsen

Supplemental material

Supplemental material for this article is available online.

Johan Simonsen Abildgaard is a senior researcher at the National Research Centre for the Working Environment, Copenhagen, Denmark. His research focuses on implementing and evaluating organizational interventions using mixed-methods approaches, change management, organizational change, and applying organization theory to work environment intervention research. He is currently on the editorial board of Scandinavian Journal of Work and Organizational Psychology. He has published his research in journals such as Work & Stress, Human Relations and Economic and Industrial Democracy [Email: jss@nfa.dk]

Karina Nielsen is the Chair of Work and Organisational Psychology at the Institute of Work Psychology, Sheffield University Management School at the University of Sheffield, UK. She is a research affiliate at Karolinska Institutet, Sweden. Her research interests lie within the changing organizations. She is particularly interested in the evaluation of organizational interventions and ways to develop methods to understand how and why such interventions succeed or fail. She is currently on the editorial boards of Human Relations, The Leadership Quarterly and Journal of Business and Psychology, and is an associate editor of Work & Stress and Applied Psychology: An International Review. She has published her work in journals such as Human Relations, Work & Stress, The Leadership Quarterly and Journal of Occupational Health Psychology. [Email: k.m.nielsen@sheffield.ac.uk]

Christian Dyrlund Wåhlin-Jacobsen is a researcher at The National Research Centre for the Working Environment, Copenhagen, Denmark. His main research interests include employee voice and participation, organizational health and safety interventions, and social interaction in the workplace. His work is published in journals such as Scandinavian Journal of Management, Management Communication Quarterly and Scandinavian Journal of Work, Environment and Health. [Email: cdj@nfa.dk]

Thomas Maltesen is an industrial PhD student at the Section of Biostatistics in the Department of Public Health at the University of Copenhagen, Denmark writing a PhD at Lundbeck A/S. [Email: thma@sund.ku.dk]

Karl Bang Christensen is Associate Professor at the Section of Biostatistics in the Department of Public Health at the University of Copenhagen, Denmark. His research interests are application of latent variable models in psychology and health. He has published his work in journals such as Psychometrika, Applied Psychological Measurement, Work & Stress and Scandinavian Journal of Work Environment and Health. [Email: kach@sund.ku.dk]

Andreas Holtermann is a Professor at the National Research Centre for the Working Environment, Copenhagen, Denmark. His main field of research is on occupational physical activity and health, and prevention of musculoskeletal pain and sickness absence at the workplace. [Email: aho@nrcwe.dk]

References

Abildgaard

Nielsen

Sverke

(2018) Can job insecurity be managed? Evaluating an organizational-level intervention addressing the negative effects of restructuring. Work & Stress 32(2): 105–123.

Albertsen

Garde

Nabe-Nielsen

, et al. (2014) Work-life balance among shift workers: Results from an intervention study about self-rostering. International Archives of Occupational and Environmental Health 87(3): 265–274.

Argyris

(1991) Teaching smart people how to learn. Harvard Business Review 69(3): 99–109.

Arneson

Ekberg

(2005) Evaluation of empowerment processes in a workplace health promotion intervention based on learning in Sweden. Health Promotion International 20(4): 351–359.

Aust

Rugulies

Finken

, et al. (2010) When workplace interventions lead to negative effects: Learning from failures. Scandinavian Journal of Public Health 38(3 suppl.): 106–119.

Avolio

Bass

(2004) Multifactor Leadership Questionnaire. Menlo Park, CA: Mind Garden.

Bandura

(1997) Self-Efficacy: The Exercise of Control. New York: W H Freeman.

Bass

Riggio

(2012) Transformational Leadership. New York, NY and Hove, UK: Psychology Press.

Bentler

(1990) Comparative fit indexes in structural models. Psychological Bulletin 107(2): 238–246.

10.

Björklund

Grahn

Jensen

, et al. (2007) Does survey feedback enhance the psychosocial work environment and decrease sick leave? European Journal of Work and Organizational Psychology 16(1): 76–93.

11.

Bourbonnais

Brisson

Vinet

, et al. (2006) Development and implementation of a participative intervention to improve the psychosocial work environment and mental health in an acute care hospital. Occupational and Environmental Medicine 63(5): 326–334.

12.

Bresó

Schaufeli

Salanova

(2011) Can a self-efficacy-based intervention decrease burnout, increase engagement, and enhance performance? A quasi-experimental study. Higher Education 61(4): 339–355.

13.

Brinkmann

Kvale

(2015) Interviews: Learning the Craft of Qualitative Research Interviewing. Thousand Oaks, CA: SAGE.

14.

Busch

Koch

Clasen

, et al. (2017) Evaluation of an organizational health intervention for low-skilled workers and immigrants. Human Relations 70(8): 994–1016.

15.

Carless

Wearing

Mann

(2000) A short measure of transformational leadership. Journal of Business and Psychology 14(3): 389–405.

16.

Chen

(2007) Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal 14(3): 464–504.

17.

Coyle-Shapiro

JA-M

(1999) Employee participation and assessment of an organizational change intervention a three-wave study of total quality management. The Journal of Applied Behavioral Science 35(4): 439–456.

18.

Creswell

Plano Clark

(2011) Designing and Conducting Mixed Methods Research (2nd edn). Los Angeles, CA: SAGE.

19.

Dahl-Jørgensen

Saksvik

PØ

(2005) The impact of two organizational interventions on the health of service sector workers. International Journal of Health Services 35(3): 529–549.

20.

Daniels

Gedikli

Watson

, et al. (2017) Job design, employment practices and well-being: A systematic review of intervention studies. Ergonomics 60(9): 1177–1196.

21.

Demerouti

Bakker

Nachreiner

, et al. (2001) The job demands-resources model of burnout. Journal of Applied Psychology 86(3): 499–512.

22.

Edwards

Webster

Van Laar

, et al. (2008) Psychometric analysis of the UK Health and Safety Executive’s Management Standards work-related stress indicator tool. Work & Stress 22(2): 96–107.

23.

EU-OSHA (2000) Research on Work-related Stress. Luxembourg: Office for Official Publications of the European Communities.

24.

Fernet

Trépanier

S-G

Austin

, et al. (2015) Transformational leadership and optimal functioning at work: On the mediating role of employees’ perceived job characteristics and motivation. Work & Stress 29(1): 11–31.

25.

Field

(2013) Discovering Statistics Using IBM SPSS Statistics. London: SAGE.

26.

Greenhalgh

Wong

Jagosh

, et al. (2015) Protocol—the RAMESES II study: Developing guidance and reporting standards for realist evaluation. BMJ Open 5(8): e008567.

27.

Guastello

(1993) Do we really know how well our occupational accident prevention programs work? Safety Science 16(3–4): 445–463.

28.

Gupta

Wåhlin-Jacobsen

Henriksen

, et al. (2015) A participatory physical and psychosocial intervention for balancing the demands and resources among industrial workers (PIPPI): Study protocol of a cluster-randomized controlled trial. BMC Public Health 15: 274.

29.

Haslam

Jetten

Postmes

, et al. (2008) Social identity, health and well-being: An emerging agenda for applied psychology. Applied Psychology: An International Review 58(1): 1–23.

30.

Havermans

Schlevis

Boot

, et al. (2016) Process variables in organizational stress management intervention evaluation research: A systematic review. Scandinavian Journal of Work, Environment & Health 42(5): 371–381.

31.

Hayes

(2009) Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs 76(4): 408–420.

32.

Holman

Axtell

(2016) Can job redesign interventions influence a broad range of employee outcomes by changing multiple job characteristics? A quasi-experimental study. Journal of Occupational Health Psychology 21(3): 284–295.

33.

Bentler

(1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6(1): 1–55.

34.

ILO (2001) Guidelines on Occupational Safety and Health Management Systems. Geneva: International Labor Office.

35.

Johns

(2006) The essential impact of context on organizational behavior. Academy of Management Review 31(2): 386–408.

36.

Judge

Piccolo

(2004) Transformational and transactional leadership: A meta-analytic test of their relative validity. Journal of Applied Psychology 89(5): 755–768.

37.

Kompier

Cooper

Geurts

(2000) A multiple case study approach to work stress prevention in Europe. European Journal of Work and Organizational Psychology 9(3): 371–400.

38.

Lacouture

Breton

Guichard

, et al. (2015) The concept of mechanism from a realist approach: A scoping review to facilitate its operationalization in public health program evaluation. Implementation Science 10(1): 153.

39.

Lamontagne

Keegel

Louie

, et al. (2007) A systematic review of the job-stress intervention evaluation literature, 1990–2005. International Journal of Occupational and Environmental Health 13(3): 268–280.

40.

Le Blanc

Hox

Schaufeli

, et al. (2007) Take care! The evaluation of a team-based burnout intervention program for oncology care providers. The Journal of Applied Psychology 92(1): 213–227.

41.

Llorens

Schaufeli

Bakker

, et al. (2007) Does a positive gain spiral of resources, efficacy beliefs and engagement exist? Computers in Human Behavior 23(1): 825–841.

42.

Lundmark

Hasson

Schwarz

von

, et al. (2017) Leading for change: Line managers’ influence on the outcomes of an occupational health intervention. Work & Stress 31(3): 276–296.

43.

MacQueen

McLellan-Lemal

Bartholow

, et al. (2008) Team-based codebook development: Structure, process, and agreement. Handbook for Team-based Qualitative Research: 119–135.

44.

McQuiston

(2000) Empowerment evaluation of worker safety and health education programs. American Journal of Industrial Medicine 38(5): 584–597.

45.

Marchal

van Belle

van Olmen

, et al. (2012) Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation 18(2): 192–212.

46.

Mikkelsen

Gundersen

(2003) The effect of a participatory organizational intervention on work environment, job stress, and subjective health complaints. International Journal of Stress Management 10(2): 91–110,

47.

Mikkelsen

Saksvik

PØ

Landsbergis

(2000) The impact of a participatory organizational intervention on job stress in community health care institutions. Work & Stress 14(2): 156–170.

48.

Mowbray

Wilkinson

Tse

(2015) An integrative review of employee voice: Identifying a common conceptualization and research agenda. International Journal of Management Reviews 17(3): 382–400.

49.

Muthén

Satorra

(1995) Complex sample data in structural equation modeling. Sociological Methodology: 267–316.

50.

Nastasi

Hitchcock

Sarkar

, et al. (2007) Mixed methods in intervention research: Theory to adaptation. Journal of Mixed Methods Research 1(2): 164–182.

51.

Nielsen

(2013) Review article: How can we make organizational interventions work? Employees and line managers as actively crafting interventions. Human Relations 66(8): 1029–1050.

52.

Nielsen

(2017) Leaders can make or break an intervention – But are they the villains of the piece? In: Kelloway

Nielsen

Dimoff

(eds) Leading to Occupational Health and Safety: How Leadership Behaviours Impact Organizational Safety and Well-Being. Chichester: John Wiley & Sons Ltd, 197–209.

53.

Nielsen

Abildgaard

(2013) Organizational interventions: A research-based framework for the evaluation of both process and effects. Work & Stress 27(3): 278–297.

54.

Nielsen

Miraglia

(2017) What works for whom in which circumstances? On the need to move beyond the ‘what works?’ question in organizational intervention research. Human Relations 70(1): 40–62.

55.

Nielsen

Noblet

(2018) Introduction: Organizational interventions: Where we are, where we go from here? In: Nielsen

Noblet

(eds) Organizational Interventions for Health and Well-Being - A Handbook for Evidence-Based Practice. Abingdon, UK and New York, NY: Routledge, 1–22.

56.

Nielsen

Randall

(2012) The importance of employee participation and perceptions of changes in procedures in a teamworking intervention. Work & Stress 26(2): 91–111.

57.

Nielsen

Fredslund

Christensen

, et al. (2006) Success or failure? Interpreting and understanding the impact of interventions in four similar worksites. Work & Stress 20(3): 272–287.

58.

Nielsen

Randall

Yarker

, et al. (2008) The effects of transformational leadership on followers’ perceived work characteristics and psychological well-being: A longitudinal study. Work & Stress 22(1): 16–32.

59.

Nielsen

Yarker

Randall

, et al. (2009) The mediating effects of team and self-efficacy on the relationship between transformational leadership, and job satisfaction and psychological well-being in healthcare professionals: A cross-sectional questionnaire survey. International Journal of Nursing Studies 46(9): 1236–1244.

60.

Nielsen

Randall

Christensen

(2010) Does training managers enhance the effects of implementing team-working? A longitudinal, mixed methods field study. Human Relations 63(11): 1719–1741.

61.

Ohno

(1988) Toyota Production System: Beyond Large-Scale Production. Portland, OR: Productivity, Inc.

62.

Pawson

(2013) The Science of Evaluation: A Realist Manifesto. London: SAGE.

63.

Pawson

Tilley

(1997) Realistic Evaluation. Thousand Oaks, CA: SAGE.

64.

Podsakoff

Organ

(1986) Self-reports in organizational research: Problems and prospects. Journal of Management 12(4): 531–544.

65.

Podsakoff

MacKenzie

Podsakoff

(2012) Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology 63: 539–569.

66.

Poulsen

Ipsen

Gish

(2015) Applying the chronicle workshop as a method for evaluating participatory interventions. International Journal of Human Factors and Ergonomics 3: 271–290.

67.

QSR International (2015) NVivo Qualitative Data Analysis Software. Melbourne, Australia: QSR International Pty Ltd.

68.

Rallis

Rossman

(2003) Mixed methods in evaluation contexts: A pragmatic framework. In: Tashakkori

Teddlie

(eds) Handbook of Mixed Methods in Social and Behavioral Research. Thousand Oaks, CA: SAGE, 491–512.

69.

Randall

Griffiths

Cox

(2005) Evaluating organizational stress-management interventions using adapted study designs. European Journal of Work and Organizational Psychology 14(1): 23–41.

70.

Randall

Nielsen

Biron

, et al. (2012) Does the intervention fit? An explanatory model of intervention success or failure in complex organizational environments. In: Improving Organizational Interventions for Stress and Well-Being. London and New York: Routledge, 120–134.

71.

Salanova

Llorens

Cifre

, et al. (2003) Perceived collective efficacy, subjective well-being and task performance among electronic work groups: An experimental study. Small Group Research 34(1): 43–73.

72.

Salanova

Llorens

Schaufeli

(2011) ‘Yes, I can, I feel good, and I just do it!’ On gain cycles and spirals of efficacy beliefs, affect, and engagement. Applied Psychology 60(2): 255–285.

73.

Saldaña

(2015) The Coding Manual for Qualitative Researchers (3rd edn). London: SAGE.

74.

Schaubroeck

Lam

Xie

(2000) Collective efficacy versus self-efficacy in coping responses to stressors and control: A cross-cultural study. Journal of Applied Psychology 85(4): 512–525.

75.

Schurman

Israel

(1995) Redesigning work systems to reduce stress: A participatory action research approach to creating change. In: Murphy

Hurrell

Sauter

Keita

(eds) Job Stress Interventions. Washington, DC: American Psychological Association, 235–263.

76.

Schwarzer

Jerusalem

(2010) The general self-efficacy scale (GSE). Anxiety, Stress, and Coping 12: 329–345.

77.

Semmer

(2011) Job stress interventions and organization of work. In: Quick

Tetrick

(eds) Handbook of Occupational Health Psychology. Washington, DC: American Psychological Association, 299–318.

78.

Teddlie

Tashakkori

(2009) Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. Los Angeles: SAGE.

79.

Tucker

Lewis

(1973) A reliability coefficient for maximum likelihood factor analysis. Psychometrika 38(1): 1–10.

80.

von Thiele Schwarz

Nielsen

Stenfors-Hayes

, et al. (2017) Using kaizen to improve employee well-being: Results from two organizational intervention studies. Human Relations 70(8): 966–993.

81.

Wåhlin-Jacobsen

(2018) Valid and taken seriously? A new approach to evaluating kaizen-inspired (and other) intervention tools. In: Nielsen

Noblet

(eds) Organizational Interventions for Health and Well-being: A Handbook for Evidence-based Practice. Abingdon, UK: Routledge.

82.

Wåhlin-Jacobsen

Henriksen

Abildgaard

, et al. (2017) Styrk Arbejdsmiljøet Med Medarbejderne - En Metode Til at Skabe Forbedringer i Arbejdsmiljøet Ved at Inddrage de Ansatte. København: Det Nationale Forskningscenter for Arbejdsmiljø. Available at: http://www.arbejdsmiljoforskning.dk/~/media/Boeger-og-rapporter/SAM-manual/SAM-manual.pdf (accessed 13 August 2019).

83.

Weiner

Lewis

Linnan

(2009) Using organization theory to understand the determinants of effective implementation of worksite health promotion programs. Health Education Research 24(2): 292–305.

84.

Wilson

Haines

(2006) Participatory ergonomics. In: Karwowski

(ed.) International Encyclopedia of Ergonomics and Human Factors (2nd edn), 3 volume set. Boca Raton, FL: CRC Press, 2226–2232.

85.

Yeung

(2004) The paradox of control in participative decision-making: Facilitative discourse in banks. Text 24(1): 113–146.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB