Sage Journals: Discover world-class research

Abstract

Many approaches to work-related stress risk assessment suggest the integration between a phase where objective data are collected and analyzed, and a phase where results of data collection and analysis are discussed and compared with information coming from the workers. However, stress researchers have criticized the use of self-report job stress measures, because of their potential distortions, and have called for an approach based on the use of objective measures. The Italian law for work-related stress risk assessment, closer to the latter approach, prescribes a two-stage procedure: first, a set of objective measures and then, conditionally to the outcome of the first stage, a set of subjective measures. In this article, we analyze, on the basis of psychometric principles, the tool used for the objective stage in the most adopted method in Italy. Such a tool is a checklist for which we discuss a number of issues, suggesting it is not methodologically well founded. Given the fact that assessment outcomes have a sensible impact on workers’ safety measures, we conclude these weaknesses affect the practice of work-related stress risk assessment.

Keywords

Introduction

Many papers have been reporting, at least since 1999, that the costs of occupational illness in general, and in particular of work-related stress, are not negligible. We now briefly describe what we feel are the most relevant studies referring to Europe, from the oldest to the more recent.

The European Agency for Safety and Health at Work (EU-OSHA, 1999) surveyed the costs (both in absolute value and in percentage of the Gross National Product) of occupational illness in 15 member states of the European Union (EU-15). The total amounts to a value between 185 and 289 billion euros a year. The European Commission (2000) conservatively estimated the costs of work-related stress at 20 billion euros a year for the EU-15. A report on work-related stress of the European Foundation for the Improvement of Living and Working Conditions (2007) analyzed costs of work-related stress for the Netherlands and Germany. The EU Executive Agency for Health and Consumer (2013) estimated that 14% of employed individuals who suffer from stress will go on to develop depression, and the total costs of depression in all 27 member states of the EU is 617 billion euros.

Also, international studies have been highlighting that work-related stress is one of the most widespread occupational illnesses. Hoel, Sparks, and Cooper (2001) reported that stress accounts for up to 30% of all work-related illness annually, on the basis of a number of reliable studies based on large population samples from the United States, Europe, and Australia. EU-OSHA (2014) provided estimates for socioeconomic costs in a number of European and non-European countries. Its conclusions stated “there is evidence suggesting that appropriately planned and implemented workplace interventions focusing on preventing stress, improving psychosocial work environment and promoting mental health are cost effective” (p. 23).

The need of an approach able to evaluate, prevent, and mitigate stress and psychosocial risks at work is, therefore, well founded on objective data. Stavroula and Aditya (2010), in a report prepared for the World Health Organization (WHO) and based on about 500 research papers, have suggested that the most accurate assessment of work-related stress consists in the integration and correlation among objective measures of working conditions (observational measures) and information coming from workers (e.g., self-report questionnaires).

Background

The normative background in Italy for work health and safety is the Legislative Decree 81/2008 (Ministry of Labour, 2008), which has transposed into the Italian law the Framework Directive of EU-OSHA (1989). This Legislative Decree states (art. 28, para. 1) the obligation for all public and private employers to assess, inter alia, work-related stress of their workers according to the content of the European Framework Agreement on Work-Related Stress (2004). This agreement does not contain an exhaustive list of potential stress indicators but describes the necessity of analyzing both objective and subjective factors. The need for such an integrated multi-source approach rests on a number of studies, such as, for example, Frese and Zapf (1988), Hurrell, Nelson, and Simmons (1998), and Giorgi, Leon-Perez, Cupelli, Mucci, and Arcangeli (2014).

The above cited Legislative Decree states also (para. 1bis) that such an assessment has to comply with the guidelines in Permanent Consultative Committee of the Ministry of Labour for Workplace Health and Safety (2010). These guidelines have defined a methodological approach aiming at providing the minimum level of implementation for employers to comply with the legislative duties for work-related stress risk assessment. Such an approach prescribes an evaluation method organized in two sequential phases. The first one (defined preliminary assessment) is mandatory, while the second one (defined in-depth assessment) is required only if both the preliminary assessment has revealed risk elements requiring mitigation and the adopted mitigating actions have been ineffective. The preliminary assessment phase requires measuring only objective factors, as long as they belong to at least three categories: sentinel events, work content, and work context. Such a measurement may be carried out by means of checklists compiled by the health and safety representatives. The in-depth assessment phase allows for the use—on homogeneous groups of workers—of tools, such as self-report questionnaires, focus groups, semi-structured interviews, again investigating at least the same three above cited categories.

The staged assessment defined in the above cited guidelines implicitly prescribes a hierarchical subordination of subjective measures to objective ones. The consequence is that if the first stage reveals a low level of risk, the assessment process terminates without carrying out the second stage (Inter-Regional Technical Coordination Committee of Prevention in Workplaces, 2012, see question G.1). This situation is a clear departure from what the European Framework Agreement on Work-Related Stress (2004) has prescribed and from the literature (Frese & Zapf, 1988; Giorgi et al., 2014; Hurrell et al., 1998; Stavroula & Aditya, 2010). Regarding this Italian approach, Zoni and Lucchini (2012) have already recognized that “A limitation of this approach is represented by the predominant relevance given to the assessment of objective factors in the first steps of the evaluation” (pp. 47-48).

The previous discussion clarifies that a correct and reliable execution of the preliminary assessment phase is essential in the Italian context. A failure at detecting a risk situation in such a phase would make it impossible to reveal risky situations, as no further assessment phase would be performed. The survey conducted in 2013 on the stress level in Italian workers observed that stress affects often or always more than 30% of workers (Italian Institute for Political, Social and Economic Studies, 2014). This outcome is not far from the value found in a 2005 survey by EU-OSHA (2009, Foreword), declaring work-related stress affects 22% of workers of the EU-27.

Note that the guidelines of the Permanent Consultative Committee have described the need for surveying the results of their application 2 years after their entering into force. As of September 2015, we are not aware of any publication reporting these results. Instead, some authors have criticized these guidelines. For example, Curzi, Fabbri, and Nardella (2013) stated the guidelines have “both diagnosis capability . . . faulty due to their incoherence with respect to the European Framework Agreement on Work-Related Stress and a preventive potential . . . inadequate in terms of identification of corrective measures of organizational nature” (p. 1). Also, Galli, Mencarelli, and Calzolari (2013) discussed these guidelines and stated that the Italian Union of Labour (UIL) is

strongly critical, particularly with respect to the under-evaluation of workers’ role and to the optional role of the assessment of workers’ perception, evaluating the proposed methodology incoherent both with respect to the European Framework Agreement on Work-Related Stress and to the most elementary principles of relevant national and international literature. (pp. 2-3)

Method

The most used method in Italy for the work-related stress risk assessment (Guglielmi, Depolo, & Violante, 2013) is the one designed by the National Institute for Insurance Against Accidents at Work (INAIL), a public non-profit entity safeguarding workers against physical injuries and occupational diseases. The user manual for Italian companies to comply with the obligations deriving from the above cited Legislative Decree 81/2008 (INAIL, 2011) describes the INAIL method. An English translation is available on their website (INAIL, 2013) and describes the method as “based on the Management Standards model of the Health and Safety Executive [HSE]” (Health and Safety Executive, 2004, Preface and Introduction). The INAIL method has proposed a checklist for the preliminary assessment phase and an indicator tool consisting of a 35-item questionnaire for the in-depth assessment, in accordance with the Italian methodological approach described above. The questionnaire is a translation to Italian of the HSE Management Standards Indicator Tool (Health and Safety Executive, 2004), but the INAIL method provides no indication of how the checklist is related to the HSE Management Standards model.

The HSE approach to stress has developed the idea of standards for managing work-related stress in terms of organizational states to be achieved and has discussed how their achievement can be assessed by workers (Cousins et al., 2004; Mackay, Cousins, Kelly, Lee, & McCaig, 2004). Moreover, the HSE approach has emphasized the involvement and the perception of workers, highlighting the need for discussing the outcome of data collection with them by means of focus groups. Issues revealed from the data collection phase may not turn out to be the most important ones for workers and, conversely, new (and possibly more important) issues can emerge in focus groups that have not been revealed by the data collection phase. Instead, in the INAIL method, the data collected during the preliminary assessment phase by means of the checklist are not subject to a focus group discussion but merely to a consultation with health and safety representatives.

The INAIL checklist is structured into three areas:

Sentinel events, collecting the trends of 10 indicators in the sub-areas of injury percentage, sick leaves, staff absence percentage, untaken leaves percentage, internal turnover percentage, external turnover percentage, disciplinary sanctions, unplanned health examinations, formal complaints, and judicial claims filed for downgrade/dismissal/harassment;

Work content, analyzing 36 items in the four sub-areas of work environment, task planning, workload, and working hours; and

Work context, analyzing 30 items in the six sub-areas of organizational culture, role in the organization, career development, autonomy and control, inter-personal relationships, and home-work interface.

The goal of this study is to investigate whether the various methodological steps of the INAIL checklist have been designed according to the state of the art principles for the definition of psychometric tools. Hence, in the remainder of this section, we describe how these methodological steps have been defined in the INAIL method’s user manual (INAIL, 2011; INAIL, 2013). Then, in the section “Discussion,” we critically review each of the steps in the light of psychometric principles.

Therefore, we now first analyze how data are collected and a risk level is computed for each area of the checklist, and then how the results of these data collection and risk evaluation are combined to produce the final outcome of the preliminary assessment phase. A methodological choice, common to all areas and to the construction of the final outcome, is that data collected result in a score, and the risk level is computed depending on the ratio of the obtained score to the maximum attainable score. This partition rule prescribes that if the obtained ratio is less than or equal to 25%, there is a low (non-relevant in the Italian version, this term is used in the following text) risk level; if it is higher than 25% and less than or equal to 50%, there is a medium risk level; and otherwise there is a high risk level (see headers of colored columns in Figures 1, 2, and 3, extracted from INAIL, 2013).

Figure 1.

Scores and risk levels for Area 1 (sentinel events).

Figure 2.

Scores and risk levels for Area 2 (work content).

Figure 3.

Scores and risk levels for Area 3 (work context).

Sentinel Events

For each sentinel event but the last two, it is observed whether the value of indicator decreased, remained stable, or increased, and a corresponding score of 0, 1, 4 is noted down. The absence or presence of events of the last two categories directly produce a score of 0 or 4, respectively. Decrease, stability, or increase have to be computed with respect to the average of the previous 3 years. For the indicators expressed in percentage, the method prescribes to compare the last year value, while for indicators expressed in absolute value, no specification is given.

The score of the area is eventually obtained by means of the following conversion process (see Figure 1):

0 ≤ sum of scores ≤ 10 results in a non-relevant risk level and an area score of 0;

11 ≤ sum of scores ≤ 20 results in a medium risk level and an area score of 2;

21 ≤ sum of scores ≤ 40 results in a high risk level and an area score of 5.

Work Content

The method investigates each of the 36 items through a yes/no question producing a score of 0 or 1.

The score of the area is exactly equal to the sum of scores and results in the following risk levels (see Figure 2):

0 ≤ area score ≤ 13: non-relevant risk level;

14 ≤ area score ≤ 25: medium risk level;

26 ≤ area score ≤ 36: high risk level.

Note that for this area, the boundaries between risk levels, set at 13 and 25 instead of at 9 and 18, do not respect the partition rule for mapping scores to risk levels.

Risk levels may be computed also for the four sub-areas. Also for these sub-areas, boundaries do not respect the partition rule.

Work Context

The method investigates each of the 30 items through a yes/no question producing a score of 0 or 1. The area score is computed by first summing only the first 26 items. Next, the sum of the four last ones is computed. If the latter sum is greater than 0, it is discarded, and the former one is the area score. Otherwise, 1 is subtracted from the former sum, and the result is the area score. No motivation is given for treating these last four items (making the whole of the sub-area “home-work interface”) in a different way.

The area score results in risk levels according to the following (see Figure 3):

0 ≤ area score ≤ 8: non-relevant risk level;

9 ≤ area score ≤ 17: medium risk level;

18 ≤ area score ≤ 26: high risk level.

Note that for this area, boundaries between risk levels, set at 8 and 17 instead of at 6 and 13, do not respect the partition rule.

Risk levels may be computed also for the first five of the six sub-areas. Also for these sub-areas, boundaries do not respect the partition rule.

Construction of the Overall Outcome

The method constructs the overall outcome by summing the scores of the three areas. The result is interpreted as follows (see Figure 4, extracted from table at p. 51 in INAIL, 2013):

0 ≤ sum of area scores ≤ 17: non-relevant risk level;

18 ≤ sum of area scores ≤ 34: medium risk level;

35 ≤ sum of area scores ≤ 67: high risk level.

Figure 4.

Scores and risk levels for the overall outcome.

Discussion

In this section, we discuss some of what we feel are critical methodological issues of the INAIL checklist with respect to its psychometric properties and to the quality of data organization and accessibility (Aiken, 1996; Kaplan & Saccuzzo, 2012; Nunnally, 1978). Critical points common to the three investigated areas are:

No justification is provided for converting the sum of scores to an area score for the first area (sentinel event) and not for the other two (work content and work context).

No objective evidence of a correlation with the measured phenomena is provided for the choice of the thresholds adopted for dividing risk levels.

All items have equal significance. For example, the “Diffusion of the organizational chart” (INAIL, 2013, p. 45, question n. 37) has the same weight for the final risk assessment as the “Misconduct of top managers and colleagues are properly managed” (p. 48, n. 61).

One of the publicly available Risk Assessment Documents required to comply with Italian legislative requirements (City of Imola, 2013) explicitly stated “The empirical basis of the method used to compute and weigh scores assigned to items is unknown” (p. 24).

In the following sub-sections, we further discuss critical methodological issues, first those that are specific to each of the three investigated areas and then those related to how the overall outcome is constructed.

Sentinel Events

The most relevant weakness is that increase or decrease is measured using absolute values and not as a percentage. Even for those indicators expressed in percent, depending on the magnitude of the base level, a big difference may exist between an increase of 1 point and 5 points. Moreover, if there is an increase, a 100% increase should weigh more than a 1% increase. Next, we note the method gives little emphasis to the need for providing supporting documentation demonstrating the revealed trends. Finally, we observe that it would have been more appropriate to compare values of sentinel events with reference values for the industrial sector.

Work Content and Work Context

A critical element for both areas is that it is difficult to provide an objective answer to some questions, because they often investigate issues with a qualitative nature and their meaning depends on the subjective interpretation of the compilers. For example, expressions such as “Adequacy of equipment resources to accomplish the task” (INAIL, 2013, p. 42, question n.15), “Particularly monotonous works” (p. 42, n.16), or “Roles are clearly defined” (p. 46, n. 49) are not objectively interpretable (Barattucci & Sarchielli, 2013).

Next, the yes/no questions are not able to properly detect elements of risk because the simple positive or negative answer does not give indications on the quality of investigated aspects. For example, a yes/no answer to a question such as “Meetings between management and employees” (INAIL, 2013, p. 45, question n.43) does not allow evaluating conditions, frequency, and quality of the meetings, which are instead highly relevant aspects to be evaluated for the sub-area of organizational culture.

Construction of the Overall Outcome

In this sub-section, we discuss the construction of the overall outcome of the preliminary assessment, focusing on three steps where we have found methodological weaknesses that undermine the significance of the outcome itself.

Area scores are summed

The most critical element of the INAIL method is that the summing of area scores tends to hide risk levels for some of the areas. Consider, for example, a situation where the area scores for the three areas are, respectively, 5 (high), 3 (non-relevant), and 9 (medium). Then the overall risk score is 17, with an overall risk level of non-relevant. Note that the methodological guidelines in case of an overall risk level of non-relevant conclude that, in such a case, the checklist “does not reveal specific conditions that can determine the presence of work-related stress” (INAIL, 2013, p. 51). Hence, on the basis of the staged assessment prescribed by the Italian methodological approach (section “Background”), there is no need of an in-depth assessment.

For such a delicate process, it would be more appropriate to observe the concurrence and concordance of results in the various areas than to simply compute their sum, which, as is well known, has a smoothing effect on the overall result. The construction of a synthesis indicator should be done using the appropriate logical-mathematical combinations with the ability to correct possible distortions of single area indicators (Lazarsfeld, 1966). The INAIL method leads to a contrary result: the synthesis indicator hides the outcome of single area indicators.

Prescriptions for medium/high overall risk levels are ambiguous

When discussing the overall assessment outcomes of medium and high risk levels, the INAIL method advises that corrective actions need to be taken for those sub-areas of work content and work context “with the highest risk level” and, if ineffective, an in-depth assessment must be performed (INAIL, 2013, pp. 24-25).

These prescriptions present some ambiguity. First of all, they do not motivate as to why no intervention is prescribed in the area of sentinel events: It is true that these describe objective facts that cannot be altered, but the prescriptions could have suggested to investigate at least possible correlation/dependencies among them and the work content/context sub-areas with the highest risk levels. Next, these prescriptions do not justify why corrective actions need to be taken only for the sub-areas with the highest risk level. Nor is this indication formulated in an operational way: Does it refer to the two highest or three highest or how many? Finally, these prescriptions suggest the same corrective approach in the two cases of overall risk level of medium and high: Which is, then, the difference between the two situations?

As these prescriptions appear under the paragraphs describing the overall risk level of medium and high (INAIL, 2013, pp. 24-25), it is clear that nobody would apply them in the case of an overall risk level of non-relevant. The need for addressing the critical situations in those sub-areas will, thus, be neglected.

Adaptation of the checklist is not discussed

A third important critical element is that is not clear how the same checklist may be applied to every kind of company, because not all items in the checklist are applicable to all companies. Contrast this procedure with the need for tailoring the tool to the type of company, which is explicitly required, for example, by Satzer and Gerey (2009) and by Alis, Dumas, and Poilpot-Rocaboy (2010). The INAIL method (INAIL, 2011, 2013) has provided neither any guidance for the adaptation of the checklist to the specific sector of an organization nor any discussion on the reliability of such a modification. For example, do threshold values separating risk levels in various areas keep their validity independently of the organization’s sector? Moreover, methodological indications should be provided on how to manage the items that are not applicable to the examined company: How do these affect each area score and the overall result of the preliminary assessment? The INAIL method (INAIL, 2011, 2013) has not addressed this highly relevant problem. Finally, it has not provided any normative database, for purpose of comparison and interpretation (Aiken, 1996; Kaplan & Saccuzzo, 2012; Nunnally, 1978).

Validity and implications

INAIL declares (INAIL, 2013, Preface) that its methodological path “has been merged with the experiences of the” Inter-Regional Technical Coordination Committee of Prevention in Workplaces (2012), containing guidelines for the correct risk management within companies and for oversight activities of public health agencies. These prescribe that checklists used for the preliminary assessment need to be:

scientifically valid with respect to:

Evaluated stressors

Objective and verifiable elements examined to estimate stressors

Criteria to assign scores and compute risk level. (p. 18)

But in the INAIL method (INAIL, 2011, 2013), there is no evidence of a scientific validation able to prove the INAIL checklist is a useful, valid, and reliable method (Aiken, 1996; Kaplan and Saccuzzo, 2012; Nunnally, 1978). Ronchetti et al. (2014) reports of a convergent validity between the checklist and the INAIL indicator tool, but the published details of the study are not enough to understand the reliability of these reported findings.

Psychometric properties

The previous discussion shows that the INAIL checklist has severe methodological weaknesses, and there is no evidence in the literature that it satisfies the following main psychometric principles:

Internal coherence, as an aspect of reliability;

Stability, as an aspect of reliability;

Content validity;

Criterion validity.

Only in regard to the area of construct validity is there a very preliminary result (discussed above) regarding convergent validity.

The psychometric property of reliability has two characterizations: internal coherence (referring to the fact that the various parts of the tool provide highly correlated indications) and stability (referring to the fact that reusing the tool after some time over an unchanged sample provides the same results). A tool whose reliability has not been established makes it impossible to derive meaningful consequences from its measurements.

Content validity refers to which degree the various components of a tool are appropriate to measure the construct and cover the entire construct domain. Its absence means the tool is not measuring what the tool should measure or is measuring only part of it.

Criterion validity refers to various ways of evaluating the tool as a good operationalization of the psychological construct to be measured (Trochim & Donnelly, 2006). Its absence means the tool is not a proper operationalization of the construct.

Implications

The various methodological weaknesses of the INAIL checklist previously discussed in this section are highly relevant in the light of the fact that their combined effect might result in an under-valuation of the actual risk levels. Considering that the INAIL method is freely and widely available because of the institutional role of such an organization, it is clear the potential consequences of these weaknesses affect a large part of the Italian workforce. Moreover, since workers’ safety measures are defined on the basis of assessment outcomes, we conclude these weaknesses have a clear bearing on the practice of work-related stress risk assessment.

Conclusion

The most widely used European models for work-related stress evaluation emphasize an approach centered on workers’ perception. But, given the widely discussed potential distortions of self-report perceptions (see, for example, Ostry, Kelly, Demers, Mustard, & Hertzman, 2003), several attempts afforded the development of observational methods. We analyzed, as an example of these attempts, an objective tool developed in Italy that declares itself to be inspired by the HSE approach. Zoni and Lucchini (2012) have already observed that the INAIL checklist has departed from HSE’s spirit, because of the “predominant relevance given to the assessment of objective factors in the first steps of the evaluation” (pp. 47-48). Indeed, the use of a checklist as a closed system of measurement and not as a process of evaluation shifts the balance of the assessment method toward objective measures.

We have found that the INAIL checklist has a number of methodological weaknesses in terms of psychometric principles, analyzed and discussed in sections “Method” and “Discussion.” We therefore conclude it is not methodologically well founded. A preliminary version of our work is in Corradini, Marano, and Nardelli (2014). Given the fact that assessment outcomes have a sensible impact on workers’ safety measures, we conclude these weaknesses have a clear bearing on the practice of work-related stress risk assessment. Given the relevant literature and the complexity of involved phenomena, we think it should be mandatory to evaluate work-related stress risk by means of the integrated use of both well founded objective measures and adequate workers’ involvement, through focus groups, questionnaires, and similar tools, as suggested, among others, by Albini, Zoni, Parrinello, Benedetti, and Lucchini (2011); Corradini, Marano, and Nardelli (2015a, 2015b); Panari, Guglielmi, Ricci, Tabanelli, and Violante (2012).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research and/or authorship of this article.

Author Biographies

Isabella Corradini is a work and organizational psychologist with a specialization in forensic psychopathology and clinical criminology. She is scientific director of the Themis Research Centre and adjunct professor of social psychology in the University of L’Aquila, where she also cooperates in international projects in social sciences within the Tempus Program. She is co-founder of the “Link&Think” research lab on socio-technical analysis of information technologies. She is expert in the psychosocial approach to workplace issues regarding safety (stress, techno-stress, mobbing, harassment, stalking) and security (crime prevention, profiling in physical security, and cybercrime). She is expert in psychological issues in communication and a court expert for psychology and criminology. She is also a consultant and a trainer in these areas for leading Italian companies. She has written many papers and books in the expertise areas, and she is the editor of a book series in reputation with a major Italian publishing house (Franco Angeli).

Assunta Marano has a PhD in cognitive psychology, psychophysiology, and personality, a postgraduate specialization in assessment psychology and counseling, and an advanced postgraduate specialization in statistical methods for biomedical research and practice. She has been adjunct professor at Sapienza University of Rome and the University of Salerno, and is currently adjunct professor in developmental psychology at the University of L’Aquila. She has been a research fellow analyzing Educational Achievement/Educational Testing Service (IEA/ETS) International Databases at the University of Roma Tre. She is currently responsible for methodology in a project on tablet use in learning disabilities for the Local Health Agency Roma B, responsible for methodology in a project on the evaluation of risk and protective factors for the prediction of academic and professional achievement at the University of L’Aquila, and responsible for methodology at the Themis Research Centre. She has several national and international publications in the above areas as well as in the psychological testing and assessment area.

Enrico Nardelli is full professor of informatics in the Department of Mathematics at the University of Roma Tor Vergata. He has more than 140 international refereed papers in the areas of algorithms and data structures, databases and geographical information systems, tools and environments for interaction with information systems, cooperative and distributed information systems, bioinformatics, and digital art. He is vice-president and member of the executive board of Informatics Europe, the association of informatics departments and research laboratories in Europe and neighboring areas. He is member of the executive board of the European Quality Assurance Network for Informatics Education (EQANIE). He is past president and member of the executive board of the Italian Association of University Professors of Informatics (GRIN). He is co-founder of the “Link&Think” research lab on socio-technical analysis of information technologies.

References

Aiken

L. R.

(1996). Rating scales and checklists: Evaluating behavior, personality, and attitude. New York, NY: John Wiley.

Albini

Zoni

Parrinello

Benedetti

Lucchini

(2011). An integrated model for the assessment of stress-related risk factors in health care professionals. Industrial Health, 49, 15-23. Retrieved from http://www.jniosh.go.jp/en/indu_hel/doc/IH_49_1_15.pdf

Alis

Dumas

Poilpot-Rocaboy

(2010). Risques: et souffrance au travail: nouvelles contraintes, nouveaux remèdes [Risks and pain in the workplace: New constraints, new solutions]. Paris, France: Dunod.

Barattucci

Sarchielli

(2013). Stress lavoro-correlato: metodologia e strumenti di valutazione [Work-related stress: Methodology and tools for its assessment]. In Consiglio Nazionale dell’Ordine degli Psicologi [Board of the Italian Professional Association of Psychologists] (Eds.), Rischio stress lavoro-correlato: le competenze dello psicologo nella valutazione e gestione [Work-related stress risk: psychologist’s expertise in its assessment and management] (pp. 55-103). Napoli, Italy: Liguori.

City of Imola. (2013). Valutazione del rischio stress lavoro correlato: estratto della relazione finale [Work-related stress risk assessment: Synthesis of the final report]. Retrieved from http://trasparenza.nuovocircondarioimolese.it/imola/amministrazione-trasparente/performance/benessere-organizzativo/valutazione-del-rischio-stress-lavoro-correlato/valutazione-del-rischio-stress-lavoro-correlato/view

Corradini

Marano

Nardelli

(2014). Work-related stress risk assessment: A critical review based on psychometric principles of an objective tool [Working Paper]. Social Sciences Open Access Repository. Retrieved from http://nbn-resolving.de/urn:nbn:de:0168-ssoar-441923

Corradini

Marano

Nardelli

(2015a, July). Information technology as an emerging risk for work-related stress: An integrated risk assessment approach. Paper presented at the 14th European Congress of Psychology, Milan, Italy. Retrieved from http://www.ecp2015.it/wp-content/uploads/2015/07/ECP-2015-Abstract-Book_upd27jul1.pdf

Corradini

Marano

Nardelli

(2015b). PRISMA-RA: A set of tools for work related stress risk assessment. In Pracana

(Ed.), Proceedings of the International Psychological Applications Conference and Trends (pp. 118-122). Lisbon, Portugal: World Institute for Advanced Research and Science.

Cousins

Mackay

C. J.

Clarke

S. D.

Kelly

P. J.

McCaig

R. H.

(2004). “Management standards” work-related stress in the UK: Practical development. Work & Stress, 18, 113-136.

10.

Curzi

Fabbri

T. M.

Nardella

(2013). Lo stress lavoro-correlato: dalla valutazione alle misure organizzative di prevenzione [Work-related stress: From assessment to prevention organizational measures] (Saggi 6/2013). Quaderni Fondazione Marco Biagi. Modena, Italy: Fondazione Marco Biagi, Università di Modena e Reggio Emilia. Retrieved from http://www.fmb.unimore.it/on-line/home/ricerca/pubblicazioni/quaderni-fondazione-marco-biagi/documento8860.html

11.

Directorate General for Employment, Social Affairs and Inclusion of the European Commssion. (2004). Framework Agreement on Work-Related Stress. Retrieved from http://ec.europa.eu/social/BlobServlet?docId=1479&langId=en

12.

EU Executive Agency for Health and Consumer. (2013). Economic analysis of workplace mental health promotion and mental disorder prevention programmes and of their potential contribution to EU health, social and economic policy. Retrieved from http://ec.europa.eu/health/mental_health/docs/matrix_economic_analysis_mh_promotion_en.pdf

13.

European Agency for Safety and Health at Work. (1989). Directive 89/391/EEC - OSH “Framework Directive.” Retrieved from https://osha.europa.eu/en/legislation/directives/the-osh-framework-directive/1

14.

European Agency for Safety and Health at Work. (1999). Economic impact of occupational safety and health in the member states of the European Union. Retrieved from https://osha.europa.eu/en/publications/reports/302

15.

European Agency for Safety and Health at Work. (2009). Occupational safety health in figures: Stress at work—facts and figures (Foreword by the Director). Retrieved from https://osha.europa.eu/en/publications/reports/TE-81-08-478-EN-C_OSH_in_figures_stress_at_work/view

16.

European Agency for Safety and Health at Work. (2014). Calculating the costs of work-related stress and psychosocial risks—A literature review. Retrieved from https://osha.europa.eu/en/publications/literature_reviews/calculating-the-cost-of-work-related-stress-and-psychosocial-risks/view

17.

European Commission. (2000). Guidance on work-related stress: Spice of life or kiss of death. Luxembourg: European Communities. Retrieved from https://osha.europa.eu/data/links/guidance-on-work-related-stress

18.

European Foundation for the Improvement of Living and Working Conditions. (2007). Work-related stress. Retrieved from http://www.eurofound.europa.eu/ewco/reports/TN0502TR01/TN0502TR01.pdf

19.

Frese

Zapf

(1988). Methodological issues in the study of work stress: Objective vs. subjective measurement of work stress and the question of longitudinal studies. In Cooper

Payne

(Eds.), Causes, coping and consequences of stress at work (pp. 375-411). Chichester, UK: Wiley.

20.

Galli

Mencarelli

Calzolari

(2013). La valutazione dello stress-lavoro correlato: potenzialità e limiti di un obbligo legislativo [Work-related stress assessment: And limitations of legislative requirements] (Saggi 5/2013). Quaderni Fondazione Marco Biagi. Modena, Italy: Fondazione Marco Biagi, Università di Modena e Reggio Emilia. Retrieved from http://www.fmb.unimore.it/on-line/home/ricerca/pubblicazioni/quaderni-fondazione-marco-biagi/documento8859.html

21.

Giorgi

Leon-Perez

J. M.

Cupelli

Mucci

Arcangeli

(2014). Do I just look stressed or am I stressed? Work-related stress in a sample of Italian employees. Industrial Health, 52, 43-53.

22.

Guglielmi

Depolo

Violante

(2013). The evaluation of psychosocial risks in the workplace: The case of Italy. In Navarrete

C. M.

Vicente

E. G.

(Eds.), Evaluation development of psychosocial risks in Europe: The state of scientific research and institutional experiences. International Yearbook on psychosocial risks prevention and quality of life at work. (pp. 71-93). Secretary of Labour Health and Environment UGT-CEC. Retrieved from http://portal.ugt.org/saludlaboral/observatorio/publicaciones/new2013/Anuario2013ING.pdf

23.

Health and Safety Executive. (2004). HSE management standards indicator tool. Retrieved from http://www.hse.gov.uk/stress/standards/pdfs/indicatortool.pdf

24.

Hoel

Sparks

Cooper

C. L.

(2001). The cost of violence/stress at work and the benefits of a violence/stress-free working environment. International Labour Organization. Retrieved from http://www.ilo.org/safework/info/publications/WCMS_108532/lang–en/index.htm

25.

Hurrell

J. J.

Jr. Nelson

D. L.

Simmons

B. L.

(1998). Measuring job stressors and strains: Where we have been, where we are, and where we need to go. Journal of Occupational Health Psychology, 3, 368-389.

26.

Inter-Regional Technical Coordination Committee of Prevention in Workplaces. (2012). Indicazioni per la corretta gestione del rischio e per l’attività di vigilanza alla luce della lettera circolare del 18 Novembre 2010 del Ministero del Lavoro [Recommendations for correct risk management and for oversight activity in the light of the Circular Letter of 18th November 2010 of Italian Ministry of Work]. Retrieved from http://centrostresslavoro-lazio.it/slc_beta/wp-content/uploads/2015/02/CTIPLL-Indicazioni-per-la-corretta-valutazione-del-rischio-SLC.pdf

27.

Italian Institute for Political, Social and Economic Studies. (2014). Rapporto Italia, 2014 [The 2014 Italy Report]. Retrieved from http://www.eurispes.eu/content/eurispes-rapporto-italia-2014

28.

Kaplan

R. M.

Saccuzzo

D. P.

(2012). Psychological testing: Principles, applications, and issues (8th ed.). Boston, Mass.: Cengage Learning.

29.

Lazarsfeld

P. F.

(1966). Concept formation and measurement in the behavioral sciences: Some historical observation. In Di Renzo

G. J.

(Ed.), Concepts, theory, and explanation in the behavioral sciences (pp. 144-202). New York, NY: Random House.

30.

Mackay

C. J.

Cousins

Kelly

P. J.

Lee

McCaig

R. H.

(2004). A “Management Standards” and work-related stress in the UK: Policy background and science. Work & Stress, 18, 91-112.

31.

Ministry of Labour. (2008). Testo Unico sulla salute e sicurezza sul lavoro [Unified text on workplace health and safety]. Retrieved from http://www.lavoro.gov.it/priorita/pagine/testo-unico-sulla-salute-e-sicurezza-sul-lavoro.aspx/

32.

National Institute for Insurance Against Accidents at Work. (2011). Valutazione e gestione del rischio stress lavoro-correlato (in Italian). Retrieved from https://www.inail.it/cs/internet/docs/alg-valutazione-gestione-rischio-stress-lavoro-correlato-ita.pdf

33.

National Institute for Insurance Against Accidents at Work. (2013). Managing and assessing the risk for work-related stress (English ed.). Retrieved from https://www.inail.it/cs/internet/docs/alg-valutazione-gestione-rischio-stress-lavoro-correlato-ing.pdf

34.

Nunnally

J. C.

(1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill.

35.

Ostry

A. S.

Kelly

Demers

P. A.

Mustard

Hertzman

(2003). A comparison between the effort-reward imbalance and demand control models. BMC Public Health, 3, 10. Retrieved from http://www.biomedcentral.com/1471-2458/3/10

36.

Panari

Guglielmi

Ricci

Tabanelli

M. C.

Violante

F. S.

(2012). Assessing and improving health in the workplace: An integration of subjective and objective measures with the STress Assessment and Research Toolkit (St.A.R.T.) method. Journal of Occupational Medicine and Toxicology, 7, 18. Retrieved from http://www.occup-med.com/content/7/1/18

37.

Permanent Consultative Committee of the Ministry of Labour for Workplace Health and Safety. (2010). Circolare del 18 novembre 2010: indicazioni per la valutazione dello stress lavoro correlato [Circular Letter of 18th November 2010: Guidelines for work-related stress evaluation]. Retrieved from http://www.lavoro.gov.it/documenti-e-norme/normative/Documents/2010/LC-18novembre2010.pdf

38.

Ronchetti

Persechino

Ballottin

Di Tecco

Russo

Castaldi

Iavicoli

(2014). Evaluating the psychometric properties of two tools for measuring work-related stress risk. In Andreou

N. J. A.

Jain

Hollis

Hassard

Teoh

(Eds.), Proceedings of the 11th Conference of the European Academy of Occupational Health Psychology (pp. 226-227). London, England: University of London. Retrieved from http://www.eaohp.org/uploads/1/1/0/2/11022736/eaohp2014_book_of_proceedings.pdf

39.

Satzer

Gerey

(2009, May). Stress—Mind—Health: The START procedure for the risk assessment and risk management of work-related stress (Arbeitspapier: 174). Gesundheit und Qualität der Arbeit, Hans-Böckler-Stiftung. Retrieved from http://www.boeckler.de/pdf/p_arbp_174.pdf

40.

Stavroula

Aditya

(2010). Health impact of psychosocial hazards at work: An overview. World Health Organization. http://whqlibdoc.who.int/publications/2010/9789241500272_eng.pdf

41.

Trochim

W. M. K.

Donnelly

J. P.

(2006). The research methods knowledge base (3rd ed.). Cincinnati, OH: Atomic Dog.

42.

Zoni

Lucchini

R. G.

(2012). European approaches to work-related stress: A critical review on risk evaluation. Safety and Health at Work, 3, 43-49. doi:10.5491/SHAW.2012.3.1.43

Work-Related Stress Risk Assessment

Abstract

Keywords

Introduction

Background

Method

Sentinel Events

Work Content

Work Context

Construction of the Overall Outcome

Discussion

Sentinel Events

Work Content and Work Context

Construction of the Overall Outcome

Area scores are summed

Prescriptions for medium/high overall risk levels are ambiguous

Adaptation of the checklist is not discussed

Validity and implications

Psychometric properties

Implications

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

Author Biographies

References