BACKGROUND: In many enterprises the semi-quantitative approach
turns out to be the available and most suitable technique to perform a risk
assessment. Despite its advantages, we cannot disregard the existing gap in
terms of validation of this type of applications.
OBJECTIVE: This paper reports a study about risk assessments'
reliability, namely both inter-coder (reproducibility) and intra-coder
(stability) reliability of the semi-quantitative approach.
METHODS: This study comprised 4 fundamental stages. Data collection
relied on free and systematized observations and made use of video
recording, documental research, analysis grids and questionnaires
specifically developed for this purpose. A set of different analysts were
asked to use four semi-quantitative risk assessment methods (in two
different moments) to estimate and assess six risks identified in two tasks
accomplished to produce Airbags. The Krippendorff's Alpha Coefficient
(α K) was the agreement measure selected to evaluate both
inter-coder and intra-coder consensus.
RESULTS: The preliminary results revealed a general low concordance
(α K < 0.6) for both reliability evaluations. Apparently there
are no relevant differences between the risk assessment results obtained by
individuals with different levels of experience or expertise.
CONCLUSIONS: This study revealed that the use of the
semi-quantitative approach should be done with caution.
