Human reliability analysis (HRA) examines human errors or human failure events (HFEs) and their probabilities (Human Error Probabilities, HEPs) under specific task conditions using various models. Given the scarcity of empirical data, expert judgments remain vital in the HRA community. Despite prior efforts to model the expert elicitation process using Bayesian methods, there is still a lack of comprehensive understanding of the overall process and its key components in the Bayesian context, which undermines the effectiveness and applicability of these models. To address this gap, this study starts by investigating the sources of uncertainties in various HRA constructs and elements of the expert elicitation process, aiming to form rigorous definitions in probabilistic terms. Furthermore, practical elicitation approaches are modeled and common misunderstandings about Bayesian modeling in past HRA research are examined and addressed. Based on this foundation, a Bayesian model is developed to integrate the different elements with the HRA constructs, and the probabilistic relationships between them are analyzed and elucidated. Two direct applications of the model are also provided to demonstrate the model’s practical applicability, highlighting its strengths and limitations. The proposed model not only offers a formal approach to rigorously incorporating expert judgments into HRA but also enhances researchers’ and practitioners’ understanding of the uncertainties related to the expert elicitation process, thereby providing a solid theoretical foundation for the development of future Bayesian models.
LiuPZhangRYinZ, et al. Human errors and human reliability. In: SalvendyGKarwowskiW (eds) Handbook of human factors and ergonomics. John Wiley & Sons, Ltd, 2021, pp.514–572.
2.
LiuPLiuJ. Combined effect of multiple performance shaping factors on human reliability: multiplicative or additive?Int J Hum Comput Interact2020; 36(9): 828–838.
3.
KolaczkowskiAForesterJLoisE, et al. Good practices for implementing Human Reliability Analysis (HRA). Technical Report NUREG-1792, U.S. Nuclear Regulatory Commission, 2005.
4.
LoisEDangVNForesterJ, et al. International HRA empirical study, phase 1 report: description of overall approach and pilot phase results from comparing HRA methods to simulator performance data. Technical Report NUREG/IA-0216, Vol. 1, U.S. Nuclear Regulatory Commission, 2009.
5.
BoringRL. A review of expertise and judgment processes for risk estimation. In: Risk, reliability and societal safety, three volume set (proceedings of the European Safety and reliability conference, ESREL 2007), Stavanger, Norway, 25–27 June 2007, pp.1901–1908. London: Taylor & Francis.
6.
YinZGongBLiuZ, et al. Digesting human-related incidents in nuclear power plant commissioning – Part I: an integrated methodology. Reliab Eng Syst Saf2025; 262: 111164.
7.
YinZGongBLiuZ, et al. Digesting human-related incidents in nuclear power plant commissioning – Part II: results and implications. Reliab Eng Syst Saf2025; 262: 111170.
8.
HamDHParkJ. Use of a big data analysis technique for extracting HRA data from event investigation reports based on the Safety-II concept. Reliab Eng Syst Saf2020; 194: 106232.
9.
ChangYJBleyDCriscioneL, et al. The SACADA database for human reliability and human performance. Reliab Eng Syst Saf2014; 125: 117–133.
10.
JungWParkJKimY, et al. HuREX–A framework of HRA data collection from simulators in nuclear power plants. Reliab Eng Syst Saf2020; 194: 106235.
11.
PandyaDPodofilliniLEmertF, et al. Quantification of a human reliability analysis method for radiotherapy applications based on expert judgment aggregation. Reliab Eng Syst Saf2020; 194: 106489.
12.
EkanemNMoslehAShenSH, et al. Phoenix–A model-based human reliability analysis methodology: data sources and quantitative analysis procedure. Reliab Eng Syst Saf2024; 248: 110123.
13.
XingJParryGPresleyM, et al. An integrated human event analysis system (IDHEAS) for nuclear power plant internal events at-power application, Volume 1. Technical Report NUREG-2199, U.S. Nuclear Regulatory Commission, 2017.
14.
XingJMorrowS. White paper: practical insights and lessons learned on implementing expert elicitation. Technical report, U.S. Nuclear Regulatory Commission, 2016.
15.
SpurginAJ. Human reliability assessment theory and practice. CRC Press, 2009.
16.
O’HaganABuckCEDaneshkhahA, et al. Uncertain judgements: eliciting experts’ probabilities. John Wiley & Sons, Ltd., 2006.
17.
GrecoSFPodofilliniLDangVN. A Bayesian model to treat within-category and crew-to-crew variability in simulator data for Human Reliability Analysis. Reliab Eng Syst Saf2021; 206: 107309.
18.
XingJChangYJDeJesus SegarraJ. The general methodology of an integrated human event analysis system (IDHEAS-G). Technical Report NUREG-2198, U.S. Nuclear Regulatory Commission, 2021.
19.
SwainAD. Human reliability analysis: need, status, trends and limitations. Reliab Eng Syst Saf1990; 29(3): 301–313.
20.
KirwanBMartinBRycraftH, et al. Human error data collection and data generation. Int J Qual Reliab Manage1990; 7(4): 34–66.
21.
HallbertBGertmanDLoisE, et al. The use of empirical data sources in HRA. Reliab Eng Syst Saf2004; 83(2): 139–143.
22.
PreischlWHellmichM. Human error probabilities from operational experience of German nuclear power plants. Reliab Eng Syst Saf2013; 109: 150–159.
23.
KimYParkJJungW. A survey of data-based human reliability analysis approaches. In: Proceedings of the 1st Asian conference on ergonomics and design (ACED 2014), Jeju Island, Korea, pp.360–365. Seoul, Republic of Korea: Ergonomics Society of Korea.
24.
MoslehABierVMApostolakisG. A critique of current practice for the use of expert opinions in probabilistic risk assessment. Reliab Eng Syst Saf1988; 20(1): 63–85.
25.
CooperSERamey-SmithAMWreathallJ, et al. A technique for human error analysis. Technical Report NUREG/CR-6350, U.S. Nuclear Regulatory Commission, 1996.
26.
GertmanDBlackmanHMarbleJ, et al. The SPAR-H human reliability analysis method. Technical Report NUREG/CR-6883, U.S. Nuclear Regulatory Commission, 2004.
27.
PesmeHLe BotPMeyerP. Little stories to explain human reliability assessment: a practical approach of the MERMOS method. In: 2007 IEEE 8th human factors and power plants and HPRCT 13th annual meeting, 26–31 August 2007, Monterey, CA, pp.284–287. IEEE: New York.
28.
EkanemNJMoslehAShenSH. Phoenix – a model-based human reliability analysis methodology: qualitative analysis procedure. Reliab Eng Syst Saf2016; 145: 301–315.
29.
PyyP. Human reliability analysis methods for probabilistic safety assessment. Technical Report VTT Publications 422, Valtion teknillinen tutkimuskeskus, 2000.
30.
YinZLiZLiuZ, et al. Collection of IDHEAS-based human error probability data for nuclear power plant commissioning through expert elicitation. Ann Nucl Energy2023; 181: 109544.
31.
BoringRLBlackmanHS. The origins of the SPAR-H method’s performance shaping factor multipliers. In 2007 IEEE 8th human factors and power plants and HPRCT 13th annual meeting, 26–31 August 2007, Monterey, pp.177–184. IEEE: New York.
32.
LaumannKRasmussenM. Suggested improvements to the definitions of Standardized Plant Analysis of Risk-Human Reliability Analysis (SPAR-H) performance shaping factors, their levels and multipliers and the nominal tasks. Reliab Eng Syst Saf2016; 145: 287–300.
33.
LiuPQiuYHuJ, et al. Expert judgments for performance shaping factors’ multiplier design in human reliability analysis. Reliab Eng Syst Saf2020; 194: 106343.
34.
KimYParkJPresleyM. Selecting significant contextual factors and estimating their effects on operator reliability in computer-based control rooms. Reliab Eng Syst Saf2021; 213: 107679.
35.
KimY. Expert elicitation for estimating PSF effects on HEPs in computer-based control rooms. In: BeerMZioE (eds.) Proceedings of the 29th European safety and reliability conference, ESREL 2019. Research Publishing Services, 2019, pp.248–252.
36.
KimYKimJParkJ, et al. An HRA method for digital main control rooms–Part II: estimating the failure probability due to cognitive error. Technical Report KAERI/TR-8065/2020, Korea Atomic Energy Research Institute, 2020.
37.
SvensonO. On expert judgements in safety analyses in the process industries. Reliab Eng Syst Saf1989; 25(3): 219–256.
38.
ForesterJKolaczkowskiACooperS, et al. ATHEANA user’s guide. Technical Report NUREG-1880, U.S. Nuclear Regulatory Commission, 2007.
39.
CookeRM. Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, 1991.
40.
JeffreyR. Probability and the art of judgment. Cambridge University Press, 1992.
41.
MeyerMABookerJM. Eliciting and analyzing expert judgment: a practical guide. Technical Report NUREG/CR-5424, U.S. Nuclear Regulatory Commission, 1990.
42.
PodofilliniLDangVN. A Bayesian approach to treat expert-elicited probabilities in human reliability analysis model construction. Reliab Eng Syst Saf2013; 117: 52–64.
43.
CelikMLavasaniSMWangJ. A risk-based modelling approach to enhance shipping accident investigation. Saf Sci2010; 48(1): 18–27.
44.
GholamizadehKZareiEOmidvarM, et al. Fuzzy sets theory and human reliability: review, applications, and contributions. In: YazdiM (ed.) linguistic methods under fuzzy information in system safety and reliability analysis. Springer International Publishing, 2022, pp.91–137.
45.
PanXZuoDZhangW, et al. Research on human error risk evaluation using extended Bayesian networks with hybrid data. Reliab Eng Syst Saf2021; 209: 107336.
46.
SeaverDAWilliamSG. Procedures for using expert judgment to estimate human error probabilities in nuclear power plant operations. Technical Report NUREG/CR-2743, U.S. Nuclear Regulatory Commission, 1983.
47.
BleyDKaplanSJohnsonD. The strengths and limitations of PSA: where we stand. Reliab Eng Syst Saf1992; 38(1): 3–26.
48.
KaplanS. ‘Expert information’ versus ‘expert opinions’. Another approach to the problem of eliciting/combining/using expert knowledge in PRA. Reliab Eng Syst Saf1992; 35(1): 61–72.
49.
BoringRLGertmanDI. Atomistic and holistic approaches to human reliability analysis in the US nuclear power industry. Saf and Reliab2005; 25(2): 21–37.
50.
HollnagelE. Cognitive reliability and error analysis method. Elsevier, 1998.
51.
ForesterJBleyDCooperS, et al. Expert elicitation approach for performing ATHEANA quantification. Reliab Eng Syst Saf2004; 83(2): 207–220.
52.
WilsonJRRutherfordA. Mental models: theory and application in human factors. Hum Factors1989; 31(6): 617–634.
53.
ComerMKSeaverDAStillwellWG, et al. Generating human reliability estimates using expert judgment, Volume 1: main report. Technical Report NUREG/CR-3688, U.S. Nuclear Regulatory Commission, 1984.
54.
BrookerP. Experts, Bayesian belief networks, rare events and aviation risk estimates. Saf Sci2011; 49(8): 1142–1155.
55.
KahnemanDSlovicPTverskyA. Judgment under uncertainty: heuristics and biases. Cambridge University Press, 1982.
56.
HuangJBusemeyerJR. Models of human probability judgment errors. PsyArXiv Preprints2024.
57.
WyerRS. An investigation of the relations among probability estimates. Organ Behav Hum Perform1976; 15(1): 1–18.
58.
AbelsonRPLeddoJGrossPH. The strength of conjunctive explanations. Pers Soc Psychol Bull1987; 13(2): 141–155.
59.
BusemeyerJRPothosEMFrancoR, et al. A quantum theoretical explanation for probability judgment errors. Psychol Rev2011; 118(2): 193–218.
60.
CostelloFWattsP. Surprisingly rational: probability theory plus noise explains biases in judgment. Psychol Rev2014; 121(3): 463–480.
61.
ZhuJQSanbornANChaterN. The Bayesian sampler: generic Bayesian inference causes incoherence in human probability judgments. Psychol Rev2020; 127(5): 719–748.
62.
KleinGCalderwoodRMacGregorD. Critical decision method for eliciting knowledge. IEEE Trans Syst Man Cybern1989; 19(3): 462–472.
63.
HallbertBKolaczkowskiA. The employment of empirical data and Bayesian methods in human reliability analysis: a feasibility study. Technical Report NUREG/CR-6949, U.S. Nuclear Regulatory Commission, 2007.
64.
van de SchootRDepaoliSKingR, et al. Bayesian statistics and modelling. Nat Rev Methods Primers2021; 1(1): 1.
65.
KimMCSeongPHHollnagelE. A probabilistic approach for determining the control mode in CREAM. Reliab Eng Syst Saf2006; 91(2): 191–199.
66.
GrothKMSwilerLP. Bridging the gap between HRA research and HRA practice: a Bayesian network version of SPAR-H. Reliab Eng Syst Saf2013; 115: 33–42.
67.
AbrishamiSKhakzadNHosseiniSM, et al. BN-SLIM: a Bayesian Network methodology for human reliability assessment based on Success Likelihood Index Method (SLIM). Reliab Eng Syst Saf2020; 193: 106647.
68.
KimYParkJ. Incorporating prior knowledge with simulation data to estimate PSF multipliers using Bayesian logistic regression. Reliab Eng Syst Saf2019; 189: 210–217.
69.
GrothKMSmithCLSwilerLP. A Bayesian method for using simulator data to enhance human error probabilities assigned by existing HRA methods. Reliab Eng Syst Saf2014; 128: 32–40.
70.
ApostolakisGMoslehA. Expert opinion and statistical evidence: an application to reactor core melt frequency. Nucl Sci Eng1979; 70(2): 135–149.
71.
MoslehAApostolakisG. Models for the use of expert opinions. In: WallerRACovelloVT (eds) Low-probability/high-consequence risk analysis. Springer, 1984, pp.107–124.
72.
MoslehA. Bayesian modeling of expert-to-expert variability and dependence in estimating rare event frequencies. Reliab Eng Syst Saf1992; 38(1–2): 47–57.
73.
MoslehAApostolakisG. The assessment of probability distributions from expert opinions with an application to seismic fragility curves. Risk Anal1986; 6(4): 447–461.
74.
BaraldiPContiMLibrizziM, et al. A Bayesian network model for dependence assessment in human reliability analysis. In: BrišRSoaresCGMartorellS (eds) Reliability, risk, and safety, three volume set. CRC Press, 2009. pp.223–230.
75.
ZhaoYSmidtsC. CMS-BN: a cognitive modeling and simulation environment for human performance assessment, part 1—methodology. Reliab Eng Syst Saf2021; 213: 107776.
76.
DroguettELGroenFMoslehA. The combined use of data and expert estimates in population variability analysis. Reliab Eng Syst Saf2004; 83(3): 311–321.
77.
GelmanACarlinJBSternHS, et al. Bayesian data analysis. CRC Press, 2014.
78.
GrecoSFPodofilliniLDangVN. A Bayesian two-stage approach to integrate simulator data and expert judgment in human error probability estimation. Saf Sci2023; 159: 106009.
79.
NilssonNJ. Reasoning with uncertain information. In: NilssonNJ (ed.) Artificial intelligence: a new synthesis. Morgan Kaufmann, 1998, pp.317–342.
80.
BoringRL. How many performance shaping factors are necessary for human reliability analysis? In Proceeding of the 10th international conference on probabilistic safety assessment & management 2010 (PSAM 10), Seattle, WA, 7–11 June 2010, pp. 1479–1487. Red Hook, NY: International Association for Probabilistic Safety Assessment and Management (IAPSAM).
81.
EaswaranK. Conditional probabilities. In: PettigrewRWeisbergJ (eds) The open handbook of formal epistemology. PhilPapers Foundation, 2019, pp.131–198.
82.
PosnerMIBoiesSJ. Components of attention. Psychol Rev1971; 78(5): 391–408.
83.
MillerGA. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev1956; 63(2): 81–97.
84.
Johnson-LairdPN. Mental models: towards a cognitive science of language, inference, and consciousness. Harvard University Press, 1986.
85.
NormanD. The design of everyday things. Basic Books, 2013.
86.
BataneroCHenryMParzyszB. The nature of chance and probability. In: JonesGA (ed.) Exploring probability in school: challenges for teaching and learning. Springer, 2005, pp.15–37.
87.
PaglioniVPGrothKM. Dependency definitions for quantitative human reliability analysis. Reliab Eng Syst Saf2022; 220: 108274.
88.
HolmbergJEBladhKOxstrandJ, et al. The application of the enhanced Bayesian THERP in the HRA methods empirical study using simulator data. In: Proceedings of 9th international probabilistic safety assessment and management conference (PSAM9), Hong Kong, China, 18-23 May 2008, pp.1497–1503. Red Hook, NY: International Association for Probabilistic Safety Assessment and Management (IAPSAM).
89.
HeckermanD. A tutorial on learning with Bayesian Networks. In: HolmesDEJainLC (eds) Innovations in Bayesian networks: theory and applications. Springer Berlin Heidelberg, 2008, pp.33–82.
90.
LittleRJRubinDB. Maximum likelihood for general patterns of missing data: introduction and theory with ignorable nonresponse. In: Statistical analysis with missing data. Wiley Online Library, 2002, pp.164–189.
91.
Arroyo-FigueroaGSucarLE. A temporal Bayesian network for diagnosis and prediction. ArXiv, 1999.
92.
LiCMahadevanS. Sensitivity analysis of a Bayesian network. ASCE-ASME J Risk and Uncert Eng Syst Part B Mech Eng2017; 4: 011003.
