Gathering Meaningful Data From Novices and or in Simplified Operating Environments to Inform Us About Highly Complex Operational Environments

Abstract

Much of the basis for current U.S. Nuclear Regulatory Commission (NRC) Human Factors Engineering (HFE) guidance comes from data from other domains (e.g. aviation, defense), qualitative data from operational experience in NPPs, and a limited amount from empirical studies in a nuclear environment. A simulator is one tool that can be used to gather more empirical nuclear specific human performance data. Although this may seem like a simple undertaking, getting trained operators for human-performance testing can be very challenging. In addition, when operators can be secured for human performance testing the operator sample is quite small, allowing for only qualitative analysis or limited quantitative analysis.

Thus, the NRC initiated research to determine: 1) if novices can successfully perform realistic operator tasks, 2) if a simulator can create a cognitively similar environment to that which NPP operators face, and if 1 and 2 are successful, 3) how the resulting performance data can be used and interpreted. For our research, we determined that the environment needed to be simplified in such a way that would induce participants to experience both the complexity and cognitive requirements incurred by trained operators. In other words, the methodological approach adhered to the principal of different but equal; the roles, procedures and interface are different, but they are different in such a way that is controlled and meant to induce the same type of cognition and level of workload that would be experienced by an operator population.

The simplification of the NPP environment has been very challenging and there have been many lessons learned. The panelists are challenged to make recommendations for investigators for best practices for gathering meaningful data from novices and or in simplified operating environments to inform us about highly complex operational environments. Some discussion questions relevant to this topic include: What type of research questions should we ask of the novice population? How can we use the data in a meaningful way? What type of research questions should be avoided with this population? How can we use novices to inform us about the “human” (not the trained operator) piece of performance? How can we simplify operational environments and still gain valuable data and insights? How can we use a trained operator population in these simplified environments?