Situation assessment is a critical skill amongst electricity distribution controllers but can be difficult to assess in practice. In the present research, we adapted a psychometric approach based on the principle that expert situation assessment engages cue-based associations from memory. Using consortium norming, we acquired performance data from 320 network control operators for five network control-related tasks that comprised the electricity network service provider edition of the EXPERT Intensive Skills Evaluation (EXPERTise 2.0) software. Operators were distributed across 11 Distributed Network Service Providers (DNSP), with data collected over three phases. Data were examined across the three phases of data collection and suggest that DNSPs can monitor the situation assessment of network controllers at an individual and at an organisational level, affording the opportunity to identify changes in performance and implement remedial strategies as appropriate.
Get full access to this article
View all access options for this article.
References
1.
BlaschE.KadarI.SalernoJ.KokarM. M.DasS.PowellG. M.CorkillD.D.RuspiniE. H. (2006). Issues and challenges of knowledge representation and reasoning methods in situation assessment (Level 2 Fusion). In Signal Processing, Sensor Fusion, and Target Recognition XV (vol. 6235, p. 623510). International Society for Optics and Photonics.
2.
DrewT.VoM. L. H.OlwalA.JacobsonF.SeltzerS. E.WolfeJ. M. (2013). Scanners and drillers: Characterizing expert visual search through volumetric images. Journal of Vision, 13(10), 3-3.
3.
GreitzerF. L.PodmoreR.RobinsonM.EyP. (2010). Naturalistic decision making for power system operators. International Journal of Human–Computer Interaction, 26(2-3), 278-291.
4.
KleinG. (2008). Naturalistic decision making. Human Factors, 50(3), 456-460.
5.
LipshitzR.KleinG.OrasanuJ.SalasE. (2001). Taking stock of naturalistic decision making. Journal of Behavioural Decision Making, 14, 331-352.
6.
LovedayT.WigginsM.FestaM.SchellD.TwiggD. (2013). Pattern Recognition as an Indicator of Diagnostic Expertise. In Latorre CarmonaP.SanchezJ. S.FredA. L. N. (Eds.), Pattern recognition - Applications and methods (vol. 204, pp. 1-11): Berlin: Springer.
7.
LovedayT.WigginsM.W.HarrisJ.SmithN.O’HareD. (2013). An Objective Approach to Identifying Diagnostic Expertise Amongst Power System Controllers. Human Factors, 55, 90-107.
8.
LovedayT.WigginsM.W.SearleB. (2014). Cue utilisation and broad indicators of workplace expertise. Journal of Cognitive Engineering and Decision-Making, 8, 98-113.
9.
LovedayT.WigginsM.W.SearleB.J.FestaM.SchellD. (2013). The capability of static and dynamic features to distinguish competent from genuinely expert practitioners in pediatric diagnosis. Human Factors, 55, 125-137.
10.
MorrisonB. W.WigginsM. W.BondN. W.TylerM. D. (2013). Measuring relative cue strength as a means of validating an inventory of expert offender profiling cues. Journal of Cognitive Engineering and Decision Making, 7, 211–226. doi:10.1177/1555343412459192
11.
PanteliM.CrossleyP. A.KirschenD. S.SobajicD. J. (2013). Assessing the impact of insufficient situation awareness on power system operation. IEEE Transactions on Power Systems, 28(3), 2967-2977.
12.
PanteliM.KirschenD. S. (2015). Situation awareness in power systems: Theory, challenges and applications. Electric Power Systems Research, 122, 140-151.
13.
PauleyK.O’HareD.WigginsM. W. (2009). Measuring expertise in weather-related aeronautical risk perception: The validity of the Cochran–Weiss–Shanteau (CWS) Index. The International Journal of Aviation Psychology, 19, 201–216. doi:10.1080/10508410902979993
14.
PetersonS. E.KulhavyR. W.StockW. A.PridemoreD. R. (1991). How map features cue associated verbal content. Bulletin of the Psychonomic Society, 29(2), 158-160.
15.
RandelJ. M.PughH. L.ReedS. K. (1996). Differences in expert and novice situation awareness in naturalistic decision making. International Journal of Human-Computer Studies, 45(5), 579-597.
16.
StreinerD. L. (2003). Starting at the beginning: an introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80, 99-103.
17.
SturmanD.WigginsM. W.AutonJ.LoftS.HeltonW.WestbrookJ.BraithwaiteJ. (2019). Control room operators’ cue utilization predicts cognitive resource consumption during regular operational tasks. Frontiers in Psychology, 10, 1967.
18.
WatkinsonJ.BristowG.AutonJ.McMahonC. M.WigginsM. W. (2018). Postgraduate training in audiology improves clinicians’ audiology-related cue utilisation. International journal of audiology, 57(9), 681-687.
19.
WeissD. J.ShanteauJ. (2003). Empirical assessment of expertise. Human Factors, 45, 104–116.
20.
WigginsM.W. (2006). Cue-based processing and human performance. In KarwowskiW. (Ed.), Encyclopedia of ergonomics and human factors (2nd ed) (pp. 641-645). London, UK: Taylor and Francis.
21.
WigginsM.W. (2015). Cues in diagnostic reasoning. In WigginsM.W.LovedayT. (Eds.), Diagnostic expertise in organisational environments (pp. 1-13). Aldershot, UK: Ashgate.
WigginsM. W.O’HareD. (1995). Expertise in aeronautical weather-related decision making: A cross-sectional analysis of general aviation pilots. Journal of Experimental Psychology: Applied, 1, 305–320. doi:10.1037/1076-898X.1.4.305
24.
WigginsM. W.O’HareD. (2003). Expert and novice pilot perceptions of static in-flight images of weather. International Journal of Aviation Psychology, 13, 173–187. doi:10.1207/S15327108IJAP1302_05
25.
WigginsM. W.StevensC.HowardA.HenleyI.O’HareD. (2002). Expert, intermediate and novice performance during simulated pre-flight decision-making. Australian Journal of Psychology, 54, 162–167. doi:10.1080/00049530412331312744
26.
YoungM. E.WassermanE. A. (2002). Limited attention and cue order consistency affect predictive learning: A test of similarity measures. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28(3), 484.
27.
ZioE. (2016). Challenges in the vulnerability and risk analysis of critical infrastructures. Reliability Engineering & System Safety, 152, 137-15.
