In future deep space exploration missions, crew will have to work more autonomously from Earth. Greater crew autonomy will increase dependence on automated systems. This study investigates the performance effects of different strategies to automate procedural work for space exploration operations.
Background
The following strategies are investigated for performing procedural work:
• Manual Work uses no procedure automation and crew performs all actions.
• Shared Work uses procedure automation to perform some actions within a procedure while crew performs other actions.
• Supervised Work uses procedure automation to perform procedure actions while crew supervises the automation.
Method
Twenty-seven participants participated in a planetary habitat scenario-based simulation using electronic procedures with automatable actions to investigate the effect of these strategies on situation awareness (SA) and workload. This study used a modification of the Situation Presence Assessment Method to measure SA and the Bedford Workload Scale to measure subjective workload.
Results
Mean response times and accuracy for SA queries show no significant difference among the three strategies. Bedford Workload ratings compared across the three strategies indicate that participants rated their workload as highest in the Manual Work condition, followed by the Shared Work condition, and lowest in the Supervised Work condition.
Conclusion
The study hypothesized that increased levels of automation would lead to lower subjective workload and decreased SA. Although no significant difference in SA was observed, subjective workload was lower in automation strategies. Based on subjective ratings, 93% of participants preferred some form of automation, with 56% preferring the Shared Work automation condition.
BillmanD.SchreckenghostD. (2018). Usability issues and guidance for flexible execution of procedural work. In AAAI Spring Symposium, The Design of the User Experience for Artificial Intelligence (the UX of AI). Stanford University. [Accessed March 2018].
2.
BillmanD.SchreckenghostD.BillinghurstS. (2015). Problem solving when procedure conditions are not met: using procedure-automation software for support. The Annual Meeting of the Human Factors and Ergonomics Society (HFES), Los Angeles, CA, Oct 26-30, 2015.
3.
BillmanD.SchreckenghostD.PardisM. (2014). Assessment of alternative interfaces for manual commanding of spacecraft systems: Compatibility with flexible allocation policies. In Human Factors and Ergonomics Society Annual Meeting, Chicago, IL, Oct 27-31, 2014.
4.
BradshawJ. M.FeltovichP.JungH.KulkarniS.TaysomW.UszokA. (2004). Dimensions of adjustable autonomy and mixed-initiative interaction. In NicklesM.RovatsosM.WeissG. (Ed.), Agents and computational autonomy: Potential, risks, and solutions (pp. 17–39). Springer-Verlag, Lecture Notes in Computer Science 2969.
5.
CarmodyM. A.GluckmanJ. P. (1993). Task specific effects of automation and automation failure on performance, workload and situational awareness. In JensenR. S.NeumeisterD. (Ed.), Proceedings of the Seventh International Symposium on Aviation Psychology (pp. 167–171). Department of Aviation, The Ohio State University.
6.
ClarkH. H. (1985). Language use and language users. In LindzeyG.AronsonE. (Eds.), Handbook of social psychology. (3rd ed., pp. 179–231). Wiley.
7.
DursoF. T.HackworthC. A.TruittT.CrutchfieldJ.ManningC. A. (1998). Situation awareness as a predictor of performance in en route air traffic controllers, Air Traffic Quarterly, 6(1), 1–20. https://doi.org/10.2514/atcq.6.1.1
8.
EndsleyM. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 32-64. https://doi.org/10.1518/001872095779049543.
9.
EndsleyM. R.KirisE. O. (1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37(2), 381–394. https://doi.org/10.1518/001872095779064555
10.
JamiesonG. A.SkraaningG. (2019). The absence of degree of automation trade-offs in complex work settings. Human Factors, 62(4), 516–529. https://doi.org/10.1177/0018720819842709
11.
KaberD. B.EndsleyM. R. (2003). The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task. Theoretical Issues in Ergonomic Science, 5(2), 113-153. https://doi.org/10.1080/1463922021000054335.
12.
KaberD. B.OnalE.EndsleyM. R. (2000). Design of automation for telerobots and the effect on performance, operator situation awareness, and subjective workload. Human Factors and Ergonomics in Manufacturing & Service Industries, 10(4), 409–430. https://doi.org/10.1002/1520-6564(200023)10:43.0.CO;2-V
13.
KleinGWoodsD. D.BradshawJ. M.HoffmanR.FeltovichP. J. (2004). Ten challenges for making automation a “team player” in joint human-agent activity. IEEE Intelligent Systems, 19(6), 91–95. https://doi.org/10.1109/MIS.2004.74.
14.
KortenkampD.BellS. (2003). Simulating advanced life support systems for integrated controls research. In Proceedings 33rd International Conference on Environmental Systems (ICES),Vancouver, BC, CA., July7-10, 2003.
15.
KortenkampD.BonassoR. P.SchreckenghostD.DalalK.VermaV.WangL. (2008). A procedure representation language for human spaceflight operations. In The 9th International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS-08).Pasadena, CA, Feb 26-29, 2008.
16.
KortenkampD.SchreckenghostD.BonassoR. P. (2000). Adjustable control autonomy for manned space flight. In IEEE Aerospace Conference, Big Sky,MT, March 2000.
17.
KramerA. F.SirevaagE. J.BrauneR. (1987). A psychophysiological assessment of operator workload during simulated flight missions. Human Factors, 29(2), 145–160. https://doi.org/10.1177/001872088702900203
NASA Technology Roadmaps. (2015). TA 7: Human exploration destination systems.
20.
OnnaschL.WickensC. D.LiH.ManzeyD. (2014). Human performance consequences of stages and levels of automation: An integrated meta-analysis. Human Factors, 56(3), 476–488. https://doi.org/10.1177/0018720813501549
21.
ParasuramanR.SheridanT. B.WickensC. D. (2000). A model of types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics–Part A, 30(3), 286–297. https://doi.org/10.1109/3468.844354
22.
PritchettA.KimS. Y.FeighK. M. (2014). Modeling Human–Automation function allocation. Journal of Cognitive Engineering and Decision Making, 8(1), 33–51. https://doi.org/10.1177/1555343413490944
23.
RileyV. (1989). A general model of mixed-initiative human-machine systems. Proceedings of Human Factors Society Annual Meeting, 33(2), 124–128. https://doi.org/10.1177/154193128903300227
24.
RoscoeA. H. (1984). Assessing pilot workload in flight. Royal Aircraft Establishment.
25.
SchreckenghostD.BillmanD.MilamT. (2015). Effectiveness of strategies for partial automation of electronic procedures during NASA HERA analog missions. In International Joint Conferences on Artificial Intelligence, Proceedings of AI in Space Workshop. Buenos Aires, Argentina, Jul 25-26,2015.
26.
SchreckenghostD.BillmanD.MilamT. (2014). Human performance with procedure automation to manage spacecraft systems. In Proceedingsof IEEE Aerospace. Big Sky, MT, Mar 1-8, 2014.
27.
SchreckenghostD.BonassoR. P.KortenkampD.BellS.MilamT.ThronesberyC. (2008). Adjustable Autonomy with NASA Procedures. In The 9th International Symposium on Artificial Intelligence, Robotics and Automation in Space,Pasadena, CA, Feb 26-29, 2008.
28.
SchreckenghostD.MilamT.KortenkampD.NguyenA. (2021) Near realtime computation of task performance using electronic procedures. In 16th International Conference on Space Operations, Cape Town, South Africa, 3–5 May 2021.
29.
SheridanT. B. (2002). Humans and automation: System design and research issues, John Wiley and Sons.
30.
SheridanT. B. (2021) Human supervisory control of automation. In SalvendyG.KarwowskiW. (Ed.), Handbook of Human Factors and Ergonomics (5th ed., Vol. 13. pp. 736–760). John Wiley & Sons Inc.
31.
VarghaA.DelaneyH. D. (2000). A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25(2), 101–132. https://doi.org/10.2307/1165329Vidulich
32.
VidulichM. A. (2000). Testing the sensitivity of situation awareness metrics in interface evaluations. In EndsleyM. R.GarlandD. J. (Eds.), Situation awareness analysis and measurement (pp. 227–246). Lawrence Erlbaum Associates.
33.
WickensC. D.LiH.SantamariaA.SebokA.SarterN. B. (2010). Stages and levels of automation: An integrated meta-analysis. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 54(4), 389–393. https://doi.org/10.1177/154193121005400425
