Assessment of Decision Support Systems for Envisioned Human Extravehicular Activity Operations: From Requirements to Validation and Verification

Abstract

This work presents the results of a laboratory-based study of prototype decision support systems (DSS) for envisioned human extravehicular activity (EVA). A central feature of this work is demonstrating the explicit linkages between DSS design requirements derived from work domain demands with the validation and verification process to examine the utility of specific DSS design solutions. Two DSS prototypes were developed—Baseline and Advanced—that addressed the same set of requirements derived through a cognitive systems engineering (CSE) definition process. The Baseline design was constructed as a minimum derivation from present-day technological standards while the Advanced design incorporated novel software solutions that currently do not exist in the EVA work domain. A representative future domain of human EVA operations was constructed and utilized to evaluate the DSS designs. Both DSS prototype designs were verified to satisfy their design requirements. Furthermore, each design was validated in favor of the Advanced DSS, which outperformed the Baseless DSS in nearly all measures of performance. This work illustrates how the same set of requirements can be satisfied in multiple ways to realize effective DSS solutions.

Keywords

envisioning extravehicular activity decision support system development human in the loop simulation requirements validation and verification

Get full access to this article

View all access options for this article.

References

Baayen

R. H.

Davidson

D. J.

Bates

D. M.

(2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412. doi:10.1016/j.jml.2007.12.005

Bates

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi:10.18637/jss.v067.i01

Bell

Coan

Oswald

(2006, June 19–23). A discussion on the making of an EVA: What it really takes to walk in space. Paper presented at the SpaceOps 2006 Conference, American Institute of Aeronautics and Astronautics, Reston, VA. doi:10.2514/6.2006-5570

Dekker

S. W. A.

Woods

D. D.

Mooij

(2002). Envisioned practice, enhanced performance: The riddle of future (ATM) systems. Journal of Applied Aviation Studies, 2, 23–32.

Donmez

Cummings

M. L.

Brzezinski

A. S.

Graham

H. D.

(2010, September 28–30). Modified Cooper Harper scales for assessing unmanned vehicle displays. Paper presented at the Proceedings of the 10th Performance Metrics for Intelligent Systems Workshop, Baltimore, MD.

Elm

W. C.

Gualtieri

J. W.

McKenna

B. P.

Tittle

J. S.

Peffer

J. E.

Szymczak

S. S.

Grossman

J. B.

(2008). Integrating cognitive systems engineering throughout the systems engineering process. Journal of Cognitive Engineering and Decision Making, 2, 249–273. doi:10.1518/155534308X377108

Hoffman

R. R.

Klein

G. L.

(2017). Challenges and prospects for the paradigm of naturalistic decision making. Journal of Cognitive Engineering and Decision Making, 11(1), 97–104. doi:10.1177/1555343416689646

Judd

C. M.

Westfall

Kenny

D. A.

(2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103, 54–69. doi:10.1037/a0028347

Klein

(2008). Naturalistic decision making. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50, 456–460. doi:10.1518/001872008X288385

10.

Klein

G. A.

Orasanu

J. E.

Calderwood

R. E.

Zsambok

C. E.

(1993). Decision making in action: Models and methods. Norwood, NJ: Ablex.

11.

Kovacs

G. T. A.

Shadden

(2017). Analysis of age as a factor in NASA astronaut selection and career landmarks. PLoS ONE, 12(7), 1–23. doi:10.1371/journal.pone.0181381

12.

Liskowsky

D. R.

Seitz

W. W.

(2010). Human integration design handbook (HIDH) (No. NASA/SP-2010-3407). Retrieved from https://ston.jsc.nasa.gov/collections/TRS/_techrep/SP-2010-3407.pdf

13.

Marquez

J. J.

Hillenius

Deliz

Zheng

Kanefsky

Gale

(2019, March 2–9). Enabling communication between astronauts and ground teams for space exploration missions. Presented at the 2019 IEEE Aerospace Conference, Big Sky, MT. doi:10.1109/AERO.2019.8741593

14.

McBarron

I. I.

(1994). Past, present, and future: The US EVA program. Acta Astronautica, 32(1), 5–14.

15.

McGeorge

Hegde

Berg

R. L.

Guarrera-Schick

T. K.

LaVergne

D. T.

Casucci

S. N.

. . . Bisantz

(2015). Assessment of innovative emergency department information displays in a clinical simulation center. Journal of Cognitive Engineering and Decision Making, 9, 329–346. doi:10.1177/1555343415613723

16.

Militello

L. G.

Dominguez

C. O.

Lintern

Klein

(2009). The role of cognitive systems engineering in the systems engineering design process. Systems Engineering, 13, 261–273. doi:10.1002/sys.20147

17.

Miller

M. J.

(2017). Decision support system development for human extravehicular activity (Doctoral thesis). Georgia Institute of Technology, Atlanta, GA.

18.

Miller

M. J.

Claybrook

Greenlund

Marquez

J. J.

Feigh

K. M.

(2017). Operational assessment of Apollo Lunar Surface Extravehicular Activity (No. NASA/TP–2017–219457). Retrieved from https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20170007261.pdf

19.

Miller

M. J.

Feigh

K. M.

(2019). Addressing the envisioned world problem: A case study in human spaceflight operations. Design Science, 5, e3. doi:10.1017/dsj.2019.2

20.

Miller

M. J.

McGuire

K. M.

Feigh

K. M.

(2015, March 7–14). Information flow model of human extravehicular activity. Paper presented at the Proceedings of the IEEE Aerospace Conference, Big Sky, MT. doi:10.1109/AERO.2015.7118942.

21.

Miller

M. J.

McGuire

K. M.

Feigh

K. M.

(2017). Decision support system requirements definition for human extravehicular activity based on cognitive work analysis. Journal of Cognitive Engineering and Decision Making, 11, 136–165. doi:10.1177/1555343416672112

22.

Miller

M. J.

Pittman

C. W.

Feigh

K. M.

(2017, September 25–29). Next-generation human extravehicular spaceflight operations support systems development. Paper presented at the 68th International Astronautical Congress, Adelaide, Australia.

23.

NASA Space Flight Human-System Standard Volume 1, Revision A: Crew Health. (2015). (No. NASA-STD-3001, Vol. 1, pp. 1–83). Retrieved from https://standards.nasa.gov/standard/nasa/nasa-std-3001-vol-1

24.

Stanton

N. A.

Rafferty

L. A.

Salmon

P. M.

Revell

K. M. A.

McMaster

Caird-Daley

Cooper-Chapman

(2010). Distributed decision making in multihelicopter teams: Case study of mission planning and execution from a noncombatant evacuation operation training scenario. Journal of Cognitive Engineering and Decision Making, 4, 328–353.

25.

Woods

D. D.

Dekker

S. W. A.

(2000). Anticipating the effects of technological change: A new era of dynamics for human factors. Theoretical Issues in Ergonomics Science, 1, 272–282. doi:10.1080/14639220110037452

26.

Woods

D. D.

Sarter

N. B.

(2010). Capturing the dynamics of attention control from individual to distributed systems: The shape of models to come. Theoretical Issues in Ergonomics Science, 11(1–2), 7–28. doi:10.1080/14639220903009896

27.

Zave

Jackson

(1997). Four dark corners of requirements engineering. ACM Transactions on Software Engineering and Methodology, 6, 1–30.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB