Our research (reported in two parts) improved software for a NASA Mission Controller group for the International Space Station and provided evidence for a key factor we believed contributed to the expected improvement. This factor is the degree of alignment of the technology to the structure of the work it is intended to support, or its fitness-for-purpose. This paper, Part 1, reports our needs analysis and software redesign, which provide specific and more general contributions. The specific contribution was new prototype software for planning work of the Attitude Determination and Control Officer group, who schedules trajectory and orientation changes of the International Space Station, with its Russian counterparts. The general contribution was a new needs analysis method, product-document analysis, a general design-process benefit. Product document analysis is a method complementary to task analysis and work domain analysis. Our needs analysis method characterized the high-level structure required of acceptable plans in terms of the plan components and their relations and constraints. The redesigned software was better aligned with the structure of work, as captured by needs analysis. We discuss conditions when product-document analysis may be useful.
BeyerH.HoltzblattK. (1997). Contextual design: Defining customer-centered systems (1st ed.). San Francisco: Morgan Kaufmann.
2.
BillmanD.ArsintescuL.FearyM.LeeJ.SmithA.TiwaryR. (2011). Benefits of matching domain structure for planning software: The right stuff. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2521-2530). New York: ACM Press.
3.
BillmanD.ArsintescuL.FearyM.LeeJ. C.TiwaryR. (2015). Needs analysis and technology alignment evaluation: Evaluation case study of alternative software for controller planning work–Part 2. Journal of Cognitive Engineering and Decision Making. 9(2), 186-207.
4.
BillmanD.FearyM.SchreckenghostD.SherryL. (2010). Needs analysis: The case of flexible constraints and mutable boundaries. In Proceedings of the 28th of the international conference extended abstracts on human factors in computing systems (pp. 4597–4612). New York: ACM Press.
5.
BrillE.MarcusM. (1992). Automatically acquiring phrase structure using distributional analysis. Berkeley, CA: Association for Computational Linguistics. doi:10.3115/1075527.1075561
6.
ButlerK. A.ZhangJ. (2009). Design models for interactive problem-solving. In Proceedings of the 27th international conference extended abstracts on human factors in computing systems - CHI EA ‘09. Boston: ACM. doi:10.1145/1520340.1520659
7.
ButlerK. A.ZhangJ.EspositoC.BahramiA.HebronR.KierasD. (2007). Work-centered design: A case study of a mixed-initiative scheduler. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 747-756). Boston: ACM.
Defense Science Board Washington DC. (2012). Defense Science Board Task Force report: The role of autonomy in DoD systems (no. ADA566864). Retrieved from http://handle.dtic.mil/100.2/ADA566864
EgglestonR. G. (2003). Work-centered design: A cognitive engineering approach to system design. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 47(3), 263-267. doi:10.1177/154193120304700303
12.
EllisS. R. (2000). Collision in space. Ergonomics in Design: The Quarterly of Human Factors Applications, 8(1), 4-9.
13.
EmamK. E.KoruA. G. (2008). A replicated survey of IT software project failures. IEEE Software, 25(5), 84-90. doi:10.1109/MS.2008.107
FeighK. M.PritchettA. R. (2013). Requirements for effective function allocation: A critical review. Journal of Cognitive Engineering and Decision Making. doi:10.1177/1555343413490945
HarrisZ. S. (1951). Structural linguistics (1963 4th impression.). Chicago: University of Chicago.
18.
HofmannH. F.LehnerF. (2001). Requirements engineering as a success factor in software projects. IEEE Software, 18(4), 58-66. doi:10.1109/MS.2001.936219
19.
Institute of Medicine: Committee on Quality of Health Care in America. (2000). To err is human: Building a safer health system (KohnL. T.CorriganJ. M.DonaldsonM. S., Eds.). Washington, DC: National Academy Press. Retrieved from http://books.nap.edu/openbook.php?isbn=0309068371
20.
KierasD. (1996). Task analysis and the design of functionality. In TuckerA. (Ed.), Handbook of computer science and engineering (pp. 1401-1423). Boca Raton, FL: CRC Press.
21.
KirwanB.AinsworthL. K. (1992). A guide to task analysis. London: Taylor & Francis.
22.
LeeJ.BillmanD. (2011). Modeling performance differences across systems, tasks, and strategies. In CarlsonL.HölscherC.ShipleyT. (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 3489-3495). Austin, TX: Cognitive Science Society.
23.
LeffingwellD. (1997). Calculating the return on investment from more effective requirements management. American Programmer, 10(4), 13-16.
24.
LevesonN. (1995). SafeWare: System safety and computers. Reading, MA: Addison-Wesley.
25.
MarisK.DulacN.LevesonN. (2004). Beyond normal accidents and high reliability organizations: The need for an alternative approach to safety in complex systems (ESD Symposium, Massachusetts Institute of Technology). Retrieved from sunnyday.mit.edu/papers/HRO-final.doc
26.
McCurdyM. (2009). Planning tools for Mars surface operations: Human-Computer Interaction lessons learned. IEEE, 1-12. doi: 10.1109/AERO.2009.4839639
27.
McCurdyM.LudowiseM.MarquezJ.LiJ. (2009). Space human factors engineering report: Crew scheduling lessons learned. Moffett Field, CA: NASA Ames Research Center.
28.
McLeodL.MacDonellS. G. (2011). Factors that affect software systems development project outcomes. ACM Computing Surveys, 43(4), 1-56. doi:10.1145/1978802.1978803
29.
NaikarN.HopcroftR.MoylanA. (2005). Work domain analysis: Theoretical concepts and methodology. Melbourne, Australia: Australian Defence Science and Technology Organisation.
30.
NasirM. H. N.SahibuddinS. (2011). Critical success factors for software projects: A comparative study. Scientific Research and Essays, 6(10), 2174-2186.
31.
PewR. W.MavorA. S. (Eds.). (2007). Human-system integration in the system development process. Washington, DC: National Research Council, The National Academies Press.
32.
Pinheiro da SilvaP.PatonN. (2000). UMLi: The Unified Modeling Language for Interactive Applications. In EvansA.KentS. (Eds.), Proceedings «UML» 2000,LNCS, Vol. 1939. New York: Springer-Verlag.
33.
ProcaccinoJ. D.VernerJ. M.LorenzetS. J. (2006). Defining and contributing to software development success. Communications of the ACM, 49(8), 79-83. doi: 10.1145/1145287.1145291
34.
RothE. M. (2008). Uncovering the requirements of cognitive work. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(3), 475-480. doi: 10.1518/001872008X288556
35.
SarsfieldL. P.StanleyW. L.LebowC. C.EttedguiE.HenningG. (2000). Safety in the skies: Personnel and parties in NTSB aviation accident investigations: Master volume (no. MR-1122/1-ICJ). Retrieved from http://www.rand.org/pubs/monograph_reports/MR1122z1.html
36.
SavolainenP.AhonenJ. J.RichardsonI. (2012). Software development project success and failure from the supplier’s perspective: A systematic literature review. International Journal of Project Management, 30(4), 458-469. doi: 10.1016/j.ijproman.2011.07.002
SherryL.FearyM. (2004). Task design and verification testing for certification of avionics equipment. In Proceedings of the 23rd Digital Avionics Systems Conference (Vol. 2, pp. 101-110). Piscataway, NJ: IEEE.
39.
SherryL.MedinaM.FearyM.OtikerJ. (2008). Automated tool for task analysis of NextGen automation. IEEE, 1-9. doi: 10.1109/ICNSURV.2008.4559185
40.
VallacherR. R.WegnerD. M. (1987). What do people think they’re doing? Action identification and human behavior. Psychological Review, 94(1), 3-15. doi: 10.1037/0033-295X.94.1.3
