Software design teams need methods to evaluate plans as part of agile development processes. The premortem is a cognitive, structured, analytic technique that supports team plan evaluation and re-planning. Few empirical or longitudinal studies involving premortems exist. Ten game development teams (n=68 members) conducted premortems early on during a year-long game development project. Teams provided initial ratings of their game design plan, then conducted a premortem, and revised their plans. In the premortems, teams identified 17.8 unique reasons on average for project failure and 16.7 mitigations to those failures. Reasons for project failure focused mainly on game design execution, team communication, and game complexity (e.g., too many levels, branching). While most teams identified solutions for these challenges, surprisingly few teams revised their plans to scale back the game design complexity.
BettinB.SteelmanK. S.WallaceC.PontiousD.VeinottE. S. (2022, September). Identifying and Addressing Risks in the Early Design of a Sociotechnical System through Premortem. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 66, No. 1, pp. 1514-1518). Los Angeles, CA: SAGE.
2.
BuehlerR.GriffinD.RossM. (1994). Exploring the "Planning fallacy”: Why people underestimate their task completion times. Journal of Personality and Social Psychology, 67(3), 366-381.
3.
CardA.WardJ.ClarksonP. J. (2012). Successful risk assessment may Not always lead to successful risk control: a systematic literature review of risk control after root cause analysis. Journal of Healthcare risk management, 31, 6-12.
4.
DybåT.DingsøyrT. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10), 833-859. doi:10.1016/j.infsof.2008.01.006
5.
DykstraJ.ShortridgeK.MetJ.HoughD. (2022, September). Opportunity Cost of Action Bias in Cybersecurity Incident Response. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 66, No. 1, pp. 1116-1120). Sage CA: Los Angeles, CA: SAGE Publications.
6.
GallopD.WillyC.BischoffJ. (2016). How to catch a black swan: Measuring the benefits of the premortem technique for risk identification. Journal of Enterprise Transformation, 6(2), 87-106.
HunterA.RandallT.ColbertH. (2018, September). Intuitive Insights For Course Of Action Development. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 62, No. 1, pp. 212-216). Sage CA: Los Angeles, CA: SAGE Publications.
9.
JoshiA.MehendaleA. (2021). Project Pre-Mortem Using Prospective Hindsight: An Unexplored Tool to Address Healthcare Projects Implementation Barriers. Indian Journal of Community Health, 33, 415-416.
10.
KleinG. (2007, September). Performing a project premortem. Harvard Business Review, 85(9), 18-19.
11.
KohnN.PaulusP.ChoiY. (2011). Building on the ideas of others: An examination of the idea combination process. Journal of Experimental Social Psychology, (47), 554-561
12.
MeissnerP.WulfT. (2013). Cognitive benefits of scenario planning: Its impact on biases and decision quality. Technological Forecasting and Social Change80, 801-814.
13.
MillsJ. K.McKimmJ. (2017). Pre-empting project failure by using a pre-mortem. British Journal of Hospital Medicine, 78(10), 584-585.
14.
MitchellD. J.RussoJ. E.PenningtonN. (1989). Back to the future: Temporal perspective in the explanation of events. Journal of Behavioral Decision Making, 2, 25-38
15.
MullenB.JohnsonC.SalasE. (1991). Productivity loss in brainstorming groups:A meta-analytic integration. Basic and applied social psychology, 12(1), 3-23.
16.
MullinixG.GrayO.ColadoJ.VeinottE.LeonardJ.PapautskyE. L.… FlachJ. (2013, September). Heuristica: Designing a serious game for improving decision making. In 2013 IEEE international games innovation conference (IGIC) (pp. 250-255). IEEE.
17.
OlsonJ. S.TeasleyS. (1996, November). Groupware in the wild: Lessons learned from a year of virtual collocation. In Proceedings of the 1996 ACM conference on Computer supported cooperative work (pp. 419-427). ACM.
18.
ParnesS. J.MeadowA. (1959). Effects of ‘brainstorming’ instructions on creative problem solving by trained and untrained subjects. Journal of Educational Psychology, 50(4), 171-176.
19.
PeabodyM.VeinottE. (2017). Focus shift: Differences in reasons generated using Premortem and Worst Case Scenario plan evaluation methods. Naturalistic Decision Making, 259-261.
20.
PutmanV. L.PaulusP. B. (2009). Brainstorming, brainstorming rules, and decision making. Journal of Creative Behavior, 43, 23-39.
21.
SunsteinC. R. (2009). Worst-case scenarios. Harvard Press.
22.
TverskyA.KahnemanD. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and uncertainty, 5, 297-323.
23.
VeinottB.KleinG.WigginsS. (2010). Evaluating the effectiveness of the premortem technique on plan confidence. 7th International ISCRAM Conference. Seattle.
