To validate prior agent-based models, public disclosure of model code is necessary, but not sufficient. Conceptual model replication, involving independent reproduction of a model without referring to the originator’s code, is crucial for detecting nonconformity between the publication text and the model implemented. We frame and evaluate four scenarios to expose the shortfalls of replication efforts that use the original study’s program code and neglect the published model description. We identify, evaluate, and offer recommendations regarding disparities between the text and the program code that generated the results reported in March’s classic study of exploration and exploitation in organizational learning. Based on our experience replicating his model, we suggest a two-step method of model verification—beginning with replicating the model from the published description, then turning to the program code of the original implemented model to account for divergent results. Exclusive reliance on either the published model description or the original program code does disservice to theory advancement, because disparities between the original conceptual and implemented models go unaddressed.
AdnerRPólosLRyallM, et al. (2009) The case for formal theory. Academy of Management Review34(2): 201–208.
2.
AxtellRAxelrodREpsteinJM, et al. (1996) Aligning simulation models: A case study and results. Computational & Mathematical Organization Theory1(2): 123–141.
3.
BendorJMoeTMShottsKW (2001) Recycling the garbage can: An assessment of the research program. American Political Science Review95(1): 169–190.
4.
BlaschkeSSchoenebornD (2006) The forgotten function of forgetting: Revisiting exploration and exploitation in organizational learning. Soziale Systeme11(2): 99–119.
5.
BurtonRM (2003) Computational laboratories for organization science: Questions, validity and docking. Computational & Mathematical Organization Theory9(2): 91–108.
6.
CarleyKM (1995) Computational and mathematical organization theory: Perspective and directions. Computational & Mathematical Organization Theory1(1): 39–56.
7.
CarleyKMSvobodaDA (1996) Modeling organizational adaptation as a simulated annealing process. Sociological Methods & Research25(1): 138–168.
8.
ChandaSS (2017) Inferring final organizational outcomes from intermediate outcomes of exploration and exploitation: The complexity link. Computational & Mathematical Organization Theory23(1): 61–93.
9.
ChandaSSRayS (2015a) Optimal exploration and exploitation: The managerial intentionality perspective. Computational & Mathematical Organization Theory21(3): 247–273.
10.
ChandaSSRayS (2015b) Formal theory development by computational simulation modeling: A tale of two philosophical approaches. Decision42(3): 251–267.
11.
ChandaSSRaySMcKelveyB (2018) The continuum conception of exploration and exploitation: An update to March’s theory. M@n@gement21(3): 1050–1079.
12.
CohenMMarchJOlsenJ (1972) A garbage can model of organizational choice. Administrative Science Quarterly17(1): 1–25.
13.
DavisJPEisenhardtKMBinghamCB (2007) Developing theory through simulation methods. Academy of Management Review32(2): 480–499.
DonohoDL (2010) An invitation to reproducible computational research. Biostatistics11(3): 385–388.
16.
EdmondsBHalesD (2003) Replication, replication and replication: Some hard lessons from model alignment. Journal of Artificial Societies and Social Simulation6. Available at: http://jasss.soc.surrey.ac.uk/6/4/11.html (accessed 20 December 2017).
17.
FangCLeeJSchillingMA (2010) Balancing exploration and exploitation through structural design: The isolation of subgroups and organizational learning. Organization Science21(3): 625–642.
18.
FiorettiG (2013) Agent-based simulation models in organization science. Organizational Research Methods16(2): 227–242.
19.
GalánJMIzquierdoLRIzquierdoSS, et al. (2009) Errors and artefacts in agent-based modelling. Journal of Artificial Societies and Social Simulation 12. Available at: http://jasss.soc.surrey.ac.uk/12/1/1.html (accessed 20 December 2017).
20.
GavettiGGreveHRLevinthalHA, et al. (2012) The behavioral theory of the firm: Assessment and prospects. The Academy of Management Annals6(1): 1–40.
21.
HalesDRouchierJEdmondsB (2003) Model-to-model analysis. Journal of Artificial Societies and Social Simulation 6(4). Available at: http://jasss.soc.surrey.ac.uk/6/4/5.html (accessed 20 December 2017).
22.
HarrisonJRLinZCarrollGR, et al. (2007) Simulation modeling in organizational and management research. Academy of Management Review32(4): 1229–1245.
23.
HollandJH (1975) Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. Ann Arbor, MI: University of Michigan Press.
24.
InceDCHattonLGraham-CummingJ (2012) The case for open computer programs. Nature482(7386): 485–488.
25.
KaneGCAlaviM (2007) Information technology and organizational learning: An investigation of exploration and exploitation processes. Organization Science18(5): 796–812.
26.
KauffmanSWeinbergerE (1989) The NK Model of rugged fitness landscapes and its application to the maturation of the immune response. Journal of Theoretical Biology141(2): 211–245.
27.
KimTRheeM (2009) Exploration and exploitation: Internal variety and environmental dynamism. Strategic Organization7(1): 11–41.
28.
LevinthalDA (1997) Adaptation on rugged landscapes. Management Science43(7): 934–950.
29.
MaguireSMcKelveyBMirabeauL, et al. (2006) Complexity science and organization studies. In: CleggSRHardyCLawrenceTB, et al. (eds) The SAGE Handbook of Organization Studies, 2nd edn.London: SAGE, pp. 165–214.
30.
MarchJG (1991) Exploration and exploitation in organizational learning. Organization Science2(1): 71–87.
31.
MidgleyDMarksRKunchamwarD (2007) Building and assurance of agent-based models: An example and challenge to the field. Journal of Business Research60(8): 884–893.
32.
MillerKD (2015) Agent-based modeling and organization studies: A critical realist perspective. Organization Studies36(2): 175–196.
33.
MillerKDMartignoniD (2016) Organizational learning with forgetting: Reconsidering the exploration-exploitation tradeoff. Strategic Organization14(1): 53–72.
34.
MillerKDZhaoMCalantoneRJ (2006) Adding interpersonal learning and tacit knowledge to March’s exploration-exploitation model. Academy of Management Journal49(4): 709–722.
35.
MuldoonR (2007) Robust simulations. Philosophy of Science74(5): 873–883.
36.
NosekBASpiesJRMotylM (2012) Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science7(6): 615–631.
37.
PosenHELevinthalDA (2012) Chasing a moving target: Exploitation and exploration in dynamic environments. Management Science58(3): 587–601.
38.
RodanS (2005) Exploration and exploitation revisited: Extending March’s model of mutual learning. Scandinavian Journal of Management21(4): 407–428.
39.
SchillingMAFangC (2014) When hubs forget, lie, and play favorites: Interpersonal network structure, information distortion, and organizational learning. Strategic Management Journal35(7): 974–994.
40.
TsangEWKKwanKM (1999) Replication and theory development in organizational science: A critical realist perspective. Academy of Management Review24(4): 759–780.
41.
VancouverJBWeinhardtJM (2012) Modeling the mind and the milieu: Computational modeling for micro-level organizational researchers. Organizational Research Methods15(4): 602–623.
42.
WildenRHohbergerJDevinneyTM, et al. (2018) Revisiting James March (1991): Whither exploration and exploitation?Strategic Organization16(3): 352–369.
43.
WilenskyURandW (2007) Making models match: Replicating an agent-based model. Journal of Artificial Societies and Social Simulation10(4). Available at: http://jasss.soc.surrey.ac.uk/10/4/2.html (accessed 20 December 2017).
