Limits of Automata—Then and Now: Challenges of Architecture,Brittleness,and Scale

Abstract

Two trajectories underway transform human systems. Processes of growth/complexification have accelerated as stakeholders seek advantage from advances in connectivity/autonomy/sensing. Surprising empirical patterns also arise—puzzling collapses of critical valued services occur against a background of growth. In parallel, new scientific foundations have arisen from diverse directions explaining the observed anomalies and breakdowns, highlighting basic weaknesses of automata regardless of technology. Conceptual growth provides laws, theorems, and comprehensive theories that encompass the interplay of autonomy/people and complexity/adaptation across scales. One danger for synchronizing the trajectories is conceptual lag as researchers remain stuck in stale frames unable to keep pace with transformative change. Any approach that does not either build on the new conceptual advances—or provide alternative foundations—is no longer credible to match the scale and stakes of modern distributed layered systems and overcome the limits of automata. The paper examines longstanding challenges by contrasting progress then as the trajectories gathered steam, to situation now as change has accelerated.

Keywords

autonomy scale complexity < topics adaptive capacity joint cognitive system brittleness limits of autonomy

Get full access to this article

View all access options for this article.

References

Carlson

J. M.

Doyle

J. C.

(2000). Highly optimized tolerance: Robustness and design in complex systems. Physics Review Letters, 84, 2529–2532. https://doi.org/10.1103/PhysRevLett.84.2529

Dunbar

(1993). Concept discovery in a scientific domain. Cognitive Science, 17(3), 397–434. https://doi.org/10.1207/s15516709cog1703_3

Farjadian

A. B.

Thomson

Annaswarmy

A. M.

Woods

D. D.

(2021). Resilient flight cotnrol: An architecture for human supervision of automation. IEEE Transactions on Control Systems Technology, 29(1), 29–42. https://doi.org/10.1109/TCST.2019.2959542

Hollnagel

Woods

D. D.

(1983). Cognitive systems engineering: New wine in new bottles. International Journal of Man-Machine Studies, 18(6), 583–600. https://doi.org/10.1016/S0020-7373(83)80034-0

Hutchins

(1995). Cognition in the wild. MIT Press.

Johnson

Bradshaw

J. M.

Feltovich

P. J.

Jonker

C. M.

van Riemsdijk

M. B.

Sierhuis

(2014). Coactive design: Designing support for interdependence in joint activity. Journal of Human-Robot Interaction, 3(1), 43–69. https://doi.org/10.5898/JHRI.3.1.Johnson

Klein

Feltovich

P. J.

Bradshaw

J. M.

Woods

D. D.

(2005). Common ground and coordination in Joint activity. In Organizational Simulation (pp. 139–184). John Wiley & Sons.

Nakahira

Liu

Sejnowski

T. J.

Doyle

J. C.

(2021). Diversity-enabled sweet spots in layerred architectures and sped-accuracy trade-offs in sensorimotor control. Proceedings of the National Academy of Science PNAS, 118(22). https://doi.org/10.1073/pnas.1916367118

Peierls

(1992). Where Pauli made his ‘wrong’ remark. Physics Today, 45(12), 112. https://doi.org/10.1063/1.2809934

10.

Roth

E. M.

Bennett

K. B.

Woods

D. D.

(1987). Human Interaction with and “intelligent” machine. International Journal of Man-Machine Studies, 27(5–6), 479–525. https://doi.org/10.1016/S0020-7373(87)80012-3

11.

Roth

E. M.

Woods

D. D.

(1988). Aiding human performance: I. Cognitive analysis. Le Travail Humain, 51(1), 39–64. https://www.jstor.org/stable/40657481

12.

Wiener

(1950). the human use of human beings: Cybernetics and society. Doubleday.

13.

Woods

D. D.

(1985). Cognitive Technologies: The design of joint human-machine cognitive systems. AI Magazine, 6(4), 86–92. https://doi.org/10.1609/aimag.v6i4.511

14.

Woods

D. D.

Ed. (2017). STELLA report from SNAFU catchers workshop on coping with complexity (March). SNAFU Catchers Consortium. https://stella.report

15.

Woods

D. D.

(2018). The theory of graceful extensibility. Environment Systems and Decisions, 38, 433–457. https://doi.org/10.1007/s10669-018-9708-3

16.

Woods

D. D.

(2024). The science and pragmatics of resilience engineering through the lens of complexification. In Series on Foundations of Resilience Enginnering (March). Cognitive Systems Engineering Laboratory. https://www.youtube.com/watch?v=R0BEQETj9D0&t=36s

17.

Woods

D. D.

Alderson

D. A.

(2021). Progress toward Resilient Infrastructures: Are we falling behind the pace of events and changing threats? Journal of Critical Infrastructure Policy, 2(2), 5–18. https://doi.org/10.18278/jcip.2.2.2

18.

Woods

D. D.

Hollnagel

(2006). Joint cognitive systems: Patterns in cognitive systems engineering. Taylor & Francis.

19.

Woods

D. D.

Roth

E. M.

(1988). Aiding human performance: II. From cognitive analysis to support systems. Le Travail Humain, 51(2), 139–171. https://www.jstor.org/stable/40657488

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

3.91 MB