Human–Automation Interaction in Self-Organizing Sociotechnical Systems

Abstract

Models of human–automation interaction concerned with “who should do what” or “who should be responsible for what” fail to appreciate how complex cognitive work is carried out in sociotechnical systems. Such systems, which have complex problem spaces with high levels of instability, uncertainty, and unpredictability, are necessarily self-organizing. In these systems, actors’ behaviors and structures are closely interconnected, or correlated, and vary continuously in ways that are fitted, or adapted, to the demands of a constantly evolving work environment. To preserve the capacity of sociotechnical systems for self-organization, and thus their capacity for dealing with the complexity of their work environments, the design of human–automation interaction must be concerned fundamentally with identifying the limits on “who can do what” and providing constrained flexibility in the workplace.

David Kaber (2018 [this issue]), in his article in this issue of the Journal of Cognitive Engineering and Decision Making, considers a variety of issues relating to models of human–automation interaction in complex systems, including the need for specifications of what the human and automation will do, and how, for the purposes of effective design. In this commentary, I argue that models of human–automation interaction that focus on “who should do what” or on “who should be responsible for what” fail to appreciate how complex cognitive work is carried out in sociotechnical systems, which are necessarily self-organizing. To this end, I discuss some of the essential characteristics of sociotechnical systems, empirical observations of how work unfolds in these contexts, and the kinds of models that are needed for creating effective designs for self-organizing systems.

Some of the most demanding aspects of performing cognitive work in sociotechnical systems is that they tend to present very large and complex problem spaces characterized by high degrees of instability, uncertainty, and unpredictability (Naikar, 2013; Naikar & Brady, in press). Workers must contend with dynamic operating conditions, in which the problems, demands, and pressures they are faced with change or evolve constantly. For example, the particular challenges posed by a wildfire to emergency management workers may fluctuate widely depending on the current and anticipated weather conditions or population sizes and infrastructure in areas to which the fire is spreading. Furthermore, workers must deal with considerable uncertainty, especially in relation to the accuracy or completeness of the information available to them. In the naval domain, for instance, imperfections in acoustics sensors for detecting objects submerged underwater may mean that the true situation can never be established with complete certainty. Finally, adding to this complexity is the inherent unpredictability of the work requirements. Workers must contend with events that have not been—and cannot be—foreseen or specified fully by analysts or designers (e.g., Leveson, 1995; Perrow, 1984; Rasmussen, 1968a, 1968b, 1969; Reason, 1990; Vicente, 1999), such as a new kind of military threat (Herzog, 2011; Reich, Weinstein, Wild, & Cabanlong, 2010), an unexpected reaction of a patient to an anesthetic during surgery (Hoppe & Popham, 2007), or an unforeseen chain of supplier collapses in the wake of a natural disaster (Park, Hong, & Roh, 2013).

These fundamental characteristics of sociotechnical systems make adaptation necessary for their viability. Workers cannot rely solely on executing well-established procedures or trained task sequences or on recognizing suitable solutions from their prior experiences, even if they have extensive know-how (Rasmussen, 1986; Rasmussen, Pejtersen, & Goodstein, 1994; Vicente, 1999). Instead, workers must be capable of continuously and reliably adapting their actions to the evolving nature of the problem, sometimes exercising significant ingenuity or creativity.

A variety of empirical studies demonstrate that adaptations in sociotechnical systems can be observed at the levels of the behaviors of individual actors and the structures of multiple actors (Naikar & Brady, in press; Naikar & Elix, 2016). In other words, workers in these systems function within groups or teams, or within a social context, and they adapt not only their individual behaviors but also their structures, or organization, in line with the evolving situation (e.g., Bigley & Roberts, 2001; Bogdanovic, Perry, Guggenheim, & Manser, 2015; Rochlin, LaPorte, & Roberts, 1987). For example, in a field study of a widespread approach to emergency management in the United States (Bigley & Roberts, 2001), some of the kinds of behavioral adaptations that were observed included improvisations in the use of tools, such as those available on a fire truck at the scene of an emergency, and adjustments in standard routines, such as those for “hose laying” or “ladder throwing.” Moreover, sometimes the adaptations included departures from rules, so that workers were directly breaching standard operating procedures. For instance, one procedure prohibits firefighting teams from approaching a fire from opposite positions, as one group can push the fire into another. However, firefighters discussed a situation in which “opposing hose streams” was in fact used as the primary tactic.

Field studies of sociotechnical systems also provide strong evidence of structural adaptation. As an example, Rochlin et al. (1987) found that the work organization on naval aircraft carriers shifts spontaneously between formal and informal structures. The formal structure, that which is documented on paper, is rigid, hierarchical, and centralized, and typically governs operations on a ship. However, during complex operations, a different kind of organizational structure is exhibited. This structure is informal in that it is not officially documented, and it is flat and distributed. For instance, faced with significant time constraints, lower-ranked personnel can make critical decisions without the approval of officials with higher rankings. The work organization on the ship is flexible in that there is no prespecified plan for when the shifts between formal and informal structures will occur. Moreover, the informal structure that is adopted on any one occasion is emergent. That is, there is no single, fixed mapping between people and roles. Instead, the mapping changes with the circumstances.

In this context, the concept of self-organization is important (Naikar & Brady, in press; Naikar, Elix, Dâgge, & Caldwell, 2017). This concept can provide an explanation of how behavioral and structural adaptations in sociotechnical systems can be achieved spontaneously, continuously, and relatively seamlessly, consistent with the observations of field studies (e.g., Bigley & Roberts, 2001; Bogdanovic et al., 2015; Rochlin et al., 1987). Specifically, field studies demonstrate that new organizational structures may be observed in sociotechnical systems that are not planned a priori, centrally coordinated, or imposed by external agents but instead appear to be a spontaneous response of the system itself to challenges posed by its environment. The self-organization concept (e.g., Fuchs, 2004; Haken, 1988; Heylighen, 2001; Hofkirchner, 1998) suggests that a system’s structure may constrain its response in ways that are unsuitable or ineffective when particular circumstances are encountered (Figure 1). However, in responding to the local conditions, individual, interacting actors may engage in spontaneous behaviors from which novel structures emerge that enable the system to respond appropriately to the circumstances. A new structure may be suitable for a time, constraining and enabling behaviors in ways that are appropriate to the circumstances—until the situation changes—and the spontaneous actions of individual, interacting actors result in further structural changes.

Figure 1.

Social self-organization. Adapted from Hofkirchner (1998).

Both actors’ structures and behaviors, then, may evolve continuously to deal with the challenges posed by the environment. Moreover, although this self-organizing process may not be perfect, it may be relatively seamless, particularly in well-established systems, and in any case, it is necessary. Significant events occur too quickly and conditions are too unstable, uncertain, and unpredictable for a priori planning, centralized coordination, or external intervention to provide feasible strategies consistently. Instead, flexibility and adaptation within the system are required to resolve—in time and in context of the local operating conditions—the “proper, immediate balance” between the system’s safety and productivity goals (Rochlin et al., 1987, pp. 83–84).

The phenomenon of self-organization has significant implications for the design of human–automation interaction in sociotechnical systems. Designs based on normative or prescriptive models of “who should do what” may compromise performance by limiting the capacity of a system for self-organization, and thus its capacity for dealing with instability, uncertainty, and unpredictability. In addition, models of human–automation interaction focused either at the level of behavior or structure will not be sufficient. Designs based predominantly on the behaviors of individual actors may have unintended, negative consequences for the structural relationships between these actors, and the converse is also true. Instead, human–automation interaction must be conceived in the context of both structure and behavior—in an integrated manner—and thus in the context of self-organizing systems.

A suitable starting point for achieving these design objectives is offered by the diagram of work organization possibilities (or WOP diagram), a modeling tool recently proposed as an extension to the cognitive work analysis framework, specifically the social organization and cooperation dimension (Naikar & Elix, 2016). This tool models the structural possibilities of multiple actors and the behavioral opportunities of individual actors within a single, integrated representation in a way that is consistent with the phenomenon of self-organization (Figure 2; Naikar et al., 2017). The set of work organization possibilities encapsulated in such a model is independent of specific circumstances, so that the model is relevant to recurring situations or novel or unforeseen events. Thus this tool has the potential to foster the development of human–automation interaction designs that do not limit artificially a system’s capacity for self-organization and therefore its capacity for dealing with a dynamic and ambiguous work environment.

Figure 2.

Relationship of the diagram of work organization possibilities to self-organization (cf. Figure 1).

Specifically, a key design intent is to preserve the inherent structural possibilities of multiple actors and behavioral opportunities of individual actors, so that new or different temporal, spatial, or functional structures can emerge from the spontaneous behaviors of interacting actors, constrained by a set of boundary conditions for safe and productive performance, consistent with the cognitive work analysis framework. Notably, the actors in a WOP diagram may encompass both human workers and automated agents, so that the emergent structures may encompass a mix of human and machine actors. Ultimately, such a design approach reveals the limits on “who can do what” rather than specifying “who should do what,” which can be conceived only in the context of events that are known or can be anticipated in detail.

The boundaries or limits on action are established by analyzing the organizational constraints, or the relationships between the action possibilities afforded by the work context, or the environment in which actors are situated, and the actors themselves (Naikar et al., 2017). As a result, the WOP diagram simultaneously reveals the behavioral opportunities of individual actors and the structural possibilities of multiple actors. The structural possibilities are interconnected with actors’ behavioral opportunities in a manner that accommodates the observation that, in self-organizing systems, the structural forms of multiple actors emerge from the flexible actions of individual, interacting actors while at the same time constraining or enabling their behaviors, in ways that are fitted to the circumstances (cf. Figure 1). Thus, it is proposed that designs based on this diagram may facilitate the emergence of novel temporal, spatial, or functional structures from the flexible behaviors of individual, interacting actors, such that the system has greater capacity for adapting to a constantly evolving work environment.

A particular design goal within this approach is to facilitate effective interrelationships between human and machine actors as a central means for promoting emergence and adaptation of the system to the work environment. The relational possibilities of human and machine actors must be comprehended in view of each actor’s full, although meaningfully constrained, spaces of opportunities for action. The specific intent is to provide actors with pathways for interaction that can preserve each actor’s degrees of freedom for behavior within the specified boundaries of effective performance, as revealed by means of a WOP diagram.

Considerable further research, however, is necessary to realize the ideals of this framework. First, empirical studies to develop a deeper appreciation of the nature of self-organization in sociotechnical systems, especially those with both human and sophisticated automated agents, are needed. Second, techniques for analyzing and modeling self-organizing sociotechnical systems with both human and machine actors require attention. Here, it is suggested that the WOP diagram and, more broadly, the concepts of cognitive work analysis provide a suitable starting point, but further research in this area would be worthwhile. Finally, design strategies for promoting self-organization in sociotechnical systems require further investigation. Such investigations should consider means for enriching the interactions needed between human and machine actors, which may not be of precisely the same kinds that are natural among human actors, if the respective capabilities of those actors are to be exploited successfully.

This research on human–automation interaction will contribute to the overarching design goal of creating integrated system designs (Naikar & Elix, 2016) that are compatible with how complex cognitive work is carried out in sociotechnical systems. Such systems are necessarily self-organizing, at least during complex, nonroutine operations, so that specifications of “who should do what” are likely to lead to uncompromising designs that do not facilitate and, worse still, deliberately inhibit flexibility in the workplace, forcing workers to improvise in an ad hoc manner, which could lead to unsafe or unproductive outcomes. Instead, actors must be supported systematically in adapting both their structures and behaviors, in an integrated manner, if we are to recognize that self-organization occurs in sociotechnical systems and that this phenomenon is important for dealing with a dynamic and ambiguous work environment. This design goal requires a change in perspective in which the emphasis is placed squarely on revealing the limits on “who can do what” and promoting constrained flexibility in the workplace.