How Can Authorities Support Distributed Improvisation During Major Crises? A Study of Decision Bottlenecks Arising During Local COVID-19 Vaccine Roll-Out

Abstract

Despite the increased importance attributed to distributed improvisation in major crises, few studies investigate how central authorities can promote a harmonic, coordinated national response while allowing for distributed autonomy and improvisation. One idea implicit in the literature is that central authorities could help track and tackle common decision bottlenecks as they emerge across “improvising” local authorities as a result of shared, dynamic external constraints. To explore this idea we map central functions needed to roll-out vaccines to local populations and identify and classify bottlenecks to decision-making by local authorities managing COVID-19 vaccine roll-out in Norway. We found five bottlenecks which emerged as vaccine roll-out progressed, three of which could feasibly have been addressed by changing the local authorities’ external constraints as the crisis developed. While the national crisis response strategy clearly allowed for distributed improvisation, our overall findings suggest that there is potential for central authorities to address external constraints in order to ease common bottlenecks as they emerge across local authorities responding to the crisis. More research is to explore alternative centralized response strategies and assess how well they effectively balance centralized and distributed control. The study contributes to the growing literature examining the interaction between local and centralized response in crisis management.

Keywords

COVID-19 vaccine cognitive systems engineering governance cognitive task analysis decision-making local authorities central authorities

Introduction

Major public crises require coordinated responses from authorities at multiple levels. The COVID-19 pandemic is a salient example, leading several to question whether international, national, regional, and local authorities would be able to coordinate their activities to roll-out vaccine effectively and fairly at unprecedented scale (EU, 2021; Baharmand et al., 2021; Mills & Salisbury, 2021). Authorities in many countries faced a changing, uncertain situation involving different vaccine types and suppliers, complex networks of government, and health workers under strain (Engel et al., 2022; Guttieres et al., 2022; Weintraub et al., 2021). Managing such crises requires that authorities strike a dynamic balance between the centralized control needed for a harmonized national response and the distributed autonomy needed to adapt effectively to local situations. The current article studies this issue through the eyes of decision-makers involved in local COVID-19 vaccine roll-out in Norway.

Centralized Versus Distributed Crisis Control

Major crises involve time pressure, uncertainty, high stakes, and stakeholders with diverse perspectives (Hannah et al., 2009; Mendonça, 2007). The increasing scale and complexity of modern crises has led to the concept of the transboundary crisis, which traverses geographical boundaries and policy areas and requires multi-level actors with conflicting responsibilities to work together to develop and implement new solutions (Ansell et al., 2010). Faced by such crises, authorities rely on centralized preparedness to coordinate a rapid and effective response at scale. For optimal response, however, they must also shape the conditions for distributed improvisation, which allows for local-level adaptations to unforeseen situations, surprises, and local variations (Ley et al., 2012; Mendonça, 2007). Being closer to the “sharp end,” local actors are well positioned to understand adverse situations, and respond by selecting from appropriate resources and actions. They can apply new insights as they see the effects of their actions – a sort of “muddling through” that can outperform detailed centralized planning (e.g., Flach & Voorhorst, 2020; Lengnick-Hall & Beck, 2005; Ley et al., 2012; Lindblom, 1959; Vicente, 1999). Compared to hierarchical teams, networked teams have also been found to share more knowledge and resolve challenges faster and more accurately in complex scenarios (Schraagen et al., 2010). Given an increasing emphasis on the importance of distributed improvisation, central authorities may need to develop the role that they play in major crisis response. This concerns both how to develop centralized rules, plans, and procedures that do not constrain adaptative local responses, and how to ensure a harmonic, coordinated national response alongside distributed improvisation (Moorkamp et al., 2020).

In the field of cognitive systems engineering, the challenge of balancing concentrated and distributed action is recognized in accounts of laws of cognitive work and the concept bounded effectiveness (Hoffman & Woods, 2011; Woods, 2002). While autonomous local actions are often more effective, effectiveness is bounded since greater distribution of autonomy by central authorities increases the efforts and resources needed to harmonize distributed actions, for instance, by establishing a shared language among local authorities, explaining varying perspectives, or developing agreement on how to deal with conflicting goals. To limit the extra costs of coordinating distributed centers of control, there is a need to balance the extent of a local, flexible response with the more efficient coordination afforded by centralized response (Hoffman & Woods, 2011).

Common Bottlenecks Emerge During Crisis Response

The fields of resilience engineering and naturalistic decision-making also support the effectiveness of local improvisation as part of successful crisis response (e.g., Hollnagel et al., 2006; Klein, 2017). Resilience is improved if people in key roles have an impressive range of response repertoires, enabling them to adapt successfully to developing situations. Selection of skillful responses requires that people can recognize and negotiate different constraints in the problem space in order to achieve their goals effectively (Cook & Nemseth, 2006; Naikar, 2013; Woods & Hollnagel, 2006). While certain constraints will be specific to the unique situations of local actors, distributed local decision-makers will also share external constraints as they respond to national crises (Wilson & Game, 2011). If such decision-makers try to tackle the same problem by negotiating their shared constraints independently of each other, inefficiencies can arise in crisis response; where the resulting local strategies vary in quality or effectiveness, public perceptions of unfairness can arise (Boin, 2019). In transboundary crises like the COVID-19 pandemic, shared external constraints can also change as the crisis develops; central authorities reformulate priorities, resource levels change, and so on (Christensen & Lægreid, 2020). The shared challenges or “common bottlenecks” that dynamic external constraint sets present will often not emerge until response progresses across local actors (e.g., Flach & Voorhorst, 2020; Woods, 2003). The challenge for central authorities is how to engage with local authorities to detect and address common response bottlenecks as they emerge during the crisis, so that solutions might be coordinated.

Coordination Emerges During Crisis Response

Though there has been relatively little focus on the dynamics of management across levels and organizations, the literature on crisis management also finds that improvisation, communication, and coordination are central aspects of effective response to crises (Williams et al., 2017). According to this literature, actors improvise not only to make decisions, but to mobilize resources and to communicate and coordinate to establish order. The literature on organizational resilience also emphasizes the importance of enhancing improvisation, coordination, and adaptive communication processes for successful crisis response, in this case across teams, disciplines, and organizational levels (Raetza et al., 2021). Here, authors recognize a need to understand developing patterns in communication and coordination that form part of a resilient response to dynamic crises involving distributed multiple actors, teams, and organizations (Lengnick-Hall & Beck, 2009, 2016). Finally, the literature on multi-team systems finds that coordination is needed to allow teams to attend to intra-team goals while ensuring their actions are cohesive with overarching system goals (Brown et al., 2021). If teams are not familiar with each other or the situation being managed, component teams must engage proactively to establish coordinated actions (Smith-Jentsch et al., 2009). Thus, coordination can emerge as part of local improvised responses, suggesting that central authorities could also engage with local authorities as response progresses in order to harmonize emerging coordination efforts.

Emergent Coordination Through Sensemaking

Collective sensemaking across authorities – vertically and horizontally – can act as an important mechanism for emergent coordination in complex, dynamic crises (Boin, 2019). Collective sensemaking recognizes that (i) local actors make response choices through social interactions with other actors; (ii) information can be selected from various sources and interpreted across distributed actors and used to construct narratives (Boin, 2019); and (iii) narratives inform collective action (van der Giessen, Langenbusch, et al., 2022). In major crises, collective sensemaking will have a key role in the development of shared situational awareness – the degree to which team members have the same situational awareness on shared situational awareness requirements (Endsley, 2015). Shared situational awareness among distributed actors is one of the main aims of coordination and lies at the heart of successful coordination in response to major crises (Cornelissen et al., 2014; Endsley & Jones, 2012; Moorkamp et al., 2020; Quinn & Worline, 2008; Weick, 1995). This implies that higher-level authorities faced by centralized coordination challenges could promote an emergent coordinated response among local authorities by establishing communication mechanisms that help diverse actors make sense together across local domains (Boin, 2019).

How Can Central Authorities Facilitate Distributed Improvisation?

The literature therefore suggests that increasingly complex crises require central authorities to (i) distribute control to enable improvised actions at local level and (ii) facilitate the coordination of local actions as responses develop. While (i) implies a top-down process in which central authorities set constraints that guide while allowing for improvised actions, (ii) suggests a bottom-up process in which higher authorities respond to improvised solutions and coordination needs as they emerge. By facilitating collective sensemaking, central authorities could promote shared situational awareness across distributed actors and save local authorities from developing solutions in parallel that end up being similar due to the common external constraints that they share. Shared situational awareness would also make local authorities less likely to improvise different solutions to the same problem, causing confusion or perceptions of unfairness among members of the public. Thus, the sort of preparedness transboundary crises demand of central authorities is not only about rehearsed behavioral plans and actions, but about engaging with cognitive orientation and emergent processes at local level as responses develop (Lengnick-Hall & Beck, 2005, 2016).

Engagement with decision-makers in local authorities would also allow central authorities to identify bottlenecks to decision-making and action as they emerge across local authorities. It could then establish which bottlenecks are (i) shared by local authorities due to the common constraints that they face or (ii) specific to individual local authorities. While the latter might best be eased by local “muddling through,” the former might be eased either by communication and coordination processes that central authorities facilitate, or by addressing external constraints that central authorities could help change (e.g., Chewning & Doerfel, 2013).

Knowledge Gap and Contribution

Despite the increased importance attributed to distributed improvisation in major crisis, little has been done to investigate what central authorities can do to promote a harmonic, coordinated national response while allowing for distributed autonomy and improvisation (Boin, 2019; Moorkamp et al., 2020). One way to contribute would be to assess the validity of the idea that central authorities could do more to facilitate coordination and identify and tackle common bottlenecks arising. An interesting question in this regard is, to what extent do common bottlenecks emerge across distributed actors during crisis management, as a result of dynamic, shared external constraints? Methods from cognitive systems engineering and naturalistic decision-making have been found to be useful for systematic identification of bottlenecks to decision-making (Cook & Nemseth, 2006; Hollnagel & Woods, 2005; Klein, 2017; Militello et al., 2010). We therefore set out to use these methods to identify and classify decision-making bottlenecks faced by local authorities managing COVID-19 vaccine roll-out in Norway. First, we mapped the problem space delimiting possibilities for local authority activity in resilience using Rasmussen’s means-ends hierarchy (Naikar, 2013; Woods, 2003). Second, we used this problem space to frame and understand the results of cognitive task analysis interviews, in order to identify common bottlenecks to decision-making faced by people in key roles for vaccine roll-out at local level (Hoffman et al., 1998; Klein et al., 1982; Militello & Hutton, 1998). Finally, we attempted to classify common bottlenecks according to (i) whether they were present before the crisis or emerged during the crisis and (ii) how central authorities might feasibly have helped addressed them, for example, by facilitating coordination among local authorities or by addressing external constraints. In this paper we describe our findings and discuss the implications for central authorities wishing to improve future multi-level responses to transboundary crises.

Method

Case Background

Norwegian plans for handling national pandemics align with four principles of preparedness: responsibility, proximity, likeness, and cooperation. These outline how crises should be managed at the lowest possible level using organizational structures normally responsible for day-to-day operations, with each actor responsible for achieving optimal outcomes in cooperation with others (DSB, 2010, cf. Decouttere, Banzimana, Davidsenet al., 2021; Ministry of Health and Care Services, 2018; Norwegian Government, 2014). In line with this, Norway’s local authorities arranged COVID-19 vaccine roll-out to their populations during 2021, with the Institute for Public Health (IPH) and national health authorities together setting important external constraints across local authorities (Figure 1; Baharmand et al., 2020; CONTRA, 2021). The main vaccine used in Norway was the mRNA vaccine manufactured by Pfizer.

Figure 1.

Vaccine roll-out in Norway is the result of an interdependent relationship between local and national authorities. KPI = Key Performance Indicator.

Institute for Public Health guidelines include recommendations on roles and responsibilities, and local authorities set up core teams to manage vaccine roll-out accordingly. In larger local authorities the team includes a Chief Doctor, a Vaccine Coordinator and/or a Vaccine Center manager, with perhaps a communications and logistics manager. In smaller local authorities, roles and tasks are consolidated, perhaps involving a Vaccine Coordinator and Chief Doctor working together on vaccine roll-out. The Chief Doctor is responsible for decisions on medical use and administration of the vaccine to the local population, deciding whether vaccine is viable, number of doses it is defensible to extract from each vial, approving administration strategy, etc. The Vaccine Coordinator will organize distribution, storage, call in patients, and so on.

During the roll-out period covered (Jan–Sep 2021), central authorities made several changes to plans. Figure 2 serves to illustrate events affecting how COVID-19 vaccination was organized at local authority level in Norway.

Figure 2.

Timeline of key events in vaccine roll-out by local authorities (LA) in Norway. IPH = Institute of Public Health; GP = general practitioner.

Participants

We selected 21 local authorities representing different population sizes, densities, and locations across Norway, and invited them to participate by email; nine authorities agreed to take part. Together study participants represented seven of Norway’s 356 municipalities, and two out of about 50 city district authorities. In the case of three of the municipalities, we interviewed several core team members to learn how people in key roles in the same municipality collaborated to organize COVID-19 vaccination. The final group of participants represent a range of key roles, and diverse local authorities with a collective population of ca. 500,000 people, just under 10% of the population in Norway (Table 1).

TABLE 1:

Overview of Participants With Role, Location in Norway and Population Size.

Interview	Role	Municipality (M) or district (D)	Area of Norway	Population
1	Chief doctor	M1	Eastern	25–50,000
2	Vaccine coordinator
3	Communications manager
4	Chief doctor	M2	Eastern	>75,000
5	Vaccine coordinator and center leader
6	Logistics manager
7	Vaccination coordinator and center leader	M3	Eastern	0–10,000
8	Chief doctor	M4	Southern	>75,000
9	Vaccine coordinator	M5	Northern	50–75,000
10	Vaccination center leader	M6	Eastern	25–50,000
11	Vaccination coordinator	M7	Eastern	50–75,000
	Vaccination center leader	M7	Eastern	50–75,000
12	Chief doctor	D1	Oslo	50–75,000
13	Chief doctor	D2	Oslo	50–75,000

The remainder of this section outlines our approach based on cognitive systems engineering, as described by Woods (2003) and Cook and Nemseth (2006), which has been shown to allow for the systematic extraction of bottlenecks to decision-making in complex situations. To assist in identification of decision-making or other macro-cognitive bottlenecks, the results of cognitive task and critical decision analyses were used to construct an account of the functional problem space faced by local decision-makers. This problem space was then used to give context for accounts of decision-making and other macro-cognitive functions and processes so that cognitive bottlenecks to goal achievement could be identified.

Interview Procedure

Interviews were conducted with 14 local authority representatives in September 2021, each lasting 80–160 min. To prepare, we reviewed key documents and notes from preliminary interviews with a county-level Chief Doctor; the head of vaccine roll-out at IPH; and a representative from each of two central authorities (Directorate of Health and Ministry of Health and Care). To prepare interviews with local authority representatives, we used framing questions from Crandall et al. (2006), and adapted questions on cognitive task analysis from Crandall et al. (2006), Applied Cognitive Task Analysis (Militello & Hutton, 1998) and the Critical Decision Method (Hoffman et al., 1998; Klein et al., 1982) (Appendix 1). In the first interview, questions from Cognimeter were used to confirm the suitability of cognitive task analysis for analysis of key tasks identified (Chrenka et al., 2001; Hutton et al., 2003). Tasks identified by each role type scored highly on cognitive dimensions for situational awareness, course of action development and evaluation, course of action monitoring and planning, teamwork and coordination, and information attention and management. Interviews were recorded and transcribed, the transcripts anonymized, and then coded in N-VIVO12 using a scheme developed in analysis of the first interview transcript. Procedures were ethically approved by NSD, the Norwegian Centre for Research Data.

Transcript Analysis

Interview transcripts were analysed in three sweeps:

1. Problem space/work domain analysis (Naikar, 2013). We coded text on Purposes of the roll-out system; Values people have as they act to achieve system purposes; System functions (goal-related); and Components (materials, tools, and infrastructure that in combination with other resources perform system functions). Texts describing Actors/roles and main Activities were also coded.

2. Decision-making and other forms of macrocognition (Crandall et al., 2006; Klein & Wright, 2016; Woods, 2003). We captured information on goal-related cognitive demands and challenges faced by decision-makers, including those relating to communication and coordination within and between teams and organizations, by coding text on Social organization; Constraints; Information (used or needed); Decision-making (goal-related); Expectations (goal-related); Planning (for actions); Adapting or re-planning; Coordinating; and Sensemaking.

3. Critical decision analsysis. Text describing 10 critical decisions, one each from 10 participants, were also coded using the above nodes. Critical decisions studied were as follows: How to train staff arriving at the vaccination center due to start in half an hour (identified by two participants); How to deal with changing dose intervals (1 participant); How to prioritize health personnel (2 participants); Whether to throw away or use vaccine doses (3 participants); and How to handle IT system crash at vaccination centers (2 participants).

The total number of passages coded under each node are given in Appendix 2. Coding was performed by a single researcher. Text for a randomly selected single interview (M5, Table 1) was also coded by an experienced human factors professional after a short explanation of the codes, and general agreement confirmed.

Developing the Problem Space

The results of the first sweep were used together with a review of relevant documents (DSB, 2010, 2018; Ministry of Health and Care Services, 2018; NIPH, 2016; Norwegian Government, 2014) to develop our understanding of the common work domain of decision-makers (Naikar, 2013; Rasmussen, 1986). Text on Purposes, Values, and Components were consolidated into categories and used to populate the problem space (Results, Figure 3). In a final step, system functions (middle row, Figure 3) were identified by considering (i) relevant coded text and (ii) how abstract functions act as a link between components and their specific functions on the one side, and values and functional purposes of the system on other (in line with Naikar, 2013).

Figure 3.

Functional domain or “problem space” analysis for local organization of vaccination, developed from interviews with 14 representatives from nine local authorities. IPH = Institute of Public Health, HD = Directorate of Health, LA = Local Authority.

Identifying Bottlenecks

The final coded text was then reviewed to identify common goal-related cognitive challenges (bottlenecks) and associated decision needs and re-grouped accordingly by pasting into a document. To assist in this process, we analyzed coded responses in light of the problem space (see Figure 3) to try to understand how decision-makers navigated constraint-shaped demands and dilemmas as they tried to achieve goals for vaccine roll-out in line with their values. This process is evident in the descriptions of emergent bottlenecks in the Results.

Results

Problem Space

From the coded text, we identified five distinct functional purposes that were common across local authorities’ roll-out systems (top row, Figure 3). While representatives described different strategies used to achieve goals, seven functional values – preferred ways of achieving goals or assessing progress – were common across local authority representatives (second row, Figure 3). Not wasting doses was a dominant value, salient especially when doses were going out of date and priority patients were not available for vaccinating.

Components used to carry out tasks were grouped for the purposes of clarity and summarized at the bottom of Figure 3. Finally, 16 central system functions identified are also presented. On analyzing the interviews, we noted that these functions align well with questions participants sought to answer at the outset of vaccine roll-out. For example, several participants wondered how they would map priority groups (function 2) or call in patients to match available doses at short notice (function 4).

Emergent Bottlenecks

Though local authorities organized vaccine roll-out independently, our analysis revealed seven bottlenecks to vaccine roll-out that were common across local authorities due to their shared constraints (Table 2). Five of these bottlenecks were emergent, that is they only became apparent as the crisis developed (column 5, Table 2). Our analysis shows that each of these five emergent bottlenecks could feasibly have been improved by increased shared situational awareness vertically and horizontally across authorities (column 6). Three out of five emergent bottlenecks could feasibly have been addressed by addressing external constraints as the crisis progressed (column 7). We refer to the latter as the three main emergent bottlenecks and present them in more detail below (the remaining two emergent bottlenecks are presented in Appendix 3).

TABLE 2:

Common Bottlenecks During Vaccine Roll-Out by Local Authorities (LA).

Bottleneck		Difficult decisions	Constraints		Emergent bottleneck?	Could be addressed by shared situational awareness?	Could be addressed by changing external constraints?
Bottleneck		Difficult decisions	Functional (mid-row Figure 3)	External	Emergent bottleneck?	Could be addressed by shared situational awareness?	Could be addressed by changing external constraints?
1	Extracting doses from vials	-Should we extract 5, 6, or 7 doses per vaccine vial?	7,12,16	-Guidelines on recommended no. doses per vial	Yes	Yes, need awareness across LAs of practices and consequences	Yes, recommended seven-dose extraction method, with conditions for when it should be used
				-Internet articles on extraction techniques
				-Various social communication networks
2	Dealing with changing dose intervals	-Should we rearrange vaccination appointments to account for changes to dose interval by central authorities?	3,4,9,16	-Dose intervals recommended by central authorities	Yes	Yes, make visible for IPH effects of changing vaccine dose intervals on actual practice by LAs, and different ways in which for public trust can be affected	Yes, changes in dose intervals account for LA concerns
				-Developing scientific knowledge of most effective dose intervals
				-Unpredictable vaccine supply
3	Prioritizing patients	-Which health-critical personnel should we prioritize?	2,4	-List of priority groups issued by central authorities	Yes	Yes, discussion on critical health personnel to develop shared understanding	Yes, clearer criteria defining critical health personnel – developed in collaboration with local authority panel
		-How do we assess borderline cases consistently and fairly?
		-Is it ok to vaccinate those almost matching prioritization criteria if much more efficient to do so?
4	Deciding whether to dispose of vaccine or use it	-How can we find people matching priority criteria in short time to use vaccine?	1,2,4,10, 11	-List of priority groups issued by central authorities	Yes	Yes	No, best dealt with at local level
				-Distance to and availability of prioritized patients
		-If we can’t do this, how can we find people who almost match priority criteria?		-Vaccine – short lifetime, limited supply
		-Can we use vaccine even if it is close to lifetime?		-Contact details – prioritized patients and staff
5	Coordinating patients, vaccines and staff	-How do we coordinate at short notice while (i) vaccinating safely, efficiently and transparently; (ii) not wasting doses, and (iii) in order of priority?	1,4,5,6,16	Many LAs given unpredictable, very limited supply of short-lived vaccine at short notice, to be administered to prioritized patients in dispersed locations	Yes	Yes, vertical awareness to inform central authorities about consequences of external constraints on LAs	No, assuming constraints were politically determined; vaccine distribution strategies best determined at local level
6	Establishing data systems for calling in patients and distributing vaccines	-How do we set up a system to identify, call in and confirm vaccine appointments with priority patients at short notice?	2,3,4,5,10, 11,16	Various data systems available to coordinate patients, vaccines, staff	No	Yes, increased awareness needed across LAs of data systems and consequences	Yes, provide generically available, tailorable systems for patient identification/call-in; and distribution logistics
6		-How do we know which centers need how many vaccines for who, where and when?	2,3,4,5,10, 11,16		No
7	Balancing the need for operational staff with the need to train staff	-How do we find qualified staff at short notice?	6,10,11,12, 13, 14,16	-COVID-19 infection levels	No	Yes	No, best dealt with at local level
		-How do we train staff at short notice?		-LA policy – can staff be recruited from other departments?
				-Vaccine – unpredictable, limited supply at short notice
				-List of human resources, qualifications and contact details
				-e-learning software

Extracting Doses From Vials

At the start of roll-out in January 2021 vaccine demand greatly exceeded supply, so the number of doses extracted from each vial had a direct influence on both the speed at which people could be vaccinated (“as fast as possible”) and the overall availability of vaccines (“as many as possible”) (cf. Figure 3). At the start of vaccine roll-out local authorities were advised by IPH to take out five doses from each vial of Pfizer vaccine, but after a few weeks a six-dose extraction was recommended. Several participants described how Chief Doctors learned of descriptions of how to use advanced techniques to take out seven doses, and how their core team had then “muddled through” to achieve seven-dose extractions that helped them better achieve functional purposes in line with values (Figure 3). In contrast, other participants reported discussing extensively whether to move from five to six doses, and whether extracting seven doses was defensible. Ultimately, IPH left the decision about whether seven doses should be extracted up to each municipality’s Chief Doctor, so each local authority had effectively to decide itself whether it condoned “off-label” use of vaccine (i.e., non-regulatory use). The result was that local authorities extracted different amounts from their Pfizer vials. While some managed to get several thousand extra doses out of their vials after they had independently improvised a seven-dose extraction method, others did not.

Asked how core teams learned about the possibility of extracting seven doses, several commented on hearing “rumors,” or gathering information from emails or “reading this or that” in the media about how others in Norway or other countries had succeeded, and searching to find the method they had used. One described how they bought a magnifying glass with a fitted light from a superstore to develop their extraction method, after reading about how others had done so. Another described finding documents linked via Facebook groups for doctors in Norway, and then learning more through a chance conversation with a friend who worked for one of the first local authorities to report extracting seven doses. Yet another found out through their work on vaccine preparation with pharmaceutical technicians. Opinions varied as to technique for extracting seven doses, with one commenting that the methods available from other countries were complicated involving the use of “several needles,” but that “we tried to simplify, and it ended up being quite simple.”

In several cases we heard how Chief Doctors championed six or seven doses had worked to convince the managers responsible for the people who would have to extract the extra doses. One such manager described the initial cause for concern about extracting seven doses:

“So we had a lot discussion for and against going for it. I am happy I criticized and asked questions at the start so that [the Chief Doctor] understood that this was a process for the staff who would work with the vaccine. It was not about changing procedure and that’s that, because there is a safety margin that…when you extract using the new method you don’t see completely clearly how much is in the injection so you have to trust that whoever has extracted the vaccine has done it right.”

Vaccine Coordinator, M1

The same manager said, “it took weeks to convince me,” but it was a “healthy” process in which the decision was made together. A similar process was evident in the other municipalities. Note how these discussions effectively involve core team members negotiating the problem space in Figure 3 and arriving conclusions after negotiating values and goals (e.g., “safe vaccination” vs. “as many as possible” or “do not waste doses”). Respondents reported they were not aware of similar negotiations in other local authorities unless they had actively tried to find out, for example, by contacting an acquaintance.

Positive consequences assessed on deciding to extract seven doses included getting “more out of the vaccine we get” and that more vaccine sooner would be “better for the population.” A clear negative consequence considered was the extra training burden. Champions were careful, however, to ensure patients received full worthy doses of vaccine. For this reason, and because of the advanced medication technique, some saw an advantage in centralizing operations within the local authority:

“We began quickly to extract “off-label”, for the rumours flew that you get out more than five doses, but there was also a nursing home that couldn’t manage to get out five doses, some managed only four, so … we decided quickly that the extraction was complicated, we should pull everything together.”

Vaccination Centre Leader, M7

Reflecting on whether IPH could have done more to encourage local authorities to extract seven doses, most seemed to think that IPH had got it about right to let each local authority decide, so that they could make decisions based on locally available competence and equipment. Not everyone agreed, however, some pointing to lack of shared understanding about how many doses the authorities “allowed” them to take out, others calling for a unifying strategy at national level, in which conditions for extracting seven doses in terms of method, equipment, and technical competence were laid out, and where each local authority was not left to find out how to do things itself. One downside of allowing each local authority to decide itself was that those who succeeded in getting out more doses were uncertain how to use them when neither they nor IPH had planned for them. Another downside was that vaccination went at different speeds in adjacent local authorities, a threat to fairness.

Based on the respondent accounts, increased shared understanding about practices on dose extraction across local authorities could have saved time and resource used on independently improvising solutions to a common problem. While central authorities could not recommend seven-dose extraction, they could have responded to the emerging bottleneck by issuing a protocol with a description for conditions to be met before using it. The ability to track how many doses per vial were being extracted by each local authority would also have allowed them to track the emerging bottleneck.

Dealing With Changing Dose Intervals

Many of the participants commented that changing recommended intervals between vaccine dose 1 and 2 was difficult for the local authorities to handle:

“One of the most notable changes we had to deal with was dose intervals, from when we started with short intervals between doses, which were then changed…from 21 to 28 days to six weeks, three months, back again, it affected us in terms of how we planned for patients and the staffing of the vaccination.”

Vaccine Coordinator, M5

One health nurse was surprised that interval changes applied to those who had already received a first dose, such that appointments for dose 2 of the vaccines – already agreed and confirmed – had to be altered. A vaccine coordinator described how this was difficult to make sense of. IPH sent out doses so that the second dose could be given according to the new intervals they had set, assuming local authorities could reschedule appointments. Changes became so routine that some local authorities started predicting them in order to cope:

“Then I judged that…for we aren’t talking about so many people and in a way the dose supply had increased a lot, so I assumed the interval would change again, and anyway I could save up spare doses from the week before so I could give people their second dose at the originally agreed time even though in IPH’s eyes it was too early.”

Vaccine Coordinator, M3

Several local authorities were keen to stick to the dose interval agreed with patients so as not to “mess people about,” and to avoid a great deal of extra telephone work in making new appointments. Changed dose intervals also created a need for more medical assessments of individual cases (e.g., when the interval was increased from 3 to 6 weeks, the 3-week interval was kept for those with immunodeficiencies). A further consequence was that local authorities who had innovated and worked hard to get more doses out of each vial were caught short.

“We got out [many] extra doses [so] we vaccinated more people than IPH thought we had. So when the first message came that we should go to a 12-week dose interval, I had to ring IPH and say that’s all very nice, but I don’t have patients for more than eight weeks more of vaccinating according to what you say you will send us in the coming weeks. And then you get a message, “then keep the 8-week interval”, but the public still think they will get 12 weeks so they ask, why not 12 weeks? Then 2-3 weeks go by and [the dose interval] goes down to nine weeks. Luckily, we hadn’t moved to 12 weeks yet, so we didn’t have to move all the patients, but after only ten days we are up to 12 weeks again! Then we got told that even though we wouldn’t have patients to vaccinate in the last three weeks, we had to do it, we had to move to 12 weeks, and that meant we had to swap 12,000 appointments. And then some weeks pass by and we get more doses and must move the same group again to earlier appointments! Then we struggle with those who aren’t too bothered about exactly when they get vaccinated, they have rearranged plans once and don’t want to be bothered to change plans again. They think, ok I could get it in August but I will get it in September anyway, it doesn’t matter. But we in the municipality have to move them, because we will get the doses earlier, and we can’t store them or they’ll be unusable.”

Chief Doctor, M1

Ultimately, participants worked around official recommendations since they were concerned about the effects on public trust. When the changes frustrated people, it also increased the load of enquiries to be dealt with by the local authority, who already had increased burden from re-arranging potential appointments. It is likely that the effects of so many changes to the dose intervals were not fully visible to central authorities, implying a need for increased shared situational awareness to allow central authorities to account for conflicting functional purposes caused by changes to an important external constraint (cf. Figure 3).

Prioritising Patients

With a strain on hospital resources, concern in society, and few doses arriving in the first few months of roll-out, it was important for authorities to prioritize (i) in a way that would most effectively prevent serious illness and reduce the treatment burden and (ii) fairly. IPH issued clear higher-level descriptions of at-risk priority groups before vaccine roll-out, based on the findings of an expert group (Appendix 2). In addition, IPH gave local authorities a share of doses to be used on frontline health personnel as the crisis progressed. Local authority representatives working in the field reported that they were still left with “more granular” prioritization decisions that were one of the most difficult challenges they faced during roll-out. These related mainly to (i) whether “near-priority” people in the same location as prioritized groups should also be vaccinated for the sake of effectiveness and (ii) which health personnel should be prioritized as critical. Two of the local authorities reviewed procedures after reports that the “wrong” people had been vaccinated.

“It wasn’t clear which health personnel we should prioritize. There are many different types of health personnel, who are they in relation to the legislation, in relation to function or what did they mean? We used a lot of time finding out who we should prioritize.”

Vaccination Centre Leader, M7

One described how they tried and failed to get hold of people, who then got upset when they found out that vaccine had been used on others. Several local authorities asked family doctors to provide lists of patients ranked in terms of priority but found it difficult to ensure that the family doctors applied priority criteria consistently. It was clear that local authorities had used time and effort on questions of priority. Some described how they made sense of the situation and adapted in order to deal with it:

“…they have 3000 staff in Health and Wellbeing, so it’s easy to pick out and prioritise the most critical 100 or 200, but if you have 1000 doses and limited time, we would have had to have a list of names and said [randomly] yes and no to each one, so we had to delegate the prioritization to each section.”

Chief Doctor, D2

“each family doctor couldn’t call in patients [randomly] based on the number of vaccines allocated, so we had a lot of discussion about which illnesses…how at the centre we have fixed criteria and say no to people who don’t match them”

Vaccination Centre Leader, M2

“those that worked with developmentally challenged persons set themselves high up on the priority list, but drop-in medical service (legevakt) lower down…but we see the whole picture and of course ranked drop-in services as the most important in this system here. We gave clear guidance on how to prioritize but of course the needs of those like us are perhaps more visible to us…”

Logistics Manager, M2

“it was terribly difficult, should we prioritize care assistants…vulnerable people are wholly dependent on them for their most basic needs, but they are a group without education and a weak voice…on the other hand dentists – who have strong representation – are protected, but then again dentists are exposed to infection.”

Chief Doctor, M2

Priority decisions were too complex to be based on a logical comparison of pros and cons, but rather solutions were found that they believed matched the circumstances well. The challenge of prioritization decisions was that once made, they were visible to all, and there were consequences for fairness of prioritizing differently within or across local authorities. There was, however, little central support to coordinate difficult decisions, just a message that “local authorities are different so you have to prioritize yourselves.” This approach fails to account for patients seeing others like them in the neighboring municipalities getting vaccinated first, or for health personnel living in one local authority working in another local authority where colleagues have been vaccinated. Several representatives therefore felt a need for clearer guidance or coordination on how to prioritize groups to limit the consequences for fairness of varying assessments. One representative sought more discussion about priority decisions among neighboring municipalities to base decisions on a broader awareness. Local authorities also sought clearer direct central guidance during roll-out specifically about which health personnel should be prioritized.

Discussion

We interviewed 14 decision-makers in key roles working for different local authorities in Norway in the critical COVID-19 vaccination period from January to September 2021. Bottlenecks to decision-making were identified from accounts of cognitive tasks in the context of a shared vaccine roll-out “problem space.” Central to the problem space are 16 system functions needed to organize roll-out so that as many people as possible are vaccinated as fast and as fairly as possible, in the order set out in authority guidelines, and while maintaining public trust in the authorities’ handling of the pandemic. The 16 functions are needed to ensure that the “three flows” of vaccines, patients, and staff (Decouttere, Vandaele, et al., 2021) are aligned with system values and goals. They should be supported when designing improved roll-out systems. The 16 functions also align well with questions decision-makers have at the start of roll-out, so can also form a basis for training or decision support.

In the context of the problem space, we identified seven common bottlenecks to decision-making by local authorities during COVID-19 vaccine roll-out. Each bottleneck was shaped by the local authorities’ shared external constraints. Five of the bottlenecks were not apparent to authorities at the outset but emerged only as the roll-out progressed. The three main emergent decision bottlenecks centered around extracting doses from vials, dealing with changing dose intervals and prioritizing patients. We observed several examples of how local authorities improvised to deal with these bottlenecks and changing external constraints in the face of local conditions. Most notably, some core teams searched for ways to extract extra vaccine doses from vials as they learned from social and other information sources that this was possible, and then “muddled through” to perfect an advanced extraction technique in the context of local constraints. Similarly, local actors improvised solutions to challenges posed by changing dose intervals and unpredictable, short-notice vaccine supply, and were willing to go outside of procedures and guidelines in order to conserve important goals. In real-world contexts, improvised data systems, the use of pensioned health workers and extra “off-label” doses afford much-needed flexibility to adapt in the face of unpredictable vaccine supply, staff shortages, and changing dose intervals. These affordances could have been threatened if national strategy had not allowed for distributed improvisation – which it clearly did.

Despite this, we wonder if the positive effects of distributed improvisation might have been improved further if central authorities had detected the three main emergent bottlenecks as they arose, and then responded by changing external constraints. Most notably, detection of varying dose extraction levels among different local authorities might have prevented IPH making frequent, sweeping changes to dose intervals that failed to account for those local authorities who had innovated to extract seven doses from their vials. In addition, all five emergent bottlenecks could arguably have been eased if local authorities had had a greater level of shared situational awareness, which national or regional authorities could have helped facilitate.

Thus, while central and local authorities appear to have implemented distributed improvisation effectively, there was less evidence that central authorities facilitated the coordination of local improvised actions as the response progressed. The need for coordination of emerging local responses was evident in calls from some respondents for IPH to disseminate a recommended seven-dose extraction method along with conditions to be met before use; and in the need for IPH and other local authorities to learn about the potential negative effects on public trust that frequent changes to dose intervals could have. A need for clearer prioritization of health personnel for vaccination also emerged, and clearer stakeholder-centered guidelines could have been worked out in response.

A recent commission into the overall crisis response by Norwegian authorities concluded that the roll-out response was largely successful, and we should emphasize that this was reflected by the comments of our participants (NOU, 2022). A willing, proactive, and cooperative work culture among core teams at local level helped harness the distributed autonomy local authorities were given. Initial high levels of public trust in the authorities was another key element in a successful crisis response (Christensen, 2020). With the aim of improving responses still further, however, it is important to ask whether high levels of public trust could have been better maintained, and the effectiveness of the roll-out optimized, if improvised actions across local authorities had been harmonized. An evaluation of a previous vaccine roll-out crisis in Norway supported this idea by concluding that local variation in decisions on vaccine administration, logistics, and the development of public trust could have limited the efficiency and perceived fairness of national vaccine roll-out (DSB, 2010).

Turning to how local improvised actions might be harmonized, we saw several examples of coordination emerging across local authorities. Chief Doctors sought out information on seven-dose extraction methods from acquaintances in similar positions in other municipalities, and nearby local authorities set up groups to borrow vaccines from each other to deal with unpredictable supplies and unexpected problems. Engagement by central authorities to identify, support, and expand on such emergent coordination by local actors could lead to supportive mechanisms where they are needed most. Given differing interpretations by local authorities on key aspects of vaccine roll-out (e.g., number of doses it was okay to extract, which health personnel groups to prioritize, importance of attending to recommended dose intervals), collective sensemaking needs to be facilitated. This would increase shared awareness among crisis actors representing different local and central authorities, of common challenges, good solutions and discrepant approaches, and thus help avoid suboptimal solutions, confusion, or perceptions of unfairness among the local authorities’ inhabitants. In the context of the multi-team literature, developing optimal joint cognitive systems for nationwide vaccine roll-out is not just about local authorities aligning activities and goals with those of national authorities, but a two-way iterative process in which national authorities and IPH coordinate to ensure alignment with the different strategies local authorities use to achieve nationwide goals. The challenge for IPH and national health authorities is how to coordinate, communicate, and understand the varying strategies and affordances of 356 local authorities in this way.

Study Limitations

The picture we have drawn is neither perfect nor complete. One limitation is that we have had restricted access to participants busy with important work and have therefore built our picture from limited views of the “system elephant.” While we have not interviewed IPH or authority representatives in depth, they were involved in preliminary discussions and workshops for the project. Given that our approach has been to look at what shapes decision bottlenecks for local responders – to look at the system through the eyes of local decision-makers – we do not regard this as a serious limitation.

While we have attempted to be as explicit as possible about the process used to generate observations and patterns to be functionally accounted for, it has not been possible to describe explicitly how we identified bottlenecks insight about the “deeper dynamics” of complex social systems. This is related to the difficulties of warrant in functional analysis and the use of insight in cognitive systems engineering, as discussed by Woods (2003). On a more technical point, some researchers might question the conciseness of our approach to cognitive task analysis, and not least our use of a curtailed critical decision method, but we found the described process to be efficient and effective given the restricted time we had with the people we spoke to. Finally, while only one researcher coded the text, we do not think this has compromised the study because the nodes used are easy to understand and coding of a randomly selected interview by a second rater showed close agreement.

Conclusion

Analysis of cognitive interviews with decision-makers involved in local COVID-19 vaccine roll-out in Norway identified seven bottlenecks to decision-making that were common across local authorities due to their shared external constraints. Five of these bottlenecks emerged as the crisis progressed. To improve management of major crises in the future, we speculate that central authorities might consider how they can facilitate coordination to improve shared situational awareness with and among local authorities and track and address common decision bottlenecks as they emerge. More research is needed, however, to explore alternative centralized response strategies and assess how well they effectively balance centralized and distributed control.

This article is part of the CONTRA project (https://contra.uia.no) financed by the Research Council of Norway.

Footnotes

Acknowledgments

We would like to thank the participants for giving their time and engagement in a very demanding period. We would also like to thank Knut Jønsrud at IPH for helpful comments on the first draft of the article. Any remaining factual errors are the sole responsibility of the authors.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Norges Forskningsråd (312773).

ORCID iD

Ross Owen Phillips

Appendix 1 CTA Questions

1. Can you tell us about your background and current position in the municipality?

2. Concerning how vaccination is organized by the municipality:

a. What are the municipality’s goals?

b. Can you describe what the municipality has to do to organize vaccination?

c. Who is responsible for what?

d. Who does what?

3. Concerning your role in organizing the vaccination:

a. What have been your main goals?

b. What have you done to achieve these goals?

c. Can you break this down into four to six main tasks?

4. For each task identified, as appropriate:

a. Can you give me the big picture?

b. What questions do you have as you approach this task?

c. How do you answer? Which information do you need? What do you have to follow up or monitor?

d. What makes things go well?

e. Has anything happened to cause problems? What are the most critical challenges?

5. Startup:

a. How did you find out what to do?

b. How were tasks distributed among roles?

c. Was there any training involved?

6. Decision-making:

a. What are the most important decisions you have had to take?

b. Which have been the most challenging?

c. What did you think about at the time?

7. Information:

a. Can you describe an instance where you have had to act based on limited information?

b. How did you look to review the situation to account for new information?

8. Collaboration:

a. Which other actors are important for you in your work?

b. How is your work influenced by communication or actions of actors outside the municipality?

9. Sustainability:

a. Can you describe the work in terms of efficiency? Were priority groups vaccinated as quickly and effectively as possible?

b. Can you describe the work in terms of waste avoidance?

c. Can you describe the work in terms of fairness? Were all people in a priority group equally informed and have access to vaccine?

d. How were each of these factors influenced by communication and action of actors outside the municipality?

10. Shortened Critical Decision Method based on Crandall et al. (2006):

a. Can you think of an incident or difficult situation in which you had to make a critical decision, a decision where you lacked sufficient information or knowledge on which to base the decision? It can be a decision involving other actors or organizations.

Choose most suitable decision for analysis

b. Can you tell a story about the events in the order in which they happened? What happened when?

Identify key decision point(s) and run through or select from following probes:

If you had to describe the situation to another at that moment, how would you summarize the situation?

What were your goals at the time, what was the most important thing you were trying to do?

What did you know? What didn’t you know?

What information did you use to make your decision? Where did it come from? How did you use it?

What other decisions could you have made at the time?

How did you choose to act as you did?

Was this a familiar situation?

Can you describe any training relevant to the situation or decision?

Was your experience from other situations relevant?

What told you that was the right decision?

How long did it take to make the decision? Was there time pressure?

Did you consult with others?

How did you know whether to trust any advice or guidance you got?

What information could have helped you?

TABLE A1.

Coding Scheme With Number of Coded Text Passages Entered Under Each Code.

Role	Interview	Goals	Values	Actors, roles	Functions (goal-related)	Actions (tasks that provide functions)	Resources, tools, affordances (provide actionable possibilities)	Organization and infrastructure	Constraints	Information	Decision-making (goal-related)	Expectations (goal-related)	Planning (for Actions)	Adapting, re-planning	Coordinating actions	Sensemaking	Total
Chief doctor M1	1	19	16	29	12	19	10	10	14	17	5	3	11	7	25	9	206
Vaccine coordinator M1	2	11	5	17	5	16	14	2	17	35	22	2	12	8	17	21	204
Communications manager M1	3	5	4	6	0	4	2	0	6	11	3	2	0	0	8	8	59
Chief doctor M2	4	13	7	11	4	1	2	6	3	14	9	0	4	4	19	13	110
Vaccine coordinator and center leader M2	5	7	2	12	2	1	5	7	5	17	14	1	3	11	13	13	113
Logistics manager M2	6	6	10	5	6	11	4	9	7	13	8	0	7	8	22	9	125
Vaccination coordinator and center leader M3	7	5	6	6	1	3	6	5	15	7	3	1	1	9	10	6	84
Chief doctor M4	8	4	3	6	1	4	2	4	0	8	3	0	2	5	14	3	59
Vaccine coordinator M5	9	7	5	8	1	5	4	4	3	6	9	5	4	5	10	5	81
Vaccination center leader M6	10	5	4	6	3	4	11	0	1	9	6	0	6	9	8	2	74
Vaccination coordinator M7
Vaccination center leader M7	11	3	5	7	0	4	12	2	9	21	7	1	7	9	25	8	120
Chief doctor D1	12	6	4	6	1	0	7	1	3	13	13	0	6	9	14	6	89
Chief doctor D2	13	0	13	11	2	0	2	3	1	16	9	0	1	4	13	5	80
Total		91	84	130	38	72	81	53	84	187	111	15	64	88	198	108	1404

Appendix 2 Priority Groups

Order of priority of vaccination for groups with increased risk of serious outcomes 2020-12-v2-anbefalinger-og-prioriteringer-2-utgave-korrigert-forside.pdf (fhi.no)

1. Care home inhabitants (n = 40.000)

2. Age 85 years and over (n = 120.000)

3. Age 75–84 and over (n = 290.000)

4. Age 65–74 years (a), people aged 18–64 years with high risk (b) (n = 1.540.000)

5. Age 55–64 years with underlying illness/condition (n = 175.000)

6. Alder 45–54 years with underlying illness/condition (n = 125.000)

7. Alder 18–44 years with underlying illness/condition (n = 160.000)

8. Alder 55–64 years (n = 650.000)

9. Alder 45–54 years (n = 745.000)

Appendix 3 Analysis of Two Other Emergent Bottlenecks

Ross Owen Phillips is a Senior Researcher at TØI in Norway, where he applies cognitive systems engineering and naturalistic decision making to study transport and logistics systems. He is a Chartered Psychologist with the British Psychological Society.

Hossein Baharmand is Associate Professor in Project and Supply Chain Management at the School of Business and Law, University of Agder (UiA), Norway. Hossein conducts mixed-methods research that deals with supporting policy makers in the context of health and humanitarian logistics with evidence-based decision support systems. He serves as the Publications and Academic Standards Chair in the International Society for Crisis Response and Management (ISCRAM.org).

Nico Vandaele holds a degree Business Engineering and obtained a PhD in Operations Research and Operations Management from KU Leuven in 1996. As full professor at KU Leuven, he is holder of the GSK Research Chairs on Access-to-Medicines, the Janssen Pharmaceutica Research Chair on Pandemic Preparedness and co-leads the Access-to-Medicines Research Center.

Decouttere is also at KU Leuven where she co-leads the Access-to-Medicines Research Center.

Lise Boey is a postdoctoral researcher at the Access-To-Medicines Research Centre ((KU Leuven, Belgium). She has a master degree in Biomedical Sciences (KU Leuven, 2016) and obtained a PhD in Biomedical Sciences in 2020 (KU Leuven). Her PhD concerned vaccination in at-risk. Lise's current research focusses on health systems concerning future African vaccine manufacturing, vaccine hesitancy, vaccine allocation, HIV/TB care, and COVID-19 antivirals development.

References

Ansell

Boin

Keller

(2010). Managing transboundary crises: Identifying the building blocks of an effective response system. Journal of Contingencies and Crisis Management, 18(4), 195–207. https://doi.org/10.1111/j.1468-5973.2010.00620.x

Baharmand

Balcik

Boey

Decouttere

Phillips

R. O.

Saeed

Vandaele

(2020). CONTRA project report number 1: Requirements identification and system mapping. University of Agder. Resultat #1852426 - CONTRA Project report #1: Requirements identification and system mapping - Cristin.

Baharmand

Balcik

Boey

Decouttere

Phillips

R. O.

Saeed

Vandaele

(2021). Toward a decision support system for COVID-19 vaccine allocation inside countries, in: No. 2 NOKOBIT Norsk Konferanse for Organisasjoners Bruk Av IT. [Paper presentation] Presented at the Norsk IKT-Konferanse for Forskning og Utdanning, NOKOBIT21, Trondheim, Norway, 27 November 2021.

Boin

(2019). The transboundary crisis: why we are unprepared and the road ahead. Journal of Contingencies and Crisis Management, 27(1), 94–99. https://doi.org/10.1111/1468-5973.12241

Brown

Power

Conchie

S. M.

(2021). Communication and coordination across event phases: A multi-team system emergency response. Journal of Occupational and Organizational Psychology, 94(3), 591–615. https://doi.org/10.1111/joop.12349

Chewning

L. V.

Doerfel

M. L.

(2013). Integrating crisis into the organizational lifecycle through transitional networks. International Journal of Humanities and Social Science, 3(21), 39–52.

Chrenka

Hutton

R. J. B.

Klinger

D. W.

Anastasi

(2001). The cognimeter: Focusing cognitive task analysis in the cognitive function model. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 45(24), 1738–1742. https://doi.org/10.1177/154193120104502408

Christensen

Lægreid

(2020). Balancing governance capacity and legitimacy - how the Norwegian government handled the COVID -19 crisis as a high performer. Public Administration Review, 80(5), 774–779

CONTRA (2021). CONTRA. Effective, efficient, sustainable and fair COVID-19 vaccine distribution. CONTRA Project Website. https://contra.uia.no/about/project-summary/ (accessed December 15 2021).

10.

Cook

R. I.

Nemseth

(2006). Taking things in one’s stride: Cognitive features of two resilient performances. In Resilience engineering: Concepts and precepts. CRC Press Taylor & Francis Group.

11.

Cornelissen

J. P.

Mantere

Vaara

(2014). The contraction of meaning: The combined effect of communication, emotions and materiality on sense making in the Stockwell shooting. Journal of Management Studies, 51(5), 699–736. https://doi.org/10.1111/joms.12073

12.

Crandall

Klein

Hoffman

R. R.

(2006). Working minds. A practitioner’s guide to cognitive task analysis. MIT Press.

13.

Decouttere

Banzimana

Davidsen

Van Riet

Vandemeulen

E. M.

Jalali

M. S.

Vandaele

(2021). Insight into vaccine hesitancy from system thinking, Rwanda. Bulletin of the World Health Organization.

14.

Decouttere

Vandaele

De Boeck

Banzimana

(2021). A systems-based framework for immunization system design: Six loops, three flows, two paradigms. Health Systems. https://doi.org/10.1080/20476965.2021.1992300

15.

Decouttere

Vandaele

Lemmens

Bernuzzi

(2016). The vaccine supply chain multathlon: the reconciliation of technology, economy and access to medicines. In Zobel, C., Altay, N., Haselkorn, M. (Eds.), Advances in Managing Humanitarian Operations. International Series in Operations Research and Management Science, pp. 205–227. Springer: Cham. https://doi.org/10.1007/978-3-319-24418-1_10

16.

DSB (2010). Ny influensa A (H1N1): Gjennomgang av erfaringene i norge (DSB rapport No. 10). Direktoratet for samfunnsikkerhet og beredskap (DSB).

17.

DSB (2018). Guide to the regulation on municipal preparedness duty. The Norwegian Directorate for Civil Protection and Emergency Planning (DSB).

18.

Endsley

(2015). Situational awareness misconceptions and misunderstandings. Journal of Cognitive Engineering and Decision Making, 9(1), 4–32. https://doi.org/10.1177/1555343415572631

19.

Endsley

M. R.

Jones

(2012). Designing for situational awareness. CRC Press Taylor & Francis Group.

20.

Engel

Van Riet

BernuzziDietrich

M. G.

De Boeck

Decouttere

Vandaele

(2022). Throughput assessment of a Covid-19 vaccine manufacturing network: A COVAX vaccine supply case study. Submitted to Flexible Services and Manufacturing.

21.

EU (2021) Securing Access to Vaccines. EU Website Https://Ec.Europa.Eu/: https://ec.europa.eu/info/live-work-travel-eu/coronavirus-response/public-health/eu-vaccines-strategy_en

22.

Flach

Voorhorst

(2020). A meaning processing approach to cognition, Resources for ecological psychology. Routledge.

23.

Guttieres

Decouttere

Vandaele

(2022). Total system preparedness: A framework to design vaccine manufacturing and supply resilient to future pandemic. Under review.

24.

Hannah

S. T.

Uhl-Bien

Avolio

B. J.

Cavarretta

F. L.

(2009). A framework for examining leadership in extreme contexts. The Leadership Quarterly, 20(6), 897–919. https://doi.org/10.1016/j.leaqua.2009.09.006

25.

Hoffman

R. R.

Crandall

Shadbolt

(1998). Use of the critical decision method to elicit expert knowledge: A case study in the methodology of cognitive task analysis. Human Factors, 40(2), 254–276. https://doi.org/10.1518/001872098779480442

26.

Hoffman

R. R.

Woods

D. D.

(2011). Beyond Simon’s slice: Five fundamental trade-offs that bound the performance of macrocognitive work systems. IEEE Intelligent Systems, 26(6), 67–71. https://doi.org/10.1109/MIS.2011.97

27.

Hollnagel

Woods

D. D.

(2005). Joint cognitive systems, foundations of cognitive systems engineering. CRC Press.

28.

Hollnagel

Woods

D. D.

Leveson

(2006). Resilience engineering: Concepts and precepts. CRC Press.

29.

Hutton

R. J. B.

Klinger

D. W.

Crandall

(2003). The CognimeterSM: A screening tool for CTA. Klein Associates Inc.

30.

Klein

(2017). Sources of power: How people make decisions (20th Anniversary Edition). MIT Press.

31.

Klein

Calderwood

MacGregor

(1982). Critical decision method for eliciting knowledge. IEEE Transactions on Systems, Man, and Cybernetics, 19(2), 462–472. https://doi.org/10.1518/001872098779480442

32.

Klein

Wright

(2016). Macrocognition: From theory to toolbox. Frontiers in Psychology, 7(January 2016), 54. https://doi.org/10.3389/fpsyg.2016.00054

33.

Lengnick-Hall

C. A.

Beck

(2016). Resilience capacity and strategic agility: Prerequisites for thriving in a dynamic environment. In Resilience engineering perspectives (Vol. 2). CRC Press.

34.

Lengnick-Hall

C. A.

Beck

T. E.

(2005). Adaptive fit versus robust transformation: How organizations respond to environmental change. Journal of Management, 31(5), 738–757. https://doi.org/10.1177/0149206305279367

35.

Lengnick-Hall

C. A.

Beck

T. E.

(2009). Resilience capacity and strategic agility: Prerequisites for thriving in a dynamic environment. In Nemeth

Hollnagel

Dekker

(Eds.), Preparation and restoration. Ashgate.

36.

Ley

Pipek

Reuter

Widenhoefer

(2012). Supporting improvisation work in inter-organizational crisis management. In Procedeedings of the SIGCHI Conference On Human Factors In Computing Systems, Toronto, Canada, 26 April–01 May 2014, pp. 1529–1538. https://doi.org/10.1145/2207676.2208617

37.

Lindblom

. (1959). The Science of Muddling Through. Public Administration Review, 19(2), 79–88

38.

Mendonça

(2007). Decision support for improvisation in response to extreme events: Learning from the response to the 2001 world trade center attack. Decision Support Systems, 43(3), 952–967. https://doi.org/10.1016/j.dss.2005.05.025

39.

Militello

Hutton

(1998). Applied cognitive task analysis (ACTA): A practitioner’s toolkit for understanding cognitive task demands. Ergonomics, 41(11), 1618–1641. https://doi.org/10.1080/001401398186108

40.

Militello

L. G.

Dominguez

C. O.

Lintern

Klein

(2010). The role of cognitive systems engineering in the systems engineering design process. Systems Engineering, 13(3), 261–273. https://doi.org/10.1002/sys.20147

41.

Mills

Salisbury

(2021). The challenges of distributing COVID-19 vaccinations. EClinicalMedicine, 31, 100674. https://doi.org/10.1016/j.eclinm.2020.100674

42.

Ministry of Health and Care Services (2018). National health preparedness plan (In Norwegian). Ministry of Health and Care Services.

43.

Moorkamp

Torenvlied

Kramer

E.-H.

(2020). Organizational synthesis in transboundary crises: Three principles for managing centralization and coordination in the corona virus crisis response. Journal of Contingencies and Crisis Management, 28, 169–172. https://doi.org/10.1111/1468-5973.12294

44.

Naikar

(2013). Work domain analysis. Concepts, guidelines and cases. CRC Press.

45.

Norwegian Government (2014). National preparedness plan – pandemic influenza (In Norwegian). www.regjeringen.no

46.

Norwegian Institute of Public Health (2016). Plan guidance for mass vaccination against pandemic influenza in municipalities and health centres (In Nowegian). Norwegian Institute of Public Health Collaboration with Norwegian Medicines Authority.

47.

NOU (2022). The governments handling of the Corona pandemic – Part 2. Report from the Corona commission (In Norwegian) (p. 5). Norges Offentlig Utredninger.

48.

Quinn

R. W.

Worline

M. C.

(2008). Enabling courageous collective action: Conversations from United Airlines flight 93. Organization Science, 19(4), 497–516. https://doi.org/10.1287/orsc.1070.0331

49.

Raetza

Ducheck

Maynard

Kirkman

B. L.

(2021). Resilience in organizations: An integrative multilevel review and editorial introduction. Group & Organizational Management, 46(4), 607–656. https://doi.org/10.1177/10596011211032129

50.

Rasmussen

(1986). Information processing and human-machine interaction: An approach to cognitive engineering. North Holland.

51.

Schraagen

J. M.

Huis in ‘t Veld

De Koning

(2010). Information sharing during crisis management in hierarchical vs. network teams. Journal of Contingencies and Crisis Management, 18(2), 117–127. https://doi.org/10.1111/j.1468-5973.2010.00604.x

52.

Smith-Jentsch

K. A.

Kraiger

Cannon-Bowers

J. A.

Salas

(2009). Do familiar teammates request and accept more backup? Transactive memory in air traffic control. Human Factors, 51(2), 181–192. https://doi.org/10.1177/0018720809335367

53.

van der Giessen

Langenbusch

Jacobs

Cornelissen

(2022). Collective sensemaking in the local response to a grand challenge: Recovery, alleviation and change-oriented responses to a refugee crisis. Human Relations, 75(5), 903–930. https://doi.org/10.1177/00187267211004688

54.

Vicente

(1999). Cognitive work analysis. Toward safe, productive and healthy computer-based work. LEA.

55.

Weick

K. E.

(1995). Sensemaking in organizationas. Sage Publications Inc.

56.

Weintraub

R. L.

Subramanian

Karlage

Ahmad

Rosenberg

(2021). COVID-19 vaccine to vaccination: Why leaders must invest in delivery strategies now. Health Affairs, 40(1), 33-41. https://doi.org/10.1377/hlthaff.2020.01523

57.

Williams

T. A.

Gruber

D. A

Sutcliffe

K. M.

Shepherd

D. A.

Zhao

E. Y.

(2017). Organizational response to adversity: Fusing crisis management and resilience research streams. Academy of Management Annals, 11(2), 733–769. https://doi.org/10.5465/annals.2015.0134

58.

Wilson

Game

(2011). Local government in the UK. Palgrave Macmillan.

59.

Woods

D. D.

(2002). Steering the reverberations of technology change on fields of practice: Laws that govern cognitive work. In Proceedings 24th Annual Meeting of the Cognitive Science Society, 7–10 August 2002, pp. 1–14.

60.

Woods

D. D.

(2003). Discovering how distributed cognitive systems work. In Handbook of cognitive task design, human factors and ergonomics (pp. 37–53). Lawrence Erlbaum Associates Publishers. https://doi.org/10.1201/9781410607775.ch3

61.

Woods

D. D.

Hollnagel

(2006). Joint cognitive systems: Patterns in cognitive systems engineering. CRC Press.