Abstract
Misalignment between the different levels of complex societal systems (e.g. socio-technical, socio-ecological) are often a barrier to transitioning these systems toward more sustainable behavior. One way to understand alignment is through the Multilevel Perspective Theory of sustainable transitions in sociotechnical systems (MLP) which posits that when a regime (the current “ways of doing” something) exhibits behavior that negatively impacts its surrounding economic, social and environmental landscape, that landscape will exert pressure on the regime to change. This in turn opens the window of opportunity for niche developments such as novel technology adoption or policy changes. Therefore, understanding the relationships between the landscape, the regime and emerging niches can help align these different levels of perspective to facilitate a successful transition toward more sustainable practices. This paper describes a methodology for mapping complex systems such as agricultural systems based on MLP theory, and incorporates a fourth “user” level (e.g. the farm) into this multilevel framework (MLP+F). The methodology is illustrated with a case study example of North Carolina’s lagoon and sprayfield swine waste management regime. By integrating well-established techniques for data collection and analysis, the case study engages a diverse panel of local experts through a modified Delphi approach to identify the relevant concepts and their cause-effect relationships for the current regime. These concepts and relationships are then categorized and organized into a multilevel, multi-perspective conceptual model using fuzzy cognitive maps. The conceptual model identifies how stakeholder groups align in their perception of the system, and provides a foundation for future qualitative, semi-quantitative, and visual analyses of the system and how it can be changed through alternative scenarios. Although this methodology was initially developed for agricultural applications, it has potential for application to other complex societal issues where understanding stakeholder alignment and the potential for sustainability transitions through change are needed.
Keywords
Introduction
In the agriculture industry, operation ownership largely remains with individual families spread across geographically rural regions (Nikolitch, 1969; Whitt et al., 2019). Thus, agricultural systems have important social and cultural components that both influence and are influenced by various factors at different scales, such as global trade, climate change, national food security, state and local rules, the regional communities and ecosystems surrounding farms, and the individual farmer’s agency in their own decision-making processes. There is a growing imperative to transition agricultural systems toward greater sustainability and resilience that protects the natural resources and climate conditions upon which agriculture itself depends (Rockström et al., 2017; Struik et al., 2014). Yet, persistent barriers such as established practices, sunk costs, regulation, and existing infrastructure continue to hinder transitions to more sustainable technology and practices (Deviney et al., 2021). To be successfully adopted, these strategies and technologies must functionally align with both the drivers for change and all of the different levels of the system affected by change.
Sustainability transitions depend on the interconnectedness of multilevel sociotechnical systems, where the elements that comprise the system involve “technology, science, regulation, user practices, markets, cultural meaning, infrastructure, production and supply networks” (Geels and Kemp, 2007). Understanding how sociotechnical systems function and evolve has garnered a significant body of research spanning multiple fields of interest including transportation, healthcare, business, economics, agriculture, climate change, and government policy (El Bilali, 2019; Geels, 2002; Li et al., 2015; Pereno and Eriksson, 2020). Yet, many of these transition studies do not provide useful insights for individual technology designers, researchers, or decision-makers working to introduce change at the user level.
A key challenge is understanding how to integrate an innovation or technology into an existing system, as well as its potential impacts. For example, markets must be identified and developed to support the technology, consumers educated on its use and benefits, and policy-makers convinced to adjust legal infrastructure to facilitate adoption (Kemp, 1994). Innovation must also overcome the inertia of an entrenched system with its established set of rules, actors, and ways of doing that drive system behaviors along the current trajectory. This system and its trajectory are collectively known as a sociotechnical regime (Geels and Schot, 2007). Driven by negative interactions with the landscape in which it is embedded, it is the regime which most innovations attempt to modify or replace. However, the transition process may be impeded by a lack of in-depth knowledge about how a system functions, uncertainty among designers regarding what users actually need or require, and the marketability of a technology when market potential for new products and services is not easy to define or quantify (Deviney et al., 2021; Kemp, 1994).
Multilevel perspective theory as a framework for modeling complex systems
Problems that arise in complex systems are often referred to as “wicked” because they are difficult to define and the solution space is often limited to how the problem is perceived, which itself may only be a symptom of other issues (Rittel and Webber, 1973). Wicked-type challenges have been studied in the context of complex sociotechnical systems (Geels and Kemp, 2007). In these systems, technology—the application of science for practical purposes—evolves in response to societal needs, and society’s embrace of technology facilitates its successful adoption. The Multilevel Perspective Theory of Innovation (MLP) was developed to examine this evolutionary process and what drives technology adoption from different levels of perspective within a sociotechnical system (Geels, 2002; Rip and Kemp, 1998). These levels of perspective include the current sociotechnological regime, the landscape surrounding the regime, and the niche space in which innovation is conceived, developed, and eventually emerges to alter or supplant the regime (Geels, 2002). Analysis of historical transition processes suggests that the alignment of purpose and behavior among the three levels facilitates successful technological transitions.
MLP has since become a focal theory for understanding how the interactions of social, economic and environmental needs, as well as policy, innovation, and destabilization work to transition sociotechnical regimes toward more sustainable behavior (Geels, 2019). Current sustainability transitions differ from historical sociotechnical transitions in that there is a need to incentivize the private sector to respond for the public good (Geels, 2010). Additionally, the sheer number of potential niche (“green”) technology options strain both the alignment and directionality of the system.
MLP+F: The argument for a fourth MLP level in agricultural systems
Within agricultural systems, the farm is often the end user of any technological advancement or innovation. Yet, the MLP is somewhat unclear regarding this user position within its framework. A frequently applied description of the regime includes the user in a broader group of social actors who contribute to behaviors that drive that regime’s trajectory (Geels and Kemp, 2007). However, the importance of user preferences to the transition process is also emphasized, despite that regimes are portrayed as internally stable systems driven to change through landscape pressure (Geels, 2002). This implies that the user stands apart from the regime and should therefore be part of the landscape.
As the user of the regime technology, it is conceivable that the farm could exert pressure on an existing regime to meet evolving user preferences. However—and particularly where environmental sustainability is concerned—farms are not necessarily the drivers for change but, like the regime, are recipients of landscape pressure to change. While this suggests that the farm should indeed be part of the regime level, the pressure exerted by the landscape is not to alter the farm’s purpose. Rather, it is the farm’s use of a particular regime technology that the landscape is pressuring to change.
Additionally, Christen et al. (2015) illustrated the lack of farmer agency in decision processes through a research study regarding the ineffectiveness of agricultural environmental regulation due to lack of farmer compliance. In their study the authors described non-farmers as relevant stakeholders “not involved in the farm practices themselves, but that can influence one way or another, the way that regulation is designed or communicated to farmers.” In comparing fuzzy cognitive maps developed by farmers and non-farmers, both separately and combined, the authors were able to show significant differences between the two groups’ perspectives and understanding of system dynamics. Particularly noteworthy was the discovery of how little consideration non-farmers gave to cost and complexity as barriers to uptake of regulatory measures by farmers in terms of funding access or maintaining business viability.
Thus, it is often the case that the farm itself may have little influence on either the landscape or the regime. Nor is the farm typically a driver for or against change unless that change impacts the farm’s ability to meet its own needs. Where the farm may exert influence is as the prospective user of a niche technology. Therefore, a successful niche-level innovation must meet the needs and constraints imposed by farm operations, while also satisfying the landscape’s demand for change in regime behavior. To do this, the innovation must somehow accommodate, integrate into, or supplant the existing regime without negatively impacting either the farm or the landscape. Separating the farm into its own level within the MLP framework (Figure 1) can give designers and decision-makers a more nuanced understanding of what an innovation or change must be able to do, and how it is constrained by the complex relationships within the system.
Relationships between MLP+F levels in an agricultural sociotechnical system.
The argument for including a farm level in the MLP framework is not a challenge to the validity of the MLP as a sociotechnical transition theory. Rather, the purpose for using the MLP as a theoretical frame for the methodology described herein is to exploit the practicality of the MLP’s multilevel approach as a foundation for conceptualizing and visualizing a complex agriculture system. Adding the farm level is a means to facilitate this process and to help define those farm-centric needs that, in addition to challenges at the other levels of MLP, introduce potential barriers to adoption for novel technology.
Purpose
Examining complex systems through the lens of MLP theory may help identify barriers and potential leverage points that are otherwise outside the scope of the typical problem definition and solution space. The MLP+F is used as a framework upon which to build a mixed methods research strategy for analyzing complex systems, and specifically those in agriculture. The strategy integrates known methods for stakeholder identification and engagement, data collection, and system analysis. With this methodology, a conceptual model of the system is constructed from relevant concepts (nodes) and relationships (edges) obtained from the diverse expertise and experience that already exists within the system. The model may then be used to analyze system relationships and simulate alternative scenarios.
Methodology
This section details a three-part methodology for the development and use of a conceptual model to improve awareness and understanding of complex agricultural systems by:
■ Applying the levels of MLP+F to the system of interest to identify the scope of the landscape, the specific regime, a niche problem space, farm type(s), any assumptions or information gaps related to the problem, and the expertise needed to address those assumptions or gaps.
■ Using a Delphi approach to collect, organize and categorize data from experts into a series of subsystems represented by fuzzy cognitive maps (FCMs) that can be combined to create the conceptual model organized on the MLP framework.
■ Analyzing system behavior or define and run alternative scenarios to identify potential change pathways across the different MLP levels that support technology adoption.
The methodology is illustrated through a case study to develop a conceptual model of North Carolina’s swine waste management system (NC SWM).
System setup using MLP+F
The first step of the methodology establishes the system of interest, its scope and scale, how the system relates to the four levels of MLP+F, associated assumptions about the current system, and potential gaps in knowledge regarding the system. It uses this information to identify stakeholder groups who have relevant expertise and experience to describe the system.
The landscape, as the largest and most diffuse level, defines the scope of the system to be studied. Landscape components should be limited to a boundary that is relevant but exogenous to the regime (Geels and Schot, 2007). It is important when defining the landscape to adequately capture the diversity across the landscape’s sphere of influence. The boundary may encompass different factors and institutions that directly impact or are impacted by the regime but are not exclusively in service to the regime (e.g. academic institutions or government agencies). Landscape components may benefit from the regime (e.g. associated industry), be negatively impacted (e.g. ecosystems), or both (e.g. local communities). Landscape components are not confined to human enterprise and can include the natural environment, geographic regions, land use, climate, and ecological factors as well as cultural boundaries, economic factors, industries, communities, and social institutions (e.g. political, academic, religious, financial, government). While it may be useful to group closely allied components as a single entity to reduce the number of components, grouping should be done with caution because dynamics within the landscape play an important but often overlooked role in both pressure on the regime and niche development (El Bilali, 2019). These dynamics arise from internal conflicts between landscape entities or synergies between such entities. When aligned, landscape components can accelerate niche development and catalyze regime change. When misaligned, landscape drivers work against each other, muting pressure on the regime and hindering sustainable transitions.
The regime can be defined in the context of the established landscape boundaries. The sociotechnical regime includes the routine, regulations and standards, adaptations, and infrastructure built around an established technology (Geels and Schot, 2007). The regime supports the underlying technology at the root of what is negatively impacting the landscape and driving the need for change. It is important not to conflate the technology itself or its regime with an industry that utilizes the technology, a niche technology that is not yet established as part of a regime, or the user of a technology, particularly if that user can elect to use other technologies (whether or not they exist). Interrelated components or concepts between the landscape and regime are expected. For example, a regulatory agency itself may be part of the landscape, but a particular regulation enforced by that agency may be designed to control some aspect of the regime.
When the regime and landscape have been established, the investigator may choose to focus on a specific niche or even a technology within that niche to address a problem derived from regime behavior. A niche is the “window of opportunity” within the landscape for innovation to develop that occurs when a regime is unable to adapt its behavior to the landscape’s needs (Nykvist and Whitmarsh, 2008). Per MLP theory, if the niche space is sufficiently protected and supported by the landscape, it may produce innovative solutions to transition the regime toward a more sustainable path. However, successful technological transitions require a critical alignment of goals across the landscape, the regime, and the niche (Geels and Schot, 2007). In agriculture systems, if the innovation will be deployed at the farm level, then the technical and economic feasibility for the individual farm must also be included in the overall system alignment.
The farm level provides a more focused understanding of the needs and decision making processes specific to the farm operation itself. Even among operations producing the same commodity (e.g. corn, cotton, pork) there is diversity in geography, farm size, ownership, and management styles that have a strong influence on farmer or producer responsibilities, decision-making, risk aversion, diversification, and interest in innovation, among other factors (Karan, 2011). Farmer values also play a key role in the decision-making process and reach beyond just economic or business considerations to include traditions, ethical behaviors, community pride, and personal traits (O’Connell et al., 2017).
Assumptions and missing information
Identifying the potential pathways and barriers to both stakeholder group and stakeholder-system alignment is one of the challenges this process seeks to address through a multi-perspective view of the current system. To begin that process, it is necessary to understand what assumptions investigators are making about the system itself and how an intervention or technology they may want to introduce could drive change within that system. A list of assumptions should be created and examined across the MLP+F levels. Critical to this list are the perceived problems with the current regime, any proposed niche solutions, and how these solutions integrate into the system, particularly at the farm level. The list should also include any assumptions about landscape components, how they relate to each other, and how they might impact the other levels. If any gaps in knowledge are apparent, these should also be recognized.
Listing assumptions and knowledge gaps is a form of reflexivity that can help the investigator identify the types of expertise and stakeholder groups needed, as well as their own bias during stakeholder identification, data collection, and analysis. Acknowledging bias is important to the validity of qualitative data and how it is presented (Johnson, 1997). In this methodology, there are two key areas where investigator bias may affect the outcome. The first is related to the Delphi approach and study participant selection, particularly if past experience or personal networks are used (Schmalz et al., 2021). The second is bias in interpreting and analyzing data. The list of assumptions can help the investigator identify where confirmation bias may develop during the analysis. To counter this tendency, the investigator can seek and compare alternative explanations through opposing viewpoints (Merriam, 2009: 219).
Stakeholder groups
This methodology uses a Delphi approach for data collection which requires the assembly of an expert panel. To ensure an adequate breadth of expertise and experience is included in the analysis, all stakeholder groups relevant to the system of interest should be identified. A stakeholder is essentially anyone who has the power to influence a system’s behavior or outcome, or is impacted by those decisions or outcomes (Reed et al., 2009). Operations research provides a useful analogy for complex agricultural systems through its analysis of large complex organizations, where those with the ability to influence a system may be defined as (1) persons who are experiencing the issue and will judge an outcome, (2) persons who are key actors, (3) sources of expertise, knowledge, or protectors of important data, and (4) someone who can act as a partner to the investigator (Pidd, 2004: 148–150). In sustainability issues related to agriculture systems, the perspective of environmental impact often correlates to the first category, which may in turn affect multiple communities of stakeholders. This aggrieved perspective is typically the source of the “problem” that the investigator is attempting to better understand and potentially mitigate through some intervention. The second and third types of stakeholders are also of interest to this methodology because they are the key actors engaged with the sociotechnical system and those with expertise about the system and its behavior. These are the agents whose knowledge base will help define the system across the levels of MLP+F. The fourth group—a partner to the investigator—is for the purpose of this methodology the conceptual model itself, which will become the tool used to link the investigator’s niche perspective with insights about the system.
Thus, the pool of potential stakeholders for a Delphi study can be quite broad. Reed et al. (2009) provide a detailed review of methods for identifying stakeholders and challenges associated with stakeholder identification. They note, for example, the tendency in stakeholder analysis literature to presume that the stakeholders in a system are already known, and so focus more on categorizing stakeholders and their interactions. As Reed et al. (2009) point out, researchers must first have knowledge of the system in order to know who has a stake in that system.
Since the purpose of this methodology is to develop that system knowledge, a simple but comprehensive approach is needed to identify stakeholder groups beyond “the usual suspects” (Reed et al., 2009), and to avoid the investigator bias previously mentioned. Using the operations research analogy, the key actors and relevant expertise, as well as those impacted by the perceived problem, will align closely with the system components that were identified within the MLP+F framework in the first step of the methodology. For example, if the landscape includes a government component, a stakeholder group representing that component may be regulators. While not all regulators are associated with the regime of interest, specific agencies that oversee the regime become a source for identifying individuals to participate in the expert panel, who then represent a part of the government component. Some components may not be part of an institution or other easily identifiable agent but can still find representation in social organizations. For example, negative environmental impacts imposed on the landscape by the regime may be represented by environmental and social justice organizations actively engaged in seeking recognition of the problem and those impacted by it.
The initial list of stakeholder groups should include representatives related to all system components identified through the MLP+F levels and the list of assumptions. As the steps of the methodology progress, additional iterations may be required to reach a satisfactory list that is broad enough to encompass the required diversity of perspectives, yet specific enough to reduce—but not necessarily eliminate—overlap and potential conflict of interest or perceptions within groups. Some overlap between perspectives is expected because some groups may appear to extend across more than one MLP+F level and individual participants can self-identify with more than one stakeholder group. Indeed, this “blurriness” is characteristic of the complexity of the system being analyzed. Thus, both the MLP+F frame and the effort to “group” stakeholders are somewhat arbitrary assignments designed to help break down a complex system into manageable chunks for visualization and analysis, but not into separate or self-contained parts.
The Delphi process
In the second phase of the methodology the data are collected, compiled into FCMs, and the maps consolidated into the overarching conceptual model of the system of interest. Data collection is done using a Delphi approach, in which a panel of experts is engaged to provide and validate information about the system. Introduced in the 1950s, the Delphi technique was developed as a method for using policy and national security experts to forecast how technology could impact the Cold War (Lund, 2020). Delphi has since become a preferred method for gathering and using expertise in scholarly research because of its participant anonymity and feedback aspects, and its flexibility in application (Gordon, 1994). Researchers across multiple disciplinary realms, including policy, medicine, ecology, and business, have used Delphi for not only forecasting outcomes, but also finding issues, identifying opposition, and in decision-making (Lund, 2020; Mukherjee et al., 2015; Okoli and Pawlowski, 2004; Yousuf, 2007).
Although the goal of most Delphi studies is to build consensus within a panel of experts, it is not necessarily the goal of consensus itself that validates the use of Delphi, but rather certain properties of data collection needs that cannot otherwise be fulfilled. Linstone and Turoff (1975) specifically note such occasions where individuals with diverse backgrounds are needed to contribute to the understanding of broad or complex issues, where the heterogeneity of the participants must be preserved (i.e. not dominated by any individual), or when (potential) disagreement between individuals requires a mediated and anonymous process. These properties are inherent to the data collection goal for this methodology, which is to capture the broadest spectrum of concepts and relationships in the system of interest as possible within the constraints of available time and resources.
Delphi uses multiple rounds of data collection, consolidation, and feedback such that the identity of individual participants are kept anonymous from the rest of the panel, but each participant is able to comment on the whole panel’s input. The general idea is that a composite of expert opinions about a subject or system will be more accurate and complete than the input of any individual (Gordon, 1994). There are numerous resources in the literature for how to conduct a Delphi study, evolving over the decades since conception to include modifications tailored to individual studies regarding expert identification, methods of data collection, analysis, and validation (Gordon, 1994; Linstone and Turoff, 1975; Lund, 2020; Yousuf, 2007). Because the context of a study is key in determining the details regarding methods of inquiry and data interpretation, the procedure recommended for this study follows, including the rationale for the options selected. Specifically, rather than pursue consensus, data should reflect the panelists’ differences, which in turn can be used to track tendencies toward or away from alignment within and across the MLP+F levels.
Selecting an expert panel
Scholz (2018) considers expertise in terms of its social relevance, such that experts are defined by their understanding of a particular domain, affording them certain characteristics such as pattern recognition, situational discrimination, cognitive authority for a lay audience, and the ability to make reliable forecasts, among other traits. Experts may have special training, or they may be laypersons with extensive experience related to the system of interest, but are generally thought to be people who have substantially more knowledge and experience than average in a specific area (Jetter, 2006). This interpretation of expertise fits the purpose of this methodology, in which experts are individuals who exist within the system of interest and who regularly engage with the regime, or are impacted by the regime at the farm, niche, or landscape level.
Table 1 lists several methods of stakeholder identification to facilitate the process of finding prospective participants. The potential for researcher bias in panel selection is a common concern in Delphi (Schmalz et al., 2021; van Zolingen and Klaassen, 2003). However, combining search methods can improve the quality of the expert panel and reduce this type of bias. Using the stakeholder group list to create a matrix of needed expertise and then matching each group with qualified experts sourced from known connections is an example (Okoli and Pawlowski, 2004). Participants should self-identify which stakeholder groups they align with to reduce panel bias that might result from the investigator’s assumptions about an individual’s group affiliation.
Methods of stakeholder identification.
An important caveat of Delphi and of this methodology is that the data collected are not intended to produce evidence with any statistical significance. Rather, the value of the data generated lies within the collective judgment of the experts engaged. Unlike statistical study which requires a representative sample of a population, Delphi panels are often relatively small (Gordon, 1994). Because the findings drawn from Delphi are taken as expert opinion rather than factual evidence (Powell, 2003), this presumed subjectivity may challenge data validity. However, well-cited studies consider expert judgment not only a useful resource when means are limited, but essential under circumstances where a situation is unique, uncertain, or other data are simply unavailable (Burgman et al., 2011; Mukherjee et al., 2015).
Therefore, the purpose for the modeling enterprise must be considered. Conventional quantitative modeling methods struggle with the imprecision, uncertainty, and nonlinearity of complex systems. Soft-computing techniques like FCM have an advantage for modeling complex systems due to their ability to incorporate fuzzy logic, neural networks, and knowledge based systems (i.e. expert systems) (Stylios and Groumpos, 2002). These techniques are able to capture and utilize the qualitative knowledge of diverse experts, each contributing to a part of the overall problem space and describing the portion of the system about which they are most comfortable (Perusich, 2020).
There are few guidelines for how many panelists are needed, however at least two perspectives for each stakeholder group are necessary to make any comparison of the data used to estimate alignment within and across the MLP+F levels. To maximize the use of resources, adequate representation may be achieved in part by engaging individuals who identify expertise across multiple stakeholder groups. In an ideal scenario, by the final input no new information about the system is being introduced (Merriam, 2009: 80; Özesmi and Özesmi, 2004). In practice, this rule is rarely followed. Lund (2020), in a review of peer-reviewed classical and modified Delphi studies concluded that between 14 and 36 panelists is sufficient. One FCM case study was demonstrated using input from only two experts (Amer et al., 2011). Panel continuity is not required—where every panelist participates in all rounds from the beginning of the study—in that it is the diversity among stakeholder group perspectives and not the individual participants that matters. An investigator may add panelists between rounds, panelists may elect or be invited to participate in some rounds but not others, or completely different panels may be used for each round (de Loë et al., 2016). One advantage of this method of data collection is the ability to engage with stakeholders remotely through online or virtual platforms.
The interview round
The first round of this Delphi is a one-on-one, semi-structured interview between the investigator and each individual participant, guided by a set of exploratory questions to describe the system of interest and any desired change to the system. By preserving individual anonymity, each participant is given the opportunity to express themselves freely without the influence of other members of the panel and to describe the system’s elements and relationships in their own terms. The investigator may probe concepts brought up by the participant if they represent novel insights.
Interviews do place a burden on the investigator as the research instrument with regard to interviewer-respondent interaction and the potential influence of researcher bias (Merriam, 2009, Ch. 5). Preparation is therefore essential (e.g. practicing interviews with non-participants) so that the investigator can focus on the participant and the situation rather than the interview process itself. A prepared investigator will better manage the unexpected, whether that occurs as a technical difficulty or a sensitive reaction to a seemingly neutral question (Mikėnė et al., 2013). After each interview, a peer-debriefing session with someone whose opinion is trusted but impartial can further help the investigator identify how their personal perspective may have influenced the questioning and its outcomes (Spall, 1998). The complementary practices of reflexivity and debriefing support the credibility of both the investigator as a research instrument and the resulting data (Merriam, 2009: 219–220).
Interview coding takes an exploratory perspective, adopting the four-part methodology presented by Carley and Palmquist (1993). Data are represented by a scheme containing four basic objects: concepts, relationships, statements, and maps. Concepts are simply ideas that represent elements in the system and are typically nouns (Sperry and Jetter, 2019). Relationships link two concepts together and will take on several traits including directionality, sign, and strength. A statement consists of two concepts and the relationship between them. A map is a network of statements. This coding scheme is designed to elicit mental models from qualitative data and therefore works well with the underlying premise of fuzzy cognitive mapping. A detailed set of rules for coding data from documents to create cognitive maps can be found in Wrightson (2015).
The four-step process of the Carley and Palmquist (1993) methodology is: (1) identify concepts, (2) define relationships, (3) extract statements, and (4) create maps. Several challenges may present while coding the interviews. For example, not every concept mentioned is involved in a causal relationship. To be a cause or effect, a concept must be a variable that can be identified by its ability to take alternative values (e.g. increase or decrease). Concept alternatives should be made explicit if not clearly implied by the formulation of the concept (Wrightson, 2015). Similarly, relationships are often implied or must be inferred from what participants say. Because relevant concepts are variables, identifying what will cause the variation is sometimes enough to describe the relationship. It may be helpful in reviewing transcripts to consider the first three steps simultaneously (i.e. identifying concepts and their relationships as statements) to make meaningful connections between causes and effects.
The same coding process can be applied to the second part of the interview as well. The key difference is that effect concepts are the desired change(s) to the system stated in the interview, and if provided, cause concepts are how the participant would enact the change(s). These statements are kept separate from the system statements as they will be used later to develop alternative scenarios for analysis.
It is not necessary to wait until all interviews are complete to begin coding. Coded data should be tagged with a unique participant identification code (e.g. P1, P2, . . ., Pn) that does not explicitly identify the participant but can be used to track their input in the event that interview transcripts need to be revisited for clarification. It is permissible and encouraged to identify common concepts across interviews, although care must be taken to preserve the intent and to keep each set of interview data separated at this phase. The final step is to develop a fuzzy cognitive map (FCM) for each participant’s set of statements. Figure 2 illustrates a simple FCM where the edges between nodes have discrete values of {−1, 0, 1}. The individual maps created in this step represent each participant’s mental model of the system.
A simple FCM created in Mental Modeler (a) and the associated adjacency matrix (b).
Every node in each individual map must be connected, directly or indirectly, to every other node. No node or subset of nodes can be independent of the rest of the map (Ackoff, 1971). The individual maps are used to ensure that the statements presented by each panelist are, indeed, related. For statements that are not connected to the map, additional relationships must be inferred by the investigator. Links between unconnected statements may be obvious, or they may require additional consideration such as whether to combine similar concepts or add an intermediate concept to make a logical connection. Alizadeh and Jetter (2017) provide detailed guidance for identifying and closing gaps between unconnected elements in a map. Visually drawing the maps is the best way to identify missing connections. Individual maps typically contain between 50 and 100 statements, so it is beneficial to use a software tool that supports this type of mapping (e.g. Mental Modeler (www.mentalmodeler.com, Gray et al., 2013)).
Statements based on the FCM created for each interview are member-checked by returning them to the respective panelist in the form of a survey. Member-checking allows each participant to review the statements from their individual maps and verify whether those statements agree with their perspective. The member check also serves to limit investigator bias that may have been introduced during the extraction of data from the interview transcripts (Birt et al., 2016; Merriam, 2009: 217–219). Statements are written: “The presence of or an increase in x increases/decreases y,” where x is the cause concept, which always increases, and y is the effect concept, which either increases or decreases depending on the sign of the relationship. Participants are asked to agree or disagree with each statement, and if they disagree, to delete or alter the statement as needed. Likewise, the participants’ recommended changes collected in the second part of the interview are also presented in the survey as cause-effect statements for verification. For each set of statements (system description and change), participants are also given an opportunity to add new statements. In this way, any inferences made by the investigator regarding the participant’s interview statements or the map connections are validated by the participant.
Thematic categories that begin to emerge within the list of system concepts represent different perspectives within the MLP+F structure of the system and will form the basis of subsystem maps for the model. Using subsystems in multi-level FCMs has been demonstrated as a useful method of managing complexity and improving the resolution of large system models (Christoforou and Andreou, 2017; Mateou and Andreou, 2005). It is important to recognize that subsystems are not independent entities from the system as a whole and must be connected to other subsystems within the model.
The number of concepts assigned to a subsystem varies, although subsystems with many concepts can become burdensome on panelists in the second round survey. Each category should affiliate with a particular stakeholder group as the groups are representative of those perspectives, and each subsystem should align with one of the MLP+F levels. It is likely, however, that the concepts and statements derived from each expert’s input will reach across multiple categories. This tendency speaks to the interconnected nature of the system concepts, of the MLP+F levels, and the fact that individuals often view the system from multiple perspectives.
After the member check, concepts and statements should be aggregated into a final list, with duplicates consolidated. It is no longer necessary to track the panelists’ individual contributions. Assigning each concept a unique identifier by category is useful in the analysis phase when full concept descriptions can be visually or analytically distracting. Change statements may also be grouped or consolidated. Tracking how often a particular goal or recommendation was suggested may be useful for identifying potential pathways to change.
The final step is to construct subsystem FCMs in the same fashion as the individual maps, ensuring that all concepts and relationships within the map—and ultimately the conceptual model—are connected. As with the individual FCMs, any statements that lack connections must be analyzed to determine how they belong in the system. To capture the connections between subsystems, each subsystem FCM should include only causal connections (drivers of change) to other subsystems. This prevents duplication of relationships when subsystems are combined.
The survey round to determine FCM values
Subsystem FCMs provide the statements for the second round survey. The survey protocol is divided into “blocks” with each block representing a subsystem. Each block contains three question sets. The first set contains the subsystem statements for which panelists estimate the amount of influence a cause concept has on the effect concept, using a Likert scale from none to major. The second question set presents only the subsystem concepts for panelists to rate how easily the concept could be altered by human intervention on a Likert scale from impossible to effortless. The third question gives panelists an opportunity to comment on the subsystem statements. Each block also provides the actual FCM as a visual representation of the subsystem. Panelists can reference these maps to help them make sense of the concepts and statements which, after aggregation and consolidation, may appear different from their individual member check surveys. Change statements are not included in the second round survey.
Responses are converted from a qualitative scale to quantitative values in the range of [0, 1] for the concept ratings and [−1, 1] (depending on decrease/increase) for the relationship statements. The values are then added to the concept and statements spreadsheets. Values for each statement or concept are averaged, the median value calculated, and the range of the difference between the highest and lowest values determined. These centrality measures become the fuzzy weighted values of the subsystem FCMs. If alternative scenarios that change one or more system concepts are developed for the model, the statement spreadsheet can be used to create an adjacency matrix representing the full set of cause-effect statements to calculate relative changes in concepts and infer possible outcomes of change to the system.
Case study
The system of interest for the case study is the lagoon and sprayfield swine waste treatment regime typically used in North Carolina. There is ongoing landscape pressure to shift swine operations away from the use of this regime, yet despite decades of research and investment, lagoon and sprayfield continues to be the primary waste treatment method for commercial hog operations in the state (Deviney et al., 2021).
The case study scope is limited to pork production and swine manure management within North Carolina. This constrains the landscape to factors and institutions within the geographic boundary of the state, except where external influences may have a localized impact, such as federal policy incentives related to sustainable agriculture or the companies that contract with swine farms (i.e. integrators) whose operations are critical to but also extend beyond North Carolina. Farms permitted prior to a 1997 moratorium on new or expanded farms are allowed to continue to use anaerobic lagoons but any new swine farms must meet five strict environmental performance standards for waste treatment (NCAC, 2009). Poultry production was not included in the new rules for wet waste management because most poultry operations use a dry litter system. However, North Carolina now ranks among the top three states for total swine and poultry production, and poultry is the top agriculture industry in the state (North Carolina Poultry Federation, 2021). The combination of concentrated high-density livestock operations, integrated production management, and relatively strict government oversight of anaerobic lagoons all contribute to the unique context of North Carolina’s pork production and swine manure management system. The impact this high density of animal operations has on rural, low-income, and minority communities has fostered ongoing environmental justice concerns. Studies have correlated several negative human physical and mental health issues as well as quality of life and property value impacts to living near CAFOs (e.g. Casey et al., 2015; Schaffer et al., 2008; Wing et al., 2008). These issues often culminate in legal action from both residents and activist organizations against the industry at both the state and federal level (Longest, 2005). However, a history of agriculture-friendly policies and right-to-farm legislation often hinders the effectiveness of these legal pressures (Smart, 2016).
The sociotechnical regime is the lagoon and sprayfield system of swine waste management. The lagoon and sprayfield regime has evolved as the dominant technology for swine manure management in North Carolina, with robust supporting physical, regulatory, and economic infrastructure that is resistant to change despite multiple landscape factors exerting pressure to eliminate the use of anaerobic lagoons on swine farms. The anaerobic lagoon is a large open-air and typically earthen structure used to biologically treat and store animal waste that is flushed from barns at daily or weekly intervals. The products of anaerobic digestion are methane, carbon dioxide, and a nutrient-enriched effluent that is land-applied to nearby fields and crops (sprayfield). Recalcitrant material and non-soluble nutrients settle at the bottom of the lagoon as sludge. Aside from the technology itself, the lagoon and sprayfield regime also includes regulatory aspects, equipment and operation maintenance processes, education resources, and technical assistance.
A niche development in the case study is renewable natural gas (RNG) production. The technology recovers energy from swine waste to significantly reduce greenhouse gas emissions from swine farms but does not manage nutrient recovery from animal waste differently than the lagoon and sprayfield regime. Digestate—the residual waste that remains after biogas recovery—is typically returned to the old open lagoon for storage until land application, so that potential environmental impacts from nutrient loss (e.g. ammonia volatilization) and flooding remain.
The farm level is focused on agricultural operations that include raising swine as part of their enterprise. Today’s swine farms typically specialize in one or more pig growth stages, from sow farms where piglets are produced to finishing farms where weaned pigs are grown to slaughter weight. In 2020, the NC Department of Environmental Quality listed approximately 3500 lagoons (NC DEQ, 2020), with over 50% located in just two counties in the southeastern part of the state. The average North Carolina swine farm has approximately 50 acres of permitted sprayfields (Christenson and Serre, 2017), although farms may have more acreage for other production activities. Over 80% of North Carolina swine farms are family owned and 87% of farms operate under contracts with pork integrators, but are not integrator owned or operated (USDA NASS, 2021a, 2021b).
In implementing the methodology, assumptions associated with the investigator’s understanding of North Carolina’s swine production system and waste management regime span all MLP+F levels (e.g. alternative technologies exist that can replace the current regime and reduce environmental impacts of swine operations; there is a desire among multiple stakeholder groups, including producers, to find cost-effective alternatives to lagoon-sprayfield waste management). The initial system analysis also included many stakeholder groups, such as the following at the landscape level:
■ local community members, including residents and businesses impacted by the regime
■ academic institutions researching and quantifying regime ecosystem and social impacts
■ agents who engage in outreach or facilitate communications about the regime’s environmental impacts
■ the pork industry in which the regime is embedded
■ the broader agricultural community utilizing or affected by regime outputs
■ societal factors such as economic drivers, land use, social and environmental justice activism, and customer preferences
■ the natural environment, local ecosystems, and climate, whose “interests” may be represented through several of the aforementioned parties such as regulators, researchers, and activists
A total of 10 stakeholder groups were identified based on assumptions and knowledge gaps associated with the case study (Table 2). During recruitment, a matrix was used to ensure that a minimum of two panelists represented each stakeholder group throughout the Delphi process, even though some additional recruitment was required due to attrition between the interview and survey rounds. Experts were identified using a combination of online searches of agencies or organizations associated with the stakeholder groups, individuals associated with the investigator’s network, and snowballing. A recruitment survey was used to allow panelists to self-identify their expertise in one or more of the stakeholder groups. This survey identified one additional group for biogas production. Tracking responses helped to alleviate potential mis- or under-representation of an individual’s expertise because of investigator assumptions.
Final list of stakeholder groups identified for the NC pork production & swine manure management case study.
From approximately 30 invitations, a total of 17 experts were recruited for the Delphi panel over the course of the case study. Fourteen panelists participated in the interview round and 15 in the survey round. Participant recruitment and data collection for the case study were conducted in 2021 while some organizations and institutions in North Carolina were still subject to pandemic-related restrictions. Thus, solicitations of and communications with stakeholders were kept to social-distancing compliant methods (e.g. emails, video conferencing, phone calls) rather than in-person. Each interview was recorded and transcribed except for two interviews where recordings were not available. For those interviews, field notes taken during the interviews were used in lieu of transcriptions.
Individual FCMs were developed from the interviews but were not provided to participants for the member-check because they were not part of the initial interview process. Participants were given 2 weeks to respond to the member-check survey, with notification that a lack of response would result in the assumption that they agreed with the statements and had no changes to make. Ten of the 14 interviewees returned their member-check surveys. Although one participant deleted a significant portion of their survey statements, most chose to keep the majority of their statements. Several participants took the time to elaborate on some statements in substantial detail to clarify particular concepts. Often this involved adding language that contributed to the fuzzy nature of the concepts themselves. For example, the concept of “misapplication of swine sludge” was elaborated to, “(potential for) inaccurate or misapplication of swine sludge.” These elaborations increased the overall number of concepts, but also created greater nuance and depth to the system model.
Concepts were then categorized and consolidated to remove duplication or overlap after the member-check. A cutoff metric of roughly 100 concepts per subsystem was used to determine when a subsystem map might be too large to work with. However, even 100 concepts in a map proved unwieldy in the second round survey, suggesting a lower cutoff number is preferable.
Each of the 11 subsystem maps was hand drawn in Mental Modeler (mentalmodeler.com, Gray et al., 2013). The FCMs created were then converted to text searchable pdf files for distribution with the second round survey. A spreadsheet table was developed to determine how to distribute the 11 subsystem maps among the participants so that (1) each participant’s survey was relatively similar in length, (2) each participant reviewed data relevant to their expertise, and (3) each subsystem is given to at least three participants to improve the likelihood that there are at least two responses per subsystem for comparison, even if some participants fail to respond to the survey. The table assigned subsystems to participants based on the total number of concept and relationship statements in each, the categories associated with each participant’s self-selected stakeholder group(s), and the contribution each participant made to each subsystem’s concepts and relationships. Analysis of these three metrics was then used to assign each participant two or three maps related to their expertise and input and each map was assigned to three participants.
A unique and expansive data set was curated from the expert panel regarding North Carolina’s swine manure management system. The data set for the subsystem maps consists of 553 concepts and 1214 relationships among them. Interestingly, few panelists utilized the commentary section of the survey (the third question in each block), and no significant changes were made to the concepts or relationships based on comments made there. However, after the second Delphi round four concepts were eliminated by virtue of the panelists assigning a null value to the relationships which included those concepts in the system, with a final tally of 1185 system connections.
Alignment in the conceptual model
While the cognitive model is not a complete representation of the system, it embodies the current understanding of the system from the diverse expertise that contributed to it. A key tenet of MLP theory is that system alignment among these perspectives fosters the conditions that promote sustainable transitions. Alignment is when all of the agents, elements, and processes in a sociotechnical system are working together toward common purpose in “configurations that work” (Rip and Kemp, 1998: 330). Alignment is not the same as consensus, wherein stakeholders agree on an issue or course of action. Instead, alignment is about understanding where differences in perceptions lie and how far apart those difference are. The extent of the alignment within and between the MLP levels determines the speed and type of transition that occurs (Geels and Schot, 2007). The aggregated data can therefore be used to examine how the panelists’ perspectives align within and across the MLP+F levels, providing insights into the possibility for transitioning the regime to a more sustainable trajectory.
Alignment may be gaged by how well participants agree in their assessment of the influence of one concept on another and of an individual concept’s ability to change. This agreement is evaluated using the values participants assigned each statement or concept. Alignment is highest when respondents give the same value for influence or change and decreases as the range in responses increases. Low levels of alignment indicate inconsistency in system understanding, revealing potential barriers to sustainable transitions. High levels of alignment may indicate a more established understanding of system dynamics and possible opportunities to promote certain drivers of change.
Figure 3 visualizes how case study panelists’ individual stakeholder group perspectives aligned within each of the MLP+F levels with regard to the influence a cause concept has on an effect concept. It is important to note that Figure 3 does not indicate the perceived amount of influence in statements, only how similar the panel responses were to each other. Looking at the four MLP+F levels, the regime appears to have the greatest amount of alignment, while the niche level has the lowest. The regime is well-established and well-regulated, so high alignment regarding influence behaviors is expected, though there is less agreement regarding the ability to change regime-related concepts (not illustrated), particularly regarding existing infrastructure and practices. The opposite is true for the niche level, where there is strong alignment regarding the ability to activate niche-level nodes, but much less agreement on how much influence change would have on the system. Unlike the regime, a niche is generally fraught with uncertainty as new interventions or technologies strive for a foothold in the system to either transform or replace the existing regime. Disagreement in this niche correlated most to the confidence respondents had in biogas production and RNG initiatives to influence sustainable manure management and potential farm income. Despite this, there was considerable alignment regarding how easy or difficult concepts in this level would be to activate. Within the landscape level, it appears that generating greater alignment within and between the society and government subsystems could help foster the necessary support from the landscape level needed to implement sustainable change.
Alignment in participant responses to the influence of cause-effect statements within each of the MLP+F levels. Count is the number of statements. Total = all responses are the same, high = slight spread in responses, some = moderate spread in responses, and low = large spread in responses, na = no response provided.
Summary and future work
This paper details a methodology for using MLP+F as a frame for constructing a cognitive model of an agriculture-based complex sociotechnical system. The methodology collects and organizes data from an expert panel whose individual perspectives each reflects a unique piece of the overall system puzzle. By combining these pieces, an investigator can develop a more comprehensive portrait of the sociotechnical regime, the surrounding landscape it affects and is affected by, the farm as the regime user, and niche opportunities for change.
A model created with this methodology is a “snapshot in time,” designed to capture the current understanding of the system of interest from those who experience it. It is not a quantified representation of the system which is typically a more resource-intensive endeavor. The distinction is that sociotechnical systems are dynamic, constantly updating inputs, processes, and outputs in real time, while perceptions of these systems and their behaviors—especially across levels—tend to lag real-time changes, especially if channels of communication are weak or closed. This leads to observations of the system based in localized knowledge and experiences, which may not reflect changes in other parts of the system. Thus, decision-making relies heavily on stakeholder understanding of the system and its behavior rather than its actual state.
To accommodate this fluid understanding of complex systems, the methodology presented lays out a path to developing a flexible and dynamic conceptual model. Nodes can be added, refined, combined, divided, clustered into smaller subsystems, or recategorized as needed to make sense of the relationships. Multiple iterations of the model can exist simultaneously. The model can also be updated periodically to reflect advancements or changes in any of the four MLP+F levels. Hence, the methodology presented herein is intended as a tool in the systems thinking toolbox for researchers and investigators across multiple disciplines. The resulting model can improve understanding of complex social systems and their interactions and providing insights to achieving greater alignment between stakeholders to achieve more sustainable outcomes.
Although the focus of this methodology has been on sociotechnical agricultural systems and the integration of a farm level, the techniques discussed herein, including the addition of a user level to the MLP theoretical frame, could be applied to any sociotechnical system or complex social setting. For example, in lieu of the farm level, a “family unit” level might be considered as the user of a particular technological or cultural regime. With this multilevel approach, drivers of change and barriers to niche opportunities can be examined at both the regime level and through the interactions within the landscape. This in turn can help identify impacts on individual agents who engage with and are impacted by both the regime and the landscape and who will ultimately be the ones to adopt or adapt to niche alternatives.
Finally, the case study presented herein demonstrates how the methodology can be applied to animal waste management, developing the NC SWM model and using it to identify areas of misalignment within the system, which is often the result of opposing goals, lack of access to information, or a delay in information transfer. The niche level demonstrated the greatest lack of alignment, indicating that a key challenge is creating common ground between stakeholder groups regarding the types of technologies and best management practices that support sustainable swine waste management. Future work will use the NC SWM model developed to test alternative scenarios identified from the panel’s recommendations for change using inference techniques for fuzzy cognitive mapping.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
