Abstract
Wireless sensor networks are widely used in industrial cyber-physical system installations, where high reliability and the need for real-time data are the two main characteristics. A large amount of real-time data can be transmitted to its destination on time using a reasonable periodic allocation of a node’s transmission slots. However, a flow may miss its deadline when flow conflicts occur. When such missed deadlines occur regularly, system performance may degrade, and when the flow is critical, such data losses can result in errors or cause disasters. To address this issue, we introduce multi-user multiple-input and multiple-output technology and a mixed-critical system into an industrial cyber-physical system. When an error occurs or when demand changes, the multi-user multiple-input and multiple-output nodes can switch their transmission mode, changing to a high-criticality configuration to meet the system’s new needs. Hence, we first propose a heterogeneous multi-user multiple-input and multiple-output system model. Based on this model, we propose a slot analyzing algorithm that guarantees system schedulability by reallocating slots for each node after replacing conflict nodes with multi-user multiple-input and multiple-output nodes. By considering both system schedulability and cost, the slot analyzing algorithm also reduces the number of multi-user multiple-input and multiple-output nodes required. Then, to further reduce the number of multi-user multiple-input and multiple-output nodes in an industrial cyber-physical system, we propose a priority inversion algorithm that improves schedulability by adjusting slot allocations before replacing conflict nodes with multi-user multiple-input and multiple-output nodes. By reducing the use of multi-user multiple-input and multiple-output nodes, the priority inversion algorithm achieves better performance than the slot analyzing algorithm when the system is in a high-criticality mode. Evaluation results show the effectiveness and efficacy of our approaches.
Keywords
Introduction
Industrial cyber-physical systems (ICPSs) are emerging as the fourth industrial revolution for industrial system.1–3 Compared to a conventional process control system, an ICPS mainly focused on system costs and reliability.4,5 Based on the features of ICPS, industrial standards are used extensively, such as ISA100, 6 WirelessHART, 7 and WIA-PA. 8
High degree of reliability and real-time communication requirements are two primary indicators in industrial network, and traditional cyber-physical systems (CPSs) cannot be applied directly. Instead, transmission slots and channels must be allocated for each node before the system works. Numerous studies have focused on this issue and attempted to improve the schedulability of ICPSs.9,10 However, the current scheduling methods cannot always guarantee the schedulability of an ICPS; for example, when several flows conflict at one node, lower-priority flows may miss their deadlines.
To ensure the schedulability of ICPSs, we introduce multi-user multiple-input and multiple-output (MU-MIMO) technology into ICPSs. A MU-MIMO node with two antennas can both receive and send packets simultaneously.11–13 By replacing conflict nodes with MU-MIMO nodes, several flows can transmit without delays. If we were to replace all the conflict nodes with MU-MIMO nodes, that would eliminate transmission conflicts in the system; consequently, when the number of MU-MIMO nodes is sufficient, we can guarantee system schedulability. However, considering the energy consumption and cost of MU-MIMO nodes, it is advantageous to be able to guarantee ICPS schedulability using fewer MU-MIMO nodes. 14
Furthermore, we must also consider schedulability when an error occurs or system demands change (usually, one ICPS involves several demands). Therefore, we introduce a mixed-critical system into ICPSs to address this issue.15–17 The existing works mainly consider the systems with single criticality, and these systems only reflect important by their priority. However, in many situations, the priority cannot represent importance. For example, as shown in Figure 1, the cement production process requires capturing temperature and detecting excessive pressure. The nodes in this system communicate through wireless mesh network. For tasks such as measuring temperature, which has a low criticality but high priority, only a general quality certification is required. In contrast, tasks such as excessive pressure detection require a strict safety certification (excessive steam pressure may cause boiler explosions); however, such pressure measurements have a lower priority than temperature measurements. When an accident or unusually sensor date occurs, we must improve schedulability of flows in high-criticality mode. Hence, we need to consider both the characters of priority and criticality to guarantee the ICPS schedulability. Therefore, it is necessary to introduce a mixed-criticality concept into ICPSs and analyze system schedulability under different criticality modes. Here, we assume that there are two criticality modes (low and high).

An example of a mixed-critical system.
There are two main challenges in our work: (1) reducing the number of MU-MIMO nodes required to guarantee the schedulability of an ICPS and (2) analyzing the schedulability of our algorithms for ICPSs in different criticality modes. To solve these problems, we propose two algorithms, a slot analyzing algorithm (SAA) and a priority inversion algorithm (PIA), to improve ICPS schedulability. The SAA first allocates slots without considering transmission conflicts and replaces conflict nodes with MU-MIMO nodes to guarantee future schedulability in low-criticality mode. In contrast, the PIA first optimizes slots allocation to reduce the number of transmission conflicts; then, we replace conflict nodes with MU-MIMO nodes only when the system cannot be scheduled. This study makes the following key contributions:
To the best of our knowledge, this is the first study to solve the transmission conflict problem in ICPSs using MU-MIMO technology.
We first propose an algorithm to obtain the candidate node set (CNSA) and then propose the SAA to improve system schedulability. The SAA can guarantee that a system can be scheduled when it satisfies Theorem 1. Furthermore, the SAA also reduces the number of MU-MIMO nodes in a candidate node set through slot analysis.
We propose the PIA to reduce the number of transmission conflicts before replacing the overlap nodes with MU-MIMO nodes. When a flow cannot be scheduled, priority inversion occurs when it satisfies Theorem 3.
We analyze system schedulability in high-criticality mode and prove that the PIA achieves better schedulability than SAA in Theorem 4.
Simulation results show that our algorithms can guarantee the schedulability of an ICPS with MU-MIMO nodes and that the PIA achieves a better performance than the SAA in high-criticality mode.
The rest of the article is organized as follows. Section “Related works” reviews previous works. Section “System model” presents the system model, which includes a network model, a mixed-critical system, and a fixed-priority (FP) scheduling policy in an ICPS. Section “Problem statement” states the problem, and we present our methods and describe the analysis in section “Scheduling algorithms.” Finally, section “Experiments” provides performance evaluation results, and section “Conclusion” concludes the article.
Related works
The first real-time transmission scheduling in ICPSs was studied in Saifullah et al. 18 This approach first formulated the end-to-end real-time transmission scheduling problem based on the characteristics of WirelessHART networks; then, the authors proved that the problem is NP-hard. Finally, they proposed several scheduling algorithms. Based on this study, Saifullah et al. 9 and Wu et al. 10 analyzed the end-to-end delay of WirelessHART networks. The schedulability of mixed-criticality industrial networks was analyzed in Xia et al. 19 In addition, Zheng et al. 20 proposed a new network model aiming at improving user experience by pushing the scheduling problem to the task layer; the work in Cheng et al. 5 improves the performance of wireless sensor network (WSN) by extracting kernel dateset form big sensory date. However, all these studies are based on traditional nodes. The sensor nodes in these networks can receive or send only one packet at a time in a single time slot.
In Kong and Liu, 21 the authors proposed an m-zig technique to improve the performance of sensor node. Based on m-zig, a sensor node can receive several packets during a single time slot. Later, a prototype of Cross-Zig was implemented for the low-power IEEE 802.15.4 specification in a software-defined radio platform. 22 L Cheng et al. 23 proposed an incast-collision-free data collection protocol named i-Core to address the many-to-one collision problem in low-duty-cycle WSNs. However, m-zig cannot both receive and send several packets at the same time.
The feasibility of employing MIMO techniques in WSNs was envisioned in Yuan et al. 24 To reduce the transmission time for mobile data collection, 25 the concept of incorporating MU-MIMO nodes into WSNs is introduced. However, only a few studies have focused on applying MU-MIMO technology to WSNs. Our research, therefore, addresses the schedulability of ICPSs using MU-MIMO technology.
Mixed-criticality systems have led to new research direction for real-time ICPSs. S Vestal 26 first proposed the concept of mixed criticality and then mixed criticality rapidly becoming a significant concept in CPSs.27–31 However, these works have mainly focused on Sandy Bridge, which is a kind of intel microprocessor architecture. For communication media, only a few related works on mixed criticality. X Jin et al. 32 studied real-time communication in industrial networks by mixed criticality. Xia et al. 19 proposed a resource analysis method based on an earliest deadline first (EDF) scheduling approach. However, all these existing works on mixed-criticality systems were performed using traditional CPSs; consequently, ICPSs may not be scheduled. Thus, existing solutions cannot be used in our model. Instead, there is an urgent need to propose scheduling methods under MU-MIMO for ICPSs to solve this issue.
System model
We consider a heterogeneous ICPS consisting of field devices (both MU-MIMO sensor nodes and traditional sensor nodes), one centralized controller and one gateway. Our system includes three aspects. First, we propose a network model and then we introduce MU-MIMO technology and mixed-criticality system into ICPSs. Finally, we derive FP scheduling scheme in ICPS.
Network model
In this subsection, we propose our system model based on ISA100 and WIA-PA. There are several features as follows: (1) limiting network size, (2) time division multiple access (TDMA), (3) route and spectrum diversity, and (4) handling internal interference.
33
The number of channels is denoted as
Industrial systems are a form of CPS applications that have higher requirements of real-time performance and reliability. Here, we introduce the MU-MIMO technique into ICPS. MU-MIMO nodes are promising for wireless transmissions because they can improve the average user spectral efficiency. 34 By mounting multiple antennas on a single sensor node, the node can receive and transmit simultaneously (or improve the packet acceptance ratio when all the antennas work on the same channel). The feasibility of employing MU-MIMO technology in ICPSs was envisioned in Zhao et al. 25
Hence, our model includes two types of sensor nodes: ordinary nodes and MU-MIMO nodes. Each ordinary node in our system is equipped with a half-duplex omni-directional radio transceiver whose status can alternate between transmitting and receiving. In contrast, a MU-MIMO node has two working modes: (1) when the antennas work on different channels, the node can both receive and transmit packets from several paths (the transmission capacity of a MU-MIMO node depends on the number of antennas it has); and (2) when the antennas work on the same channel, the node can reduce signal interference and improve the acceptance ratio because several antennas receive a packet from the same flow. Each packet in our system is transmitted through the network under source routing.
Figure 2 shows an example of a MU-MIMO node in the first mode (where antennas work on different channels) to address a transmission conflict in which

An example of MU-MIMO with a node handling a transmission conflict.
Mixed-critical system
The flow is a periodic end-to-end communication between a source and its destination. There are
At the beginning, packets are transmitted in low critical mode. To provide more resources, each MU-MIMO node’s antennas work on different channels. The system switches when accident occurs. To enhance system reliability in high-criticality mode, each MU-MIMO node’s antennas are reconfigured to work on the same channel.
FP scheduling in ICPS
We provide a summarization of FP scheduling under ICPS to analyze the schedulability of system in this subsection. FP scheduling is a commonly adopted scheme in practice for CPSs and real-time CPU scheduling. 35 Each job priority is pre-allocated by the network controller, and transmissions are scheduled based on this priority.
We assume that the priority of each flow is the same as its number. That is,
We define a system as schedulable when all the flows in a system can be scheduled (reach the destination before its deadline). Then, the definition of network schedulability is whether or not all flows in a network are schedulable. It is worth noting that when the system is in high-criticality mode, we no longer focus on the schedulability of low-criticality flows. When we repeat Z experiments, and only z experiments the emergency flow can be scheduled, then, the schedulability ratio is
Problem statement
Given the flow set
When the system is deployed, we can easily determine flows that miss their deadlines. However, each flow’s schedulability is interrelated with others. Thus, how should we decide which nodes should be replaced with MU-MIMO nodes?
As described in the previous section, the power consumption and cost of a MU-MIMO node are much higher than those of a normal node. Therefore, it is unreasonable and not cost-effective to deploy many MU-MIMO nodes when the system can be scheduled. Hence, the problem of determining the smallest number of MU-MIMO nodes that can meet the system requirements in low-criticality mode is another challenge.
The MU-MIMO nodes’ transmission mode changes when the system switches to high-criticality mode. Therefore, the scheduling algorithm also needs to consider schedulability in high-criticality mode.
We need to analyze the system’s schedulability under our proposed method.
Scheduling algorithms
In this section, we first study the issue of how to improve system schedulability with a small number of MU-MIMO nodes. First, we identify the nodes that may be replaced with MU-MIMO nodes to define candidate nodes as follows.
Definition 1 (Candidate Node)
We define a candidate node as a node at which transmission conflicts (intersection or overlap nodes) can occur. As Figure 2 shows, the paths of
Flows may conflict when they have path overlaps. To facilitate candidate node identification, we assume that, at most, one part of a path overlaps between two flows. As Figure 3 shows, two periodic flows transmit in an ICPS that conflict at the second and third slots in the first period. There are two methods for addressing transmission conflicts. The first method is to reallocate the slots for each node after replacing the conflict nodes with MU-MIMO nodes (as Figure 3 shows, the system can be scheduled when we replace node

The analysis of conflicts.

An example for un-schedulable conflict.
Therefore, for a system in which each link’s transmission slot has been allocated, we propose the SAA to improve system schedulability under a traditional FP policy. When the system cannot be scheduled by MU-MIMO nodes, the SAA reallocates system resources to guarantee system schedulability (increase the transmission speed of unscheduled data flows). By considering the characteristics of mixed-critical systems and MU-MIMO nodes, we then propose the PIA algorithm, which adjusts slot allocations before replacing intersection nodes with MU-MIMO nodes (the second method to address transmission conflict). By optimizing slots allocation, PIA can guarantee system schedulability with fewer MU-MIMO nodes. To guarantee the schedulability of the ICPS, PIA will also reallocate slots when the system cannot be scheduled. It is important to note that initially both the SAA and PIA algorithms work in low-criticality mode. We will analyze the schedulability of these two algorithms later.
Finding candidate nodes
An ICPS consists of numerous sensor nodes. In this subsection, we study how to select candidate nodes to guarantee the system can be scheduled. Transmission conflicts occur at the path overlaps of flows. As Figure 5 shows, there are two types of overlaps (without considering the flow’s direction).

Transmission conflict.
Lemma 1
When flow paths have an overlapping region, the overlap nodes are candidates for MU-MIMO nodes. We denote the candidate node set as
Proof
Transmission conflict can obviously only occur at an overlapping region. We can solve this issue by MU-MIMO nodes. We can account for the conflicts caused by the first type as shown in Figure 2. For the transmission conflict caused by the second type, as shown in Figure 5, there are two nodes (
Hence, we propose a Candidate Node Searching Algorithm (CNSA) to search for the set of candidate node as follows.
We search the candidate nodes by traversing the flow paths of the entire system. Nodes on more than one flow path are added to the candidate node set
SAA
After obtaining the candidate node set, we reduce the number of nodes in this set to reduce the system’s cost. Because not all the candidate nodes can experience transmission conflicts, we propose the SAA to analyze the schedulability of the ICPS under a traditional FP policy in one superframe (a superframe is the lowest common multiple of
Since SAA is used after the CNSA, we do not consider the slot allocations for each flow; we just replace conflict nodes with MU-MIMO nodes when a transmission conflict occurs.
When the system cannot be scheduled, we need to improve the schedulability using MU-MIMO nodes. Because MU-MIMO nodes can both receive and send packets in a single time slot, we first allocate slots for each node without considering transmission conflicts (the network controller allocates the channel for each transmission using the FP scheduling policy). Then, we replace conflict nodes with MU-MIMO nodes to guarantee system schedulability. Obviously, it is both unnecessary and not cost-effective to replace all the nodes in the candidate node set with MU-MIMO nodes. Therefore, SAA reduces the number of MU-MIMO nodes in
We reallocate the transmission time slots for each node by the schedulability of each flow (lines 1–8). If the flow cannot be scheduled, we find the intersection nodes and reallocate the slots for this flow without considering transmission conflicts. Otherwise, we retain the original allocation. Then, we analyze the transmission slot for each node in
Theorem 1
ICPS can be scheduled with SAA when the number of channels is no less than the number of flows
Proof
When
PIA
In this section, we study how to reduce the number of MU-MIMO nodes by the second method, the PIA scheduling algorithm, which performs optimal allocation of resources before the system runs. Through optimal allocation, PIA can decrease the number of transmission conflicts at intersection nodes and further reduce the number of required MU-MIMO nodes. As shown in Figure 6,

Another slot allocation for the example in Figure 3.
Initially, the PIA allocates slots based on the traditional FP policy. If the system cannot be scheduled, the network controller obtains each flow’s arrival time at its destination, which can be denoted as
Theorem 2
When
Proof
The priority of
where
When
Based on Theorem 2, PIA accelerates the unscheduled flow until it can be scheduled or no longer influences the other flow’s schedulability. However, Theorem 2 is not suitable for all overlaps. When
Definition 2 (EOR)
When
Hence, Theorem 2 can be extended as follows.
Theorem 3
When
Proof
Based on Theorem 2, we extend priority inversion to the last part of the EOR of
To guarantee the second constraint condition in Theorem 3, we denote the number of overlaps after the last EOR in
where
Based on the above discussion, the PIA can reduce the number of transmission conflicts before adding MU-MIMO nodes to the system. Furthermore, the PIA can also improve the schedulability of ICPSs under a high-criticality mode. When the system cannot be scheduled under the PIA, the nodes at flow intersections will be replaced by MU-MIMO nodes to guarantee the schedulability of ICPS. The pseudo code of the PIA is as follows.
PIA first allocates slots under the FP policy (line 1) and initializes the number of MU-MIMO nodes (line 2). Then, it judges whether the system can be scheduled or not. If the system can be scheduled, it returns
When the system performs the priority inversion operation at the last EOR, the conflict is removed. Hence, that part of the overlap is no longer an EOR. The system repeats this process until either the system can be scheduled or no flow satisfies Theorem 3.
Algorithm analysis in high-criticality mode
In this section, we analyze the schedulability of the SAA and the PIA. Since both the SAA and PIA can be scheduled by adding MU-MIMO nodes in low-criticality mode, we analyze only the schedulability in high-criticality mode.
Theorem 4
PIA has a higher schedulability than SAA when the system is in high-criticality mode.
Proof
When the system switches to high-criticality mode, low-criticality flows are abandoned and the high-criticality flow period changes to
For
We denote the number of MU-MIMO nodes in the SAA and PIA as
when
Hence, the PIA has a higher schedulability than does the SAA when the system is in high-criticality mode.
Experiments
We evaluate the performance of our proposed methods by experiments. We compare our approaches with the traditional FP algorithm without MU-MIMO nodes. We compare both the acceptance rate and the number of MU-MIMO nodes for each criticality mode. We use the acceptance rate to represent the schedulability of an ICPS. When all flows can be scheduled, the acceptance rate is
Simulation parameters.
Low-critical mode
We first compare the performances of the algorithms in low-critical mode. As shown in Figure 7(a), the system acceptance rate is decreased by the FP scheduling policy (n = 15, N = 50, m = 16), which occurs because the idle resources decrease as the system utilization increases. The latency tolerance of a packet is reduced along with the idle resources. The ICPS can be scheduled under both SAA and PIA in any situations by increasing the number of MU-MIMO nodes when

(a) Relationship between the acceptance rate system utilization and (b) relationship between the number of MU-MIMO nodes and system utilization.
We repeat this simulation for the situation in which all flows can be scheduled as shown in Figure 8. The number of candidate nodes is fixed when we increase system utilization by only adjusting the flow period (because there is only one test in this simulation, and the flow transmission paths do not vary). Initially, no MU-MIMO nodes exist in the system because it can be scheduled without any MU-MIMO nodes. However, when we increase the system utilization, transmission conflict occurs. To guarantee the schedulability of system, we need to add MU-MIMO nodes to the system. In addition, the number of MU-MIMO nodes required by the PIA is always less than the number required by the SAA. This occurs because by optimizing slots allocation, the PIA can reduce the number of transmission conflicts. Hence, the number of MU-MIMO nodes required by the PIA is always less than the number required by the SAA.

Relationship between the number of MU-MIMO nodes and system utilization.
The relationship between the acceptance rate/number of MU-MIMO nodes and the number of flows is shown in Figure 9 (U = 0.3, N = 50, m = 16). The acceptance rate is reduced along with the number of flows under the FP scheduling policy. The acceptance rates of the SAA and PIA are reduced when the number of flows becomes larger than the number of channels. This result occurs because when

(a) Relationship between the acceptance rate/number of MU-MIMO nodes and the number of flows and (b) relationship between the number of MU-MIMO nodes and the number of flows.
Figure 10 shows the relationship between the acceptance rate/number of MU-MIMO nodes and the number of nodes (U = 0.3, n = 15, m = 16). All these results indicate that the SAA and the PIA can guarantee the schedulability of the system. The number of MU-MIMO nodes in both the SAA and the PIA is no larger than the number of candidate nodes regardless of the conditions. In addition, when the number of nodes increases to 90, the number of MU-MIMO nodes in the PIA and the SAA is identical because the PIA transmutes into the SAA when it cannot resolve the conflict.

(a) Relationship between the acceptance rate and the number of flows and (b) relationship between the number of MU-MIMO nodes and the number of flows.
High-critical mode
When the system switches to high-critical mode, the number of MU-MIMO nodes is no longer our main concern (the number of MU-MIMO nodes is the same as in low-critical mode). Instead, we are more concerned with the performances of SAA and PIA in high-critical mode.
Figure 11 shows the relationship between the system acceptance rate and system parameters such as the number of nodes and the number of flows. In these conditions, we set

(a) Relationship between the acceptance rate and the number of nodes in high-critical mode and (b) relationship between the acceptance rate and the number of flows in high-critical mode.
Figure 12 shows the relationship between the system acceptancerate and system utilization. To study the relationship between these two elements, we increase system utilization by adjusting only the flow period. Subsequently, we obtain the same result as Figure 11, which also verifies the correctness of Theorem 4.

Relationship between the number of MU-MIMO nodes and system utilization in high-critical mode.
Conclusion
In this article, we make key contributions to real-time transmission scheduling in ICPSs. We first introduce MU-MIMO technology and the concept of a mixed-critical system into ICPSs and discuss the characteristics of these two features. Then, we analyze the transmission paths and obtain the candidate node set. Next, based on the characteristics of MU-MIMO nodes, we propose the SAA and the PIA to guarantee the schedulability of ICPS in low-criticality mode. By considering system cost, these two algorithms help to reduce the number of MU-MIMO nodes used. Finally, we analyze the schedulability of these two algorithms when the system switches to high-criticality mode. The simulation results show that our scheduling algorithms and analysis perform better than the existing scheduling policy. In future work, we plan to study ICPS schedulability under conditions with limited numbers of antennas and to implement our approach in a real-world ICPS.
Footnotes
Handling Editor: Zhipeng Cai
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was partially supported by the National Natural Science Foundation of China (grants 61502474, 61233007, and 61672349) and by the Youth Innovation Promotion Association CAS.
