Sage Journals: Discover world-class research

Abstract

In order to meet the two crucial requirements of Internet of Things (support of a large number of Internet of Things devices and low power consumption), this article proposes a novel and efficient virtual bandwidth aggregation scheme for Internet of Things data. Since many Internet of Things applications require reliable Internet of Things data transmission, a Transmission Control Protocol is chosen as a transport protocol in this article. The proposed scheme consists of three salient functions: packet scheduler, estimator, and ACK buffer. The packet scheduler provides an efficient heterogeneous link selection using the proposed scheduling algorithm. The estimator provides an accurate delay and bandwidth estimation of heterogeneous wireless links using network layer information. Finally, the ACK buffer substantially boosts the performance of the scheduler and estimator with a simple ACK buffering scheme. Our extensive analysis using a numerical method and simulation corroborates the outstanding performance of the proposed scheme. Because of the salient functions of the proposed scheme, it can provide maximum capacity for future Internet of Things data transmission.

Keywords

Transmission Control Protocol heterogeneous networks Internet of Things dual connections bandwidth aggregation

Introduction

Nowadays, with the emergence of communication-capable devices, such as smart-phones and tablets, computing and networking have become the norm. This ubiquitous computing and networking have been enhanced to become the Internet of Things (IoT), which is expected to change the computing paradigm of our daily lives. IoT is a network of a large number of communication-capable devices. There are two crucial IoT requirements: support of a large number of IoT devices and low power consumption.

In order to meet the requirement to support a large number of IoT devices, various types of wireless networks should be utilized. In order to meet the requirement of low power consumption, packet transmission efficiency in erroneous wireless environments should be maximized. Therefore, the future IoT environment is anticipated to be heterogeneous wireless networks of various types such as 3rd Generation Partnership Project (3GPP) Long-Term Evolution (LTE) and IEEE 802.11-based wireless local area networks (WLANs), which should provide the IoT Quality of Service (QoS) level in terms of delay and throughput for IoT packet transmission efficiency.¹ There are two types of IoT devices: strictly power limited IoT devices and flexibly power limited intelligent IoT devices. The strictly power limited IoT devices are sensor type power limited IoT devices and require extremely low power consumption to provide long operation time with limited battery. The flexibly power limited IoT devices are battery powered with easy battery exchange or charging capabilities and expected to be dominant IoT device types. In most of cases, such IoT devices require both communication reliability and transmission efficiency. The most widely used transport protocols are User Datagram Protocol (UDP) and Transmission Control Protocol (TCP). While UDP is very simple transport protocol without feedback mechanism, TCP is a more reliable but sophisticated transport protocol with feedback mechanism. In this article, the main target of the proposed scheme is the flexibly power limited intelligent IoT devices with reliable communication. In the most of recent communication protocol deployment, application layer error recovery protocol is additionally provided when UDP is used as a transport protocol to provide communication reliability. When TCP is chosen as a transport protocol, its processing can be little more complicated than UDP but TCPs standardized salient reliability-related features can be used. With respect to communication overhead, under the same reliability requirement, it is hard to say that UDP with separate application layer reliability mechanism provides less overhead than TCP with intrinsic reliability mechanism. In this article, TCP is chosen over UDP in order to provide standardized TCP intrinsic reliability mechanism for easy deployment across various IoT devices.

In a heterogeneous network environment, a high QoS level in terms of delay and throughput can be provided with a simple scheme called bandwidth aggregation. The configuration of a bandwidth aggregation scheme in a heterogeneous network is shown in Figure 1. As we can see, the receiver is equipped with at least two wireless interfaces. Packets are transmitted over the multiple wireless interfaces, which leads to throughput improvement. A properly designed bandwidth aggregation scheme has many advantages: increased throughput, improved packet delivery and reliability, load balancing, low-cost capacity increase, and so on. In addition, a bandwidth aggregation scheme can be implemented in various protocol layers, from an application layer to a physical layer. Therefore, using a bandwidth aggregation scheme, packet transmission performance can be enhanced with a flexible implementation.² As described above, in order to meet the requirement to support a large number of IoT devices, heterogeneous wireless links will be utilized. In the heterogeneous network environment, the proposed virtual bandwidth aggregation (V-BAG) scheme can be advantageous especially in handling large amount of data such as video streaming–related applications. In future, IoT applications such as vehicular-related applications: autonomous connected car and public safety using drones, utilization of heterogeneous links to provide enough bandwidth and reliability will be norm, and V-BAG is anticipated to play a very important role.

Figure 1.

Bandwidth aggregation in heterogeneous networks.

The authors in the works by Luo et al.,³ Tang et al.,⁴ and Robertson and Venkatasubramanian⁵ propose a bandwidth aggregation scheme in an application layer for throughput enhancement. Application layer bandwidth aggregation schemes have the advantage of easy implementation. However, since specially designed applications are a requirement for bandwidth aggregation, an application layer bandwidth aggregation scheme is unable to provide backward compatibility. Stream Control Transmission Protocol (SCTP)⁶ and Multipath Transmission Control Protocol (MPTCP)^7,8 are transport layer bandwidth aggregation schemes. Both schemes provide good backward compatibility. However, since senders and receivers are required to use the same transport layer bandwidth aggregation scheme, the proposed schemes do not provide scalability. Link layer bandwidth aggregation schemes are proposed by Yaver and Koudouridis,⁹ Kim et al.,¹⁰ and Koudouridis et al.¹¹ The proposed schemes provide good backward compatibility and enhance the accuracy of estimation using various link layer parameters. However, implementation of the proposed schemes is difficult because of layering violations.

Network layer bandwidth aggregation schemes are proposed by Chebrolu and Rao¹² and Evensen et al.¹³ Table 1 provides the comparison among network layer schemes. Chebrolu and Rao¹² propose a scheme for real-time application, and Evensen et al.¹³ propose a scheme using Internet Protocol (IP) address translation. However, specific link estimators are not described in both the works by Chebrolu and Rao¹² and Evensen et al.,¹³ whereas¹² operating under UDP¹³ does not mention any transport layer protocols. As compared in the table, the proposed V-BAG is superior to other schemes and provides detailed designs to be implementable.

Table 1.

Comparison between network layer schemes.

	EDPF¹²	N-proxy¹³	V-BAG
Layer	Network	Network	Network
Transport	UDP	None	TCP
Estimation	None	None	Network layer estimator
Specialty	For real-time application	IP address translation	Easy implementation

EDPF: Earliest Delivery Path First; V-BAG: virtual bandwidth aggregation; UDP: User Datagram Protocol; TCP: Transmission Control Protocol; IP: Internet Protocol.

Since implementation and scalability are very important aspects of practical bandwidth aggregation schemes, in this article, a network layer that is able to provide easy implementation and good scalability is chosen as a target layer for a bandwidth aggregation scheme. However, there is one technical challenge with the network layer bandwidth aggregation scheme: a TCP packet reordering problem.¹⁴ Since TCP provides congestion control, when a packet is lost, duplicate ACKs are transmitted for the following successfully received packets. When the TCP sender receives the duplicate ACKs, it reduces its congestion window (CWND).

Normally, a bandwidth aggregation scheme employs both a scheduling algorithm (which allocates packets to the wireless link depending on the bandwidth size of the link) and an estimator (which measures the status of the wireless links). Owing to the difficulty of wireless link status estimation, out-of-order delivery of packets may occur. Since the TCP receiver transmits duplicate ACKs and the TCP sender reduces CWND upon reception of the duplicate ACKs, the TCP packet reordering problem causes severe throughput deterioration. In order to design an efficient bandwidth aggregation algorithm, the TCP packet reordering problem needs to be solved. We propose a simple and efficient mediation algorithm using an ACK buffer.

In this article, a novel and efficient V-BAG scheme for IoT data is proposed. In order to meet the two crucial requirements of the Internet of Things (support of a large number of IoT devices and low power consumption), in this article, heterogeneous wireless networks consisting of 3GPP LTE and WLANs are considered. For reliable IoT data transmission, a TCP is chosen as a transport layer. The network layer is chosen as a target layer for the proposed algorithm considering scalability and implementation.

The proposed scheme consists of three salient functions: packet scheduler, estimator, and ACK buffer. The packet scheduler provides an efficient heterogeneous link selection using the proposed scheduling algorithm. The estimator provides an accurate delay and bandwidth estimation of the heterogeneous wireless links using the network layer information. Finally, the ACK buffer substantially boosts the performance of the scheduler and estimator with a simple ACK buffering scheme.

Our extensive analysis using a numerical method and simulation corroborates the outstanding performance of the proposed scheme. Thanks to the salient functions of the proposed scheme, it provides maximum capacity for future IoT data transmission.

The rest of this article is organized as follows. In section “V-BAG system model,” V-BAG’s three salient functions (packet scheduler, estimator, and ACK buffer) are presented, and the overall architecture is demonstrated. Section “V-BAG in erroneous wireless links” provides a mathematical analysis of V-BAG in erroneous wireless links. Section “Evaluation” presents the simulation results and analysis. Finally, section “Conclusion” concludes the article.

V-BAG system model

Figure 2 shows the overall system architecture of V-BAG. V-BAG consists of three key functions: packet scheduler, estimator, and ACK buffer. In this section, each function is explained in detail. In order to maximize system performance, the V-BAG scheduler function is implemented at the sender side, and the functions of the estimator and ACK buffer are implemented at the receiver side. The estimator utilizes the arrival time of the transmitted packet ( $t 2$ ) and the departure time from the sender ( $t 1$ ), which are optional parameters in a TCP header, to calculate the bandwidth ( $b 1, b 2$ ) and delay ( $D 1, D 2$ ) of the wireless links. The ACK buffer stores duplicate ACKs in order to minimize TCP packet reordering problems. All key functions are located in the network layer.

Figure 2.

System architecture.

Packet scheduler

In V-BAG, a packet scheduler distributes packets to the proper wireless link depending on available bandwidth size. The proposed packet scheduler considers the link bandwidth and the end-to-end delay. When packets for transmission arrive at the packet scheduler, the packet scheduler uses the proposed algorithm, called the modified Earliest Delivery Path First (m-EDPF), which is designed based on the Earliest Delivery Path First (EDPF) algorithm.¹² The packet scheduler calculates the expected arrival time ( $d_{i}^{l}$ ) at the receiver for all available links l

d_{i}^{l} = MAX ({\hat{D}}_{current}^{l}, d_{i - 1}^{l} - δ) + \frac{L_{i}}{{\hat{b}}_{current}^{l}}

(1)

where ${\hat{b}}_{current}^{l}$ and ${\hat{D}}_{current}^{l}$ are the bandwidth and the delay of the link l, respectively, which are measured and transmitted by the proposed estimator of receivers. $L_{i}$ is the size of packet i. $δ$ is the processing time of the sender which is the time difference between the arrival time of the current packet and the departure time of the previous packet.

Since m-EDPF is designed based on EDPF, first, the EDPF algorithm is explained. The EDPF algorithm¹² calculates the packet arrival time, $d_{i}^{l}$ , using the following equation

d_{i}^{l} = MAX (a_{i} + {\hat{D}}_{current}^{l}, A_{t}) + \frac{L_{i}}{{\hat{b}}_{current}^{l}}

(2)

where $a_{i}$ is the arrival instance of the $i th$ packet and $A_{t}$ is the time the wireless channel becomes available for the next transmission at the sender.

The algorithm has some practical problems when it is implemented. Since the value of $a_{i}$ increases cumulatively, the calculation time of $a_{i} + {\hat{D}}_{current}^{l}$ becomes substantially long, and a memory leak problem might occur. Moreover, since estimation errors accumulate when there is an error in ${\hat{D}}_{current}^{l}$ , the accuracy of $d_{i}^{l}$ becomes unreliable. In particular, $A_{t}$ is ambiguous and very difficult to measure for implementation.

The proposed m-EDPF algorithm is designed to overcome such practical difficulties of EDPF. The proposed m-EDPF clearly designates which entity measures ${\hat{b}}_{current}^{l}$ and ${\hat{D}}_{current}^{l}$ , and all ambiguous terms are removed. The proposed m-EDPF is designed to work recursively to remove the problems of a long processing time and possible memory leak, which are caused by cumulatively increasing parameters.

In the proposed m-EDPF algorithm, every packet i has its own packet arrival time $d_{i}^{l}$ for all available links l. The packet scheduler transmits packet i through path p with the lowest packet arrival time

p = {l : d_{i}^{l} \leq d_{i}^{m}, 1 \leq m \leq N}

(3)

where N is the number of available heterogeneous wireless links in V-BAG.

Estimator

The proposed estimator, located in the receiver, measures the bandwidth and delay of the wireless links, which are used by the packet scheduler, and transmits the measured bandwidth and delay to the sender. Where many bandwidth aggregation–related studies violate layering regulations to measure wireless links, the proposed estimator can be practically implemented without layer violations using the information available in a network layer. The detailed procedure for the estimator is listed in Table 2.

Table 2.

The procedure for estimator.

Packet departure from the sender through link l:

begin

time_stamp_option \leftarrow tcp_time_stamp

end

Packet arrival at the receiver through link l:

begin

t 1_{i}^{l} \leftarrow time_stamp_option

t 2_{i - 1}^{l} \leftarrow t 2_{i}^{l}

t 2_{i}^{l} \leftarrow tcp_time_stamp

b_{i}^{l} \leftarrow \frac{L_{i}}{t 2_{i}^{l} - t 2_{i - 1}^{l}}

{\hat{b}}_{i}^{l} \leftarrow \frac{\frac{2 τ}{t 2_{i}^{l} - t 2_{i - 1}^{l}} - 1}{\frac{2 τ}{t 2_{i}^{l} - t 2_{i - 1}^{l}} + 1} {\hat{b}}_{i - 1}^{l} + \frac{b_{i}^{l} + b_{i - 1}^{l}}{\frac{2 τ}{t 2_{i}^{l} - t 2_{i - 1}^{l}} + 1}

{\hat{D}}_{i}^{l} = (1 - α) (t 2_{i}^{l} - t 1_{i}^{l} - \frac{L_{i}}{b_{i}^{l}}) + α {\hat{D}}_{i - 1}^{l}

end

ACK is generated at the receiver:

time_stamp_option \leftarrow {\hat{b}}_{i}^{l}

time_stamp_option \leftarrow {\hat{D}}_{i}^{l}

The estimator measures packet i’s arrival time ( $t 2_{i}^{l}$ ) transmitted over link l. Then, it calculates the bandwidth ( $b_{i}^{l}$ ) of link l using the following equation

b_{i}^{l} = \frac{L_{i}}{t 2_{i}^{l} - t 2_{i - 1}^{l}}

(4)

where $L_{i}$ is the size of TCP packet i. Using the calculated value of $b_{i}^{l}$ is not practical since $b_{i}^{l}$ fluctuates severely owing to wireless packet losses. Therefore, the proposed estimator employs the low-pass filter suggested in TCP westwood¹⁵ to obtain a stable $b_{i}^{l}$ value

{\hat{b}}_{i}^{l} = \frac{\frac{2 τ}{t 2_{i}^{l} - t 2_{i - 1}^{l}} - 1}{\frac{2 τ}{t 2_{i}^{l} - t 2_{i - 1}^{l}} + 1} {\hat{b}}_{i - 1}^{l} + \frac{b_{i}^{l} + b_{i - 1}^{l}}{\frac{2 τ}{t 2_{i}^{l} - t 2_{i - 1}^{l}} + 1}

(5)

where $1 / τ$ is the cut-off frequency of the filter.

The delay of wireless link l ( $D_{i}^{l}$ ) can be calculated by $t 2_{i}^{l} - t 1_{i}^{l}$ , where $t 1_{i}^{l}$ and $t 2_{i}^{l}$ are the departure times of packet i from senders 1 and 2, respectively. $t 1_{i}^{l}$ and $t 2_{i}^{l}$ can be obtained from the TCP header’s time stamp option. However, similar to $b_{i}^{l}$ , delay value $D_{i}^{l}$ also fluctuates severely owing to wireless packet losses. Therefore, the proposed estimator employs the simple low-pass filter to obtain a stable $D_{i}^{l}$ value

{\hat{D}}_{i}^{l} = (1 - α) (t 2_{i}^{l} - t 1_{i}^{l} - \frac{L_{i}}{b_{i}^{l}}) + α {\hat{D}}_{i - 1}^{l}

(6)

where $α$ is the filter parameter, which has a value between 0 and 1. When $α$ increases, ${\hat{D}}_{i}^{l}$ is dominated by ${\hat{D}}_{i - 1}^{l}$ .

The estimator measures $b_{i}^{l}$ and $D_{i}^{l}$ for all arrived packets and calculates ${\hat{b}}_{i}^{l}$ and ${\hat{D}}_{i}^{l}$ . The calculated ${\hat{b}}_{i}^{l}$ and ${\hat{D}}_{i}^{l}$ are transmitted to the sender by putting them on the time stamp option of TCP ACK. The transmitted ${\hat{b}}_{i}^{l}$ and ${\hat{D}}_{i}^{l}$ are used by the packet scheduler.

Is it enough?

Heterogeneous networks with two independent links are assumed to verify the performance of the packet scheduler and estimator. In order to be practical, the throughput of one link (link 1) is set to the typical throughput value of LTE (150 Mbps), and the throughput of the other link (link 2) is set to the typical throughput value of IEEE 802.11ac 256–quadrature amplitude modulation (QAM) (80 Mbps). To remove the effect of the TCP algorithms, it is also assumed that there are enough packets to transmit and that TCP always maintains the maximum CWND size. During the transmission of 30 packets, the performances of the packet scheduler and the estimator are observed.

In Figure 3, the first four packets are transmitted to links 1 and 2 one by one to calculate initial bandwidth and delay, and their arrival time is not ordered. Transmitting four packets is to setup initial parameter values and a few packets are fairly reasonable due to the fast convergence speed of parameter values. In addition, the packet scheduler distributes packets according to the ratio of bandwidth. We can see that the proposed packet scheduler and estimator work well in ideal heterogeneous networks.

Figure 3.

Packet scheduling results with perfectly estimated bandwidth and delay: (a) estimated bandwidth and delay and (b) packet scheduling results.

Now, it is important to investigate how the proposed packet scheduler and estimator work with TCP algorithms. The CWND of a TCP changes following the CWND control algorithm. For example, when a new TCP session is established, the TCP increases CWND by a slow-start algorithm. Packet transmission is limited by the changes in CWND. Although the estimator can obtain the bandwidth $b_{k}^{l}$ of an available wireless link using equation (4), packet transmission might be paused owing to a small CWND. Accordingly, the value of $t 2_{k}^{l}$ increases, which leads to a $b_{k}^{l}$ that is smaller than it should be. In dynamically changing TCP environments where CWND changes dynamically, inaccurate measured bandwidths and delays mislead the scheduler into inefficient packet distribution, which leads to a TCP packet reordering problem. The TCP packet reordering problem causes duplicate ACKs, which leads to a reduction in CWND size and a degradation in overall system performance.

In order to evaluate the aforementioned performance degradation problem with a TCP algorithm, CUBIC TCP,¹⁶ which is one of the most widely used TCP variants, is simulated in V-BAG using the same simulation settings. As we can see from Figure 4, the bandwidth and delay values severely fluctuate as CUBIC TCP CWND changes. Accordingly, owing to inefficient scheduling of the packet scheduler using inaccurate bandwidth and delay values, which causes a TCP packet reordering problem, the throughput of V-BAG is below 100 Mbps (≪150 Mbps). We can clearly see that V-BAG should have an additional function that alleviates the TCP packet reordering problem. As an additional function, ACK buffer is proposed.

Figure 4.

Throughput performance without ACK buffer: (a) estimated bandwidth and delay and (b) congestion window and throughput.

ACK buffer

The ACK buffer is located at the receiver side. When multiple senders transmit data to one common receiver, a separate ACK buffer is used for each sender. In practice, V-BAG uses a separate ACK buffer for each session. The ACK buffer stores all duplicate ACKs generated in the transport layer. The ACK buffer is designed to discard the stored duplicate ACKs in case of TCP packet reordering and to transmit the stored duplicate ACKs in case of wireless packet loss. When the segment numbers of the received packets through all available wireless links are higher than the segment numbers of the duplicate ACKs, the ACK buffer regards this situation as a wireless packet loss event. When the segment number of newly generated ACK is different from the segment number of duplicate ACKs, the ACK buffer regards this situation as a TCP packet reordering event. When V-BAG is operating, the V-BAG receiver should keep receiving packets from all available links. When the V-BAG receiver does not receive a packet for a certain period of time, this situation can also be regarded as packet loss event. In this case, the ACK buffer also immediately transmits the stored duplicate ACKs. If the wireless channel conditions are not good enough to send duplicate ACKs immediately, storing the duplicate ACKs during a certain period of time may affect the overall system performance. Under such bad channel conditions, since duplicate ACKs are not sent, TCP timeout would occur, which is same as the TCP Tahoe. While TCP Reno halves the CWND size for duplicate ACKs, TCP Tahoe initializes the window size. As we can see from the many previous studies, TCP Tahoe shows better performance than TCP Reno in severe channel error condition, and V-BAG can ensure that the performance is not degraded because of ACK buffer.

Using the proposed ACK buffer, as we can see from Figure 5, the throughput performance of V-BAG is substantially improved. This means the proposed ACK buffer can effectively alleviate the TCP packet reordering problem. We can also see that despite inaccurate delay measurements by the estimator, the proposed ACK buffer exhibits an excellent performance. In Figure 5, the throughput performance of V-BAG achieves over 200 Mbps (>150 Mbps).

Figure 5.

Throughput performance with ACK buffer: (a) estimated bandwidth and delay and (b) congestion window and throughput.

V-BAG in erroneous wireless links

As described in section “V-BAG system model,” V-BAG performs well in heterogeneous wireless environments. V-BAG overcomes the TCP packet reordering problem using a salient ACK buffering function. However, in order to design an efficient IoT data transmission scheme, the effects of wireless packet errors need to be investigated. In this section, in order to evaluate the performance of V-BAG, analytical modeling is proposed using Markovian analysis. Since CUBIC is one of the widely used TCP protocols, CUBIC is selected as a transport protocol for analysis.

In the work by Bao et al.,¹⁷ the authors suggest a Markovian chain model to mathematically analyze CUBIC and to derive CUBIC’s normalized throughput in these proposed wireless environments. In the proposed wireless environments, random packet loss is caused by fading or interference, and random packet loss is assumed to experience a random Poisson process at rate $λ$ . The assumption $τ_{loss}$ , which is the time duration between two consecutive packet losses, is a random variable with exponential distribution. The probability density function (pdf) of $τ_{loss}$ is

f (τ_{loss}) = λ \exp (- λ τ_{loss}), τ_{loss} > 0

(7)

Let us suppose there are two independent wireless links with same packet loss rate, $λ$ , and they are used for V-BAG. In V-BAG, the TCP is able to observe packet losses with a time duration of $τ_{v - bag}$ . Since $τ_{loss}$ of an individual wireless link is a random variable with exponential distribution, $τ_{v - bag}$ is also a random variable with exponential distribution ( $τ_{v - bag} = \min (τ_{loss}, τ_{loss})$ ). The pdf of $τ_{v - bag}$ is

f (τ_{v - b a g}) = 2 λ \exp (- 2 λ τ_{l o s s}), τ_{v - b a g} > 0

(8)

Given the time instant $t_{0}$ , the probability that the next packet loss happens within the time interval ( $t_{0} + T_{1}$ , $t_{0} + T_{2}$ ] is

P (T_{1} < τ_{v - b a g} < T_{2}) = \exp (- 2 λ T_{1}) - \exp (- 2 λ T_{2})

(9)

According to Bao et al.,¹⁷ the state index at time instant k where $k = 1, 2, \dots$ is denoted as $X_{k}$ which corresponds to the $k th$ window reduction. Then

P (X_{k + 1} | X_{k}, X_{k - 1}, \dots, X_{1}) = P (X_{k + 1} | X_{k})

(10)

Therefore, the state sequence $X_{1}, X_{2}, \dots$ is a Markov chain, and the average normalized TCP CUBIC throughput is

\bar{x} = \frac{1}{W_{v - bag}} \frac{\sum_{i = 1}^{N} \sum_{j = 1}^{N} π_{i} P_{ij} s_{ij}}{\sum_{i = 1}^{N} \sum_{j = 1}^{N} π_{i} P_{ij} τ_{ij}}

(11)

where N is the partitioned N in equal-sized intervals when the CWND increases from 0 to $W_{v - bag}$ . $W_{v - bag}$ is the maximum CWND size, which is caused by the bandwidth aggregation of two wireless links ( $W_{v - bag} = W_{1} + W_{2}$ ). $π_{i}$ is the stationary probability of the $i th$ state in Markov chain satisfying

\sum_{i = 1}^{N} π_{i} P_{ij} = π_{j}, j \in {1, 2, \dots, N}

(12)

and

\sum_{i = 1}^{N} π_{i} = 1

(13)

$P_{ij}$ is the state from $i th$ state to $j th$ state

\begin{matrix} P ({(D (a_{i}, \frac{(j - 1) W_{v - bag}}{N}))}^{+} < τ_{v - bag} \\ \leq D (a_{i}, \frac{j W_{v - bag}}{N})) \\ = \exp (- 2 λ {(D (a_{i}, \frac{(j - 1) W_{v - bag}}{N}))}^{+}) \\ - \exp (- 2 λ D (a_{i}, \frac{j W_{v - bag}}{N})) \\ = \exp (- 2 λ {(\sqrt[3]{\frac{(j - 1) W_{v - bag} / N - a_{i}}{C}} + \sqrt[3]{\frac{β a_{i}}{C}})}^{+}) \\ - \exp (- 2 λ (\sqrt[3]{\frac{j W_{v - bag} / N - a_{i}}{C}} + \sqrt[3]{\frac{β a_{i}}{C}})) \end{matrix}

(14)

where $D (x, y) = \sqrt[3]{(y - x) / C} + \sqrt[3]{β x / C}$ . $τ_{ij}$ is the average time duration of the state transition and $s_{ij}$ is the throughput area of CWND which has the value of

s_{ij} = a_{i} τ_{ij} + \frac{C}{4} ((τ_{ij} - K)^{4} - K^{4})

(15)

where $K = \sqrt[3]{W_{\max} β / C}$ .

As a result, the average normalized CUBIC throughput of V-BAG $\bar{x}$ is a function of the packet loss rate ( $λ$ ) and the maximum CWND size ( $W_{v - bag}$ ). In equation (8), the time duration ( $τ_{v - bag}$ ) of packet losses, which can be observed by TCP with V-BAG, is proven to be an exponentially distributed random variable with the expected value of $1 / 2 λ$ . This means V-BAG CUBIC experiences more severe wireless packet loss than normal CUBIC used over a wireless link. In severe packet loss environments, the maximum CWND size ( $W_{v - bag} = W_{1} + W_{2}$ ) must be maximized for better throughput performance. Therefore, the difference between the maximum CWND size of the individual wireless links ( $W_{1}$ and $W_{2}$ ) must be minimized.

Evaluation

In this section, the proposed V-BAG is evaluated in two aspects: the effect of the packet loss rate on V-BAG and the throughput difference between two individual links. In order to fully evaluate the performance of the proposed bandwidth aggregation scheme, only one device is considered in the simulator. Since V-BAG system is designed to support large scale IoT deployment by placing the complexity (packet scheduler) in the network, V-BAG can be easily expanded to support a large number of IoT devices with proper deployment of multiple packet schedulers. The V-BAG simulator consists of the four main parts: the packet generation part in the application layer, the sender part of CUBIC TCP which transmits the generated packets, the heterogeneous wireless network part modeled by the Markov chain, and the general TCP receiver part. In addition, the three main V-BAG functions, packet scheduler, estimator, and ACK buffer, are implemented in detail. The parameters used in the simulation are listed in Table 3.

Table 3.

Parameters used in the simulation.

Unit	Value
Wireless link delay	2 ms
Cut-off frequency ( $τ$ )	0.01
Cut-off frequency ( $α$ )	0.1
Cubic parameter (C)	0.4
Cubic parameter ( $β$ )	0.8

Figure 6 shows how much throughput gain can be achieved by V-BAG in comparison with an individual wireless link at various packet loss rates. The supposed LTE link can fully utilize a 20 MHz bandwidth to achieve 150 Mbps. In an application layer, abundant traffic is generated so that packets are only transferred according to CUBIC TCP CWND changes. Two individual LTE links are bandwidth aggregated for V-BAG, and the traffic of V-BAG is also transmitted according to CUBIC’s CWND changes. An automatic repeat request (ARQ) scheme that retransmits lost packets in a wireless link is also adopted. The simulation is performed for approximately 50 s, and the average throughput of V-BAG and the individual link is measured while packet loss rate ( $λ$ ) changes from 0.05 to 0.2. Since throughput stabilizes before 50 s as we can see from Figure 5, the simulation time of 50 s is chosen as a reasonable time period to measure the performance.

Figure 6.

Average throughput comparison at various packet loss rates.

The V-BAG system is evaluated by comparing its throughput to that of a single wireless link that shows the maximum throughput among the individual wireless links. The V-BAG system is effective if the throughput of the V-BAG system is greater than the throughput of a maximum single link. In Figure 6, it is observed that V-BAG outperforms the single LTE link at all points of the packet loss rate. Thus, the bandwidth aggregation system in V-BAG is effective. However, as the packet loss rate increases, the throughput of the single LTE link and V-BAG generally decreases, and the throughput of V-BAG decreases more sharply because V-BAG is affected by the severe packet loss rate rather than the LTE link. As a result, as the packet loss rate increases, overall system throughput also decreases, but V-BAG is more effective at any points of the packet loss rate.

Figure 7 also shows the average throughput of V-BAG. Now, we fix the packet loss rate to 0.05, and two individual wireless links for V-BAG system are assumed. Link 1 is assumed to be a single LTE link of 150 Mbps, which is the same value as in the previous simulation. The data rate of link 2 is changed from 150 to 40 Mbps, and it is indexed in the X-axis in Figure 7. As the data rate of link 2 decreases, the throughput of V-BAG also decreases. When the data rate of link 2 is below 60 Mbps, V-BAG can be regarded as having failed since the average throughput of V-BAG is lower than the theoretical throughput of link 1 (red line). In conclusion, we can see that the data rate difference between link 1 and link 2 should be minimized to maximize the throughput gain of V-BAG.

Figure 7.

Average throughput comparison at various data rates of the second wireless link.

With careful observation of the simulation results in this section, we can see that V-BAG is more affected by the data rate difference between two wireless links than the packet loss rate analyzed in section “V-BAG in erroneous wireless links.” Therefore, the data rate difference can be a good metric to decide whether to employ the V-BAG scheme.

Conclusion

In this article a novel and efficient V-BAG scheme for IoT data was proposed. With careful observation of the related works, the proposed V-BAG was designed to operate in network layers for easy implementation and good scalability. In order to meet the two important requirements of IoT (support of a large number of IoT devices and low power consumption), V-BAG has three key functions: packet scheduler, estimator, and ACK buffer. Each function was explained in detail, with an evaluation. The effects of wireless packet loss were mathematically investigated, and the simulation results proved that the bandwidth aggregation of the proposed scheme is quite effective. The main advantage of the proposed V-BAG system is that it increases the network capacity by bandwidth aggregation. The other advantage of V-BAG system is that it obtains transmission efficiency by simultaneously transmitting data through heterogeneous wireless networks. From an instantaneous energy consumption point of view, it is not energy efficient. However, because V-BAG scheme requires less time duration to transmit the same amount of data, IoT equipments can enter the idle state or energy-saving mode more often. Therefore, from a long-term energy consumption point of view, the V-BAG system is able to meet the low energy requirement of IoT equipments. In this article, two representative wireless links, LTE and WLAN are selected to show the benefits and the performance of V-BAG. However, since the proposed V-BAG is designed to be transparent to underlying communication technologies, other communication technologies such as IEEE-802.15.4, SigFox, and LoRa can be easily employed.

Investigation on much energy can be saved using heterogeneous wireless networks leading to short packet transmission time, real implementation of the proposed scheme in IoT devices, and performance comparison with other bandwidth aggregation schemes in real communication environments are left as future works.

Footnotes

Academic Editor: Michele Amoretti

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Gubbi

Buyya

Marusic

. Internet of Things (IoT): a vision, architectural elements, and future directions. Future Gener Comp Sy 2013; 29(7): 1645–1660.

Ramaboli

Falowo

Chan

AH.

Bandwidth aggregation in heterogeneous wireless networks: a survey of current approaches and issues. J Netw Comput Appl 2012; 35(6): 1674–1690.

Luo

Mukerjee

Dillinger

. Investigation of radio resource scheduling in WLANs coupled with 3G cellular network. IEEE Commun Mag 2003; 41(6): 108–115.

Tang

Wang

. An application layer protocol for energy-efficient bandwidth aggregation with guaranteed quality-of-experience. IEEE T Parall Distr 2015; 26(6): 1538–1546.

Robertson

Venkatasubramanian

KK.

Wireless bandwidth aggregation for wireless traffic. Worcester, MA: Worcester Polytechnic Institute, 2014.

Stewart

Ramalho

Xie

. Request For Comment 3758: stream control transmission protocol partial reliability extension, 2004, https://tools.ietf.org/pdf/rfc3758.pdf

Ford

Raiciu

Handley

. Request For Comment 6182: architectural guidelines for multipath TCP development, 2011, https://tools.ietf.org/pdf/rfc6182.pdf

Kang

Koji

Multi-path mechanism for audio/video streaming based on bandwidth estimation. Int J Comput Sci Netw Secur 2013; 13(2): 24–31.

Yaver

Koudouridis

. Performance evaluation of multi-radio transmission diversity: QoS support for delay sensitive services. In: Proceedings of the vehicular technology conference, Barcelona, 29 April 2009, pp.1–5. New York: IEEE.

10.

Kim

Davis

Ueda

. Splitting downlink multimedia traffic over WiMAX and WiFi heterogeneous links based on airtime-balance. Wirel Commun Mob Comput 2012; 12(7): 598–614.

11.

Koudouridis

Lundqvist

Karimi

. A quantitative analysis of the throughput gains and the energy efficiency of multi-radio transmission diversity in dense access networks. Telecommun Syst 2015; 59(1): 145–168.

12.

Chebrolu

Rao

RR.

Bandwidth aggregation for real-time applications in heterogeneous wireless networks. IEEE T Mobile Comput 2006; 5(4): 388–403.

13.

Evensen

Kaspar

Engelstad

. A network-layer proxy for bandwidth aggregation and reduction of IP packet reordering. In: Proceedings of the 2009 IEEE 34th conference on local computer networks, Zurich, 20–23 October 2009.

14.

Przybylski

Belter

Binczewski

Shall we worry about packet reordering. Comput Method Sci Technol 2005; 11(2): 141–146.

15.

Mascolo

Casetti

Gerla

. TCP westwood: bandwidth estimation for enhanced transport over wireless links. In: Proceedings of the 7th annual international conference on mobile computing and networking (MobiCom 01), Rome, 16–21 July 2001, pp.287–297. New York: ACM.

16.

Rhee

CUBIC: a new TCP-friendly high-speed TCP variant. ACM SIGOPS Oper Syst Rev 2008; 42(5): 64–74.

17.

Bao

Wong

VWS

Leung

VCM

. A model for steady state throughput of TCP CUBIC. In: Proceedings of the global telecommunications conference (GLOBECOM 2010), Miami, FL, 6–10 December 2010, pp.1–6. New York: ACM.