Dynamic capacity provision for wireless sensors’ connectivity: A profit optimization approach

Abstract

We model a wireless sensors’ connectivity scenario mathematically and analyze it using capacity provision mechanisms, with the objective of maximizing the profits of a network operator. The scenario has several sensors’ clusters with each one having one sink node, which uploads the sensing data gathered in the cluster through the wireless connectivity of a network operator. The scenario is analyzed both as a static game and as a dynamic game, each one with two stages, using game theory. The sinks’ behavior is characterized with a utility function related to the mean service time and the price paid to the operator for the service. The objective of the operator is to maximize its profits by optimizing the network capacity. In the static game, the sinks’ subscription decision is modeled using a population game. In the dynamic game, the sinks’ behavior is modeled using an evolutionary game and the replicator dynamic, while the operator optimal capacity is obtained solving an optimal control problem. The scenario is shown feasible from an economic point of view. In addition, the dynamic capacity provision optimization is shown as a valid mechanism for maximizing the operator profits, as well as a useful tool to analyze evolving scenarios. Finally, the dynamic analysis opens the possibility to study more complex scenarios using the differential game extension.

Introduction

The concept of Internet of things (IoT) as a revolutionary paradigm is not new. ¹ However, the wide concept of IoT that we know nowadays was not defined until the past decade. ² The number of devices connected is growing driven by this paradigm; in fact, according to Cisco, there will be 5.5 billion mobile devices connected to the Internet by 2020, ³ with a wide range of applications in several areas, such as education, healthcare, industry, infrastructures, smart homes, as well as smart cities,^4,5 among others. In this context of huge density of devices connected to wireless networks, the network capacity provision problem has been focused on optimizing the bandwidth usage using different approaches, such as algorithms and programming,^6–8 protocol modifications, ⁹ and game theory.^10–13 Nevertheless, given that the main actors in the capacity provision problem are the network operators (OP), it is also needed to justify the solutions not only from an efficiency point of view but also from an operator profit point of view.

The OP profit maximization problem has been addressed several times in the literature as a pricing problem.^14–17 Some of the papers only analyze monopolistic scenarios, ¹⁸ but it is also common to analyze competitive scenarios using game theory.^17,19 The analysis is typically solved statically, and the results are obtained in the equilibrium, where the actors have no incentive to change its decisions.^20,21 However, there are some studies that analyze dynamic problems, where the system parameters may vary over the time and the optimization is done within a time interval.^22,23 In our work, we tried to extend the scenario analyzed in the work by Sanchis-Cano et al. ²⁴ by solving a dynamic optimization problem using the price as control variable. However, the model was not controllable due to the linear dependence of the Hamiltonian function with the price. To solve this problem, we decided to analyze the profit maximization problem in an IoT scenario, using the capacity provision as control variable ¹¹ instead of the price.

Paper contributions and outline

In this article, we analyze a wireless sensors’ connectivity scenario from an economic point of view using mathematical modeling and game theory. We analyze the scenario using a static model as a first approximation and then we propose a more realistic dynamic model, using evolutionary games and optimal control theory to solve the problem of capacity provision for sensors’ connectivity. We analyze a scenario with several sensors’ clusters trying to transmit the gathered data through a network operator (OP), which provides wireless connectivity. The behavior of the sensors is modeled using a delay-sensitive utility function. The scenario is analyzed both statically and dynamically using game theory. For the static model, the sensors’ population equilibrium is found using population games, and the OP optimal leased capacity is obtained through a maximization problem. The static model is solved using backward induction, and a Nash equilibrium is found. In the dynamic model, the population behavior is modeled using the replicator dynamic, while the OP capacity decision is obtained solving an optimal control problem using the Pontryagin maximum principle (PMP).^25,26 The aim of this article is to show the feasibility of the proposed IoT scenario. To achieve this objective, we maximize the profits of the network operator in a given time interval, using the capacity provision as the maximization variable. We provide detailed mathematical procedures, not only for optimization problems with fixed parameters but also for problems where the parameters may vary over the time. In addition, we also provide graphical results, which demonstrate the efficiency of our dynamic capacity provision method for wireless sensors’ connectivity and the feasibility of the scenario.

One real-life scenario where our work may be useful is a scenario where an operator provides wireless connectivity to different kinds of sensors in a city. If the operator is able estimate the sensors’ mean life or can predict new deployments of sensors, then it can optimize the leased capacity over a long time period. In addition, if it is able to lease the capacity in advance, it may obtain a price reduction, and therefore, a reduction in its investment costs.

The main contributions of the article could be summarized by the following points:

• The provision of wireless sensors’ connectivity is shown feasible from an economic point of view for all the actors if the investment costs of the service provision are bounded (sections “Game I: static analysis” and “Results and discussion”).

• The capacity provision is a valid alternative to pricing techniques in profit maximization scenarios (section “Results and discussion”).

• The dynamic optimization using optimal control is shown more efficient than the optimization using equilibrium concepts (section “Results and discussion”).

• The dynamic optimization allows to optimize not only static but also changing IoT scenarios (section “OP optimal control and sinks’ distribution with dynamic parameters”).

The rest of this article is organized as follows: in section “General model,” we describe in detail the scenario and the behavior of the actors involved, the utility of the sinks, and the operator profit. In section “Game analysis,” the scenario is analyzed using a static and a dynamic model. The sinks’ subscription problem as well as OP profit maximization problem are solved using game theory and optimization. Section “Results and discussion” shows and discusses the results, while section “Conclusion” draws the conclusions.

General model

We consider the IoT scenario which is depicted in Figure 1 with several clusters uploading their sensing data to the Internet through a network operator (OP). The sensor nodes are grouped into clusters. Each cluster has a large number of sensing nodes connected through a multi-hop wireless network. ²⁷ Each cluster has a sink node, which transmits the data collected by all the nodes in the cluster to the Internet through a network operator (OP). Our scenario is based on the work by Sanchis-Cano et al. ²⁴ and analyzes the interaction between the sinks and the OP. The analyzed model has the following market actors:

• Sinks.

• Network operator (OP).

Figure 1.

Analyzed scenario with all the actors of the market.

Sinks

Each sink belongs to only one cluster. Each sink is responsible of transmitting all the data collected by its sensors in a cluster to the Internet. They are the clients of the wireless connectivity service offered by the OP. The number of sinks is $N$ , where $N>>1$ .

In order to model the utility perceived by the sinks that subscribe to the OP, we use a quality function $Q$ based on the previous works,^{18,24,28–31} which evaluate the service offered by the OP

$$Q\equiv c{\left(T\right)}^{-1}$$

(1)

where $c>0$ is a conversion factor and $T$ is the mean sensing-data-unit (s.d.u) service time. Note that when the service time $T$ increases, $Q$ decreases, or equivalently, the sinks perceive a worst quality when the delay of the network increases. This function has the ability to model the congestion in the wireless network, which is suitable for IoT scenarios with delay constraints. ³² We model the OP service as an M/M/1 system, and compute the mean service time $T$ as follows ³³

$$T=\frac{1}{\mu -\lambda }$$

(2)

where $\mu$ is inverse of the mean s.d.u transmission time $\tau =1/\mu$ or simply the system capacity, and $\lambda$ is the arrival rate of the s.d.u.

We propose a utility function, which models the perception of the sinks about the service offered by the OP, as the difference between the quality perceived by the sinks and the price charged by the OP. This utility function is also known as compensated utility and is commonly used in telecommunications^28,34–36

$$U_s\equiv Q-p=c(\mu -x_1\mathrm{Nr})-p$$

(3)

where we have re-written the arrival rate as the traffic generated by all the sinks being served $\lambda =x_1\mathrm{rN}$ , $r$ is the s.d.u generation rate of one sink, $p$ is the price in monetary units (m.u.) per s.d.u charged by the OP to each sink when it transmits one s.d.u and $x_1$ is the fraction of sinks being served by the OP.

The utility must be non-negative $U_s\ge 0$ or the sink will not subscribe to the service. Note that all the sinks in the system perceive the same utility. The distribution of sinks in the system is described by the vector $X_s=(x_0,x_1)$ , where $x_0$ and $x_1$ are the fraction of sinks being served and not being served by the OP, respectively, and $x_0+x_1=1$ .

Network operator

The OP offers a wireless connectivity service to the sinks that allows them to transmit the data collected and charges a price $p$ to the corresponding sink per s.d.u transmitted.

The objective of the OP is to maximize its own profit choosing the system capacity in order to provide a service ratio $\mu$ given a fixed price $p>0$ . The OP profit is as follows

$${\Pi }_{\mathrm{OP}}=x_1\mathrm{Nrp}-k{\mu }^2$$

(4)

where $\mathrm{Npr}{x}_1$ are the revenues obtained from sinks and $k{\mu }^2$ are the investment costs ³⁷ of leasing a system capacity $\mu$ , and $k$ is a cost scale factor. The convex cost factor allows us to prevent an aggressive behavior of the OP,^12,38 opening the possibility to analyze competitive scenarios in future studies.

Figure 2 shows the payment flow described in this section; we observe that the amount of money perceived by the OP is proportional to the traffic generated by all the sinks multiplied by the price that each sink pays per data unit.

Figure 2.

Model payment flow and actors involved.

Game analysis

The model described in the previous section can be analyzed as two games each one with two stages. The first game is a static analysis, while the second game is a dynamic analysis of the model. Both games have the following structure: first, an optimization stage where the OP chooses the capacity that maximizes its profits and second, a sink’s subscription stage. The games are summarized in Figure 3.

Figure 3.

Description of the game stages.

The games were solved as follows. First, the Game I was solved. A static analysis was conducted and the equilibrium solutions were obtained. Second, the Game II was solved, obtaining the optimal OP decisions and the social state as a function of time.

Both games were solved using backward induction, which allows us to find a subgame perfect Nash equilibrium (SPNE) of the proposed games. Backward induction consists in deducing backward from the end of a problem to the beginning to infer a sequence of optimal actions. Any Nash equilibrium found using backward is a Nash equilibrium for every subgame or, equivalently, an SPNE.^24,39

Game I: static analysis

This game analyzes our scenario using a static model, where all the parameters are fixed. In this game, the actors act with perfect rationality and its decisions are instantaneous. The solution of this game is a Nash equilibrium where no actor has incentive to change its own decisions.

Stage II: sinks’ subscription game

This stage is played once the OP has fixed its $\mu$ . Sinks’ equilibrium was solved using the unified framework provided by population games described by Sandholm. ⁴⁰ This framework is useful for study strategic interactions between agents with certain properties that our model satisfies.

Population game

• Strategies: $S=\{0,1\}$ , where $0$ means not to subscribe to the OP and $1$ means to subscribe to the OP.

• Social state: $X_s=\{x_0,x_1\},x_0+x_1=1$ . Sinks’ distribution between not being served and being served by the OP.

• Payoffs: $F_s(x_0,x_1)=\{F_{s0}(X),F_{s1}(X)\}=\{0,U_s\}$ , where $U_s$ is the utility of the sinks defined in equation (3), $F_{s_0}(X)$ is the utility of the sinks not subscribing to the OP, and $F_{s_1}(X)$ is the utility of the sinks subscribing to the OP.

Pure best response

The pure best response $b(X_s)$ is the best response where the actors can only choose a pure strategy. ⁴⁰ In this case, a pure strategy means that all the population of sinks choose the same strategy. The first step for solving the population game is to obtain the pure strategies that are optimal at each social state $X_s$

$$\begin{array}{c}b(X_s)\equiv \underset{i\in S}{\mathrm{argmax}}{F}_{\mathrm{si}}(X_s)\hfill \\ =\{\begin{array}{c}i=0\mathrm{if}\mu \le \frac{p}{c}+x_1\mathrm{Nr}\hfill \\ \multicolumn{1}{c}{i=1\mathrm{if}\mu \ge \frac{p}{c}+x_1\mathrm{Nr}}\end{array}\hfill \end{array}$$

(5)

where $i$ is the pure strategy chosen by all the population.

Mixed best response

The mixed best response $B(X_s)$ is the best response where the actors can choose a mixed strategy. ⁴⁰ In this case, a mixed strategy means that each sink in the population chooses its strategy based on probabilities, and therefore, the population could be split into several strategies. Once we have obtained the pure best responses, we can extend the results to include the best mixed strategies

$$\begin{array}{c}B(X_s)\equiv \{[z_0+z_1=1;z_i\in R_{+}]:z_i>0\Rightarrow i\in b(X_s)\}\hfill \\ =\{\begin{array}{c}{z}_0=1,z_1=0\mathrm{if}{x}_1\ge \frac{c\mu -p}{\mathrm{cNr}}\hfill \\ z_0>0,z_1>0\mathrm{if}{x}_1=\frac{c\mu -p}{\mathrm{cNr}}\hfill \\ z_0=0,z_1=1\mathrm{if}{x}_1\le \frac{c\mu -p}{\mathrm{cNr}}\hfill \end{array}\hfill \end{array}$$

(6)

where $z_i$ is the fraction of the population choosing the strategy $i$ .

Nash equilibrium

At this point, social state $x\in X_s$ is a Nash equilibrium of the game $F_s$ if all the agents choose a best response to $x\in X_s$

$$\begin{array}{c}\mathrm{NE}(F_s)\equiv \{x\in X_s:x\in B(X_s)\}\hfill \\ =\{\begin{array}{ccc}(1,0)\hfill & \mathrm{if}\hfill & \mu \le \frac{p}{c}\hfill \\ \multicolumn{1}{c}{(1-\frac{c\mu -p}{\mathrm{cNr}},\frac{c\mu -p}{\mathrm{cNr}})}& \mathrm{if}\hfill & \frac{p}{c}\le \mu \le \frac{p}{c}+\mathrm{Nr}\hfill \\ \multicolumn{1}{c}{(0,1)}& \mathrm{if}\hfill & \mu \ge \frac{p}{c}+\mathrm{Nr}\hfill \end{array}\hfill \end{array}$$

(7)

Stage I: OP capacity optimization

In this stage, the OP wants to maximize its profit given by equation (4) using $\mu$ as the optimization variable and considering the price $p$ fixed by a regulatory authority. Given the three cases obtained from equation (7), we analyze the case where the maximum profit is reached

$${\Pi }_{\mathrm{OP}}=\{\begin{array}{ccc}-k{\mu }^2\hfill & \mathrm{if}\hfill & \mu \le \frac{p}{c}\hfill \\ \multicolumn{1}{c}{\frac{c\mu -p}{c}p-k{\mu }^2}& \mathrm{if}\hfill & \frac{p}{c}\le \mu \le \frac{p}{c}+\mathrm{Nr}\hfill \\ \multicolumn{1}{c}{\mathrm{Nrp}-k{\mu }^2}& \mathrm{if}\hfill & \mu \ge \frac{p}{c}+\mathrm{Nr}\hfill \end{array}$$

(8)

• Case 1: $\mu \le p/c$ : in this case, the maximum profit is obtained solving the optimization problem

$$\begin{array}{cc}\underset{\mu }{\max }\hfill & {\Pi }_{O{P}_{c1}}^{*}=-k{\mu }^2\hfill \\ \multicolumn{1}{c}{\mathrm{subject}\mathrm{to}}& \mu \le \frac{p}{c}\hfill \end{array}$$

(9)

where ${\Pi }_{O{P}_{\mathrm{ci}}}^{*}$ is the profit obtained in equation (8) for the Case $i$ . The solution for the problem defined in equation (9) is as follows

$$\begin{array}{ccc}{\Pi }_{O{P}_{c1}}^{*}=0\hfill & \mathrm{with}\hfill & {\mu }^{*}=0\hfill \end{array}$$

(10)

Note that in this case, it is not possible to obtain positive profit.

• Case 2: $\frac{p}{c}\le \mu \le \frac{p}{c}+\mathrm{Nr}$ : in this case, the maximum profit is obtained solving the optimization problem

$$\begin{array}{cc}\underset{\mu }{\max }\hfill & {\Pi }_{O{P}_{c2}}^{*}=\frac{c\mu -p}{c}p-k{\mu }^2\hfill \\ \multicolumn{1}{c}{\mathrm{subject}\mathrm{to}}& \frac{p}{c}\le \mu \le \frac{p}{c}+\mathrm{Nr}\hfill \end{array}$$

(11)

The problem in equation (11) is solved using Karush–Kuhn–Tucker (KKT) conditions and its solution is as follows

$${\Pi }_{O{P}_{c2}}^{*}=\{\begin{array}{ccc}\frac{(c-4k)p^2}{4\mathrm{ck}}\hfill & \mathrm{if}\hfill & k>\frac{\mathrm{cp}}{2(p+\mathrm{cNr})}\hfill \\ \multicolumn{1}{c}{}& \mathrm{with}\hfill & {\mu }^{*}=\frac{p}{2k}\hfill \\ \multicolumn{1}{c}{\frac{c^2\mathrm{Npr}-k{(p+\mathrm{cNr})}^2}{c^2}}& \mathrm{if}\hfill & k\le \frac{\mathrm{cp}}{2(p+\mathrm{cNr})}\hfill \\ \multicolumn{1}{c}{}& \mathrm{with}\hfill & {\mu }^{*}=\frac{p}{c}+\mathrm{Nr}\hfill \end{array}$$

(12)

• Case 3: $\mu \ge \frac{p}{c}+\mathrm{Nr}$ : in this case, the maximum profit is obtained solving the optimization problem

$$\begin{array}{cc}\underset{\mu }{\max }\hfill & {\Pi }_{O{P}_{c3}}^{*}=\mathrm{Nrp}-k{\mu }^2\hfill \\ \multicolumn{1}{c}{\mathrm{subject}\mathrm{to}}& \mu \ge \frac{p}{c}+\mathrm{Nr}\hfill \end{array}$$

(13)

The problem in equation (13) is solved again using KKT conditions and its solution is as follows

$${\Pi }_{O{P}_{c3}}^{*}=\frac{c^2\mathrm{Npr}-k{(\mathrm{cNr}+p)}^2}{c^2}\mathrm{with}{\mu }^{*}=\frac{p}{c}+\mathrm{Nr}$$

(14)

Given that the first part of equation (12) is always greater than equation (14) for the problem restrictions, the OP optimal profit can be summarized as follows

$${\Pi }_{\mathrm{OP}}^{*}=\{\begin{array}{ccc}\frac{(c-4k)p^2}{4\mathrm{ck}}\hfill & \mathrm{if}\hfill & k>\frac{\mathrm{cp}}{2(p+\mathrm{cNr})}\hfill \\ \multicolumn{1}{c}{}& \mathrm{with}\hfill & {\mu }^{*}=\frac{p}{2k}\hfill \\ \multicolumn{1}{c}{\frac{c^2\mathrm{Npr}-k{(p+\mathrm{cNr})}^2}{c^2}}& \mathrm{if}\hfill & k\le \frac{\mathrm{cp}}{2(p+\mathrm{cNr})}\hfill \\ \multicolumn{1}{c}{}& \mathrm{with}\hfill & {\mu }^{*}=\frac{p}{c}+\mathrm{Nr}\hfill \end{array}$$

(15)

Analyzing the previous results, we observe that ${\Pi }_{\mathrm{OP}}^{*}>0$ if the following conditions are met

• Case $k>\mathrm{cp}/2(p+\mathrm{cNr})$

$$k<\frac{c}{4}$$

(16)

• Case $k\le \mathrm{cp}/2(p+\mathrm{cNr})$

$$k<\frac{c^2\mathrm{Npr}}{{(p+\mathrm{cNr})}^2}$$

(17)

In this case, there are two possible interpretations depending on which is more restrictive than equation (17) or $k\le \mathrm{cp}/2(p+\mathrm{cNr})$ . If $c>p/\mathrm{Nr}$ , then the case condition $k\le \mathrm{cp}/2(p+\mathrm{cNr})$ is more restrictive than equation (17) and therefore there are no additional conditions. However, if $c\le p/\mathrm{Nr}$ , then equation (17) is more restrictive and it must be met in order to obtain positive profits.

As shown in the previous analysis, the value of $k$ has a vital role in the feasibility of the system and therefore has to be bounded in order to obtain positive profits.

Game II: dynamic analysis

This game analyzes our scenario using a dynamic model, where the parameters and the decisions of the actors may change over the time. The dynamic analysis was conducted using evolutionary game theory for the sinks’ subscription game, while for the OP capacity, optimization stage optimal control theory and PMP were used.

Stage II: sinks’ evolutionary subscription game

In order to maximize the user utility described in equation (3), we define the following evolutionary game:

• Strategies: $S=\{S_0,S_1\}$ , where $S_0$ means not to subscribe to the OP and $S_1$ means to subscribe to the OP.

• Social state: $X_s(t)=\{x_0(t),x_1(t)\},x_0+x_1=1$ . Sinks’ distribution between not being served and being served by the OP.

• Payoffs: $U_s(t)=\{u_0(t),u_1(t)\}=\{0,U_s(t)\}$ , where $U_s(t)$ is the utility of the sinks defined in equation (3) as a function of time, $u_0(t)$ is the utility of the sinks not subscribing to the OP, and $u_1(t)$ is the utility of the sinks subscribing to the OP. Note that here the utility varies with the time due to the variation on the social state.

The sinks use a set of rules to update their strategies. This set of rules is known as revision protocol ⁴⁰ and determine the evolutionary dynamic. There are several revision protocols but we are interested in the imitative protocols and direct selection protocols. In the imitative protocols, the users update their strategies taking into account the strategies chosen by other users, but imitative protocols admit boundary rest points that are not Nash equilibria of the underlying game. ⁴¹ On the other hand, direct selection protocols are not directly influenced by the choice of others and this characteristic prevents the boundary rest points. In this work, we have chosen an imitative protocol, given that it is tractable analytically and widely used in the literature. However, we need to be cautious about the boundary rest points.

The revision protocol used in this work can be described by the following action:

• At the time instant $t$ , a user with strategy $S_i$ imitates the strategy $S_j(j\ne i)$ selected by other user if $u_i(t)>u_j(t)$ with probability

$${\rho }_{\mathrm{ij}}^I(t,x_j,u_i,u_j)=x_j(t)[u_j(t)-u_i(t){]}^{+}$$

(18)

The revision protocol was introduced by Schlag ⁴² in a population game context. Under this protocol, a user switches its strategy only if the other user has a better utility. The switching rate is proportional to the difference in the utility and the number of users in the destination strategy. The protocol has D2 data requirements. ⁴¹

The mean dynamic can be derived from the proposed revision protocol (equation (18)) as follows

$$\begin{array}{cc}\multicolumn{1}{c}{{\stackrel{·}{x}}_i}& =\sum _{j\in S}{x}_j{\rho }_{\mathrm{ji}}-x_i\sum _{j\in S}{\rho }_{\mathrm{ij}}\hfill \\ \multicolumn{1}{c}{}& =\sum _{j\in S}{x}_i{x}_j{[u_i-u_j]}^{+}-x_i\sum _{j\in S}{x}_j{[u_j-u_i]}^{+}\hfill \\ \multicolumn{1}{c}{}& =x_i\sum _{j\in S}{x}_j(u_i-u_j)=x_i(u_i-\sum _{j\in S}{x}_j{u}_j)\hfill \\ \multicolumn{1}{c}{}& =x_i(u_i-U_{\mathrm{AVG}})\hfill \\ \multicolumn{1}{c}{{\stackrel{·}{x}}_i}& =\delta x_i(u_i-u_{\mathrm{AVG}})\hfill \end{array}$$

(19)

where $\delta$ is the learning rate and $U_{\mathrm{AVG}}=\sum _{j\in S}{x}_j{u}_j$ is the average utility of all the users in the model. Following the mean dynamic described above, users learn progressively the best choice until the market reach a stationary point, where the action of one user has no impact in the utility of the other users and no user has an incentive to switch its strategy. When the equilibrium is reached, the utility of all the users is the same $u_i=u_j\forall i,j\in N$ . This mean dynamic is also known as replicator dynamic. Adapting equation (19) to our model, we obtain the following equation

$$\begin{array}{c}{\stackrel{·}{x}}_0=\delta x_0(u_0-x_0{u}_0-x_1{u}_1)=\delta x_0(-x_1{u}_1)\hfill \\ {\stackrel{·}{x}}_1=\delta x_1(u_1-x_0{u}_0-x_1{u}_1)=\delta x_1(u_1-x_1{u}_1)\hfill \end{array}$$

(20)

Given that $x_1=1-x_0$ , we can work only with one of the previous equations without loss of generality.

Dynamic stationary points

The dynamic reaches a stationary point when no user is willing to change its strategy or equivalently when ${\stackrel{·}{x}}_i=0$

$$\begin{array}{c}{\stackrel{·}{x}}_1=\delta x_1(u_1-x_1{u}_1)=0\hfill \\ \delta x_1{u}_1(1-x_1)=0\hfill \end{array}$$

Solving the previous equation and assuming that $\delta >0$ , we get the following steady states:

• Case 1

$$x_1=0,x_0=1$$

(21)

• Case 2

$$\begin{array}{c}1-x_1=0\hfill \\ x_1=1,x_0=0\hfill \end{array}$$

(22)

• Case 3

$$\begin{array}{c}{u}_1=c(\mu -x_1\mathrm{rN})-p=0\hfill \\ x_1=\frac{c\mu -p}{\mathrm{cNr}},x_0=1-\frac{c\mu -p}{\mathrm{cNr}}\hfill \end{array}$$

(23)

Stability of stationary points

Once we have found the stationary points, it is necessary to characterize its stability. Consider a steady state $x\in X_s$ where sinks perceive a utility $U_s(x)$ and an invader state $y\in X_s$ where some sinks move to a different strategy and they perceive a utility $U_s(y)$ . We can affirm that $x\in X_s$ is a globally evolutionary stable strategy (GESS) ⁴⁰ if

$$U_s(y)-U_s(x)<0\forall y\in X-\{x\}$$

(24)

which means that the utility perceived by the sinks which did not switch their strategy from state $x\in X_s$ is higher than the utility perceived by the sinks which switched it. An equivalent definition is that the utility of sinks which switch their strategy decreases or the utility of sinks which keep their strategy increases, while the utility of sinks which switch remains constant. ⁴³ We can apply this definition to the steady states found in the previous point

• Case 1: $X=(x_0=1,x_1=0)$ .

Consider that a number of sinks $ϵ$ migrate from strategy $S_0$ to $S_1$ , which leads us to the new social state

$$X\prime =(x{\prime }_0=1-\epsilon ,x{\prime }_1=\epsilon )$$

The utility of sinks in both states is as follows

$$\begin{array}{c}{U}_s(x_0)=0,U_s(x_1)=c\mu -p\hfill \\ U_s(x{\prime }_0)=0,U_s(x{\prime }_1)=c(\mu -\epsilon \mathrm{Nr})-p\hfill \end{array}$$

This steady state is a GESS if

$$\begin{array}{c}{U}_s(x_0)>U_s(x{\prime }_1)||U_s(x{\prime }_0)>U_s(x{\prime }_1)\hfill \\ 0>c(\mu -\epsilon \mathrm{Nr})-p\hfill \end{array}$$

For all the possible values of $ϵ\in ]0,1]$ , it is true if

$$\mu \le \frac{p}{c}$$

(25)

• Case 2: $X=(x_0=0,x_1=1)$ .

Consider that a number of sinks $ϵ$ migrate from strategy $S_1$ to $S_0$ , which leads us to the new social state

$$X\prime =(x{\prime }_0=\epsilon ,x{\prime }_1=1-\epsilon )$$

The utility of sinks in both states is as follows

$$\begin{array}{c}{U}_s(x_0)=0,U_s(x_1)=c(\mu -\mathrm{Nr})-p\hfill \\ U_s(x{\prime }_0)=0,U_s(x{\prime }_1)=c(\mu -\epsilon \mathrm{Nr})-p\hfill \end{array}$$

This steady state is a GESS if

$$\begin{array}{c}{U}_s(x_1)>U_s(x{\prime }_0)||U_s(x{\prime }_1)>U_s(x{\prime }_0)\hfill \\ c(\mu -\mathrm{Nr})-p>0||c(\mu -\epsilon \mathrm{Nr})-p>0\hfill \end{array}$$

For all the possible values of $ϵ\in ]0,1]$ , it is true if

$$\mu \ge \frac{p}{c}+\mathrm{Nr}$$

(26)

• Case 3: $X=(x_0=1-(c\mu -p/\mathrm{cNr}),x_1=c\mu -p/\mathrm{cNr})$ .

Consider that a number of sinks $ϵ$ migrate from strategy $S_1$ to $S_0$ , which leads us to the new social state

$$X=(x_0=1+\epsilon -\frac{c\mu -p}{\mathrm{cNr}},x_1=\frac{c\mu -p}{\mathrm{cNr}}-\epsilon )$$

The utility of sinks in both states is as follows

$$\begin{array}{c}{U}_s(x_0)=0,U_s(x_1)=c(\mu -\frac{c\mu -p}{\mathrm{cNr}}\mathrm{Nr})-p=0\hfill \\ U_s(x{\prime }_0)=0,U_s(x{\prime }_1)=c(\mu -(\frac{c\mu -p}{\mathrm{cNr}}-\epsilon )\mathrm{Nr})-p\hfill \end{array}$$

The necessary conditions to be a GESS are follows

$$\begin{array}{c}{U}_s(x_1)>U_s(x{\prime }_0)||U_s(x{\prime }_1)>U_s(x{\prime }_0)\hfill \\ 0>0||c(\mu -(\frac{c\mu -p}{\mathrm{cNr}}-\epsilon )\mathrm{Nr})-p>0\hfill \end{array}$$

For all the possible values of $\epsilon \in ]0,c\mu -p/\mathrm{cNr}]$ , it is true if

$$\mu >\frac{p}{c}$$

(27)

On the other hand, if we analyze the case when a number of sinks $ϵ$ migrate from strategy $S_0$ to $S_1$ , we obtain the new social state

$$X=(x_0=1-\epsilon -\frac{c\mu -p}{\mathrm{cNr}},x_1=\frac{c\mu -p}{\mathrm{cNr}}+\epsilon )$$

The utility of sinks in both states is as follows

$$\begin{array}{c}{U}_s(x_0)=0,U_s(x_1)=c(\mu -\frac{c\mu -p}{\mathrm{cNr}}\mathrm{Nr})-p=0\hfill \\ U_s(x{\prime }_0)=0,U_s(x{\prime }_1)=c(\mu -(\frac{c\mu -p}{\mathrm{cNr}}+\epsilon )\mathrm{Nr})-p\hfill \end{array}$$

The necessary conditions to be a GESS are as follows

$$\begin{array}{c}{U}_s(x_0)>U_s(x{\prime }_1)|U_s(x{\prime }_0)>U_s(x{\prime }_1)\hfill \\ 0>c(\mu -(\frac{c\mu -p}{\mathrm{cNr}}+\epsilon )\mathrm{Nr})-p\hfill \end{array}$$

For all the possible values of $\epsilon \in ]0,1(c\mu -p/\mathrm{cNr})$ , it is true if

$$\mu <\frac{p}{c}+\mathrm{Nr}$$

(28)

With equations (27) and (28), we have the sufficient conditions where this state is a GESS

$$\frac{p}{c}<\mu <\frac{p}{c}+\mathrm{Nr}$$

(29)

In the previous analysis, we have demonstrated that there is a GESS for all the possible values of the control variable $\mu$ . Furthermore, in every single population games, like in our model, it can be demonstrated that every GESS is unique and it is also a Nash equilibrium. ⁴⁰ In addition, every GESS is also an ESS and, as proven by Barron, ³⁹ it is also an asymptotically stable solution of the dynamic.

Note that when one of the steady states deduced in equations (21)–(23) is a GESS, it is unique. Figure 4 shows a particular case when the GESS is the mixed strategy equilibrium (equation (23)).

Figure 4.

Replicator dynamic convergence when the GESS is a mixed equilibrium.

Stage I: OP dynamic capacity optimization

The capacity optimization stage was solved using optimal control theory, ²⁶ which allows us to do a dynamic optimization within a time horizon and not only in the steady states. As a result of the dynamic optimization, we obtained a control function in every instant of time $t$ that optimizes the objective function within a time horizon $t\in [0,T]$ . The problem that we are going to solve is to obtain the optimal capacity that maximizes the profits of the OP, given that the behavior of sinks is modeled by the dynamic (eqaution (19))

$$\begin{array}{c}\underset{\mu }{\max }{\Pi }_{\mathrm{OP}}(\mu )=\underset{0}{\overset{T}{\int }}{e}^{-\rho t}{\Pi }_{O{P}_{\mathrm{INS}}}(\mu )\mathrm{dt}\hfill \\ \mathrm{s}.\mathrm{t}.{\stackrel{·}{x}}_i=\delta x_i(u_i-u_{\mathrm{AVG}}),X_s(0)=X_0,\mathrm{and}\mu \in ]0,{\mathbb{R}}^{+}[\hfill \end{array}$$

(30)

where $\rho$ is a given discount rate, ${\Pi }_{O{P}_{\mathrm{INS}}}(\mu )$ is the instantaneous profit of the OP defined in equation (4) and $X_0$ is the initial distribution of the population.

In order to solve the previous problem, we used the PMP, which provides the necessary conditions to find the candidate optimal strategies for the open-loop case. The Hamiltonian function of the OP is defined as follows

$$H={\Pi }_{O{P}_{\mathrm{INS}}}+\lambda {\stackrel{·}{x}}_1$$

where $\lambda$ is the adjoint variable of the OP. Rewriting the Hamiltonian in terms of our model, we have the following equation

$$\begin{array}{c}H=x_1(\delta \lambda x_1(-c(\mu +\mathrm{Nr})+\mathrm{cNr}{x}_1+p)\hfill \\ +\delta \lambda (c\mu -p)+\mathrm{Npr})-k{\mu }^2\hfill \end{array}$$

(31)

Following the PMP, all candidate optimal strategies must satisfy the necessary conditions

$${\mu }^{*}(t)=\underset{\mu \in ]0,{\mathbb{R}}^{+}[}{\max }H$$

(32)

$${\stackrel{·}{x}}_1=\delta x_1(u_1-u_{\mathrm{AVG}})$$

(33)

$$\stackrel{·}{\lambda }(t)=\lambda \rho -\frac{\partial H}{\partial x_1}$$

(34)

$$\lambda (T)=0$$

(35)

where equation (32) is the maximality condition, equation (33) is the replicator dynamic, which models the behavior of the sinks, equation (34) is the adjoint equation, and equation (35) is the transversality condition. Solving equation (32), we obtain the candidate strategy to maximize in terms of the state $x_1$ and the adjoint variable $\lambda$

$${\mu }^{*}(t)=-\frac{c\delta \lambda (x_1-1)x_1}{2k}$$

(36)

Replacing the optimal candidate strategy equation (36) in the remaining PMP conditions and with the initial state condition, we have the system of partial differential equations (PDEs) shown in equation (37)

$$\{\begin{array}{c}{\stackrel{·}{x}}_1=\frac{\delta (x_1-1)x_1(c{x}_1(-c\delta \lambda +c\delta \lambda x_1+2\mathrm{kNr})+2\mathrm{kp})}{2k}\hfill \\ \stackrel{·}{\lambda }(t)=\frac{2k(\lambda (\delta p+\rho )-\mathrm{Npr})-\delta \lambda x_1(c^2\delta \lambda +4k(p-\mathrm{cNr}))-\delta \lambda x_1{\mathrm{cx}}_1(-3c\delta \lambda +2c\delta \lambda x_1+6\mathrm{kNr})}{2k}\hfill \\ x_1(0)=x0\hfill \\ \lambda (T)=0\hfill \end{array}$$

(37)

This system is a two-boundary value problem (TBVP) and cannot be solved using traditional methods for PDEs, given that it has no initial conditions for all its variables. Instead of it, is has an initial condition and an end condition. This problem has been solved numerically using the shooting method. ⁴⁴ Given that the shooting method requires a good initial estimation for the value of $\lambda (0)$ , otherwise it may be unstable, we have solved the problem in several steps, beginning with small values of $T$ and increasing it in the following stages, using the solution of $\lambda (0)$ of the previous stage as initial estimation for the present stage.

Results and discussion

In this section, we present the numerical results for the static and dynamic games analyzed in the previous section. The results were obtained for the case when the equilibrium is a mixed strategy. The figures were calculated for the values shown in Table 1 unless otherwise specified.

Table 1.

Reference Case 1—static parameters.

Parameter	Value	Units
Quality conversion factor $(c)$	$1$	$\left[\frac{m.us}{s.d.u^2}\right]$
Sensor data generation ratio $(r)$	$1$	$\left[\frac{s.d.u}{s}\right]$
Operator price $(p)$	$0.2$	$\left[\frac{m.u}{s.d.u}\right]$
Total number of sensors $(N)$	$200$
Capacity cost scale parameter $(k)$	$\frac{\mathrm{cp}}{1.5(\mathrm{cNr}+p)}$	$\left[\frac{m.us}{s.d.u^2}\right]$
Dynamic’s learning rate $(\delta )$	0.14
Initial social state $(X_s(0))$	$\{0.05,0.95\}$
End time horizon $(T)$	$1$	$\left[\mathrm{s}\right]$
Discount rate $(\rho )$	$0.2$

OP optimal control and sinks’ distribution with static parameters

In order to study the static and dynamic results, we show the optimal capacity ${\mu }^{*}(t)$ and the fraction of sinks being served by the OP $x_1(t)$ as a function of the time $t$ , for different values of the number of sinks $N$ .

Figure 5 shows the OP optimal capacity in the static case and in the dynamic case for different values of $N$ . In both the static and the dynamic analyses, when $N$ increases, the optimal capacity increases in order to be able to serve the higher number of sinks. Comparing the static and the dynamic analyses, we observe that the provider chooses a similar strategy for low values of $t$ . It is different due to the existence of the discount rate $\rho$ . Nevertheless, when $t$ is close to $T$ , the provider decreases the reserved capacity, and when $t=T$ , the total capacity reserved is zero. This behavior makes sense given that the OP optimize its decision for a limited time interval, and it is not worthy to have costs when the OP has not to provide more services. Figure 6 shows a similar behavior. For low values of $t$ , the population learns the optimal strategy by imitation moving from the initial state to the static Nash equilibrium. The population learns faster the optimal strategy when it has a higher amount of sinks. For values of $t$ close to $T$ , the utility perceived by the sinks decreases due to the decrease in the capacity offered by the provider. The sinks start to leave the OP service but they are not able to learn fast enough and some sinks remain in the OP when $t=T$ and it offers no service at all.

Figure 5.

OP optimal capacity in the static and dynamic cases for different values of $N$ .

Figure 6.

Social state in the static and dynamic cases for different values of $N$ .

OP optimal control and sinks’ distribution with dynamic parameters

In this section, we show the evolution of the optimal capacity ${\mu }^{*}(t)$ and the fraction of sinks being served by the OP $x_1(t)$ , when the number of sinks in the system is also a function of the time $N(t)$ . The results for two different scenarios are shown. Figures 7–10 are related to the Scenario 1, while Figures 11–14 are related to the Scenario 2. The figures for each scenario were calculated for the values shown in Table 2.

Figure 7.

Scenario 1: evolution of the number of sinks $N$ as a function of $t$ .

Figure 8.

Scenario 1: OP optimal capacity in the cases with static and dynamic optimization as a function of $t$ .

Figure 9.

Scenario 1: social state in the three studied cases as a function of $t$ .

Figure 10.

Scenario 1: evolution of the OP profits for different strategies as a function of $t$ and total profits.

Figure 11.

Scenario 2: evolution of the number of sinks $N$ as a function of $t$ .

Figure 12.

Scenario 2: OP optimal capacity in the cases with static and dynamic optimization as a function of $t$ .

Figure 13.

Scenario 2: social state in the three studied cases as a function of $t$ .

Figure 14.

Scenario 2: evolution of the OP profits for different strategies as a function of $t$ and total profits.

Table 2.

Reference Case 2.1—dynamic common parameters.

Parameter	Scenarios 1 and 2	Units
Quality conversion factor $(c)$	$1$	$\left[\frac{m.us}{s.d.u^2}\right]$
Sensor data generation ratio $(r)$	$1$	$\left[\frac{s.d.u}{s}\right]$
Operator price $(p)$	$0.2$	$\left[\frac{m.u}{s.d.u}\right]$
Initial number of sensors $(N(0))$	$1200$
Dynamic’s learning rate $(\delta )$	$0.14$
Initial social state $(X_s(0))$	$\{0.25,0.75\}$
End time horizon $(T)$	$0.5$	$\left[\mathrm{s}\right]$
Discount rate $(\rho )$	$0$

In both scenarios are shown three different cases:

• Case 1. In this case, the values of ${\mu }^{*}(t)$ and $x_1(t)$ are obtained using the solutions for the static equilibrium obtained in equations (7) and (15) for each instant of time. The values of ${\mu }^{*}(t)$ and $x_1(t)$ are represented in the figures with the names “ ${\mu }^{*}$ Static” and “ $x_1^{*}$ Static,” respectively.

• Case 2. In this case, the value of ${\mu }^{*}(t)$ is obtained using the solution for the static equilibrium obtained in equation (7) for each time instant. However, the value of $x_1(t)$ is obtained from the replicator dynamic defined in equation (20). The values of ${\mu }^{*}(t)$ and $x_1(t)$ are represented in the figures with the names “ ${\mu }^{*}$ Static” and “ $x_1^{*}$ Replicator,” respectively. Note that the value of ${\mu }^{*}(t)$ is the same in the Case 1 and Case 2. This case models a more realistic model when the behavior of the sinks is not ideal and their reaction against a change in the market is not instantaneous.

• Case 3. In this case, the values of ${\mu }^{*}(t)$ and $x_1(t)$ are obtained from the solution to the optimal control problem defined in equation (37). The values of ${\mu }^{*}(t)$ and $x_1(t)$ are represented in the figures with the names “ ${\mu }^{*}$ Optimal Control” and “ $x_1^{*}$ Optimal Control,” respectively.

Scenario 1

This scenario models a decreasing number of sensors over the time due to failures in the sensors during its life, as shown in Table 3 and Figure 7. The figures were calculated for the values shown in Tables 2 and 3.

Table 3.

Reference Case 2.1—dynamic non-common parameters.

Parameter	Scenario 1 value
Evolution of number of sensors $(N(t))$	$N(0)-\frac{0.7N(0)}{T{e}^{0.8T}}t{e}^{0.8t}$
Capacity cost scale parameter $\left(k\left[\frac{m.us}{s.d.u^2}\right]\right)$	$\frac{\mathrm{cp}}{1.8(\mathrm{cN}(0)r+p)}$
Parameter	Scenario 2 value
Evolution of number of sensors $(N(t))$	$N(0)+\frac{0.7N(0)}{T{e}^{0.8T}}t{e}^{0.8t}$
Capacity cost scale parameter $\left(k\left[\frac{m.us}{s.d.u^2}\right]\right)$	$\frac{\mathrm{cp}}{2.75(\mathrm{cN}(0)r+p)}$

Due to the variation in the number of sensors N, the optimal decision for the OP over the time may vary. Figure 8 shows how the system is able to adapt its decisions to variations not only in the distribution of the sinks but also in the system parameters. The difference between the Cases 1 and 2 and the Case 3 is small for small values of $t$ but it increases when $t$ is close to $T$ . Figure 9 shows the distribution of the sinks as a function of time, while Figure 10 shows the instantaneous profit for all the cases, while the aggregated profits are $48.46$ for the Case 1, $44.36$ for the Case 2, and $45.56$ for the Case 3. We observe how the optimal control strategy, represented in the Case 3, allows to increase the OP profits compared with the Case 2 despite the lower number of sinks subscribed. This is possible, thanks to the lower value of ${\mu }^{*}$ , and therefore, there is a reduction in the investment costs. We also observe how the non-optimal behavior of the sinks caused by the replicator dynamic decreases the OP profits with respect to the Case 1; however, a scenario with instantaneous sink decisions is not realistic.

Scenario 2

This scenario models an increasing number of sensors over the time due to a progressive deployment of new sensors, as shown in Table 3 and Figure 11. The figures were calculated for the values shown in Tables 2 and 3.

As in the previous scenario, the change in the number of sensors varies the OP optimal static solution ${\mu }^{*}\mathrm{static}$ , as shown in Figure 12. However, in this case, the optimal control decision does not follow the static optimal solution. This is possible given that the OP knows in advance the evolution of $N$ over the time and can adapt its strategy to optimize not only the instantaneous profits but also the profits in all the time interval. This strategy allows the OP to maintain all the sensors subscribed during more time, as shown in Figure 13, and allows the OP to increase its profits with respect to the static optimization. Figure 14 shows the instantaneous profit for all the cases, while the aggregated profits are $81.06$ for the Case 1, $80.14$ for the Case 2, and $82.77$ for the Case 3.

Conclusion

A capacity provision scenario for wireless sensors’ connectivity has been studied using mathematical modeling. The scenario was studied using both a static model and a more complex, but also more realistic, dynamic model. The analysis was conducted using concepts such as game theory, replicator dynamics, optimal control, and optimization.

The behavior of the sensors was modeled through a utility function based on a congestion model, while the subscription decision was modeled using both the static equilibrium and the replicator dynamic. The network operator profit was modeled using the revenues obtained from the sensors and a quadratic investment cost function. The optimal profit in a defined time interval was obtained solving an optimal control problem, using the network capacity as a control variable, and compared against the static optimization.

It has been shown that the optimization using optimal control, when the users are modeled using the replicator dynamic, allows the OP to obtain higher profits than the optimization using the equilibrium solution. In addition, the dynamic optimization allowed the operator to optimize its profits not only in a scenario with fixed parameters but also in a scenario where the system parameters, like the number of sensors, change over the time. Given the obtained results, we can conclude that the proposed scenario is feasible from an economic point of view for all the actors. In addition, we show that the optimal control theory is a profitable and a powerful tool for the maximization of the network operator profits in dynamic IoT scenarios.

Future work will involve the dynamic profit optimization of more complex scenarios with several competing operators using differential games.