Sage Journals: Discover world-class research

Abstract

The worldwide generation of waste electrical and electronic equipment is continuously growing, with electric vehicle batteries reaching their end-of-life having become a key concern for both the environment and human health in recent years. In this context, the proliferation of Internet of Things standards and data ecosystems is advancing the feasibility of data-driven condition monitoring and remanufacturing. This is particularly desirable for the end-of-life recovery of high-value equipment towards sustainable closed-loop production systems. Low-Power Wide-Area Networks, despite being relatively recent, are starting to be conceived as key-enabling technologies built upon the principles of long-range communication and negligible energy consumption. While LoRaWAN is considered the open standard with the highest level of acceptance from both industry and academia, it is its random access protocol (Aloha) that limits its capacity in large-scale deployments to some extent. Although time-slotted scheduling has proved to alleviate certain scalability limitations, the constrained nature of end nodes and their application-oriented requirements significantly increase the complexity of time-slotted network management tasks. To shed light on this matter, a multi-agent network management system for the on-demand allocation of resources in end-of-life monitoring applications for remanufacturing is introduced in this work. It leverages LoRa’s spreading factor orthogonality and network-wide knowledge to increase the number of nodes served in time-slotted monitoring setups. The proposed system is validated and evaluated for end-of-life monitoring where two representative end-node distributions were emulated, with the achieved network capacity improvements ranging from 75.27% to 249.46% with respect to LoRaWAN’s legacy operation. As a result, the suitability of different agent-based strategies has been evaluated and a number of lessons have been drawnaccording to different application and hardware constraints. While the presented findings can be used to further improve the explainability of the proposed models (in line with the concept of eXplainable AI), the overall framework represents a step forward in lightweight end-of-life condition monitoring for remanufacturing.

Keywords

Multi-agent system lorawan time-slotted end-of-life remanufacturing

1. Introduction

Waste Electric and Electronic Equipment (WEEE) has become a worldwide concern, not only because of its hazardous impact on the environment and human health but also because of representing one of the fastest-growing waste streams to date. In 2019, 53.6 million metric tonnes of WEEE were generated, that is, 7.3 kilograms per capita, of which less than 18% was officially documented and managed in an environmentally-sound manner [1]. The Directive 2012/19/EU of the European Parliament and of the Council [2] provides a regulatory framework for the collection, storage and transportation of related materials, which is intended to prevent the generation of WEEE and to contribute to the efficient use of natural resources and also minimize the human health and environmental risks associated with WEEE disposal.

From all kinds of WEEE, Electric Vehicle Batteries (EVBs) are raising special interest in both industry and academia due to the exponentially-growing demand for Electric Vehicles (EVs), which is expected to result in an enormous number of battery packs reaching their End-of-Life (EoL) in the coming years [3]. While these will need to be handled accordingly to reduce their impact on the environment, life-cycle engineering strategies such as remanufacturing are gaining momentum these days in order to bring them to like-new condition and give them a second life [4].

EoL decision-making can benefit to a great extent from the availability of traceable information about the EVB’s health condition, which can also facilitate predictive maintenance, lifetime prognostics, and fault detection [5, 6]. Following an Internet-of-Things (IoT) architectural approach, the monitoring of EoL battery packs through embedded sensors prior to disassembly and inspection stages can result in time savings of up to 34% [7]. This, in turn, enables the individual virtual representation of each EVB –its so-called Digital Twin [3]–, which leverages fine-grained real-time sensor data for building and training highly-accurate predictive models [8]. While making little economic sense for low-value products, this has proved to be especially viable for complex high-value equipment [9].

The integration of accurate sensors in EVB packs is key to protecting them from damage caused by adverse operation, transportation or storage conditions. Cell-based data generated are used to estimate the state of health (SoH) and state of charge (SoC) of the battery. These indicators are useful to determine the aging and charge level of battery packs, which in turn provide valuable information to adapt EoL operational strategies for product recovery [10].

The wireless transmission of the EVB’s SoH and SoC is raising interest among reverse logistics providers as a means to expedite testing and grading operations [11, 6]. However, despite these being typically computed onboard, the complexity of estimation methods is negatively influenced by high quantities of battery cells to be monitored. Hence, there is also growing interest in the transmission of raw sensor data for remote server-side fault detection and preventive maintenance in order to improve decision-making [3, 12].

However, there are various barriers to the recovery of EoL EVBs that need to be overcome to guarantee their viability in remanufacturing. First, the presence of rare metals such as cobalt or lithium can release toxic gasses increasing the risk of fire, which requires strict compliance with local regulations during transportation that significantly increases the costs associated with their recovery [13]. Second, with local storage regulations limiting the minimum distance between battery packs and their stacking conditions, most industrial warehouses for EVBs are typically spread over large distances, which increases the infrastructure cost of real-time sensor-based monitoring.

The deployment of Low-Power Wide-Area Networks (LPWANs [14]) can bring multiple benefits while bridging IoT sensor data to the Internet over long distances, including low deployment costs (a single base station is due to serve thousands of nodes), low maintenance costs (expected lifespans of a few years under periodic data transmissions), and IoT-based interoperability with Industry 4.0 manufacturing systems [15]. Among LPWAN, the LoRaWAN standard has already become one of the most extended LPWAN solutions, whose open specification is provided by the LoRa Alliance for long-range and low-power communications [16]. However, to date, there is limited evidence on the suitability of LoRaWAN for high-traffic applications such as EVBs monitoring across industrial consolidation points for EoL recovery.

In this sense, several limitations exist associated with an inefficient use of the available spectrum caused by an Aloha-type random-access protocol [17, 18]. While time-slotted channel access represents a suitable collision-avoidance strategy to guarantee robust communication networks with delivery rates of nearly 100% [19], the lack of flexible resource-allocation mechanisms severely limits its applicability for reliable large-scale applications. In this context, the integration of knowledge-based decision-making can bring potential benefits [20].

Considering the aforementioned, this work presents a multi-agent system (MAS) framework for time-slotted LoRaWAN communications to optimize the allocation of resources in compliance with specific reliability requirements considering as well end application constraints [21]. To validate the end-to-end system in practice, a use case for the recovery and storage of EoL EVBs is addressed, where their major transportation and storage conditions are studied. These are then translated into a LoRaWAN-specific data payload design and implementation which, in turn, is used as a basis to assess scalability improvements achieved for different scenarios experimentally. As a result, this work focuses on capacity-oriented improvements by individualizing synchronization periods at the device level to take advantage of already defined guard times in the network. This is one of the first works to address the definition of variable guard times and individual synchronization periods per end node, which prevents transmission overlap due to clock skew while still ensuring large LoRaWAN cell capacities. Although a recent approach [22] has proposed the use of variable guard times in the network, its impact on the overall network scalability has not been evaluated.

The use of MAS has attracted particular interest in recent years in the fields of civil engineering [23, 24], smart home [25], and connected mobility [26]. However, to the best of our knowledge, only two works in the literature have proposed the use of MAS on top of the LoRaWAN standard. One paper [27] presented an intra-slicing resource allocation technique, while another [28] focused on implementing a deep reinforcement learning technique for improved resource allocation. However, the evaluation of these works was based on a reduced set of LoRaWAN nodes: 4 real nodes in the former and 30 simulated nodes in the latter.

The major contributions of this work are: (i) the design and deployment of a multi-agent network management system for optimal resource allocation in application-oriented time-slotted LoRaWAN networks; and (ii) the experimental validation of scalability improvements achieved through the application-oriented allocation of resources in time-slotted LoRaWAN networks.

This work is an extension of a previous paper [29], where the first preliminary results of multi-agent-enabled allocation of resources in large-scale LoRaWAN networks were provided. To the best of our knowledge, this is the first work demonstrating scalability improvements on top of LoRaWAN communications in the reverse supply chain domain, an area where the number of studies involving the use of LoRaWAN technology has increased significantly in recent years [30, 31]. In the current extended work, nevertheless, the following new contributions are provided:

1.
EoL monitoring of EVBs. By exploring the storage and transportation requirements for EoL recovery of EVBs, which is then considered as a baseline for the design of a realistic LoRaWAN frame payload to be reused in domain-specific industrial warehouses.
2.
Decision-making logic. A new decision-making logic is proposed and validated on top of resource-allocation agents based on the lessons learned from our previous work.
3.
Resource-allocation optimization. An optimization mechanism is presented to balance uplink and downlink resources efficiently in different application-specific network status scenarios, with the achieved network capacity improvements ranging from 75.27% to 249.46%.

This work is structured as follows: Section 2 reviews the condition monitoring of EoL EVBs and presents the LoRaWAN-based MAS including network fundamentals and metrics; Section 3 addresses the network design and setup conditions based on identified application-related constraints requirements; Section 4 presents the evaluation results and discussion based on experimental validation of the MAS in terms of achieved LoRaWAN network capacity improvements; finally, Section 5 highlights the major conclusions, learned lessons for the community, and future works.
2. MAS design for large-scale EoL monitoring

This section addresses the multi-agent network management system logic design and architecture enabling on-demand resource allocation for EoL condition monitoring of EVBs. To do so, first, the conditions for EoL transportation and storage of EVBs – two of the most critical stages in their reverse supply chain [32] – are reviewed in Section 2.1. Second, LoRaWAN fundamentals and the selection of network metrics are described in Section 2.2 to introduce the proposed system logic. Finally, each of the agents being part of the MAS and the defined information flows in the end-to-end system architecture integrating LoRaWAN nodes, gateways, and the network server are briefly described in Section 2.3.

2.1 Condition monitoring of EoL EVBs

Figure 1.

Case study on LoRaWAN-enabled monitoring of EoL EVBs in large-scale consolidation points to support remanufacturing.

The battery management system (BMS) is the core component of the EVB, which is responsible for balancing the performance of individual modules and cells and providing relevant information about their health condition. It is in turn able to identify abnormal operating conditions through the on-board computation of different metrics such as their SoH, State-of-Charge (SoC), or State of Power (SoP) [33]. The SoH is defined as the ratio between an actual and initial battery indicator, such as its capacity, impedance or resistance [10].

The large number of operations involved in each stage of a reverse supply chain greatly increases uncertainty and, therefore, the resulting operational efficiency. The availability of real-time information about the condition of EoL EVBs is vital to support data-driven remanufacturing, which is expected to improve the economic and operational performance of reverse supply chains by enabling the re-use of modular components on demand [6].

While there have been numerous efforts to reach a consensus technique for accurate SoH estimation, two widely-extended strategies are [10]:

Experimental measurements. These require in-situ diagnosis, such as impedance measurements or capacity measurements, or more complex laboratory equipment like spectroscopy.

Model-based methods. These involve equation tuning, curve-fitting, or machine learning (ML) models trained to estimate the battery degradation over time.

The latter are receiving special interest in industry, since their deployment in BMSs for onboard fault detection and diagnosis is becoming ever more feasible.

According to a series of interviews with third-party logistics providers and recyclers [34], collection and storage are key phases in the reverse supply chain of EVBs. While the stockpiling of EVBs is a potentially unsafe practice [5]. The low volumes of battery packs reaching their EoL these days result in non-critical ones – also referred to as green or yellow – being transported and stockpiled in consolidation points, where they remain stored for long periods of time until a certain quantity is reached and their transportation for reuse or remanufacturing becomes economically viable. However, in the case of lithium-ion batteries, most accidents take place during their storage at warehouses, with short-circuit, self-heating and ageing being three of the most common causes [32]. The large size of the facilities used for the storage of these battery packs adds complexity in terms of covering wide areas under low infrastructure and energy costs [34].

Let us consider a consolidation point, as shown in Fig. 1, where yellow and green lithium-ion battery packs are stored once their EoL is reached. Through the deployment of a single-gateway LoRaWAN network, individual condition-monitoring of EVBs is proposed to enhance the remanufacturing information infrastructure. The goal is to improve decision granularity throughout the reverse supply chain based on real-time information availability prior to remanufacturing operations. The following metrics are monitored: SoH (provided by the on-board BMS), smoke, temperature and humidity, which have a significant impact on the aging of EVBs. Monitoring periods are considered constant for every LoRaWAN node attached to an EVB (equal to 5 minutes), while on-board BMSs are responsible for providing module-level fault reports which will be appended to periodic LoRaWAN frames when required.

Table 1

Specifications of four EVBs from the marketplace – Tesla S, BMW i3, Nissan Leaf, and Audi A6 – according to [35, 5]

Model	Components		Dimensions [mm]
	Mods.	Cells	Length	Width	Height
Tesla model S ${}^{\text{a}}$	16	444	2,830	1,772	127
BMW i3 ${}^{\text{a}}$	8	12	1,660	963	265
Nissan Leaf ${}^{\text{a}}$	48	4	1,570	1,188	259
Audi A6 PHEV ${}^{\text{b}}$	8	13	918	694	110

${}^{\text{a}}$ 2014 model; ${}^{\text{b}}$ 2017 model.

Although the legal conditions for EoL storage of EVBs are highly dependent on the local authorities and municipalities, some general restrictions apply: standard distances between rows of pallets typically range from 6 to 2 meters, a maximum of 2 stacked EVBs are allowed, and the space left between layers should be, at least, of 1 meter [34]. Considering four EVBs from the marketplace, whose sizing and modular specifications are provided in Table 1, the reference dimensions of a baseline industrial warehouse are provided in Fig. 1.

2.2 LoRaWAN network metrics

LoRaWAN is an open specification for Long-Range communications [16], which is built on top of LoRa physical layer [36]. LoRaWAN defines the Medium Access Control layer (MAC) for LoRa end devices that communicate, following a star-of-stars topology, with one or more co-located gateways. These gateways forward uplink traffic to a network server via a backhaul such as Ethernet, 4G or 5G using TCP/IP, which is there de-duplicated and processed by an application server so as to decrypt data payloads. Most available multi-channel LoRaWAN gateway modems are half-duplex and, hence, not able to listen for upcoming traffic during ongoing downlink transmissions.

LoRaWAN technology operates in the unlicensed frequency bands of 433, 868 and 915 MHz in Europe, being the European Telecommunications Standards Institute (ETSI) responsible for regulating air-time usages in the different frequency bands [37]. While the ones destined for LoRaWAN end devices are, in most cases, restricted to 1-% duty cycles (DCs) –setting a maximum channel occupation of 36 seconds per hour and device–, the band used by gateways has a limit of 10% so as to support fair use of downlink capabilities. The time window for which LoRaWAN devices cannot access the channel is known as blackout period [38], which should be carefully considered in order to schedule downlink traffic in the network in compliance with the existing regulations.

LoRa is based on Chirp Spread Spectrum (CSS), where the achieved data rate depends on the selected spreading factor (SF), bandwidth, and coding rate. The SF ranges from SF7 to SF12 increasing the sensitivity of LoRa nodes at the expense of longer transmission air times; that is, lower data rates. As a result, LoRa transmissions using higher SFs are able to reach longer distances –in the order of kilometers– but also increase significantly the chances of wireless collisions, since each transmission occupies the channel for a longer amount of time. Transmissions at different SFs are quasi-orthogonal, which enables collision-free scheduling of simultaneous traffic over different SFs so as to increase the network capacity.

Table 2
LoRaWAN regional parameters in Europe for a 125-kHz bandwidth

SF	Bit rate [bits/p]	Payload [bits]	Time on air ${}^{*}$ [ms]
SF12	250	51	1,318.91
SF11	440	51	741.38
SF10	980	51	370.69
SF9	1760	115	185.34
SF8	3125	222	102.91
SF7	5470	222	56.58

${}^{*}$ Calculated for a 20-byte payload using the default settings.

The duration of a LoRa frame is computed using Eqs (1) and (2) – by Semtech [36] for SX1276 and SX1278 LoRa modems – as a function of the physical-layer transceiver (PHY) configuration and payload length. The number of chirps per symbol, in turn, is determined by $2^{\textit{SF}}$ . Table 2 shows, according to the European regional parameter specification [16], the maximum payload lengths and resulting time on air for different SF setups.

$\displaystyle T_{\textit{OA}}=(n_{\textit{preamble}}+4.25)\times T_{\textit{% sym}}+\,P_{\textit{symb}_{\textit{Nb}}}\times T_{\textit{symb}}$ (1)

where $n_{\textit{preamble}}$ is the number of preamble symbols, $T_{\textit{symb}}$ is the symbol duration (defined as $2^{\textit{SF}}$ divided by the bandwidth), and $P_{\textit{symb}_{\textit{Nb}}}$ is the number of symbols that make up the packet’s payload and header which, in turn, is computed according to Eq. (2).

$\displaystyle P_{\textit{symb}_{\textit{Nb}}}=8\;+$ $\displaystyle\quad\max\biggl{[}\left\lceil\frac{8\textit{PL}-4\textit{SF}+28+1% 6-20H}{4\times(\textit{SF}-2\textit{DE})}\right\rceil$ (2) $\displaystyle\quad\times\,(\textit{CR}+4),0\biggr{]}$

where PL is the number of payload bytes, SF the spreading factor, $H$ indicates whether header is present (0) or not (1), DE indicates whether low data-rate optimization is enabled (1) or not (0), and CR is the coding rate from 1 to 4.

The proposed system logic is built on top of time-slotted LoRaWAN communications Class A communications [39], where gateways follow a Time-Division Multiple-Access (TDMA) schema [40] while being responsible for assigning fixed-length time slots to joining end devices on demand. The allocation is based on individual application-oriented requirements (such as payload length or periodicity) and real-world constraints (such as DC limitations or clock accuracy constraints). For this, upon joining, end devices get synchronized with a global time reference at the gateway level, whose feasibility was experimentally validated in the literature achieving ten-millisecond synchronization accuracies [19].

In this work, an orthogonal allocation of LoRaWAN end-node transmission slots over different SFs is proposed, which results in six simultaneous SF-specific schedules sharing a single downlink channel.

The slot length (see Eq. (3)) is defined as the sum of a guard time (required to compensate for device resynchronization over time due to positive and negative clock drifts), the time on air (computed according to Eqs (1) and (2)), and the synchronization offset (measured as the actual time offset with the global time reference upon synchronization).

$\displaystyle S_{\textit{len}}=\left\lceil\frac{2\times T_{\textit{sk}}\times T% _{\textit{synch}}}{10^{6}}\right\rceil+T_{\textit{OA}}$ (3)

where $T_{\textit{sk}}$ the clock skew, $T_{\textit{synch}}$ is the time interval between two synchronization requests ensuring no overlap in neighbor transmissions, that is, the synchronization period, and $T_{\textit{OA}}$ is the time on air.

We would like to textitasize that while most of the work in the literature is based on the existence of homogeneous clock skew (e.g. [41, 42]), this can significantly affect the reliability performance of communication in practice due to overlapping transmission slots (see an experimental study [19]).

The role of the MAS is to balance the available resources across SF schedules so as to ensure reliable time-slotted communications while delaying network congestion as much as possible. This occurs when the MAS is no longer able to accept new joining devices because one of the existing channels has exceeded its maximum capacity, either due to a transmission in progress or a slot reservation caused by an orthogonal transmission in progress. For this, the two network metrics monitored for decision-making are: (i) uplink occupancy and (ii) downlink usage.

Let us define the uplink occupancy, according to Eq. (4), as the time percentage to be reserved in all the orthogonal SF schedules (one per SF at every gateway) to guarantee both uplink and downlink communication of every node in the network.

$\displaystyle T_{0}=\left(\frac{S_{\textit{len}}}{T_{\textit{UL}}}+\sum_{i=% \textit{SF}7}^{\textit{SF}12}\frac{\left\lceil\frac{S_{\textit{len}}^{*}}{S_{% \textit{len}_{i}}}\right\rceil S_{\textit{len}_{i}}}{T_{\textit{synch}}}\right% )\times 100$ (4)

where $T_{0_{\textit{UL}}}$ is the time interval during which the device reserves a specific channel for uplink communication, $T_{0_{\textit{DL}}}$ for downlink communication, $T_{\textit{synch}}$ is the synchronization period at a specific SF, $T_{\textit{UL}}$ is the uplink period required by the end device, $S_{\textit{len}}^{*}$ is the slot length at a specific SF-based schedule (equal to $S_{\textit{len}}$ or $\frac{S_{\textit{len}}}{2}$ for best-case and worst-case scenarios, respectively), and $S_{\textit{len}_{i}}$ is the slot length at SF equal to $i$ .

Similarly, the downlink usage is defined according to Eq. (5) as the time during which the downlink channel will not be accessible due to either an ongoing uplink or downlink transmission or black-out period compliance.

$\displaystyle U_{\textit{DL}}=\sum_{n=0}^{N}\frac{T_{\textit{OA}_{n}}+\textit{% BP}_{n}}{T_{\textit{synch}_{n}}}\times 100$ (5)

where $T_{\textit{OA}_{n}}$ is the transmission time on air of a node $n$ , $\textit{BP}_{n}$ is the blackout period of the node using such SF, and $T_{\textit{synch}_{n}}$ is its synchronization period.

2.3 MAS logic for resource allocation

The MAS is designed to comply with the following constraints:

•
Application-based constraints. These include payload size, transmission period, and end-node distance to a gateway (specified in Section 3).
•
Hardware-based constraints. These include the clock skew of end nodes and the half-duplex communication capability of LoRaWAN gateways.
•
Physical constraints. These include the gateway’s blackout period in compliance with ETSI regulations, that is, $9\times T_{\textit{OA}}$ , and in-band collisions expected to take place for two or more overlapping end-node transmissions over the same SF and bandwidth.
•
Logic design constraints. Considering the proposed TDMA-based mechanism, six simultaneous schedules (one per SF) will run in parallel at the gateway level which, given their half-duplex constraints, need to be orchestrated to guarantee collision-free downlink channel sharing.

With reference to hardware constraints, as recently found in [19], not even end devices having the same reference model and manufacturer can be expected to have the same clock hardware specifications. This phenomenon is referred to as clock diversity in this work, which is used as starting point for the design of schedule-specific time slots by considering the coexistence of various clock skews in the network. To do so, a reference time-slot length is initially established for each schedule and, upon device join, its assigned guard time and time on air are dynamically tuned by the MAS according to individual hardware and application constraints. This represents a significant contribution in this work, since the previous TDMA approaches assumed fixed guard-time lengths for the sake of simplicity.

Three stages are proposed for agent-based resource allocation in the network, the transition between which depends on the number of active end nodes in the network and the amount of traffic being ingested at the gateway level. These are specified as follows:

1.
Warm-up stage. The MAS collects metadata from the network while end nodes implement legacy Aloha-based channel access. This stage is activated under low network traffic conditions.
2.
Launching stage. Based on the available information, once a specific network traffic threshold is reached, the agents are responsible for deploying the required instances for each SF-based schedule at the gateway level. Each of these instances is referred to as an SF-based Network Synchronization and Scheduling Entity (NSSE), which is responsible for guaranteeing end-device synchronization and time-slot transition for a specific SF schedule.
3.
Joining stage. Based on application-specific and individual hardware constraints of the joining end node, the most convenient schedule and time-slot structure (including both guard time and transmission time) is assigned, which is notified to the concerned NSSE and end node via downlink so that it can request synchronization using a suitable SF. After joining, end devices follow a periodical data transmission schema until re-synchronization is required (negotiated directly with their already-assigned NSSE) or a more suitable configuration is received from the multi-agent network management system.

Based on the defined network metrics and stages, Algorithm 2.3 details the system logic implemented by the MAS in order to manage and allocate network resources on demand. According to Algorithm 2.3, during the initial warm-up stage, the MAS is responsible for collecting metadata from joining end devices as well as their clock skew ( $T_{\textit{sk}_{n}}$ ) and required time on air ( $T_{\textit{OA}_{n,\textit{SF}}}$ ), which are kept following an Aloha-like schema. The activation of the launching stage triggers the computation of baseline slot lengths for each of the SFs based on the previously-collected data, which are then used to launch parallel NSSE instances at the gateway level (see Schedule Launching routine in Algorithm 2.3). From this point on, each request received from new end devices is processed based on End-Node Joining routine in order to assign it to the most appropriate SF schedule based on the current network status, baseline slot lengths ( $S_{\textit{len}_{0}}$ ), and existing constraints. To do so, each joining node will be replied by MAS with the selected SF in order to schedule a synchronization request to be processed by the concerned NSSE.

MAS logic for end-node joining[1] Schedule Launching: $\textit{SF}\in[7,12]$ compute $T_{\textit{sk}_{\textit{SF}}}$ as a function of init.cfg compute $T_{G_{0,\textit{SF}}}$ as a function of $T_{\textit{synch}_{0}},T_{\textit{sk}_{\textit{SF}}}$ compute $T_{{\textit{OA}_{0,\textit{SF}}}}$ as a function of SF, PL compute $S_{\textit{len}_{0,\textit{SF}}}$ as a function of $T_{G_{0,\textit{SF}}},∼{}T_{\textit{OA}_{0,\textit{SF}}}$ return $S_{\textit{len}_{0,\textit{SF}}}\forall\ \textit{SF}\in[7,12]$ End-Node Joining: $\textit{SF}\in[7,12]$ $T_{\textit{OA}_{0,\textit{SF}}}>T_{\textit{OA}_{0,\textit{SF}}}$ $\textit{DC}_{n,\textit{SF}}\geqslant\textit{DC}_{\textit{max}}$ increment required_slots compute $T_{\textit{synch}_{n,\textit{SF}}}$ as a function of $S_{\textit{len}_{0,\textit{SF}}}$ , $T_{\textit{sk}_{n}}$ compute $T_{G_{n,\textit{SF}}}$ as a function of $T_{\textit{synch}_{n,\textit{SF}}}$ , $T_{\textit{sk}_{n}}$ compute $T_{0_{n,\textit{SF}}}$ , $U_{\textit{DL}_{n,\textit{SF}}}$ $\textit{SF}_{N}\leftarrow\textit{SF}_{opt}$ return $\textit{SF}_{N}$ Main Routine: wait $\textit{joining end-node}∼{}(n)$ join_request $n\leftarrow\textit{end\_node }$ $\textit{join\_request}_{n}\leftarrow T_{\textit{sk}_{n}},T_{\textit{OA}_{n,% \textit{SF}}}∼{}\forall\ \textit{SF}\in[7,12]$ $N_{\textit{devices}}<\textit{threshold}$ activate warm-up add $n$ to $N$

$N_{\textit{devices}}$ is threshold activate launchingSchedule Launching $N$ $SF\in[7,12]$ launch $\textit{NSSE}_{\textit{SF}}$

$N_{\textit{devices}}>\textit{threshold}$ activate joiningEnd-node Joining $\textit{join\_request}_{n}$ reply $n$ , $\textit{NSSE}_{\textit{SF}_{\textit{opt}}}$

The proposed multi-agent architecture is shown in Fig. 2, which consists of seven software agents that are deployed, at gateway level, on top of the time-slotted logic described and interact with two ends: first, the LoRaWAN network (including its network server, gateway, and co-located end devices) and, second, the NSSE (responsible for launching the instances that handle synchronization and scheduling tasks).

Figure 2.
Multi-agent network management system.

The proposed agents collaborate to balance the use of uplink and downlink resources in the network – e.g., uplink channel occupancies and downlink usages – by computing the required length of slots of each of the NSSE instances and assigning the most suitable SF and synchronization period to the joining nodes. These implement socket communication based on TCP (Transmission Control Protocol), which enables a distributed deployment.

The role of each agent being part of the resource-allocation network is described in the following lines:

•
Frame FWD (FWD). Standing for frame forwarding, this agent is responsible for filtering join requests from periodic uplink traffic or synchronization requests sent to a specific NSSE. These are then forwarded from the LoRaWAN-specific network server to the MAS for device registration and metadata collection.
•
Device registration. This agent is responsible for gathering request information from joining end nodes (i.e., hardware and application constraints) as well as their metadata (i.e., received signal strength indicator (RSSI), signal-to-noise ratio (SNR), requesting SF, etc.), which is ingested into a database accessible by all agents.
•
Data rate discarding. This agent leverages the current system knowledge which, combined with the information included in the end-node’s join request, is used to discard unfeasible SFs due to either physical constraints (e.g., long distance to the gateway) or agent-based scalability constraints (e.g., congestion of a specific schedule). This agent of proactive nature is expected to implement reliability-oriented criteria.
•
Payload formatting. This agent is responsible for splitting the joining end-node’s data payload into standard-format magnitudes (e.g., temperature, humidity, clock skew, etc.)
•
Time-on-air calculation. This agent computes the required uplink transmission time on air according to Eqs (1) and (2) based on the magnitudes that will be periodically reported by the end node.
•
Resource allocation. This agent has a key role in the network management system, which is responsible for proactively implementing the scalability-oriented criteria based on the current network knowledge, joining end-node’s constraints, and available uplink and downlink time-slotted resources.
•
Instance deployment. This agent is responsible for launching new NSSE instances on demand based on the consensus reached by the previous agents or notifying existing ones about coming end nodes that will be requesting time synchronization. These, in turn, will receive a downlink frame including the required metrics to do so, that is, the assigned SF and time-offset to initiate synchronization with its assigned NSSE.

3. Network emulation design and setup

A practical approach to time-slotted LoRaWAN-based monitoring of EoL EVBs is addressed in this section. To do so, the required number of nodes and their relative distance to the gateway in the baseline condition-monitoring scenario are established. This is used to validate the scalability achieved when integrating the multi-agent proposal in the described scenario, for which LoRaWAN payload frames and encoding/decoding tasks carried out by agents are justified in this section.

To validate capacity improvements for large-scale network deployments supporting the remanufacturing of EVBs, a uniform distribution ( $\rho$ ) of 100 nodes per square kilometer within a rounded gateway cell was considered for all the emulations. The packet generation intensity ( $\lambda$ ) was set to 0.2 packets per minute. This was done through minor modifications on top of ChirpStack Device Emulator,1 which enabled the emulation of a set of end nodes with heterogeneous joining (e.g. clock skew) and application (e.g. link quality or payload content) constraints.

A previous real-world reliability validation of end-to-end synchronization and scheduling concerning end nodes and a single NSSE [19] encouraged us to use such a device emulator in this work in order to focus on application-oriented network capacity improvements through the addition of multi-agent components. The remaining system components (LoRaWAN network and application servers, MAS, and NSSE instances) were implemented and launched experimentally to validate network capacity improvements.

3.1 Payload frame design

Table 3
Magnitude lookup table at payload formatting agent

Data type	Header	Bits	Range	Resolution (MSB) ${}^{\text{a}}$
Thing ID	0x00	3	[0, 7]	1 unsigned
Warehouse	0x01	3	[0, 7]	1 unsigned
Uplink period	0x02	8	[0, 255]	1 minute unsigned
Mobility	0x03	1	[0, 1]	Static/dynamic
Battery SoH	0x04	10	[0, 1,000]	0.1% unsigned
Temperature	0x05	11	[0, 1,500] ${}^{\text{b}}$	0.1 ${}^{\circ}$ C signed
Humidity	0x06	10	[0, 1,000]	0.1% unsigned
Pressure	0x07	10	[0, 800] ${}^{\text{c}}$	0.1 kPa unsigned
Altitude	0x08	16	[0, 38,004] ${}^{\text{d}}$	0.25 meters signed
Luminosity	0x09	14	[0, 16,383]	1 Lux unsigned
Fault	–	9	[0, 511]	1 unsigned

${}^{\text{a}}$ Most significant bit. ${}^{\text{b}}T=-50+0.1*v\;\forall v\in$ [0, 1,500]. ${}^{\text{c}}\!P=300+v\;\forall v\in$ [0, 800]. ${}^{\text{d}}A=-500+0.25*v\;\forall v\in$ [0, 38,004].

Based on a set of related magnitudes to be transmitted periodically, bit-wise encoding was used to generate compact payload frames so as to reduce frame payload sizes and, hence, overall transmission time on airs in the network. Table 3 details the set of magnitudes defined, their header identifiers, sizes, and their achieved bit resolution. This information is retrieved by the payload formatting agent to compute data payload lengths after a device’s registration. For the sake of simplicity, the clock skew of joining end devices is considered to be known according to their manufacturer datasheet. Hence, a multi-selector for different thing IDs was defined, i.e. groups of devices that have the same clock specifications.

Header definitions are based on Cayenne’s Low Power Payload (LPP2) resource identifiers, which conform to the IPSO Alliance Smart Objects guidelines to enable interoperability, but considerably reduce payload lengths by starting to number object identifiers from 0. In order to further reduce payload lengths, bit packing is applied to LPP’s definitions to compress data. Furthermore, new object definitions such as clock skew or thing ID were included by assigning empty identifiers.

Figure 3.

LoRaWAN payload frame design.

Two different models of data frame were defined in order to keep transmission times to a minimum, the field structure of which is detailed in Fig. 3.

On the one hand, the joining payload frame (see Fig. 3) consists of three fields:

•

Hardware constraints. It includes hardware constraints such as a device’s clock skew.

•

Application constraints. It includes application-based constraints, such as a node’s mobility condition (typically stationary) or uplink periodicity requirements based on the EVB status.

•

Data magnitudes. It serves as a declaration of the sensed magnitudes to be periodically reported by the node, such as the EVB’s SoH or its surrounding temperature.

•

Fault. It is reserved for the generation of module-level alerts.

Each magnitude, in turn, consists of three sub-fields: header identifier (16-bit header indicating the magnitude ID), availability bit (a single bit indicating whether the concerned magnitude is included in the next field) and, in case included, the magnitude itself. Generic-payload magnitudes are omitted during device registration by setting availability bits to 0, since headers suffice to compute payload sizes of future periodic data reports.

The periodical payload frame, on the other hand, omits the fields provided upon joining in order to reduce LoRaWAN transmission times and, instead, includes the content of the declared magnitudes following the same order. Additional space for module-level fault information is included in the packet, which requires the system to deal with variable payload sizes.

3.2 End-node deployment distributions

The geographic spread of EVBs determines the set of possible SFs to initiate communication with their associated gateway. For the sake of simplicity, a uniform distribution of 100 nodes per square kilometer within a round-shaped area with respect to the gateway is considered [43]). Specifically, two differently-spreading scenarios were considered to benchmark capacity improvements achieved by the MAS in rural- and urban-like deployments, namely Scenario 1 ( $\mathscr{S}1$ ) and Scenario 2 ( $\mathscr{S}2$ ). These are shown in Table 4 as cell radius thresholds (SF distance boundaries), and their resulting SF distribution percentages based on the proposed uniform distribution of end nodes.

Table 4
Percentage of joining nodes and cell radius per SF for end-node distributions in scenarios $\mathscr{S}1$ and $\mathscr{S}2$

Scenario		SF7	SF8	SF9	SF10	SF11	SF12
		$\ell_{5}$	$\ell_{4}$	$\ell_{3}$	$\ell_{2}$	$\ell_{1}$	$\ell_{0}$
$\mathscr{S}1$	[%]	41.12	23.53	11.76	5.88	5.88	11.76
	[km]	1.00	1.25	1.36	1.41	1.46	1.55
$\mathscr{S}2$	[%]	6.25	6.25	12.50	31.25	25.00	18.75
	[km]	0.24	0.35	0.50	0.75	0.90	1.00

The methodology used to obtain the distance thresholds is detailed below. Cell radius distance thresholds (SF boundaries) were determined based on the experimental LoRa-based RSSI and SNR measurements from the coverage study conducted in [44], which considered both urban and rural deployment conditions. To do this, the following criterion was used to establish SF boundaries: 5 dB $\leqslant$ $|\textit{SNR}_{\textit{min}}-\textit{SNR}|$ , where $\textit{SNR}_{\textit{min}}$ is the minimum signal-to-noise ratio (SNR) supported per SF (i.e., $-$ 7.5 dB for SF7, $-$ 10 dB for SF8, etc.). Please refer to our previous paper [44] for more details. Where these were not available for a specific SF, they were interpolated based on upper and lower SF limits, and where upper or lower limits were not available, they were determined based on linear regression using the analytical model results provided in a recent paper [45]. A brief description of both scenarios is provided below:

•

Scenario 1 ( $\mathscr{S}1$ ). This scenario emulates rural-like deployment conditions where low SFs predominate in the network.

•

Scenario 2 ( $\mathscr{S}2$ ). This scenario emulates urban deployment conditions where high SFs predominate in the network.

It should be noted that these parameters are not intended to represent unique LoRaWAN network deployment conditions, but rather two case-specific examples inspired by recent literature to validate MAS scalability achievements under different physical constraints.

Two key differences between the two scenarios can be noticed in Table 4. First, there exists a higher proportion of end nodes that need to join using higher SFs in $\mathscr{S}2$ to guarantee successful packet decoding at the gateway level while lower SFs are predominant in $\mathscr{S}1$ . Second, the resulting distance boundaries are significantly shorter in $\mathscr{S}2$ , e.g. $\ell_{0}$ is 1 km instead of 1.55 km. As a result, these two scenarios cover a range of physical constraints to be handled by the MAS, which serves to identify potential capacity-improvement strategies for varying network setup conditions. These are detailed in Section 4.

Two decision-making stages were defined at the MAS level for the proposed distribution scenarios, which are based on the application of different strategies at Launching and Joining routines from Algorithm 2.3. They are specified in Table 5.

Table 5

Decision-making strategies assessed at the MAS level

Strategy	MAS stage	Description
$L_{1}$	Launching	Balanced distribution of guard time across SFs.
$L_{2}$	Launching	Exponentially-falling distribution of guard times.
$L_{3}$	Launching	Exponentially-rising distribution of guard times.
$J_{1}$	Joining	Assign each node the lowest-possible SF.
$J_{2}$	Joining	Assign the SF that minimizes $\textit{max}[T_{0},U_{\textit{DL}}]$ .

Figure 4.

Launching strategies depending on the distribution of guard time across SFs.

$L_{1}$ to $L_{3}$ represent guard-time computation strategies, of which $L_{2}$ and $L_{3}$ are graphically represented in Fig. 4. Based on the preliminary evaluation addressed in [29], balanced and exponentially rising guard-time distributions ( $T_{G_{0,\textit{SF}}}$ in Algorithm 2.3) were identified as relevant design factors to improve network scalability deployments where multiple high SFs co-exist. In addition, $J_{1}$ and $J_{2}$ are proposed and evaluated in this work as SF allocation strategies to achieve further capacity improvements. In all cases, these are based on a set of heuristics that do not require expensive computational resources.

4. Results and discussion

The results provided in this section are divided according to the two end-node distributions proposed ( $\mathscr{S}1$ and $\mathscr{S}2$ ), with a view to providing a set of representative decision-making strategies –and their impact on the overall network capacity– for the given input constraints. Specifically, based on the preliminary results obtained in [29], the impact of the MAS pursuing $L_{1}$ to $L_{3}$ are assessed in Section 4.1, while only that of $L_{1}$ and $L_{3}$ are assessed in Section 4.2 given the limited benefits of $L_{2}$ for high-SF scenarios identified in [29].

4.1 Network capacity improvements in $\mathscr{S}1$

Considering a uniform distribution of 100 nodes per squared kilometer and cell radius from Table 4, the maximum size of the network was set to 2500 devices.

Figure 5 shows downlink channel usages and uplink occupancies per SF over time for the MAS applying Launching strategies $L_{1}$ (Fig. 5a and c, respectively) and $L_{2}$ (Fig. 5b and d) while allocating the minimum-possible SF upon device join (strategy $J_{1}$ ). The impact of unbalancing guard time distributions is especially noticeable for SF7 and SF12, which achieve a more balanced occupation across different SFs in uplink at the expense of increasing downlink usages due to more frequent resynchronizations at higher SFs. However, with the downlink usage having led to network congestion in both cases, a balanced distribution of guard times (strategy $L_{1}$ ) was able to increase the cell capacity by achieving a more efficient downlink usage.

Figure 5.

Overall downlink usages ( $U_{\textit{DL}}$ ) and uplink occupancies ( $T_{0}$ ) in the network for different launching strategies (Scenario $\mathscr{S}1$ ).

Interestingly, the joining strategy $J_{2}$ did not result in noticeable network-size improvements with respect to these shown in Fig. 5. In the case of strategy $L_{1}$ , balancing guard time distributions translates into the allocation of the SF that achieves the highest efficiency in the downlink, that is, the lowest-possible SF. In the case of $L_{2}$ , a different device allocation across SFs results in rapid downlink congestion (similarly as in Fig. 5b) so that the MAS is forced to stop allocating new end devices. Nevertheless, it increases the efficiency of uplink channel occupancy similarly as when strategy $J_{1}$ is pursued (see in Fig. 5d). In short, this first experiment shows that decision-making during the launching stage has a higher dependency on the network scalability than decision-making upon device joining for unbalanced downlink-to-uplink channel usages.

In view of the results, the decision-making criteria implemented by the MAS was extended with the computation of an optimal synchronization period in both launching and joining stages ( $T_{\textit{synch},\textit{opt}}$ ), so that uplink and downlink resources are balanced and capacity improvements when applying strategies $L_{2}$ and $J_{2}$ can be quantified. To do so, agents were set up to use the information gathered from the network during the warm-up stage (see Algorithm 2.3) to compute uplink and downlink channel utilization and to select the synchronization period that minimizes the overall use of resources in the network. From that point on, as in the previous case, agents adapt the synchronization period of each end device to the already-deployed time slot lengths based on their individual clock skew and link quality requirements.

Figure 6.

Optimal synchronization period pursuing applying strategies $L_{1}$ and $L_{2}$ (Scenario $\mathscr{S}1$ ).

Figure 6 shows the uplink and downlink channel utilization as a function of the synchronization period for the same network configurations deployed in Fig. 5. The higher the synchronization period, the longer are the guard times at the expense of increasing downlink channel utilization. When an exponentially falling distribution of guard times is applied (strategy $L_{2}$ shown in Fig. 6b), the impact of high synchronization periods is even more significant for both uplink and downlink. First, these result in longer slot lengths at lower SFs, which have a negative impact on the uplink occupancy due to slot reservation to enable resynchronization of other nodes allocated to orthogonal SFs. Second, shorter guard times at higher SFs negatively influence both the downlink usage (more frequent resynchronizations using longer slot lengths) and the uplink occupancy (increasing to a greater extent the impact of slot reservation).

Table 6 shows, for strategies $L_{2}$ and $L_{3}$ , the resulting guard times ( $T_{G}$ ), times on air ( $T_{\textit{OA}}$ ), and slot lengths ( $S_{\textit{len}}$ ) after optimal synchronization period computation for one device being assigned to each of the SFs. The shortest resulting slot lengths are highlighted in each case. To enable a fair comparison, only nodes with the same clock skew (75 ppm) are included, since the network assigned on demand a suitable synchronization period –and hence, guard time– to each node based on its individual hardware and application constraints (see Section 3.1).

Table 6

Metrics computed by agents for joining end devices when applying strategies $L_{2}$ and $L_{3}$ after optimal synchronization period computation (Scenario $\mathscr{S}1$ )

SF	Strategy $L_{2}$			Strategy $L_{1}$
	$T_{G}$	$T_{\textit{OA}}$	$S_{\textit{len}}$	$T_{G}$	$T_{\textit{OA}}$	$S_{\textit{len}}$
SF7	661.66	46.34	708.00	116.66	46.34	163.00
SF8	393.57	82.43	476.00	134.57	82.43	217.00
SF9	246.17	164.86	411.00	172.14	164.86	337.00
SF10	172.33	288.67	461.00	246.33	288.67	535.00
SF11	134.57	659.46	749.00	393.54	659.46	1053.00
SF12	116.93	1155.07	1272.00	661.93	1155.07	1817.00

Finally, Fig. 7 shows the resulting number of nodes (cell capacity) achieved when applying the MAS in the network for each of the defined launching and joining strategies. For the network distribution being deployed (scenario $\mathscr{S}1$ ), where a higher proportion of low SFs coexist (see Table 4), no significant impact on the network capacity was identified for different joining strategies being applied for the same synchronization period, since fast downlink congestion limited the network scalability. Specifically, $L_{1}$ resulted in an approximate maximum of 1415 end nodes regardless of the joining strategy applied, and similarly, $L_{2}$ resulted in 1100 nodes. By computing the optimal synchronization period in each case, the MAS achieved significant capacity improvements with respect to the baseline scenario. While agents implementing strategies $L_{1}$ during launching achieved 54.56% improvements by being able to serve 2184 coexisting devices (compared to 1413 in the case of static synchronization periods), the impact of launching strategy $L_{2}$ increased significantly after computing optimal synchronization periods (up to a 75.27% improvement). This means that, once the slopes of uplink and downlink channel utilizations are balanced through the computation of optimal synchronization periods, the benefits of implementing decision-making strategies like $J_{2}$ (alternative to LoRaWAN legacy) can be leveraged.

Figure 7.

Maximum number of co-existing devices achieved for different strategies being applied (Scenario $\mathscr{S}1$ ).

Figure 8.

Channel utilization reduction applying an optimal synchronization period ( $T_{\textit{synch},\textit{opt}}$ ) while implementing $J_{2}$ strategy for 315 devices (Scenario $\mathscr{S}2$ ).

On the whole, nevertheless, balancing guard times across SFs ( $L_{1}$ ) resulted in the most efficient usage of resources for the theoretical SF distribution being tested (see Table 4).

4.2 Network capacity improvements in

\mathscr{S}2

Given a uniform distribution of 100 end nodes per square kilometer and the maximum cell radius from Table 4, the maximum size of the network served in the $\mathscr{S}2$ end node distribution scenario was set up to 315 devices.

Figure 8 shows the impact of implementing $J_{2}$ on uplink and downlink channel utilization (similar results were obtained for $J_{1}$ ) for a maximum network size of 315 end nodes before and after applying an optimal synchronization period. As can be observed, increasing the synchronization period did not have a significant impact on uplink occupancies. This can be explained according to Fig. 6, where a low variability of uplink occupancies with respect to downlink usages was observed in the network for a range of synchronization periods lower than the optimal. Conversely, downlink channel utilization is reduced to a great extent, which prevents early network congestion.

By emulating a network with higher node densities but maintaining the same end-node distribution (Scenario $\mathscr{S}2$ ), Fig. 9 shows the maximum number of nodes that can be served from the MAS while guaranteeing collision avoidance. At a first sight, as expected, the coexistence of more end nodes in higher SFs halves the cell capacity with respect to the achieved in Fig. 7 for balanced guard time distributions ( $L_{1}$ ) and a baseline end-node allocation strategy ( $J_{1}$ ).

Figure 9.

Maximum number of co-existing devices achieved for different strategies being applied (Scenario $\mathscr{S}2$ ).

Some interesting conclusions are drawn from Fig. 9. First, exponentially-rising guard time distributions ( $L_{3}$ ) outperformed balanced ones in high-SF deployments (Scenario $\mathscr{S}2$ ) with improvements up to 31.15%. In contrast, in low-SF deployments (Scenario $\mathscr{S}1$ ), balanced guard times ( $L_{1}$ ) achieved the most efficient channel utilization. Second, adopting optimal synchronization periods in the resource-allocation engine achieves an even more significant cell capacity improvement for high-SF deployments (Scenario $\mathscr{S}2$ ) with respect to low-SF ones (Scenario $\mathscr{S}1$ ). Specifically, 100.83% compared to 54.56%. Nevertheless, in this case, it is a different launching strategy ( $L_{2}$ instead of $L_{3}$ ) which extends the network capacity. Given the high SFs that coexist in the network, the maximum capacity is limited by the downlink channel utilization when a default synchronization period is applied, which can leverage exponentially-rising guard time distributions ( $L_{3}$ ) due to less frequent resynchronizations being required. In contrast, when uplink and downlink utilization are balanced at the MAS, this launching strategy achieved the lowest channel efficiency across all SFs as a result of slot reservation requiring longer time intervals (longer slot lengths at higher SFs). These conclusions are key for MAS to understand the best decision-making strategy that needs to be implemented as a function of the known application and hardware constraints.

Figure 10.

Impact of $J_{2}$ allocation strategy on SF distributions and uplink channel utilization while implementing $L_{2}$ (Scenario $\mathscr{S}2$ ).

Figure 11.

Number of nodes per SF over time for joining strategy $J_{2}$ and different launching strategies (Scenario $\mathscr{S}2$ ).

Furthermore, the resulting SF distributions in the network while implementing $L_{2}$ before and after an optimal synchronization period is applied can be analyzed in detail. These can be observed in Fig. 10a and b while $J_{2}$ joining strategy was being applied, which obtained the best results in Fig. 9b. With the initial SF joining distributions being summarized in Table 4, Fig. 10a shows how nodes joining with lower SFs are typically re-allocated to higher SFs, which results in early downlink congestion. Dark gray areas represent uplink channel utilization, which achieves a low efficiency when this launching goal is pursued. Nevertheless, when uplink and downlink slopes are balanced over time, the agents achieve significantly higher efficiency in both uplink and downlink channels (see Fig. 10b). Such is the case that, when an optimal synchronization period is adopted, network congestion takes place due to a single uplink channel (SF12), which achieves the best network size improvements. Hence, 1514 end nodes are served while following a TDMA collision-avoidance schema, which represents a 110.57% improvement concerning a legacy scenario ( $L_{1}$ and $J_{1}$ ), and a 249.46% improvement with respect to its respective non-synchronized setup.

Finally, Fig. 11 shows the number of nodes over time for different launching strategies, and their impact on the overall network size. With SF9 resulting in the shortest slot lengths while applying $L_{2}$ , agents prioritized it over lower SFs in order to prevent early downlink congestion. To further increase the network capacity, SF schedules with uplink occupancies equal to or higher than 80% were automatically disabled by the MAS for new device allocations so that uplink channel utilization can be balanced to some extent. As a result, Fig. 11a shows how SF9 was prioritized over lower SFs, followed by SF10, SF11 and, finally, SF12. As shown in Fig. 10b, congestion took place when SF12 schedule reached a 100% of uplink occupancy. In contrast, guard time distributions implemented in Fig. 11b prioritized the minimum possible SF, which congested the network due to high SF12 uplink occupancies less co-existing nodes.

4.3 Practical insights

Finally, this section discusses various insights related to a real-world deployment in order to improve the replicability of this work through a detailed description of the hardware and software material resources required.

The presented end-to-end system architecture has been validated in practice using four B-L072Z-LRWAN1 STM32L03 end nodes (SX1272 transceivers) and an 868-MHz multi-channel LoRaWAN gateway based on Raspberry Pi and the iC880A concentrator4 (SX1301 transceiver). The approximate cost per end node is 45$ and per gateway is 200$, although depending on the size and cost requirements of the end application, the end nodes can be reduced to about 10$.

The gateway implemented the UDP packet forwarder,5 which was connected to a running instance of the ChirpStack open source network server, consisting of gateway bridge, network server, and application server services. The functionality of each agent was developed using the Python language, all of which were deployed on the same server (Intel i5-4590 CPU, 4 cores, 3.30 GHz, 16-GB RAM, Ubuntu 18.04 Bionic) as separate processes communicating internally via TCP sockets. The frame forwarder agent implemented both TCP and MQTT communication with the MAS and ChirpStack network servers, respectively. In addition, NSSE instances were built using the Click Router Framework [46] (for more details on the NSSE implementation, see our previous work [19]). Finally, the open source database PostgreSQL6 v10.23 was used to store end node metadata and network metrics.

While the previous end-to-end system was implemented to validate the proposal, the results shown in this work required the addition of the ChirpStack device emulator to increase the traffic load on the network. To do this, both the LoRaWAN gateway and the STM32 end nodes were replaced by a running instance of the ChirpStack device emulator, with payload frames designed to follow the format specified in Fig. 3 and Table 2, and SFs selected based on weighted probabilities from Table 4. This was done using the weightedrand7 implementation in the Go language.

5. Conclusions

In this work, an IoT approach to large-scale monitoring of EoL EVBs is proposed and designed on top of LoRaWAN communications. To do so, a time-slotted scheduling technique is followed for collision avoidance and the design of a multi-agent resource allocation component is introduced to optimize LoRaWAN channel access according to the available network resources, co-existing end nodes, and application-oriented constraints at the gateway level.

First, EoL storage and transportation conditions have been reviewed, which motivated the design of a single-gateway LoRaWAN communication network based on the number of monitoring nodes and the required payload magnitudes to be periodically transmitted. Second, the design and deployment of the multi-agent resource-allocation network manager are addressed following a modular design, on top of which different schedule launching and end-node joining decision-making strategies are defined at the MAS with a view to providing improvements in the maximum-achievable cell capacity for two different geographical end-node distribution scenarios: Scenario $\mathscr{S}1$ consisting of higher densities of low SFs (rural-like), and Scenario $\mathscr{S}2$ consisting of higher densities of higher SFs (urban-like).

An overview of the main scalability-oriented conclusions and lessons learned from our experimental setups is provided below, which support the role LoRaWAN communication networks for large-scale monitoring of EoL products as well as the suitability of MAS-enabled on-demand resource allocation:

The most relevant cell capacity improvements were achieved through the online computation and integration of optimal synchronization periods in the network, which guaranteed the balancing of uplink and downlink channel utilization according to individual end-node application and hardware constraints. Up to 75.27% improvements in the network capacity were achieved by the MAS for Scenario $\mathscr{S}1$ end-node distributions (higher coexistence of low SFs), and 249.46% for Scenario $\mathscr{S}2$ end-node distributions (higher coexistence of high SFs).

Once optimal synchronization periods are computed, the next decision-making stage achieving significant scalability improvements was launching. The allocation of guard times across different SF-based schedules prevented the network from early congestion at a single uplink schedule and resulted in 50.72% improvements in the maximum-achievable network size for Scenario $\mathscr{S}1$ deployment distributions and 39.15% for Scenario $\mathscr{S}2$ ones.

Lastly, additional capacity improvements were achieved for different decision-making strategies being applied upon end-node joining. These guaranteed the balancing of uplink channel utilization across the different SF schedules on top of optimal synchronization periods and suitable launching strategies having been applied. Specifically, these improvements were up to 33.06% in the case of Scenario $\mathscr{S}1$ distributions and 7.45% in that of Scenario $\mathscr{S}2$ .

As a take-home message, based on the results of this work, future contributions to LoRaWAN time-slotted resource-allocation systems should focus their efforts on optimizing the decision strategies involved in schedule initiation, which has been shown to be the stage where the most significant scalability improvements can be achieved based on contextual information collected during warm-up. That is, the design of guard times across multiple SFs in the network.

In the future, we plan to use of the selected network metrics in addition to their impact on uplink and downlink channel utilization to generate a dataset and improve the granularity of the conclusions drawn through a sensitivity study. To further improve the decision-making goals proposed in this work, the design and integration of a new context manager agent built upon ML techniques will be proposed, which will serve to automatically assess the scalability in time-slotted LoRaWAN networks through a semi-automated data ingestion pipeline from open sources of information following the open-source intelligence (OSINT) concept. Should this be the case, an end-to-end performance evaluation including agent-to-agent communication would be highly desirable.

To validate network scalability improvements under real deployment conditions, a preliminary network testbed will be deployed in an industrial setup with the aim to identify factors limiting the scalability of the proposal and design new agents to tackle them.

Footnotes

https://www.chirpstack.io/.

https://docs.mydevices.com/docs/lorawan/cayenne-lpp.

https://www.st.com/en/evaluation-tools/b-l072z-lrwan1.html.

https://lora-alliance.org/lora_products/ic880a-lora-concentrator/.

https://github.com/ttn-zh/packet_forwarder.

https://www.postgresql.org/.

github.com/mroth/weightedrand.

Acknowledgments

Grants 2019-PREDUCLM-10703 and 2022-GRIN-34056 funded by Universidad de Castilla-La Mancha and by “ESF Investing in your future”. Grant PID2021-123627OB-C52 funded by MCIN/AEI/10.13039/50110 0011033 and by “ERDF A way to make Europe”. Grant DIN2018-010177 funded by MCIN/AEI/10.13039/501 100011033.

References

Forti

Balde

Kuehr

Bel

. The global e-waste monitor 2020: Quantities, flows and resources. United Nations University, International Telecommunication Union, and International Solid Waste Association; 2020.

European Parliament and Council. Directive 2012/19/EU of the European Parliament and of the Council of 4 July 2012 on Waste Electrical and Electronic Equipment, WEEE. Official Journal of the European Union L. 2012; 197: 38-71.

Rentemeister

Badeda

Jöst

Schulte

Sauer

. Digital twin for battery systems: Cloud battery management system with online state-of-charge and state-of-health estimation. Journal of Energy Storage. 2020; 30: 101557.

Matsumoto

Yang

Martinsen

Kainuma

. Trends and research challenges in remanufacturing. International Journal of Precision Engineering and Manufacturing-Green Technology. 2016; 3(1): 129-42.

Harper

Sommerville

Kendrick

Driscoll

Slater

Stolkin

, et al. Recycling lithium-ion batteries from electric vehicles. Nature. 2019; 575(7781): 75-86.

Chen

Arsenault

Karlson

Simon

, et al. Recycling end-of-life electric vehicle lithium-ion batteries. Joule. 2019; 3(11): 2622-46.

Charnley

Tiwari

Hutabarat

Moreno

Okorie

Tiwari

. Simulation to enable a data-driven circular economy. Sustainability. 2019; 11(12): 3379.

Bhatti

Mohan

Singh

. Towards the future of smart electric vehicles: Digital twin technology. Renewable and Sustainable Energy Reviews. 2021; 141: 110801.

Kerin

Pham

. A review of emerging industry 4.0 technologies in remanufacturing. Journal of Cleaner Production. 2019; 237: 117805.

10.

Noura

Boulon

Jemeï

. A review of battery state of health estimation methods: Hybrid electric vehicle challenges. World Electric Vehicle Journal. 2020; 11(4): 66.

11.

Friansa

Haq

Santi

Kurniadi

Leksono

Yuliarto

. Development of battery monitoring system in smart microgrid based on Internet of Things (IoT). Procedia engineering. 2017; 170: 482-7.

12.

Adhikaree

Kim

Vagdoda

Ochoa

Hernandez

Lee

. Cloud-based battery condition monitoring platform for large-scale lithium-ion battery energy storage systems using Internet-of-Things (IoT). In: 2017 IEEE Energy Conversion Congress and Exposition (ECCE). IEEE; 2017. pp. 1004-9.

13.

Bergh

. Mapping the European reverse logistics of electric vehicle batteries. Master Thesis. 2020.

14.

Chaudhari

Zennaro

Borkar

. LPWAN technologies: Emerging application characteristics, requirements, and design considerations. Future Internet. 2020; 12(3): 46.

15.

Roda-Sanchez

Olivares

Garrido-Hidalgo

de la Vara

Fernández-Caballero

. Human-robot interaction in Industry 4.0 based on an Internet of Things real-time gesture control system. Integrated Computer-Aided Engineering. 2021; 28(2): 159-75.

16.

LoRaWAN

{}^{\text{\@setsize{\scriptsize}{8pt}{\viipt}{\@viipt}\texttrademark}}

1.0.3 Specification. Rev. 1.0.3. 2018.

17.

Van den Abeele

Haxhibeqiri

Moerman

Hoebeke

. Scalability analysis of large-scale LoRaWAN networks in ns-3. IEEE Internet of Things Journal. 2017; 4(6): 2186-98.

18.

Leonardi

Lo Bello

Battaglia

Patti

. Comparative assessment of the LoRaWAN medium access control protocols for IoT: Does listen before talk perform better than ALOHA? Electronics. 2020; 9(4): 553.

19.

Garrido-Hidalgo

Haxhibeqiri

Moons

Hoebeke

Olivares

Ramirez

, et al. LoRaWAN Scheduling: From Concept to Implementation. IEEE Internet of Things J. 2021.

20.

Sousa

Teixeira

Carneiro

Nunes

Novais

. Knowledge-based decision intelligence in street lighting management. Integrated Computer-Aided Engineering. 2022; 29(2): 189-207.

21.

Casado

Martinez-Tomás

Fernández-Caballero

. Multi-agent system for knowledge-based event recognition and composition. Expert Systems. 2011; 28(5): 488-501.

22.

Zorbas

Abdelfadeel

Kotzanikolaou

Pesch

. TS-LoRa: Time-slotted LoRaWAN for the Industrial Internet of Things. Computer Communications. 2020; 153: 1-10.

23.

Esmalian

Wang

Mostafavi

. Multi-agent modeling of hazard–household–infrastructure nexus for equitable resilience assessment. Computer-Aided Civil and Infrastructure Engineering. 2022; 37(12): 1491-520.

24.

Gutierrez-Soto

Adeli

. Multi-agent replicator controller for sustainable vibration control of smart structures. Journal of Vibroengineering. 2017; 19(6): 4300-22.

25.

Alexiadis

Veliskaki

Nizamis

Bintoudi

Zyglakis

Triantafyllidis

, et al. A smarthome conversational agent performing implicit demand-response application planning. Integrated Computer-Aided Engineering. 2022; 29(1): 43-61.

26.

Chen

Dong

Labi

. Graph neural network and reinforcement learning for multi-agent cooperative control of connected autonomous vehicles. Computer-Aided Civil and Infrastructure Engineering. 2021; 36(7): 838-57.

27.

Tellache

Mekrache

Bradai

Boussaha

Pousset

. Deep Reinforcement Learning based Resource Allocation in Dense Sliced LoRaWAN Networks. In: 2022 IEEE International Conference on Consumer Electronics (ICCE). IEEE; 2022. pp. 1-6.

28.

Park

Lee

Joe

. Network resource optimization with reinforcement learning for low power wide area networks. EURASIP Journal on Wireless Communications and Networking. 2020; 2020(1): 1-20.

29.

Garrido-Hidalgo

Roda-Sanchez

Olivares

Ramírez

Fernández-Caballero

. Multi-agent LoRaWAN Network for End-of-Life Management of Electric Vehicle Batteries. In: International Work-Conference on the Interplay Between Natural and Artificial Computation. Springer; 2022. pp. 505-14.

30.

Ponis

Plakas

Aretoulaki

Tzanetou

Maroutas

. LoRaWAN for tracking inland routes of plastic waste: Introducing the smart TRACKPLAST bottle. Cleaner Waste Systems. 2023; 4: 100068.

31.

Guo

Zhong

. A customer-centric IoT-based novel closed-loop supply chain model for WEEE management. Advanced Engineering Informatics. 2023; 55: 101899.

32.

Xie

Wang

Jiang

. Fire protection design of a lithium-ion battery warehouse based on numerical simulation results. Journal of Loss Prevention in the Process Industries. 2022; 80: 104885.

33.

Berecibar

Gandiaga

Villarreal

Omar

Van Mierlo

Van den Bossche

. Critical review of state of health estimation methods of Li-ion batteries for real applications. Renewable and Sustainable Energy Reviews. 2016; 56: 572-87.

34.

Ziemba

Prevolnik

. The reverse logistics of electric vehicle batteries: challenges encountered by 3PLs and recyclers. 2019.

35.

Garrido-Hidalgo

Ramirez

Olivares

Roda-Sanchez

. The adoption of Internet of Things in a Circular Supply Chain Framework for the Recovery of WEEE: The Case of Lithium-Ion Electric Vehicle Battery Packs. Waste Management. 2020; 103: 32-44.

36.

SX1272/3/6/7/8: LoRa Modem. Rev. 1. 2013.

37.

Short Range Devices (SRD) operating in the frequency range 25 MHz to 1000 MHz. Rev. 3.1.1. 2017.

38.

Tomić

Bhatia

Breza

McCann

. The limits of LoRaWAN in event-triggered wireless networked control systems. In: 2018 UKACC 12th International Conference on Control (CONTROL). IEEE; 2018. pp. 101-6.

39.

Haxhibeqiri

Moerman

Hoebeke

. Low overhead scheduling of LoRa transmissions for improved scalability. IEEE Internet of Things J. 2018; 6(2): 3097-109.

40.

Sgora

Vergados

. A survey of TDMA scheduling schemes in wireless multihop networks. ACM Computing Surveys (CSUR). 2015; 47(3): 1-39.

41.

Chasserat

. Achieving energy efficiency in dense LoRaWANs through TDMA. In: IEEE International Symposium On a World of Wireless, Mobile and Multimedia Networks (WoWMoM). IEEE; 2020.

42.

Triantafyllou

Sarigiannidis

Lagkas

Moscholios

Sarigiannidis

. Leveraging fairness in LoRaWAN: A novel scheduling scheme for collision avoidance. Computer Networks. 2021; 186: 107735.

43.

Heusse

Attia

Caillouet

Rousseau

Duda

. Capacity of a LoRaWAN cell. In: Proceedings of the 23rd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems. 2020. pp. 131-40.

44.

Garrido-Hidalgo

Olivares

Ramirez

Roda-Sanchez

. An End-to-End Internet of Things Solution for Reverse Supply Chain Management in Industry 4.0. Computers in Industry. 2019; 112: 103127.

45.

Duda

Heusse

. Spatial issues in modeling LoRaWAN capacity. In: Proceedings of the 22Nd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems. 2019. pp. 191-8.

46.

Kohler

Morris

Chen

Jannotti

Kaashoek

. The Click modular router. ACM Transactions on Computer Systems (TOCS). 2000; 18(3): 263-97.

Internet-of-Things framework for scalable end-of-life condition monitoring in remanufacturing

Abstract

Keywords

1. Introduction

2.1 Condition monitoring of EoL EVBs

Table 2 LoRaWAN regional parameters in Europe for a 125-kHz bandwidth

3.1 Payload frame design

Table 3 Magnitude lookup table at payload formatting agent

Table 4 Percentage of joining nodes and cell radius per SF for end-node distributions in scenarios 𝒮 ⁢ 1 and 𝒮 ⁢ 2

4.1 Network capacity improvements in 𝒮 ⁢ 1

5. Conclusions

Footnotes

Acknowledgments

References

Table 2
LoRaWAN regional parameters in Europe for a 125-kHz bandwidth

Table 3
Magnitude lookup table at payload formatting agent

Table 4
Percentage of joining nodes and cell radius per SF for end-node distributions in scenarios $\mathscr{S}1$ and $\mathscr{S}2$

4.1 Network capacity improvements in $\mathscr{S}1$