A resource-oriented middleware in a prototype cyber-physical manufacturing system

Abstract

The interconnection among heterogeneous sensors and data acquisition equipment in cyber-physical systems have profound significance in achieving adaptability, flexibility, and transparency. Various middlewares have been developed in cyber-physical systems to collect, aggregate, correlate, and translate system monitoring data. Existing middleware solutions are normally highly customized, which face several challenges due to the highly dynamic and harsh production environments. The data generated by sensors can only be shared by specific applications, which prevents the reusability of sensors. Moreover, the lack of uniform access to sensors causes high cost and low efficiency in application development. To address these issues, a resource-oriented middleware architecture called ROMiddleware was proposed, and three key enabling technologies including heterogeneous sensor modeling and grouping, open application programming interfaces development, and token-based access right control mechanism have been developed. Under the guidance of the key enabling technologies, a prototype of ROMiddleware was implemented and its performance was evaluated. Finally, two applications were developed to stress the significance of ROMiddleware. The results show that ROMiddleware can meet the requirements of data acquisition in cyber-physical systems.

Keywords

Cyber-physical system resource-oriented architecture middleware intelligent manufacturing condition monitoring production data acquisition

Introduction

Computing and communication technologies will soon be embedded in almost all types of objects and structures in the physical environment. Such systems that bridge the cyber world of computing and communications with the physical world are referred to as cyber-physical systems (CPSs).¹ Most of the researchers argue that CPS originated from embedded systems which are characterized by tight integration and coordination between computation and physical processes. According to this conception, CPS can be regarded as a system of collaborating computational entities which are in intensive connection with the surrounding physical world and its on-going processes, providing and using, at the same time, data-accessing and data-processing services available on the Internet.² CPS is regarded as one of the national research priorities of the United States and European research councils in various sectors such as automotive, aerospace, civil, railways, medical, and manufacturing.³ As an intellectual challenge, CPS is about the intersection, not the union, of the physical and cyber worlds. It combines engineering models and methods from mechanical, environmental, civil, electrical, biomedical, chemical, aeronautical, and industrial engineering with the models and methods of computer science (CS). As a consequence, CPS enables a new potential for improved efficiency, accountability, sustainability, and scalability of systems.⁴

Along with the parallel development of CS and information and communication technologies (ICTs) on one hand, and manufacturing science and technology (MST) on the other in the last two decades, manufacturing industry has undergone a number of major transitions, for example, from flexible manufacturing, to agile manufacturing, to computer integrated manufacturing, to e-manufacturing, and eventually to intelligent manufacturing philosophy. Especially, compared with the Internet-based e-manufacturing which emphasizes on digitization, globalization, mobility, and collaborative work,^5,6 intelligent manufacturing or smart manufacturing covers the characteristics of e-manufacturing and aims at taking advantages of CS, ICT, and MST to enable high levels of adaptability, flexibility, and intelligence in manufacturing processes to address individual customer needs.⁷ The application of CPS in manufacturing gives a special and significant meaning in achieving not only high quality, productivity, and reduced cost, but also the ability to react quickly, responsively, and effectively to the market and to realize the goal of mass customization and mass individualization eventually.⁸ Relying on CPS technology and MST, today’s manufacturing systems would be transformed into an intelligent shop floor with near-zero downtime,⁹ adaptive smart machining,¹⁰ and intelligent decision making.¹¹

In such an intelligent system, all involved resources (e.g. machine tools and their auxiliaries, industrial robots, workpieces, and workers) are equipped with various sensors (e.g. acceleration, noise, and energy) to improve the adaptability, flexibility, and transparency. Acquiring accurate and reliable sensor data from production site is the first and foremost step for the introduction and the implementation of CPS. Generally, a CPS middleware solution which bridges heterogeneous sensors and different applications will be designed to collect, aggregate, correlate, and translate system monitoring data. However, the design and development of CPS middleware face several challenges due to the highly dynamic and harsh production environments. First, there are many applications such as production environment monitoring, machine tool monitoring, and workpiece quality control in a cyber-physical manufacturing system. By nature, the data acquisition systems will be highly customized for specific applications and use highly customized protocols and data formats, which inevitably results in a closed and private system, that is, other applications cannot obtain the sensor data directly. For example, energy sensors are initially installed on the machine tools for an energy consumption monitoring application rather than a predictive maintenance application. As a result, the latter has no access to these sensors except that developers spend time on low-level and repetitive tasks. To save developers from reinventing the wheel, a uniform data acquisition mechanism is needed. Second, sensors are made by different manufacturers that do not always comply with the same communication protocol. It is therefore difficult to define and enforce a common standard among all the diverse sensors belonging to diverse domain in CPS, raising two integration issues:¹² (1) there is no uniform operation interface due to the heterogeneity in sensors. Consequently, application developers have to take a large effort to learn about the details of sensors, which causes the high cost and long cycle of application development; (2) open application programming interface (API) packages for different programming languages are also lacking. Third, because of the diversity of the demands for monitoring data acquisition, a configurable middleware solution is required. For example, the running state prediction of machine tool requires the aggregation of multiple sensor data (e.g. acceleration and energy); different applications (e.g. energy consumption calculation of single machine and production environment monitoring) collect the same sensor data with different sampling frequencies. Finally, there is a strong need to have a fine-grained access right control mechanism for the consideration of privacy and security under an open and interconnected environment.¹³

In order to address the issues mentioned above, a resource-oriented middleware architecture for CPS, called ROMiddleware, was proposed. ROMiddleware is a hardware–software integrated platform, fundamentally providing sensor virtualization to upper applications. It has the characteristics of being distributed, loosely coupled, and reusable. Several research questions are discussed in this article and corresponding methodologies are given as follows:

How diverse sensors are deployed on the shop floor on a large scale so that an intelligent production environment is created? This article presents a resource description framework (RDF)-based sensor virtualization model and an intelligent gateway–based deployment method for modeling and grouping the heterogeneous sensors, respectively. Using these methods, each physical sensor is abstracted as a sensor node (SN) and then each SN is registered into the intelligent gateway.

How SNs are operated and shared in an effective way? This article follows representational state transfer (REST) architectural style to seamlessly integrate SNs into the Web. Each SN is represented as a standard Web resource that can be accessed through universal resource identifier (URI). Based on REST interfaces, open API packages for different program languages are developed so that developers can write applications without needing to reprogram sensors manually.

How the privacy and security of sensor resources are guaranteed in such an open and interconnected architecture? In this project, the authors proposed a fine-grained access right control mechanism for multi-application environment so that only the authorized applications can access the sensor resources.

This article contributes in several aspects. First, the concept of CPS is introduced into manufacturing systems and therefore traditional manufacturing is updated and transformed into an intelligent era. Second, a hardware–software integrated middleware solution for CPS called ROMiddleware was designed and developed. It has important capabilities for heterogeneous devices supported, across programming languages supported, loose coupling, reusability, and fine-grained access right control mechanism. Third, a prototype cyber-physical manufacturing system was developed, and the validation of ROMiddleware was carried out to show the ease and conciseness of application development.

Literature review

CPS in manufacturing systems

In the manufacturing sector, CPS played an especially central role, and much attention has been devoted to the integration of CPS with manufacturing since its emergence in 2006, when the first US National Science Foundation Workshop on CPSs was held. However, the highly dynamic and harsh production environments, different types of sensors, high reliability requirements, and security issues bring about tremendous challenges to the implementation of CPS in manufacturing. For these reasons, a lot of research has been conducted on the CPS deployment, data acquisition, and applications in manufacturing systems.

On the aspect of CPS deployment, Lee et al.¹⁴ proposed a 5C architecture consisting of connection layer, conversion layer, cyber layer, cognition layer, and configuration layer. Liu and Jiang¹⁵ proposed a CPS architecture for intelligent shop floor. The architecture provided guidelines for implementation of CPS in manufacturing systems: from the hardware interconnection, to the data acquisition, processing and visualization, and the final knowledge acquisition and learning. Monostori et al.² reviewed the implementation of CPS in manufacturing and highlighted that the integration of automation markup language and OPC unified architecture had the ability to realize the self-description and configuration of CPS and eventually achieve the goal of plug-and-work. On the aspect of real-time monitoring data acquisition and application, Selcuk¹⁶ argued that recent advances in CS and ICT such as wireless sensor network (WSN), Internet of Things (IoT), and radio frequency identification (RFID) had been enabling predictive maintenance applications to be more efficient, applicable, and affordable. Mori and Fujishima¹⁷ developed a remote monitoring and maintenance system for computer numerical control (CNC) machine tool monitoring using mobile phones or PCs. Yuan et al.¹⁸ presented an RFID-based visibility monitoring framework for work-in-process tracking in the discrete manufacturing process. Grisostomi et al.¹⁹ designed a WSN and installed it in a manufacturing cell. Based on that, they developed an online production data collection and storage framework named Web2py to collect and store online monitoring data at low cost.

In summary, the implementations of CPS in manufacturing systems are still in development. The rapid advancement of intelligent sensors, data acquisition system, and wireless communication devices can provide relatively mature solutions in respect of hardware–software deployment, real-time data acquisition, and production monitoring. However, most existing solutions are customized, private and closed, thus some requirements like the uniform interface, reusability of multi-type sensors, and ease of integration into different applications cannot be satisfied. In this project, a universal middleware was proposed to address these requirements with an efficient way.

Middlewares in CPS

CPS middleware aims to transfer the monitoring data collected from the production sites to the upper applications for analysis and send the production commands given by applications to controllers for control.¹⁵ In recent years, various middleware solutions such as WSN,²⁰ IoT,²¹ RFID,²² smart home,^13,23 product lifecycle management,²⁴ production monitoring, and control^10,19,25 have been developed. Among the Internet- or Web-based middleware solutions, there are mainly two middleware architecture styles which are highly applied: resource-oriented architecture (ROA)²⁶ and service-oriented architecture (SOA).²⁷ These two architecture styles are compared in the following for the purpose of deciding which one is more suitable for the development of the authors’ prototype cyber-physical manufacturing system.

SOA is a set of WS-* Web services (Simple Object Access Protocol (SOAP), Web Services Description Language (WSDL), etc.) and guidelines for designing and developing software in the form of interoperable services. Each service has an interface description (e.g. by WSDL) and creates a loosely coupled contract between client and server. Based on the semantic analysis, sensors can be invoked well by a service. SOA has taken the majority in many fields, especially in business because of its convenience of complex service retrieval operations. However, SOA has some deficiencies: (1) it is not suitable for complex and highly dynamic sensor network and (2) the delay time and error probability of semantic analysis increase when the number of SNs becomes excessive.²⁸

ROA is a set of principles and guidelines for designing and developing software in the form of resources with REST interfaces. These resources are identified by some sort of “address” data included with the request (e.g. HTTP URL) and can be reused for different purposes. As the implementation of ROA, REST architecture that is defined by Roy Fielding²⁹ is very suitable for building middleware architecture using “pure” Web technology (HTTP and HTML). For instance, Stirbu³⁰ proposed a semantically augmented scalable discovery infrastructure for the heterogeneous sensor and actuator networks based on REST architecture. Guinard and Trifa³¹ proposed an approach for integrating real-world devices to the Web. More recently, Hong²⁸ proposed a resource-oriented middleware framework for heterogeneous sensors in smart home scenario.

The problems that SOA addresses are similar to the ones ROA tries to solve: the integration of many heterogeneous devices through WSs. However, ROA, especially REST, achieves this in a more lightweight and simpler manner and focuses on resources rather than functions as is the case with WS-* Web services.³² Furthermore, REST is suitable for highly dynamic production environment, read-only or read-mostly data scenario,³³ which agrees well with the characteristics of cyber-physical manufacturing system. However, existing middleware solutions such as Bundle,¹³ Real-time Data Distribution Service (RDDS),³⁴ WebMed,²⁷ Smart Items Services Infrastructure (SI)^2,24 Atlas,²³ and Dynamic Integration Middleware (DIM)³⁵ have several shortcomings that limit their applicability in the context of CPS. Consequently, ROMiddleware is developed to overcome these shortcomings. Table 1 summarizes some of the important differences between ROMiddleware and other similar middleware. It can be seen from Table 1 that ROMiddleware has advantages over others in architecture, multiple programming languages support, and access right control support.

Table 1.

Comparisons between ROMiddleware and other middlewares.

Item	ROMiddleware	Bundle	RDDS	WebMed	(SI)²	Atlas	DIM
Architecture	ROA	SOA	SOA	SOA	SOA	SOA	SOA
Application	Intelligent shop floor	Smart home	Building firefighting	Car park management	Product lifecycle management	Smart home	Mobile devices
Reusability	√	√	√	√	√	√	√
Heterogeneous devices supported	√	√	√	√	√	√	√
Across programming language supported	√	×	×	√	×	×	×
Access right control supported	√	√	×	×	×	√	×

ROA: resource-oriented architecture; SOA: service-oriented architecture.

The middleware in the prototype cyber-physical manufacturing system

In this section, a resource-oriented middleware is designed for the purpose of hiding the heterogeneity of sensors and providing upper applications with a uniform way to communicate with smart items. After a system architecture overview, three key enabling technologies, that is, heterogeneous sensor modeling and grouping, development of open API, and access right control mechanism, were developed in the middleware (called ROMiddleware) of the prototype cyber-physical manufacturing system.

System architecture

The “4A” architecture for cyber-physical manufacturing system is proposed as shown in Figure 1. It is divided into four layers: asset layer, access layer, aggregation layer, and application layer.

Physical asset layer. This layer is the connection between the CPS and the physical world, and consists of various sensors that are the machine’s gateway to sense its surrounding physical environment. Using appropriate sensor installation, various production process data such as spindle acceleration, energy consumption of machine tool, and temperature and humidity of shop floor can be collected.

Device access layer. The main goal of this layer is to deal with the heterogeneity of sensors. On one hand, this layer provides a resource-oriented abstraction that unifies the management of distributed and heterogeneous sensors. Each physical sensor is virtualized as a Web resource (SN) that has the metadata relating to the corresponding physical sensor such as spatial attributes, temporal attributes, and state representation. These attributes can be measured, estimated, and manipulated through the REST operation interfaces. On the other hand, quite a few sensors do not possess capability of accessing to the Web. Therefore, a concept of intelligent gateway in this layer is proposed to provide a wired or wireless networking environment for sensor integration and management.

Resource aggregation layer. This layer aims to support the access control for the Web resources. First, it provides open API packages for upper applications based on REST interfaces, so that the barrier to real-time data acquisition for application developers can be minimized. By doing so, the reusability of sensors and the collaboration across applications can be realized. Second, it provides the functionalities of online management and state maintenance of sensors. Third, it provides access right control under multi-application environment.

Application layer. This layer allows user to create applications relating to interaction with physical sensors such as quality prediction and control, energy consumption monitoring, and machine tool monitoring. Using open APIs provided by resource aggregation layer, developers are able to acquire real-time monitoring data without needing to learn about all the details of diversity. In addition, having application layer ensures that applications can connect to the resource aggregation layer from anywhere in the world.

Figure 1.

System architecture for cyber-physical manufacturing system.

Heterogeneous sensor modeling and grouping

In order to standardize heterogeneous sensors and realize the communication between sensors and applications, an SN is formalized as

S N = {U R I, O p e r a t i o n, C o n f i g, I n f o r m a i t o n, V a l u e}

(1)

where $SN$ represents a physical sensor. $URI$ defines the network location of $SN$ . $Operation$ is a set of REST operation methods by which SN can be modified, deleted and created in an effective way. $Config$ is a set of configuration items such as the IP address, port number, and register address. It should be noted that different types of sensors have different configuration items. For instance, in the sensor network of RS-485 bus, the configuration item is the address of RS-485 bus. While in the sensor network of Wi-Fi, the configuration items include the IP address and sensor port number. $Information$ is a set of basic properties that include the physical location, the registration information, and the value information. $Value$ are the collected sensor data that include the updated value, the updated time, the updated frequency, and the sensor status.

In order to realize resource discover and enable an SN to be readable by a machine, RDF, a semantic Web data model, is used to virtualize the SN. As a metadata model of Web resources, RDF is well suited for conceptual description using a triples <resource-attribute-value>. The RDF-based conceptual model of SNs is presented in Figure 2. Based on the conceptual model, the RDF-XML templates of all SNs are designed. Figure 3 shows an example of energy sensor virtualization in the form of RDF-XML template. According to the RDF-XML templates, all the information like sensor location, measured value, sampling frequency, and sensor status can be acquired in a machine readable way.

Figure 2.

Conceptual model of SNs based on RDF.

Figure 3.

An example RDF-XML template of energy sensor virtualization.

It is impossible for all the heterogeneous sensors to communicate with each other directly and maintain the same group. Moreover, some of sensors do not have Ethernet ports, which indicates that they cannot access into the Internet directly. Therefore, intermediate devices called intelligent gateways are used to enable sensor communication and group management. In this article, an intelligent gateway refers to an embedded board that communicates between sensors and the Internet and provides local processing and storage capabilities. Moreover, the embedded board can run a more complex control program and has various kinds of extended interfaces such as USB, Bluetooth, Wi-Fi, and RS232.

For better group management, physical sensors in the shop floor are divided into several groups depending on geographic location and communication protocols. Each group consists of several SNs and an intelligent gateway. The SNs in the same group are connected to the same intelligent gateway. Compared with SNs, the intelligent gateway is more powerful in storage and computing, and it is defined as the cluster head of a group. The cluster head not only communicates with other cluster heads but also communicates with the SNs from the same group. If a sensor wants to be a member of a particular group, it must add its RDF-XML template to the cluster head. This process is called SN registration. Based on the intelligent gateway, the group management of the registered SNs like state monitoring, data acquisition, information filling, and update can be realized.

Development of open APIs

In order to facilitate programming and share the production data among different applications, open and flexible API packages for different programming languages are required, with characteristics of low threshold, light weight, open interfaces, and reusability.

REST is a resource-oriented service access architectural style of the World Wide Web, whose purpose is to induce performance, scalability, simplicity, modifiability, visibility, portability, and reliability.²⁹ According to the conception of REST, the sensor network deployed on the shop floor is viewed as a set of Web resources identified by URIs. The operation of Web resources is carried out by the combination of URIs (specified by client) and HTTP verbs. In this article, REST architecture is adopted to implement the representation and mapping of SNs. The representation of SNs is conducted in three dimensions, namely, network address, operation methods, and virtualization, as shown in Figure 4(a). There are five elements in the conceptual model of SNs. Among them, the element URI and Operation represents the sensor network address and four operation methods, respectively; the other three elements (Config, Information, and Value) constitute the RDF-XML template. Under the REST architecture, the physical sensors are mapped as SNs by filling the RDF-XML template (see Figure 4(b)). All the SNs constitute the Web resource pool. The upper applications operate the SN through the predefined operation interfaces.

Figure 4.

Web resource representation and mapping using REST architecture: (a) representation of SNs and (b) mapping between physical sensors and SNs.

Based on the REST style representation, a hierarchical model for sensor network organization is proposed, as shown in Figure 5. Each sensor network (an intelligent gateway and several physical sensors) is mapped as a three-layered model. The top layer, medium layer, and bottom layer are the cluster heads, SNs, and elements of SNs, respectively. They are identified via URIs in the form of cluster head IP address + cluster head port + cluster head identifier + sensor node identifier. The partial operation interfaces and the constitution of URIs are presented in Table 2. Considering that the operation interfaces of HTTP protocol vary with different programming languages and different applications are written with different programming languages, open API packages for different programming languages are developed by inheriting from REST operation interfaces. The partial operation functions from API package are listed in Table 3.

Figure 5.

Hierarchical model for sensor network organization.

Table 2.

Operation interfaces and the constitution of URIs.

URI	REST operation	Description
/{CHId}	GET	Get a list of SNs of the specified cluster head
/{CHId}	DELETE	Delete the SNs of the specified cluster head
/{CHId}/sensorId	GET	Get the RDF-XML template of the specified SN
/{CHId}/sensorId	PUT	Register a new SN to the cluster head
/{CHId}/sensorId	DELETE	Delete the specified SN
/{CHId}/sensorId	POST	Modify the RDF-XML template of the specified SN
/{CHId}/sensorId/value	GET	Get the value of the specified SN

URI: universal resource identifier; REST: representational state transfer; SN: sensor node.

Table 3.

Operation functions of API package and their implication.

Operation function	REST operation	Parameter	Description
AddSensorInfo()	PUT	Token, new RDF-XML template	Register a new SN to the cluster head
GetSensorList()	GET	Token	Get a list of SNs of the specified cluster head
GetSensorInfo()	GET	Token	Get the RDF-XML template of the specified SN
GetSensorValue()	GET	Token	Get the senor value of the specified SN
GetSensorState()	GET	Token	Get the sensor state of the specified SN
UpdateSensorInfo()	POST	Token, modified data	Modify the specified SN
DeleteSensorInfo()	DELETE	Token	Delete the specified SN

API: application programming interface; REST: representational state transfer; SN: sensor node.

Access right control mechanism

In the REST architecture, the URIs of SNs are transparent to the applications, which indicates that each application can access the SN via REST interfaces without any constraint, thus resulting in security and privacy problems. To solve these problems, a token-based access right control mechanism is designed. Based on the mechanism, the REST operation interfaces are not open anymore to every developer, which means that the application can access the SNs only when it gets the permission tokens.

Let $APP = {AP P_{1}, AP P_{2}, \dots, AP P_{M}}$ be the set of applications developed using ROMiddleware, $SN = {S N_{1}, S N_{2}, \dots, S N_{N}}$ be the set of SNs registered in the ROMiddleware. $M$ and $N$ are the number of applications and SNs, respectively. Tokens are generated via a many-to-many relationship between applications and SNs, denoted as $Toke n_{k} = f (AP P_{m}, S N_{n})$ , in which $AP P_{m}$ represents the ID of the mth application and $S N_{n}$ represents the URI of the nth SN. The value of token is obtained using message-digest algorithm 5 (MD5), a widely used data encryption algorithm. The detail of MD5 is outside the scope of this article. Note that if an application wants to use the registered SN, it must make a request for sensor access permission. By doing so, the token for the required SN can be acquired. Then, the token is validated when the application submits a sensor access request. If the token validation shows that the application is authorized, then the RDF-XML template with real-time sensor data will be transmitted to the application. Figure 6 presents the flowchart of request for sensor access permission and token validation.

Figure 6.

Flowchart of request for sensor access permission and token validation.

Implementation of the middleware

In this project, the implementation of ROMiddleware was carried out based on our key research contributions previously stated. The middleware performance was evaluated by experiments of stress testing.

ROMiddleware development

A small-scale shop floor in the prototype cyber-physical manufacturing system was taken as the production environment to run the proposed middleware. The prototype is composed of three workstations, as shown in Figure 7. Workstation 1 is a CNC micro lathe (C000057A lathe) which is used to finish turning process. Workstation 2 is a CNC milling machine (MM-250S3 milling) which is used to finish milling and drilling processes. Workstation 3 consists of a CNC milling machine (EMCO MILL55) and a KUKA robot. The former is used to finish milling and drilling processes and the latter is used to load/unload workpieces from a conveyer belt.

Figure 7.

Sensor deployment in the prototype cyber-physical manufacturing system.

In order to collect the monitoring data, various sensors are deployed on the production environment (see Figure 7). For example, noise sensors, light intensity sensors, and temperature and humidity sensors are deployed on the shop floor for monitoring the production environment; energy sensors and acceleration sensors are installed on the three workstations for monitoring the state of machines. To bridge the sensors and upper applications, the embedded board named Raspberry Pi (RPi) is used. The RPi is equipped with an ARM processor with 700 MHz, a memory with 256 MB. It also has four USB slots, an RS232/485 port, a 10/100 Ethernet port, HDMI, and composite video output. In this article, eight types of sensors (see Table 4) are connected to different RPis according to six different communication protocols.

Table 4.

Different types of sensors for collecting production data.

Sensor type	Communication protocol	Communication mode
Temperature sensor (WF7002)	TCP/UDP	Wi-Fi
Energy sensor (UMG604)	Modbus_TCP	Wi-Fi
Noise sensor (HBR-ZS2)	Modbus_RTU	RS232/RS485
Temperature and humidity sensor (DHT11)	1-wire	1-wire serial port
Temperature and humidity sensor (DS18B20)	1-wire	1-wire serial port
Laser displacement sensor (IG-028)	Private protocol	RS232
Light intensity sensor (QY-150A)	Modbus_RTU	RS232/RS485
Acceleration sensor (A302EX)	Zigbee	Wi-Fi

TCP/UDP: transmission control protocol/user datagram protocol.

After the deployment of sensor network, ROMiddleware is developed. First, each SN is virtualized in the form of RDF-XML template (see Figure 3). Then, the sensor drivers are programmed with C language in the light of the communication mode and data format. These drivers are coded as interfaces that can be invoked by the main program. Third, the main program is written using Tornado, a Python Web framework and asynchronous networking library. The main program is responsible for parsing RDF-XML templates, creating sensor threads, and managing sensor thread pool. Fourth, the URIs of RPis and SNs are designed and assigned (see Figure 5). Through URIs and HTTP verbs, monitoring data can be acquired directly. Finally, open APIs for different program languages (Java, C, and Python) are developed by encapsulating REST interfaces.

Performance evaluation

In order to test and verify whether the middleware can support large volumes of data requests in a short time, experiments of stress testing are carried out. Considering that the data transmission between the middleware and application is a HTTP-based request–response mode, Apache JMeter, a load testing tool originally designed for Web applications, is selected to analyze and measure the performance of ROMiddleware. Experiments are carried out by evaluating the response time when a large number of requests access the same SN simultaneously. The JMeter server is installed on a computer with two Intel^® Core™ Duo E7500 CPU 2.93 GHz and 4 GB RAM. The network bandwidth is 2 MB/S. Without the loss of generality, the computer and the sensor network are not in the same local area network.

For these experiments, n HTTP requests are established to visit the same SN. The request rate is one time per second; the duration of the experiment is 1 or 24 h. Through the experiments, the performance indexes such as the average response time (ART), 99% sample response time (99% RT), and the maximum response time (MRT) are extracted. The partial results are shown in Table 5. Based on the results, some conclusions are drawn. First, most of the HTTP requests can return the sensor value within the allowed MRT (10 s). In addition, there is no error during the request and response, which demonstrates the reliability and accuracy of ROMiddleware. Second, when the number of HTTP requests is fixed, the response time including ART, 99% RT, and MRT increases slightly with an increase in duration, which makes it possible that applications can request for the SN for a long time with very little increase in response time. Third, as shown in Figure 8, when the duration is fixed, ART and 99% RT greatly increase as the HTTP requests increase, which illustrates that the a large number of concurrent requests from applications will cause the long response time. Therefore, the number of concurrent requests must be limited for the purpose of fast response. In addition, taking the uncertain factors such as network congestion into consideration, the trend of the change of MRT does not follow that of ART and 99% RT.

Table 5.

The partial results of stress testing.

HTTP requests	Duration (h)	ART (ms)	99% RT (ms)	MRT (ms)
5	1	22	39	1520
5	24	33	56	4113
10	1	16	43	2331
10	24	33	49	8333
25	1	15	53	141
25	24	26	60	1769
50	1	17	50	312
50	24	43	72	2369

ART: average response time; 99% RT: 99% sample response time; MRT: maximum response time.

Figure 8.

Response time for changing the number of HTTP requests.

Overall system testing and results analysis

Thanks to the API packages, a wide variety of Web applications have been developed in an easy way. First, the developers make a request for sensor access permission by getting a token. Then, they download a language-specific (e.g. Java) API package and import it into the project. Finally, the developers write applications with the support of API package.

To clarify the ease and conciseness of application development and stress the great significance of ROMiddleware, two Web applications which involved real-time data acquisition are selected. The first one is the application of condition monitoring, as shown in Figure 9. On one hand, the application is used to monitor the noise, light intensity, temperature, and humidity in the shop floor. It can be seen from the monitoring panel that the environmental noise, the temperature, and the humidity are 44 dB, 27 °C, and 63% RH, respectively, which indicates that workers operate the machines in a good working environment. In addition, the energy consumption of the three workstations is also obtained, from which Workstation 2 is machining parts while Workstation 1 and Workstation 3 are idle. On the other hand, the application is used to monitor the running state of machine tools using random forest algorithm, a decision tree–based ensemble learning method for classification and prediction. The training data such as spindle acceleration (X-direction, Y-direction, and Z-direction), turret acceleration (X-direction, Y-direction, and Z-direction), power of machine tool, and machining noise are gathered through ROMiddleware. The running state of machine tool includes “normal,”“warning,” and “abnormal” state. The random forest model is trained by the historical data collected by ROMiddleware. Then, the running state of machine tools is predicted when the real-time sensor data are put into the trained model. The running state prediction results can improve the efficiency of machine tools, decrease the failure rate, and finally achieve the goal of continuous production and near-zero downtime. The second application is the energy consumption monitoring of machines, as shown in Figure 10. The Workstation 2 (MM-250S3) is drilling the hole in a shaft. Based on ROMiddleware, the real-time power is obtained with sampling frequency of 2 Hz. The energy consumption of the machining process is calculated based on the collected data.

Figure 9.

Condition monitoring of production environment and machine tool.

Figure 10.

Real-time energy consumption monitoring of single machine.

Some conclusions are drawn according to the results of the above two applications. First, to acquire the monitoring data, the developers just need to invoke the predefined methods (e.g. getSensorValue() and getSensorState()) that are encapsulated in the API package. So, the developers can develop applications in an easy way. Second, the application of production environment monitoring can obtain multiple sensor data (e.g. energy consumption, noise, light intensity, temperature, and humidity) synchronously, which indicates that ROMiddleware can support multiple concurrent requests. Third, both application 1 and application 2 are able to collect the energy consumption via the API package, which validates the reusability of sensors. In summary, the experiments demonstrate the important role of ROMiddleware in monitoring data acquisition; it not only provides uniform access into the heterogeneous sensors with an efficient way but also realizes the concurrency and reusability.

Conclusion and further work

In this article, a resource-oriented middleware called ROMiddleware in a prototype cyber-physical manufacturing system is presented. The system architecture and three key enabling technologies are described, and the performance evaluation is conducted. Finally, two Web applications are selected to validate the ease and conciseness of application development. The results show that ROMiddlware can meet the requirements of real-time data acquisition in cyber-physical manufacturing system.

Benefits of ROMiddleware include the ease and conciseness of application development across languages, supporting multiple applications using the same sensor concurrently, and supporting access right control for security and privacy. However, there are some limitations. First, all the data requests from the applications must be first submitted to the aggregation layer. After being processed, these requests go from the aggregation layer to the physical sensor via the intelligent gateway. The collected sensor data from the intelligent gateway also has to go to the aggregation layer first, before it reaches the application layer. As a result, the response time may not be as good as those the applications interact with sensors directly. Second, ROMiddleware is a request/response type (pull-based) system rather than a message-passing type (push-based) system. So, it does not support real-time data pushing under the dynamic production environment.

In the future, ROMiddleware will be extended to support local processing within ROMiddleware so that responsiveness can be improved. Based on the monitoring data, the event trigger mechanism will be designed and some production control methods will be applied so that applications like condition monitoring, tool wear prediction, and surface quality prediction can be carried out more intelligently. Moreover, push-based sensors such as RFID devices and measurement devices (e.g. vernier calipers, roughness testers, and digital dial gauges) will be integrated into ROMiddleware for workpiece tracking and tracing.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by National Nature Science Foundation of China with grant no. 71571142.

References

Zhang

Jiang

Cheng

. Configuration design of the add-on cyber-physical system with CNC machine tools and its application perspectives. Proced CIRP 2016; 56: 360–365.

Monostori

Kádár

Bauernhansl

. Cyber-physical systems in manufacturing. CIRP Ann: Manuf Techn 2016; 65: 621–641.

Gill

. From vision to reality: cyber-physical systems. In: Proceedings of the HCSS national workshop on new research directions for high confidence transportation CPS: automotive, aviation and rail, Vienna, VA, 18–20 November 2008, http://www2.ee.washington.edu/research/nsl/aar-cps/Gill_HCSS_Transportation_Cyber-Physical_Systems_2008.pdf

Frazzon

Hartmann

Makuschewitz

. Towards socio-cyber-physical systems in production networks. Proced CIRP 2013; 7: 49–54.

Cheng

Bateman

. e-Manufacturing: characteristics, applications and potentials. Prog Nat Sci 2008; 18: 1323–1328.

Cheng

Pan

Harrison

. Web-based design and manufacturing support systems: implementation perspectives. Int J Computer Integr Manuf 2001; 14: 14–27.

Davis

Edgar

Porter

. Smart manufacturing, manufacturing intelligence and demand-dynamic performance. Comput Chem Eng 2012; 47: 145–156.

Jiang

Leng

Ding

. Social manufacturing as a sustainable paradigm for mass individualization. Proc IMechE, Part B: J Engineering Manufacture 2016; 230: 1961–1968.

Lee

Kao

Yang

. Service innovation and smart analytics for industry 4.0 and big data environment. Proced CIRP 2014; 16: 3–8.

10.

Wang

Ghani

SBC

Cheng

. Adaptive smart machining based on using constant cutting force and a smart cutting tool. Proc IMechE, Part B: J Engineering Manufacture 2013; 227: 249–253.

11.

Rao

Patel

. Novel method for decision making in the manufacturing environment. Proc IMechE, Part B: J Engineering Manufacture 2011; 225: 422–434.

12.

Bandyopadhyay

Sengupta

Maiti

. Role of middleware for Internet of Things: a study. Int J Comput Sci Eng Surv 2011; 2: 94–105.

13.

Vicaire

Hoque

Xie

. Bundle: a group-based programming abstraction for cyber-physical systems. IEEE T Ind Inform 2012; 8: 379–392.

14.

Lee

Bagheri

Kao

. A cyber-physical systems architecture for industry 4.0-based manufacturing systems. Manuf Lett 2015; 3: 18–23.

15.

Liu

Jiang

. A cyber-physical system architecture in shop floor for intelligent manufacturing. Proced CIRP 2016; 56: 372–377.

16.

Selcuk

. Predictive maintenance, its implementation and latest trends. Proc IMechE, Part B: J Engineering Manufacture. Epub ahead of print 5 January 2016. DOI: 10.1177/0954405415601640.

17.

Mori

Fujishima

. Remote monitoring and maintenance system for CNC machine tools. Proced CIRP 2013; 12: 7–12.

18.

Yuan

Guo

Wei

. Radio frequency identification-enabled monitoring and evaluating in the discrete manufacturing process. Proc IMechE, Part B: J Engineering Manufacture. Epub ahead of print 17 January 2016. DOI: 10.1177/0954405415620986.

19.

Grisostomi

Ciabattoni

Prist

. Application of a wireless sensor networks and Web2Py architecture for factory line production monitoring. In: Proceedings of the 11th international multi-conference on systems, signals & devices (SSD), Barcelona, 11–14 February 2014, pp.1–6. New York: IEEE.

20.

Wang

Cao

. Middleware for wireless sensor networks: a survey. J Comput Sci Technol 2008; 23: 305–326.

21.

Bandyopadhyay

Sengupta

Maiti

. A survey of middleware for Internet of Things. In: Özcan

Zizka

Nagamalai

(eds) Recent trends in wireless and mobile networks (Proceedings of third international conferences, WiMo 2011 and CoNeCo 2011, Ankara, Turkey, 26–28 June 2011). Berlin: Springer, 2011, pp.288–296.

22.

Al-Jaroodi

Aziz

Mohamed N.

Middleware

for RFID systems: an overview. In: Proceedings of the 33rd annual IEEE international computer software and applications conference, Seattle, WA, 20–24 July 2009, pp.154–159. New York: IEEE.

23.

King

Bose

Pickles

. Atlas: a service-oriented sensor platform: hardware and middleware to enable programmable pervasive spaces. In: Proceedings of the 31st IEEE conference on local computer networks, Tampa, FL, 14–16 November 2006, pp.630–638. New York: IEEE.

24.

Anke

Müller

Spieß

. A service-oriented middleware for integration and management of heterogeneous smart items environments. In: Proceedings of the 4th MiNEMA workshop, Sintra, 2–3 July 2006, pp.7–11.

25.

Wang

Hou

. Plant running management based on wireless sensor network. In: Proceedings of the 2009 international conference on measuring technology and mechatronics automation, Zhangjiajie, China, 11–12 April 2009, pp.138–141. New York: IEEE.

26.

Zhang

Cai

Zhu

. RESTful management for cyber-physical systems. In: Proceedings of the 15th IEEE international symposium on object/component/service-oriented real-time distributed computing workshops (ISORCW), Shenzhen, China, 11 April 2012, pp.43–47. New York: IEEE.

27.

Hoang

Paik

Kim

. Service-oriented middleware architectures for cyber-physical systems. Int J Comput Sci Netw Sec 2012; 12: 79–87.

28.

Hong

. A resource-oriented middleware framework for heterogeneous Internet of Things. In: Proceedings of the IEEE international conference on cloud and service computing (CSC), Shanghai, China, 22–24 November 2012, pp.12–16. New York: IEEE.

29.

Fielding

Taylor

. Principled design of the modern web architecture. ACM T Internet Tech 2002; 2: 115–150.

30.

Stirbu

. Towards a restful plug and play experience in the web of things. In: Proceedings of the IEEE international conference on semantic computing, Santa Clara, CA, 4–7 August 2008, pp.512–517. New York: IEEE.

31.

Guinard

Trifa

. Towards the web of things: web mashups for embedded devices. In: Proceedings of the 2nd workshop on mashups, enterprise mashups and lightweight composition on the web (MEM 2009), Madrid, 20–24 April 2009, pp.1–8, http://webofthings.org/wp-content/uploads/2009/03/guinard_trifa_webofthings_mashup.pdf

32.

Guinard

Trifa

Karnouskos

. Interacting with the SOA-based Internet of Things: discovery, query, selection, and on-demand provisioning of Web services. IEEE T Serv Comput 2010; 3: 223–235.

33.

Thelin

. A comparison of service-oriented, resource-oriented, and object-oriented architecture styles. In: Proceedings of the OMG’s 2nd workshop on web services modeling, architectures, infrastructures and standards, Philadelphia, PA, 22–25 April 2003, http://www.omg.org/news/meetings/workshops/WebServEurope_Manual/03-1_Thelin.pdf

34.

Kang

Kapitanova

Son

. RDDS: A Real-Time Data Distribution Service for cyber-physical systems. IEEE T Ind Inform 2012; 8: 393–405.

35.

Sang

Park

Jeong

. An efficient dynamic integration middleware for cyber-physical systems in mobile environments. Mobile Netw Appl 2013; 18: 110–115.