Implementation of MQTT protocol based network architecture for smart factory

Abstract

The core concept of smart manufacturing is based on digitization to construct intelligent production and management in the manufacturing process. By digitizing the production process and connecting all levels from product design to service, the purpose of improving manufacturing efficiency, reducing production cost, enhancing product quality, and optimizing user experience can be achieved. To digitize the manufacturing process, IoT technology will have to be introduced into the manufacturing process to collect and analyze process information. However, one of the most important problems in building the industrial IoT (IIoT) environment is that different industrial network protocols are used for different equipment in factories. Therefore, the information in the manufacturing process may not be easily exchanged and obtained. To solve the above problem, a smart factory network architecture based on MQTT (MQ Telemetry Transport), IoT communication protocol, is proposed in this study, to construct a heterogeneous interface communication bridge between the machine tool, embedded device Raspberry Pi, and website. Finally, the system architecture is implemented and imported into the factory, and a smart manufacturing information management system is developed. The edge computing module is set up beside a three-axis machine tool, and a human-machine interface is built for the user controlling and monitoring. Users can also monitor the system through the dynamically updating website at any time and any place. The function of real-time gesture recognition based on image technology is developed and built on the edge computing module. The gesture recognition results can be transmitted to the machine controller through MQTT, and the machine will execute the corresponding action according to different gestures to achieve human-robot collaboration. The MQTT transmission architecture developed here is validated by the given edge computing application. It can serve as the basis for the construction of the IIoT environment, assist the traditional manufacturing industry to prepare for digitization, and accelerate the practice of smart manufacturing.

Introduction

In recent years, the concept of digitization, which is closely related to Industry 4.0, is to connect the automation equipment and management system in the factory by using IoT technology. All kinds of data are collected, integrated, and analyzed, which can assist the factory in preparing goods, predicting failure time, preventing and maintaining, improving production capacity, etc. Therefore, creating an IIoT environment is the common goal of the global manufacturing industry. However, one of the most important issues in building the Industrial Internet of Things (IIoT) environment is that different industrial network protocols are used with different equipment in factories, so that information in the manufacturing process may not be easily exchanged and obtained.

MQTT is an ISO standard (ISO/IEC PRF 20922:2016) message transmission protocol based on publish/subscribe mode between clients and servers. It is easy to apply because of its simple designing concept, lightweight packet, and open resource characteristic. MQTT is designed for IoT and M2M (machine-to-machine) applications in environments with low network bandwidth, low hardware resources, unstable, and high latency networks. The publish/subscribe mode is introduced in the networking architecture based on MQTT protocol proposed in this research. The broker in this mode can process information published or subscribed by different clients. The machine in the framework can be used as the client terminal of MQTT, subscribing to machine sensing information or publishing machine control information. With the above design, the difficulty of information transmission between heterogeneous machines can be solved through this networking architecture, making it more convenient and real-time for engineers to monitor or deal with problems on different machines.

In the publish/subscribe mode, there are three roles: (1) Publisher as the client; (2) Subscriber as the client; (3) Broker as the server. First, Subscribers subscribe to topics of interest. When a publisher sends a new message, the broker sends that message to all subscribers by topic. Since a broker is acting as a relay station between two clients, it is not necessary for both sides to know each other’s IP addresses. Only the address and the receiving port number of the broker are needed.

To reduce the workload of the server and reduce the delay of the network, the concept of edge computing and fog computing is also adopted in the networking architecture of this research. In the edge computing layer, various sensing data will be calculated and analyzed through the Raspberry Pi installed in the manufacturing area. The processed information is then passed to the Raspberry Pi as a broker in the fog computing layer through the MQTT protocol. Finally, the sensing information will be published by the broker to the corresponding topic. Other clients in the MQTT framework, such as the cloud data center, machine interface, etc., can obtain the required sensing information by subscribing to the topic.

Besides, this research also designed an application case of using hand gestures to control the CNC processing machines as edge computing. In this case, the user’s real-time image is captured through a webcam, and the image is processed and gesture recognition performed on the raspberry. The result of the identification will be published to the specific topic of MQTT for the CNC control system to read, and the CNC control system can perform the specified actions following the hand gestures.

Literature review

Jain and Chhajed ¹ collated and compared several popular lightweight IoT protocols. It was announced that each protocol has its specific environment and application condition. Among them, MQTT has the advantages of small message packets, high QoS levels, and high reliability. It is mainly applicable to embedded devices and microcontroller-based devices. Bezerra et al. ² tested and evaluated that MQTT is a better and more suitable IoT protocol in smart factory environment. Under the same conditions, MQTT has smaller message packets and a simpler structure, so it is faster to transmit messages. Nawandar and Satpute ³ developed a low-cost intelligent system for smart irrigation that uses MQTT and HTTP to keep the user informed about the current crop situation even from a distant location. MQTT with low power and low complexity is ideal for farmland monitoring, where the signal of network is weak. Yen ⁴ combined the IoT communication and laser guidance technology to improve the performance of existing Automated Guided Vehicle (AGV). The data packets of the status of AGV system are analyzed and converted into MQTT format packets so that the power and speed of the vehicle can be real-time monitored. Li ⁵ introduced MQTT into the equipment in the factory. The equipment combined with IIoT gateway, which developed on an embedded system, and installed around the machines for sensor information collection. They also developed a mobile application platform by using MQTT protocol for monitoring and realizing two-way information exchange. From the above literature survey, it can be found that if the MQTT protocol is used, it is expected to make packets lightweight, reduce bandwidth requirements, and enhance the performance of data collection and real-time monitoring. MQTT is rarely used in manufacturing, so it is proposed to introduce MQTT into the factory to build IIoT environment in this paper.

Edge computing and fog computing make computing closer to the local network where the data is collected, and reduce the time and network bandwidth costs for data to and from the cloud. By using multi-level cooperation between edge computing, fog computing, and cloud computing, tasks are shared with the terminal devices, and it no longer 100% relies on cloud computing. Chen et al. ⁶ mentioned that in the field of Industrial IoT, edge computing provides added benefits of agility, real-time processing, and autonomy to create value for intelligent manufacturing. Qi and Tao ⁷ proposed a smart manufacturing system based on edge computing, fog computing, and cloud computing. Edge computing provides edge services near the source of data to meet the critical requirements in agile connectivity, real-time optimization, smart applications, security, and privacy. Froiz-Míguez et al. ⁸ proposed an IoT home automation system for fog computing. Various data of actuators and sensor nodes are sent to the gateway for processing through the MQTT, and the filtered information is sent to the cloud server. The latency measurements show that fog computing can be harnessed for providing the real-time response. Chen et al. ⁹ proposed the general hierarchical structure of the smart factory and pointed out that edge computing makes full use of embedded computing of terminal equipment, thus providing the autonomy of distributed processing equipment. At the same time, edge computing in collaboration with cloud computing can make the entire manufacturing system more intelligent.

It can be seen from the above research that edge computing can provide computing processing, storage, transmission, and networking services between terminal devices and traditional cloud computing, to meet the requirements of real-time processing and bandwidth reduction, providing a favorable development foundation for smart manufacturing. Therefore, it is proposed to import the architecture and applications of edge computing and fog computing into manufacturing in this research.

In this study, a smart factory network rack based on an open communication protocol is proposed, which uses embedded devices to collect and filter sensor information to achieve the following four research aims.

1. The IoT communication protocol MQTT is imported into the network architecture, and the publish/subscribe mode is adopted between the machine tool, sensor module and website, to achieve the purpose of fast and light information transmission and solve the transmission problem of heterogeneous communication interface.

2. The concept of edge computing is imported. A Raspberry Pi is set up beside each machine, to collect and calculate sensor information, publish the result information to the broker, and storage it to the cloud server.

3. The human-machine interface and dynamically updating website are set up to display the processing data of the machines in the factory and the sensing data of Raspberry Pi on each machine. Factory staff can get the real-time status of equipment and manufacturing site status quickly for decision-making and immediate processing.

4. The function of real-time gesture recognition is developed and built on the Raspberry Pi. The gesture recognition results can be transmitted to the machine through MQTT, and the machine will carry out corresponding actions according to different gestures to achieve human-robot collaboration mode.

Methodology

Networking architecture based on MQTT protocol

The system architecture of this study is divided into four functional layers, control layer, edge computing layer, data management layer, and application layer. The overall system architecture is shown in Figure 1. The key technologies are presented in the following:

1. Motion Control Layer: The Control core of this layer is a universal controller. In addition to being the main core of the operation of the machine tool, it can also receive sensor information collected by Raspberry Pi for inspection on the human-machine interface (HMI) of the machine tool. MQTT is used at each layer of communication technology to simplify and speed up the exchange of messages.

2. Edge Computing Layer: It is planned to set up Raspberry Pi around the machine, various sensors are installed, process the received sensor information. Then publish it to the Broker in each region, send it in the client that subscribes to this topic, and store it to the management layer.

3. Fog Computing Layer: Raspberry Pi is set up in each area of the factory as the MQTT Broker, responsible for the subscribing and publishing between clients, and carry out distributed network packet transmission and communication.

4. Application Layer: A website is built to actively receive the machine and sensor information published by the control layer and the edge computing layer, to instantly display the information to be viewed.

5. Data Management Layer: Format and types of required data are designed, the data storage entities are defined, and the information of each layer is stored in the cloud data center.

Figure 1.

The architecture diagram of MQTT protocol based network architecture for smart factory.

Hardware connection planning of the factory network architecture

As shown in Figure 2, in combination with the existing management servers or data management centers in the factory or each factory, an MQTT broker is built to jointly manage. The broker is responsible for sending and receiving publish/subscribe information, and this information is stored in the data center.

Figure 2.

Hardware connection planning of the factory network architecture.

Each industrial computer is distributed and wired in the local network of the factory, and each computer is equipped with a Raspberry Pi to collect and publish sensing information wirelessly. 802.11AC is used as the wireless communication protocol by Wi-fi AP, and WEP encryption is used for LAN encryption.

MQTT is a publish/subscribe based message transport protocol, so a fixed IP address is not needed for the client, as long as the fixed IP address and the receiving port number of the broker is needed.

Function planning of motion control layer

In the motion control layer, the generalized controller is the main core of the operation of the machine tool. The machine status and current processing information can be controlled and viewed on the human-machine interface of the machine tool. In the system architecture of this study, as the client of MQTT protocol, parameters and information of the machine will regularly be published, including spindle status, feed speed, encoder position… and so on. In addition, a request can be sent to the broker to subscribe to Raspberry Pi attached to each machine and displayed on the human-machine interface.

Function planning of edge computing layer

Due to the traditional cloud computing architecture, all data is transmitted back to the central data center for processing and then transmitted back to the user device. In the case of IoT, high availability networks with minimal latency are required to process large amounts of data in real-time, hence the concept of edge computing is adopted to the network architecture in this study. As shown in Figure 3, all sensing data is processed in the endpoint of the edge device, and the final results are returned to the cloud data center for storage, reducing bandwidth usage and latency, and shortening response time.

Figure 3.

The edge computing architecture in this study.

The design of the publish/subscribe function in the system is shown in Figure 4. Raspberry Pi is set up in each area of the factory as the MQTT Broker, responsible for the subscribing and publishing between clients, and carry out distributed network packet transmission and communication. It is also planned to set up Raspberry Pi around the machine, various sensors are installed, process the received sensor information. These sensing data and machine data will be published to the broker in each region. Then, the data will be sent to the client that subscribes to this topic such as machines and websites, and stored in the management layer.

Figure 4.

The design of publish/subscribe function.

In addition, the application of edge computing in the network architecture is planned. The real-time image is captured by webcam, image processing, and gesture recognition are performed by the edge computing core Raspberry Pi. After the result is obtained, it will be published on a specific topic. Once the result of gesture recognition is received by the machine, the corresponding motion control command will be performed. The process of planning is shown in Figure 5.

Figure 5.

The process of edge computing application.

The detailed flow of image processing and hand gesture recognition in Raspberry Pi is shown in Figure 6, and the Convexity defects are mainly used to distinguish hand gestures 0, 1, 2, 3, 4, and 5. First, the captured image is converted to HSV space and binarized with a custom threshold to distinguish the hand from the background. Dilation and Gaussian Blur of the image are then used to make the Contour of the hand easier to be obtained. After taking the Contour of a hand, a convex polygon containing all the points is obtained and called Convex hull. Deviations from this hull can be considered as Convexity defects. After getting how many Convexity defects is on the hand image, it can be inferred that the actual hand gesture is 0, 1, 2, 3, 4, or 5.

Figure 6.

The image processing flow of hand gestures recognition.

Function planning of application layer

A Web Server is planned to develop in node.js environment. By using WebSocket to connect, two-way communication between the browser and the server is established, and the server is allowed to send information to the browser actively. Therefore, socket.IO is adopted to realize real-time bidirectional communication between server and browser. JSON, a lightweight data exchange format, is used to send updated data from the server. JSON-formatted messages are read and parsed by the website, and the dynamic messages are displayed instantly. The process is shown in Figure 7.

Figure 7.

The message receiving mechanism for the dynamically updating website.

Results

Edge computing module

Mosquitto, an open-source (EPL/EDL licensed) message broker, is used to build the MQTT broker in the Raspberry Pi, as the middle agent for sending and receiving messages. In addition, the MQTT Client is built on Raspberry Pi attached to the machine as an edge computing module. The paho-mqtt library by Eclipse Paho project is installed in Raspberry Pi to develop the function of the MQTT client. Functional modules in the edge computing embedded system are built, as shown in Figure 8, which is composed of the following five modules.

1. Parameter setting module: the number of Raspberry Pi, topic parameters of Raspberry Pi, MQTT related connection parameters, and database connection parameters can be set.

2. Sensor data module: the data transmitted by the sensor at the I/O pin can be read, namely temperature, hydrometer, and webcam, and the parameters will be passed to other modules.

3. Transmission module: responsible for publishing functions and uploading data to the database.

4. Schedule execution module: it is the main program that performs the overall function of Raspberry Pi according to the process sequence of sensing data reading, publishing by MQTT transmission, and uploading database.

5. Image processing and recognition module: The program of image processing is developed by using OpenCV library version 4.2.0. After extracting the real-time image, the gesture recognition result is obtained from this module instantly.

Figure 8.

Functional modules of edge computing embedded system.

Raspberry Pi is used to set up an edge computing module, and sensors and a webcam on the module receive sensing data and real-time images. After the processing of Raspberry Pi, it is published through MQTT and stored in the SQL database. In this research, hand gesture recognition is used as an application case of edge computing in the MQTT network architecture of a manufacturing factory. The real-time image is captured by a webcam, and image processing and hand gesture recognition are performed by Raspberry Pi. The results are shown in Figure 9, hand gestures zero to five of the engineer can instantly be recognized, and the result will be transmitted to the machine controller through the MQTT network architecture.

Figure 9.

Hand gesture recognition results.

Machine controller

In this research, the motion control command library (MCCL) provided by the Industrial Technology Research Institute, R.O.C. is used to implement the machine controller, and the human-machine interface is developed with Visual Studio 2015. The functions are shown in Figure 10, including the use of MCCL instructions to issue control commands to the machine, and relevant information and interactive components are displayed on the interface. Besides, MQTT communication function is integrated to subscribe to Raspberry Pi configured on the machine, and publish motion control information of the machine. By subscribing to the gesture recognition result information, different machine actions are performed.

Figure 10.

The functions of human-machine interface.

Human-machine interface

The human-machine interface of the machine tool is shown in Figure 11. The left part is the control module of the machine. There are functions of the servo on/off, the current motion position of the machine, three-axis six-directions JOG, encoder feedback position, home position returning, spindle on/off, spindle feed speed adjustment, spindle chuck control, and motion control command demonstration, etc. in the interface. The right part of the HMI is MQTT functional module. The sensing data from Raspberry Pi attached to the machine can be subscribed, and it will be published and displayed on the interface immediately. In addition, the above machine motion control status can be published to the corresponding topic. After receiving the subscribed gesture recognition result information, different actions of the machine are performed. The corresponding six-direction inching of gestures zero to five is performed, as shown in Table 1.

Figure 11.

The human-machine interface of the machine tool.

Table 1.

The corresponding movements of gestures.

Gesture recognition result	Actions of the machine
0	X+ JOG
1	X− JOG
2	Y+ JOG
3	Y− JOG
4	Z+ JOG
5	Z− JOG

Manufacturing information website

In the node.js environment, MQTT.js is used to develop a Web Server as the channel to receive MQTT messages. The Express framework and the Socket.io are integrated to enable the Web Server and the browser to communicate with each other. JSON-formatted messages are read and parsed by the website, and the dynamic messages are displayed instantly.

The real-time sensing information monitoring website is shown in Figure 12. Subscribed sensing information of the Raspberry Pi will be updated in the text as well as in the line chart below. The previous data will be accumulated, and new data will be added dynamically in this line chart. When placing the mouse on the line chart, the data of each point in detail can be viewed. The real-time machine information monitoring website is shown in Figure 13. Subscribed machine information of the machine tool will be updated dynamically in the text. There are the latest statuses of the servo on/off, the current motion position, encoder feedback position, the times of home position returning, spindle on/off, spindle feed speed, spindle chuck on/off, etc. displayed on this website in real-time. And mechanism parameters such as encoder numerical value, maximum safe speed of the motor, the gap width of the lead screw, gearbox reduction ratio, positive and negative direction over stroke software boundary, etc. are also can be viewed on this website.

Figure 12.

Real-time sensing information monitoring website.

Figure 13.

Real-time machine information monitoring website.

Responsive web design is adopted to this website, which can make the website have appropriate display when browsing on different devices, and reduce the user’s zooming, panning, and scrolling behavior. If using a smaller screen, the content on the website will be automatically lined up for easy browsing. The real-time machine information monitoring website on the mobile device is shown in Figure 14.

Figure 14.

Real-time sensing information monitoring website on the mobile device.

Application situation of introducing into the factory

In this study, Raspberry Pi is used to set up an edge computing module. The sensor on the module receives sensing data and intercepts images from Webcam. Gestures 0, 1, 2, 3, 4, and 5 can be instantly identified by image processing and gesture recognition on Raspberry Pi. The machine subscribes to the gesture recognition result, receives it through MQTT, and does the corresponding action.

The system is imported into the factory as shown in Figure 15. A three-axis machine tool is taken as an example, the edge computing module is set up beside it, and a human-machine interface is built for the user controlling and monitoring. MQTT Broker and Wi-Fi AP are installed on the plant shelf to provide wireless transmission and MQTT communication for the plant area. Users can also monitor the system through the dynamically updating website at any time and any place.

Figure 15.

The system is applied to the factory.

Conclusion and future work

In this study, this system can be applied to build the IIoT environment base, which can assist the factory transformation and development of intelligent applications. The research results are as follows:

1. The IoT communication protocol MQTT is imported to standardize and lightweight data, a smart factory network architecture is proposed, and a heterogeneous communication bridge is built among machine tools, embedded devices, and the website.

2. The concept of edge computing is imported to collect, calculate, and process the sensing information from the low-cost Raspberry Pi. The resulting information is published in the MQTT protocol format. Edge computing architecture is used to reduce data transfer time.

3. The human-machine interface and dynamically updating website are set up to display the processing data of the machines in the factory and the sensing data of Raspberry Pi on each machine. The website can also be viewed on a mobile device for mobile viewing. Factory staff can get the real-time status of equipment and manufacturing site status quickly for decision-making and immediate processing.

4. The function of real-time gesture recognition is developed and built on the edge computing module. By making gestures of zero to five in front of the webcam, the number of convexity defects is taken to recognition by Raspberry Pi. The three-axis machine tool receives gesture recognition result through MQTT and does the corresponding action according to different hand gestures to achieve human-robot collaboration mode.