Abstract
Digital twin is widely studied in the context of industry 4.0. It is expected that the application of digital twin in intelligent factory logistics can enrich connectivity, proactivity, and agility of the logistics. Architecture is one of the significant factors impacting on the selection of appropriate enabling technologies for constructing the intelligent logistics. This paper proposes a digital twin architecture for intelligent factory logistics. It mainly includes a physical layer, two cyber layers, and an interface layer. The architecture is in compliance with the architecture of the Human-Cyber-Physical System (HCPS). Moreover, the evolution of the architecture is elaborated during the planning, implement, and operation stages when construction of the digital twin. At the initial stage, human should participate in the decision-making process frequently to determine whether the results given by digital twin need to be changed. However, the data-driven model based digital twin will continually learn the human’s changing behaviors, thus constantly updating itself. It can evolution from digital model, digital shadow to digital twin with the continuous construction process. The ultimate digital twin should be able to assess operational key performance indicators (KPIs) and handle dynamic events.
Introduction
In the value chain operation environment of smart supply chain, intelligent logistics has become the core element of intelligent factories. 1 The construction of smart factory logistics is a systematic process, it refers to the overall integration of logistics nodes such as arrival, unloading, packaging, storage, distribution, and delivery. The intelligent factory realizes integration within the factory and collaboration between the supplier and the customer through the intelligent logistics system. However, the logistics system is a complex and changeable system, and its continuous improvement is increasingly important. Logistics processes generate a vast amount of data which have considerable potential for optimization.
The digital twin is defined as a virtual model of a real physical object or process. It allows one to digitalize the properties and behaviors of physical objects and processes precisely. It can extract value in the form of insights, decision-supporting information, and detailed virtual representations of physical worlds. 2 As autonomy is also one of the core characteristics of a digital twin, it will help to make logistics more and more efficient.
A logistics digital twin architecture has to be designed to deal with the dynamics and implement easily. The architecture also needs to be flexible, scalable, and able to evolve as requirements change. The internal logistics is a mixture of automation, facilities with low degree of flexibility and manually performed logistics activities. 3 Many studies have given a universal definition, technology, and implementation digital twin architecture, which can be used as the guide the construction of the digital twin system in specific scenarios. However, digital twin architecture needs further refinement during a new constructing process of intelligent factory internal logistics system, which need to consider all the whole logistics activities of internal logistics. In this study, we developed an overall digital twin architecture for intelligent factory logistics mainly based on the five-dimensional digital twin architecture, 4 followed by detailed description of how a digital twin formed from the planning and design, implement, and operation of intelligent factory logistics.
The remainder of this paper is organized as follows. “Related works” section described the research methodology and reviewed the literature related to logistics digital twin, technologies to achieve autonomy in logistics operation. “The evolution of the architecture of digital twin for intelligent factory logistics” section uncovers the basic digital twin architecture, and how to construction a digital twin from the planning stage, impletation stage, and operation stage, and also its main components. Finally, the last section provides some conclusions followed by future research directions.
Related works
Review methodology
The related works presented in this study proposes to study the developments in the logistics digital twin and its related technologies. This study search related works in Scopus, Science Direct, and Bing academic databases, searching for search Field Type Article title, Abstract, Keywords, the search applied the “AND” operator between digital, twin, framework, logistics and “OR” operator within architecture, Supply Chain Management (SCM), manufacturing, intelligent. We first identification of relevant papers by title and abstract reading and then in-depth reading the works related to the main motivation of this study. Moreover, the literatures about the main technology like IoT, MES, decision-making, simulation to build the digital twin are also considered.
Literature review
Grieves first described a digital twin with three dimensions of physical space, virtual space, and the connection between them. 5 The characteristics, key technologies, and modeling architecture of the digital twin have been wildly explored. 6 Tao et al. 4 add digital twin data and services to the 3-dimension digital twin model to form a five dimension digital twin model. Digital twin concept refers to a wide range of technologies and application fields, so it must concentrate for specific tasks. It is widely studied in manufacturing fields.7–9 Jaensch et al. 10 presented a data-driven and model-driven digital twin for manufacturing system. Zhang et al. 11 applied digital twin into Job shop scheduling problem to deal with the dynamic scheduling. Zhang et al. 12 established a nine layered model framework of manufacturing system based on the digital twin. The proposed framework is based on the four-dimensional collaborative model and is refined into entity layer, auxiliary layer, interface layer, data layer, processing layer, information layer, algorithm layer, scheduling layer, and functional layer.
SCM digital twin as an external logistics digital twin is a wildly studied topic. Creating a supply chain digital twin requires a detailed description of the facilities and processes, complex logic tasks, creating experiments, collecting statistics, and integration of the model with an ERP system and BI systems. 13 SC risk management is a common decision-making scenario in the application of SCM digital twin. Dmitry Ivanov et al. emphasis data analytics in the construction of SCM digital twin, they point out data analytics applied to disruption identification in real time. What’s more, simulation and optimization are key not only in the modeling of physical supply chain systems, but also in the modeling of cyber SC systems.14,15 As for the internal logistics digital twin, There is already have some researches on the development of architecture. Pan et al. 16 proposed a cloud-fog-edge-based digital twin control architecture. The architecture consists of a physical layer and virtual layer. The physical layer mainly includes some capability resources respected to Entity&IoT, control strategy, computing, and local optimizer dimension. While the model center, data center, mirror center, and synchronization center are the main parts of virtual layer. It enabled the capability to deal with different levels of dynamics in the production logistics. A digital twin architecture that enable the analysis and processing of large amounts of logistics data in real-time on the basis of IoT applications and big data analytics is suggested by Haße et al. The architecture is composed of on optional data acquisition layer, data visualization, data processing, and a semantic layer. It can enable the end-to-end data management from data collection through data processing up to data visualization. The combination of IoT applications and big data analytics can be realized using open source software components under the guidance of this architecture. They also pointed out that it becomes evident that due to the focus of digital twins on the area of manufacturing, further investigations are necessary with regard to logistics. 17 It can be seen that the literature mainly study the digital twin architecture of final operational stage of the internal logistics and human is rarely considered in the logistic digital twin.
In order to achieve autonomy in logistics operations, the digital twin system must contain models and algorithms to realize data processing, data fusion, simulation, prediction, decision making support, and so on. 18 Grigoriev et al. 19 pointed out that information model of production and logistics systems is the basis for development and maintenance of corresponding digital twins. They offered the structural model of the production and logistics system of the digital twin and also developed the data structure of the information model of the production and logistics system. The information model described different parameters and properties of the object, the structure of data necessary for solving the corresponding tasks, processes of their updating and the composition and interaction of information systems which contain and process the data. Spatial-temporal information is important in logistics digital twins. Location information as spatial-temporal information is one of the attributes in geometry modeling of virtual entities. Zhao etal. 20 studied the knowledge graph for production logistics resource allocation. According to the study, the digital twin spatial–temporal mapping of mobile production logistics resources is achieved through the deep neural network. The spatial–temporal value of the resources is estimated through the featured IoT signal data. They proposed a DSTKG model to extract knowledge and reveal the relationship between resources and its related operations. This model can also represent the virtual entities spatial-temporal visualization. However, DSTKG has some limitation in the real-time estimation of spatial-temporal values in large-scale deployment and thus affect the real time decision making. Machine learning as an application of artificial intelligence is wildly studied and applied in industries due to its abilities to fulfill tasks by learning from data. The application of machine learning or reinforcement learning in the intelligent logistics has begun corresponding research. Florian Jaensch et al. argue that machine learning methods can be easily used through integration of AI-Environment and data-driven modeling/ model improvements interfaces of digital twin. It is suggested to be an effective approach to solve different issues autonomously while continuously improving the digital twin continuously. Abdullah Albadrani et.al presented machine learning algorithms to improve the knowledge utilization efficiency of the data preparing and collection in the planning of inbound logistics system. The consumer preferences, policies, time, strategy, and network design can be tracked and trained using appropriate algorithm. 21 Yan et al. 22 reviewed the development and application of reinforcement learning (RL) in the field of logistics and supply chain management. They pointed out that RL algorithms may learn from historical data for prediction purposes and offer an integrated forecasting and optimization approach to delivering solutions to stochastic problems in logistics and SCM. Moreover, simulation-based environments can just be easily used as the environment that RL needed.
During the construction of intelligent logistics, the collaboration of the whole intelligent factory should be considered. There may already be data acquisition systems or information systems like Manufacturing Execution System (MES) and SCM system. The logistics digital twin should bridge other systems in time systematically and seamlessly. To deal with the production and supplier risk, Greis et al. suggested a digital twin framework to bridge production and logistics processes. The proposed framework integrated SCM, Enterprise Resource Planning (ERP), Advanced Planning and Scheduling (APS), and MES to dismiss the information silos. Machine learning based prediction, optimization, and decision making enabled the quick response to the disruptions in production and logistics. 23 Negri et al. 24 proposed a digital twin framework that can integrate a digital shadows simulation model with MES, the frameworks have been tested and validated in their Industry 4.0 laboratory. They design an intelligence layer with rules and knowledge to manage error states and trigger the real-world disassembly processes when low assembly quality is detected.
Decision making supporting is a typical application of the logistics digital twin. Baalsrud Hauge et al. 25 proposed an architecture supporting integrating a digital twin into production logistics and a corresponding application scenario. Physical objects like AGV, robot and vision system, real-time locating system, simulator, data components and visualization components. They described how the different components needed for decision making are integrated and interacting through data streaming bus, TCP, MTTQ. Jiewu Leng et al. have given a digital twin-driven joint optimization model to obtain periodic optimal decisions by using aggregated real-time data from the physical warehouse product-service system. The optimization will then be fed back to the semiphysical simulation engine in digital twin for verification. 26
As can be seen from the reviews of the related works, digital twin is widely investigated in intelligent logistics. However, most of the work has been done on logistics for production lines and warehouses, while the intermediate stages of entire construction of the intelligent factory logistics have been rarely considered. The digital twin for intelligent factory logistics still needs a detailed description from planning, implement, operation, and continuous improvement. This paper aims to reveal: (1) how to gradually establish a corresponding digital twin system at each stage of the logistics system construction process, (2) how people can be integrated into the digital architecture, and (3) how people can help form and evolve digital twins.
The evolution of the architecture of digital twin for intelligent factory logistics
Basic architecture of digital twin for intelligent factory logistics
The basic architecture of the digital twin for intelligent factory logistics is shown in Figure 1. It can be seen that the digital twin includes a physical layer, two cyber layers, and an interface layer. The cyber layer can collect data from the physical layer automatically through the IoT gateway. While the human, information system and other information sources can interact with the cyber space of the digital twin. The cyber space composed of two cyber layers processes the data in a digital twin and provides autonomous or human-interfered services. The analysis functions and frontend services are fully customizable in this layer. To achieve autonomous decision-making, the digital twin needs to have some control ability. On the one hand, it is necessary to control the virtual entities, and the most important thing is to control real entities. It is possible that there are management and control functions in the models, so the digital mapping can interact with physical world directly or there may be other systems, that is, MES, WMS, to achieve the execution of decisions. As shown in Figure 2, real-time data driven micro-services perform predictions or abnormal detections. When the set threshold is exceeded, updates of the model and data supporting the micro-service are triggered. Otherwise, processing strategies will be generated by the micro-services and validated within the simulation models. After verification passes, if the digital twin system has the control ability to the specific strategies, the digital twin system will update or generate corresponding control commands. Otherwise, it will issue tasks and interact with WMS, MES, or other systems through the interface layer, thereby indirectly interacting with the physical world. Subsequently, the physical layer provides real-time feedback to the cyber space. Thus, the closed-loop interaction between cyber space and physical space of the digital twin is realized. The digital twin system itself must be constantly compared with the real world to continuously reduce differences and iteratively converge.
The basic architecture of intelligent factory logistics digital twin.
The illustration of how the digital twin interacts with the physical world.
We propose offline modeling and online post-calibration techniques to achieve the continuous convergence of the digital twin. Due to errors in model construction and training, there will form a gap between the cyber space and physical space. Digital twins will detect the discrepancies through data analysis services based on feedback data collected from the two spaces. By the continuously self-learning of the models, the fidelity of digital twins can be improved, which can more fully describe the characteristics of the physical space. It should be mention that human knowledge is limited. Thus, the continuous convergence of the digital twin is mainly based on machine intelligence and supplemented by human intelligence. Supervised learning and unsupervised learning could be adopted to construct intelligent agent to autonomous learning, which will support the evolution of the cyber space. If the discrepancies cannot be minimized, the digital twin will seek human help through interfaces. Consequently, the model parameters will be adjusted or even the model will be modified.
Physical layer: The physical layer involves all the entities associated with intelligent factory logistics in the real world. It refers to intelligent production and logistics equipment, human, sensors, RFID, hardware for control, camera, and so on. In order to incorporate the human to the digital twin, the human should carries a GPS tracker or using human machine interface (HMI) to interact with the cyber space. Information about logistics activities and related entities can be collected using logistics information technology. The information mainly includes real-time information on goods (location, inventory), the operating status of equipment (forklifts, conveyors, packaging equipment), the picking path, the routing path and the information of the operators. It also needs to transfer the instructions generated by the digital twin to the physical layer.
Cyber layer 1: The cyber layer 1 contains three main components, the twin data, model library, and digital mapping core. Twin data include collected parameter sets, simulation data, historical data, artificial input data, and data from the Internet and other information system. The model library contains models for the specific tasks such as geometric entity model, data processing model, data analysis model, simulation model, and prediction model. A model can be built, trained, and tested by using the parameters or data in the twin data that needed for solving a specific task. While the data generated during the optimization or operation of models will be stored in the twin data for the continuous updates of the model or developing other models. It should be mentioned that it does not necessarily need a 3D visualized virtual scenarios. Digital twins only require digital representation, and three-dimensional models are only one of the models. It depends on whether there is a visualization task. What is more, the digital twin will build a digital mapping core of the physical world based on part of parameters and models according for specific tasks. The digital mapping core consist of dynamic configuration and service engine. It could organize the parameter sets and model library to provide basic support service capability to the whole system. The components of the twin data, model library, and digital mapping core cannot deliver individually value to the upper layer of the digital twin, but can participate in the creation and offering of value propositions. The twin data is too redundant to a specific task, such as decision-making, and they will be formatted and filtered into the digital engine in the digital mapping core. The digital engine uses trained models to processes the raw information according to different requirements. When the task changes, the dynamic configuration in the digital mapping core could update the configuration files and reloaded them. Thus, the digital twin system can easily and dynamically extend the capabilities it provided.
Cyber layer 2: Cyber layer 2 provides services for specific tasks. It contains micro services which combine various models to satisfy the requirements of their specific tasks such as data visualization, prediction, and early warning (for human decision-making), etc. The micro services use unified data read and write protocol. it can reduce the couple effect of different service and thus facilitate the application development and deployment. The layer transmits instructions and control strategy to the physical layer and completes the feedback from the virtual to the real. When the composition model of micro-services is updated, its service capability can also be expanded
Interface layer: To comply with the architecture of the Human-Cyber-Physical System (HCPS), we introduce an interface layer to connect the human to the digital mapping. 27 Although human involvement can make the digital twin system more complex, but it could also increases the flexibility and stability of the system. Human can use the interface to input data such as evaluation criterion, business strategy, the objective of intelligent logistics, model hyper-parameters etc. Expert knowledge can be introduced to the digital twin system through an external knowledge base through an interface. When the digital twin cannot make decision during some complex tasks or physical systems requires human to complete jobs, the results or alter messages can be visualized through interface layer. The decision made by human can be taken by cyber space and job completion status can be also feedback to the cyber space through HMI. The information systems like MES, ERP, CRM, and SCM can be integrated with digital twin using the interface layer. The digital twins can access edge, fog, and cloud computing resources through interfaces to support data analysis and model training. Moreover, different applications can be developed by getting access to different microservices.
The digital twin architecture during the planning and design of logistics
The architecture of the digital twin in the planning and design stage is concluded as Figure 3. At this stage, the digital representation of the physical entity does not interact with the physical entity as the physical layer is empty. Therefore, the digital mapping core of the digital twin can only be called as digital model core. Consequently, the main task for construct the digital twin for intelligent factory logistics is to enrich the cyber space.
The architecture of digital model of intelligent factory logistics in planning stage.
The interface layer should include management interface, storage interface, and communication interface. Storage interface is designed for storing and retrieving from external storage services. The internal state of the digital twin can be accessed by external services or applications via management interface. Communication interfaces include physical communication interfaces and digital communication interfaces. The former is used to interact with the physical device, while the latter is used to interact with the software system.
Intelligent factory logistics planning is a systematic planning process. The designer need to investigate the process of inbound logistics, internal and outbound logistics. The general planning steps include requirements survey, concept design, preliminary planning, detailed planning, and scheme verification. The digital twin data in the planning stage should include the parameter sets that can be used to develop models. In the stage of requirements survey, the planning requirements and constraints should be clarified as the input and basis for future planning. Thus, on the one hand, it is necessary to investigate and analysis the external environment such as industry analysis data, local policy, regulations, and so on. These data can be acquired from the Internet using web spider technology. On the other hand, the internal data may refer to the existing factory about material, infrastructure, production, manufacturing, logistic information, and so on. Furthermore, it is necessary to investigate information and data related to logistics operation management.
Expert knowledge is an important part for the digital twin as it need the expert to guild the development of the digital twin. Expert knowledge data model should be built in the model library. The data model could defines the data structure, data operations, and data constraints of expert knowledge. Thus, the twin data could store expert knowledge in the form of a knowledge base. The raw expertise could be input via the communication interface. If the expert knowledge already exit in an external knowledge base, it could be introduced in the digital twin system through the storage interface. When the design scheme generated by digital twins needs to be evaluated, the evaluation service will use expert knowledge to determine the evaluation metrics and weights. The operation data of existing factories are also of great value to the construction or renovation of factory logistics. Since the logistics system has not been actually existing or in operation, virtual data should be created to simulate the occurrence of dynamic events. All these data can be deposited as digital twin data.
Models combined with optimization, analysis methods, and algorithm are built based on the parameter sets described above. The models can then be used to virtual represent and realize the planning, design, and evaluation of the logistics system to be built. Some models that the digital twin needed to describe the real world in the form of geometry or behavior may be created by other applications. Thus, the external system data may also be provided to the digital twin via the data input interface. A 3D geometry model should be basically developed to map the physical equipment. The simulation models allowing to simulate various states of the logistics system. They are can also be used to analyze the characteristics of the logistics system, optimize the target parameters, and evaluate the operating efficiency of the system. Moreover, the sales forecasting models are needed to calculate the material resource demand and capacities. As for the method and algorithm, the PEEP analysis method is often used to get the requirement of intelligent logistics. Although there are no real-time logistics data in this stage, nonlinear programming, heuristics, or metaheuristics are often used to deal with some optimization tasks. However, if there are enough input or historical data, machine learning algorithms can also be used in this stage. It should be mentioned that limited sets of object parameters and models are necessary and sufficient to solve a specific task. Especially, only a 3D geometry model can also be used as a low level of adequacy description of a logistics system.
The digital model core should organize the twin data and model to form data analysis engine, simulation engine, optimization engine, and machine-learning engine to further develop task-oriented services. Based on the digital model of the logistics, it can provide different services to deal with different tasks. Such as layout design service, warehouse design, and validation service. The layout design may provide layout of the factory or warehouse and equipment selection scheme. The storage assignment model, and the order picking planning model, order picking route optimization model can be developed as warehouse design service. It may provide storage allocation, order picking and other strategies. The digital model evaluates the rationality of the design based on the simulation model and the input data. By comparing and analyzing different schemes, the design parameters can be optimized, for example, the number and capacity of the buffer stations, the number of logistics equipment, logistics path setting, package size, packing method, etc.
Finally, various detailed logistics plans, such as layout scheme, resource allocation, facilities parameters, and infrastructure parameters can be output through results output and visualization interfaces. The outputs of the digital twin in planning stage can be used as input and construction basis for subsequent implementation. It should be pointed out that it is usually up to human to make decisions at the planning stage as the problems encountered are usually multi-objective problems. Decision options should be provided via the interface to the human to make the final decision. It enables the interaction of the human with the digital twins.
There may be multiple sub-digital twins, such as the packaging digital twin, the handling digital twin, which are built for a specific task with part of the models in model library and part of the parameter sets, as shown in Figures 4 and 5. All subsystems should cooperate with each other to support the complex logistics operations. For example, you can simulate and determine the number, speed, charging station of AGVs through handling digital twin. It can also simulate the possibility of several AGV meeting at once or the possibilities of navigating intersections.
The illustration of sub-digital-twin interacts with each other in the planning stage.
An example of the detailed sub-digital-twin.
The digital twin architecture during the implement of logistics
The main purpose of the implement of the logistics is to realize the construction of the physical world. The connection between the digital model and the physical world should be established at this stage. In addition, it also needs to integrate the digital twin with existing information systems. The architecture of digital twin of intelligent factory logistics in implement stage is shown in Figure 6.
The architecture of digital twin of intelligent factory logistics in implement stage.
The construction of the physical world includes the purchase and arrangement of production equipment and corresponding logistics equipment according to the layout, warehousing, carrying, loading/unloading, handling, and packaging schemes output in the planning stage. Intelligent factory construction should give priority to intelligent production and logistics equipment that can provide reliable operation and related operation information.
Any information lags between the physical and cyber worlds may devalue the information obtained. The Internet of Things (IoT) technology is an effective way to connect the physical world with the cyber world. 28 The data in the intelligent factory include identification, sensing, and locating information. The identification of the physical entities can be enabled by RFID, NFC, QR code, and other technologies. Wireless sensor networks (WSNs) are an effective way to sense the state and performance of physical entities as well as surrounding environmental conditions. Locating or tracking data can be obtained through GPS, GPRS, GIS, and other technologies. The sensor data can be processed by the IoT middleware such as RFID middleware, sensor network middleware or embedded middleware in the edge layer of IoT. The data can be transmitted by communication technology. Short range wireless communication technologies like Wi-Fi, Bluetooth, ultra wide band (UWB), ZigBee are mainly used in the scope of intelligent factory.28,29 Messaging protocol like Message queuing telemetry transport (MQTT) can be used as the communication protocol. The acquired structured data and unstructured data can be archived in SQL (Oracle, Sybase……) or No-SQL databases (MongoDB, Cassandra……). Of course, sensing data can also be obtained from systems such as MES, ERP, or SCM through system integration, which can eliminate data silos. Extensive real-time data collection facilitates digital twins to detect variation in logistics activities. So that the digital twin can use sustainably updated information as input for further analysis.
With the implementation of IoT, more and more data will be collected to update the parameter sets. Historical data along with these data will be used to further update the models in the digital twin. The digital twin self-learning mechanism supported by deep learning, reinforcement learning, or lazy learning algorithms can use these variated data to update the algorithm parameters in long term. The digital representation should also be updated according to the actual construction situation. Since the digital model can obtain relevant data from physical entities, but not vice versa. Thus the digital representation at this stage should be called as digital shadow core. Digital shadow stores all readable parameters of the objects or systems. The perception engine should be included in the core to support data acquisition from the physical world.
In the later stage of implementation, the equipment has been fully laid out and the corresponding layout model has been adjusted according to the actual situation, so route planning can be carried out by building a route-planning model. The sematic model should also be built at this stage. The correlation and interoperability between physical devices and the digital twin can be easily established using the sematic model. This is very useful for establishing a connection between computer representations and the real world. What’s more, unified event description model can be used to fuse of multi-source data from heterogeneous equipment.
Before logistics operations, logistics tasks should be first simulated under various working conditions to find potential conflicts in logistics plans. The plans can only be dispatched to the physical world for execution after verification and revision. What’s more, the quality of data analysis can be improved through data fusion of the physical world and simulation.
The digital twin architecture during logistics operation
In the operation of the intelligent logistics, the digital twin and physical system are already connected and synchronized. Logistics operations according to production tasks. In the operational stage, the main difference between the digital twin in the operational stage with digital twin in the implementation stage is that it has the ability to support bi-direction interact within cyberspace and physical space. As shown in Figure 7, during the operation of logistics, the real-time data of the operators, materials, logistics rules, equipment, and environment can be acquired from the physical world. The conversion engine should be built in this stage. The decisions that digital twin made can be transformed into instruction using the conversion engine and transmit to the physical world via message middleware enabled by MTQQ, Kafka, etc. 30 If the digital twin is integrated with existing information systems, the decision can also be sent to control system like MES or WMS to issue the control commands to the physical world.
The architecture of digital model of intelligent factory logistics operation stage.
Logistics decisions in the operation process often require near-real-time solutions. The relatively slow computing makes linear programming or heuristic algorithm unsuitable for the quickly responding to the unexpected dynamic changes in operation environment. The reinforcement learning algorithm can rapidly offer an optimal or near-optimal solution using the pretrained model in the planning and implement stage. Moreover, the results obtained by other algorithms can be used to initialize the model in the operation stage to get a better solution faster.
The decision-making architecture that utilizes the collaborative simulation and reinforcement learning model is illustrated in Figure 8. The blue dotted box shows the training process of decision model using simulation model. Simulation model, who is a mapping of the physical logical world, can server as the environment of the reinforcement learning. It could update the model parameters by synchronizing data acquired from the physical logistics scenarios. It can provide state of the logistics process (such as equipment status, equipment utilization, transportation time, processing time, etc.) as the input of the decision model. The corresponding decision actions can be generated using the policy in the model and accordingly reward will feedback from the environment to the agent of the model.
The decision-making architecture uses the collaborative of simulation and reinforcement learning model.
When the training is completed, the relevant parameters of the decision-making model are determined, which can be used for task decision-making in real scenarios and quickly respond to various disturbances. The red dotted box shows the online decision-making and simulation process. The trained model will give decision schemes and send it to the simulation model for assessment and validation without distributing the physical system. Consequently, the optimization dynamical decision is sent to the physical world for execution. When the simulation results cannot meet the actual requirements, the decision model may be retrained through the simulation data and update the corresponding parameters.
The model of the operation stage should support the normal operation and exception handling of logistics. The logistics scheduling and management are the main activities of logistics operation. The transportation management model which encompasses route planning and optimization, scheduling is one of the highly required model. Inventory management model that can assign the storage location is also common used. When the logistics system operates for a period of time, it will get a lot of historical data and simulation data. Thus, the methods for handling some common abnormal events can be obtained through data mining, knowledge engineering, and other methods. They are stored in the form of knowledge base and continuously updated.
The digital twin is used to obtain the actual values of the logistics operation parameters, rather than the ideal values. If it is operating normally, these operating results can be analyzed and adjusted to reduce the deviation between the digital twin and the physical operation. If there are some abnormal data and events that occur, the decisions that how to deal with disturbances (e.g. equipment failures, congestion, order changes, material shortage) can be made. Then, the solutions can be evaluated by simulating models in the digital twin and the optimal one can be used to generate new logistic tasks and send guidance to the physical world. Then a closed loop between cyberspace and the physical world is formed.
The services of the digital twin mainly include the monitoring and scheduling. Services for the quality and efficiency evaluated of logistics operations are also needed. The previous dimensions and indicators will change and iterate accordingly.
Conclusions and future work
In this study, the HCPS-based intelligent factory logistics twin digital twin architecture has been proposed. The construction of an intelligent factory logistics needs to go through stages of planning and design, implementation, and operation. Therefore, in this study we elaborate the changes in the architecture to build their corresponding digital twins.
Human play an important role in the digital twin of logistics. The models in digital twin generate corresponding instructions in different stage. When the digital twin cannot make autonomous decisions, decision options that need human intervene should be given. At the initial stage, it is often decided whether to make corresponding changes through human participation. In the later stage of digital twin construction, it is more inclined to continuously optimize the digital twins to achieve autonomous decision-making. Then the model can continuously learn people’s changing behavior and update itself through learning ability.
The digital model can be used for virtual construction of a physical system in the planning stage. Moreover, we need to evaluate the effectiveness of each scheme through the human-aided decision-making services. At the implementation stage, the construction of the IoT can favor further interaction between cyber space and physical space. The digital model in the planning stage evolves into a digital shadow that can acquire data from the physical world. During the operational stage, it evolves into a final digital twin, forming a closed loop between cyberspace and the physical world. Digital twin will get the data from the WMS, SCADA, MES, or other control systems if these exist. The digital twin can directly issue instructions to the physical world via these systems in the normal operation of logistics. When abnormal encountered, the service layer of the digital twin will provide the decision and then issue the control commands. The digital twin needs to continuously detect the discrepancy between the physical world and cyberspace to iteratively update logistics digital twins powered by massive amounts of operational data.
At the planning and design stage, heuristic algorithms are often used because there is insufficient data. The traditional optimization method is challenging to solve the collaborative problem under dynamic interference in the operating stage. Moreover, more data can be collected in the operation stage, the data-driven method can be used to support real-time decision-making.
We can conclude that the architecture presented in this study is flexible, scalable, and adaptable. Firstly, the architecture adopts a modular approach to construct a digital twin cyber space, which can flexibly handle various tasks. When there are different tasks, the corresponding application or service will be requested, and this request will be further decomposed into simpler requests that are often directed to different micro-services. The micro-services will then call the corresponding model and data, so different tasks can be completed without redeployment. Secondly, the interfaces should worked in plug-in pattern. The interfaces in the interface layer are independent components. They could enhance or extend the core digital twin system with additional capabilities without the need of re-deployments. Thus, the architecture shows a highly scalability. Thirdly, self-learning, self-adaptation, and self-growing capability can be achieved based on machine intelligence. Machine learning, deep learning, and reinforcement learning approaches could highly facility the model evolution. In addition, the dynamic configuration module in digital mapping core also provides adaptability for digital twins based on the proposed architecture.
In our future work, we will extend the logistics digital twin architecture to the supply chain and consider the influence of market variability and changes on the logistics between the intelligent factory and the customer or supplier.
Footnotes
Handling Editor: Chenhui Liang
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is sponsored by the Basic Scientific Research Projects of State Administration of Science, Technology, and Industry for National Defense (JCKY2018210A001).
