Real-time scheduling methods are essential and critical to complex product flexible shop-floor due to the dynamic events in the production process, such as new job insertions, machine breakdowns and frequent rework. Recently, digital twin (DT) technology can help identify disturbances by continuously comparing physical space with virtual space, which enables real-time scheduling and greatly reduces the deviation between pre-schedule and actual schedule. However, the conventional scheduling models and algorithms cannot satisfy the adaptiveness and timeliness requirements of optimization in DT enabled shop-floor (DTS). To address above challenges, an overall framework of DT enabled real-time scheduling (DTE-RS) for complex product shop-floor is proposed to effectively reduce adverse impacts of the dynamic disturbances and minimize the makespan. Firstly, complex product flexible job shop scheduling problem (CPFJSP) is formulated as Markov Decision Process (MDP), taking into account machine breakdown and new job insertions. Then, deep Q-network (DQN) based solution is developed to achieve optimal task dispatching according to real-time production state. Finally, the case study for aircraft overhaul shop-floor is conducted to demonstrate effectiveness and feasibility of the proposed real-time scheduling method. Through experimental comparison, it is indicated that the proposed method could effectively respond to dynamic disturbances and outperform the dynamic scheduling method in terms of makespan.
HanYChenXXuM, et al. A multi-objective flexible job-shop cell scheduling problem with sequence-dependent family setup times and intercellular transportation by improved NSGA-II. Proc IMechE, Part B: J Engineering Manufacture2022; 236(5): 540–556.
2.
ZhuangCLiuJXiongH. Digital twin-based smart production management and control framework for the complex product assembly shop-floor. Int J Adv Manuf Technol2018; 96: 1149–1163.
3.
RuiZShilongWZheqiZ, et al. An ant colony algorithm for job shop scheduling problem with tool flow. Proc IMechE, Part B: J Engineering Manufacture2014; 228(8): 959–968.
4.
QiaoFLiuJMaY. Industrial big-data-driven and CPS-based adaptive production scheduling for smart manufacturing. Int J Prod Res2021; 59(23): 7139–7159.
5.
ZhangCGuPJiangP. Low-carbon scheduling and estimating for a flexible job shop based on carbon footprint and carbon efficiency of multi-job processing. Proc IMechE, Part B: J Engineering Manufacture2015; 229(2): 328–342.
6.
WangJLiuYRenS, et al. Evolutionary game based real-time scheduling for energy-efficient distributed and flexible job shop. J Clean Prod2021; 293: 126093.
7.
YangYHuangMYu WangZ, et al. Robust scheduling based on extreme learning machine for bi-objective flexible job-shop problems with machine breakdowns. Expert Syst Appl2020; 158: 113545.
8.
Scholz-ReiterBWindtKLiuH. A multiple-logistic-objective-optimized manufacturing planning and control system. Proc IMechE, Part B: J Engineering Manufacture2011; 225(4): 599–610.
9.
LuoS. Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning. Appl Soft Comput2020; 91: 106208.
10.
WardRSoulatiantorkPFinneranS, et al. Real-time vision-based multiple object tracking of a production process: Industrial digital twin case study. Proc IMechE, Part B: J Engineering Manufacture2021; 235(11): 1861–1872.
11.
ZhangJDengTJiangH, et al. Bi-level dynamic scheduling architecture based on service unit digital twin agents. J Manuf Syst2021; 60: 59–79.
12.
ZhuangCMiaoTLiuJ, et al. The connotation of digital twin, and the construction and application method of shop-floor digital twin. Robot Comput Integr Manuf2021; 68: 102075.
13.
WangLHuXWangY, et al. Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning. Comput Netw2021; 190: 107969.
14.
ShiueYRLeeKCSuCT. Real-time scheduling for a smart factory using a reinforcement learning approach. Comput Ind Eng2018; 125: 604–614.
15.
FangYPengCLouP, et al. Digital-twin-based job shop scheduling toward smart manufacturing. IEEE Trans Ind Inform2019; 15(12): 6425–6435.
16.
HuHJiaXHeQ, et al. Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.0. Comput Ind Eng2020; 149: 106749.
17.
HuLLiuZHuW, et al. Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network. J Manuf Syst2020; 55: 1–14.
18.
GrievesMVickersJ. Digital twin: mitigating unpredictable, undesirable emergent behavior in complex systems. In: KahlenJFlumerfeltSAlvesA (eds) Transdisciplinary perspectives on complex systems. Cham: Springer, 2017, pp.85–113.
19.
TaoFZhangMDigital twin shop-floor: a new shop-floor paradigm towards smart manufacturing. IEEE Access2017; 5: 20418–20427.
20.
LiuZChenWZhangC, et al. Intelligent scheduling of a feature-process-machine tool supernetwork based on digital twin workshop. J Manuf Syst2021; 58: 157–167.
NegriEPandhareVCattaneoL, et al. Field-synchronized Digital Twin framework for production scheduling with uncertainty. J Intell Manuf2021; 32(4): 1207–1228.
23.
VillalongaANegriEBiscardoG, et al. A decision-making framework for dynamic scheduling of cyber-physical production systems based on digital twins. Annu Rev Control2021; 51: 357–373.
24.
YanJLiuZZhangC, et al. Research on flexible job shop scheduling under finite transportation conditions for digital twin workshop. Robot Comput Integr Manuf2021; 72: 102198.
25.
ZhaoBGaoJChenK, et al. Two-generation Pareto ant colony algorithm for multi-objective job shop scheduling problem with alternative process plans and unrelated parallel machines. J Intell Manuf2018; 29(1): 93–108.
26.
HanZZhuYMaX, et al. Multiple rules with game theoretic analysis for flexible flow shop scheduling problem with component altering times. Int J Model Identification Control2016; 26(1): 1–18.
27.
SunDHHeWZhengLJ, et al. Scheduling flexible job shop problem subject to machine breakdown with game theory. Int J Prod Res2014; 52(13): 3858–3876.
28.
WangJZhangYLiuY, et al. Multiagent and bargaining-game-based real-time scheduling for Internet of things-enabled flexible job shop. IEEE Internet Things2019; 6(2): 2518–2531.
29.
JunSLeeSChunH. Learning dispatching rules using random forest in flexible job shop scheduling problems. Int J Prod Res2019; 57(10): 3290–3310.
30.
PriorePPonteBPuenteJ, et al. Learning-based scheduling of flexible manufacturing systems using ensemble methods. Comput Ind Eng2018; 126: 282–291.
31.
ZangZWangWSongY, et al. Hybrid deep neural network scheduler for job-shop problem based on convolution two-dimensional transformation. Comput Intell Neurosci2019; 2019: 7172842.
32.
SuCTShiueYR. Intelligent scheduling controller for shop floor control systems: a hybrid genetic algorithm/decision tree learning approach. Int J Prod Res2003; 41(12): 2619–2641.
33.
ArazOSalumL. A multi-criteria real-time control scheme based on neural networks and fuzzy inference for DRC manufacturing systems. Int J Prod Res2020; 48: 251–270.
34.
FourieCJ. Intelligent scheduling using a neural network model in conjunction with reinforcement learning. Proc IMechE, Part B: J Engineering Manufacture2005; 219(2): 229–235.
35.
HuHJiaXLiuK, et al. Self-adaptive traffic control model with behavior trees and reinforcement learning for AGV in Industry 4.0. IEEE Trans Ind Inform2021; 17(12): 7968–7979.
36.
WangYCUsherJM. Application of reinforcement learning for agent-based production scheduling. Eng Appl Artif Intell2005; 18(1): 73–82.
37.
ShahrabiJAdibiMAMahootchiM. A reinforcement learning approach to parameter estimation in dynamic job shop scheduling. Comput Ind Eng2017; 110: 75–82.
38.
WangYF. Adaptive job shop scheduling strategy based on weighted q-learning algorithm. J Intell Manuf2020; 31(2): 417–432.
39.
WanZLiHHeH, et al. Model-free real-time EV charging scheduling based on deep reinforcement learning. IEEE Trans Smart Grid2019; 10(5): 5246–5257.
40.
WaschneckBReichstallerABelznerL, et al. Optimization of global production scheduling with deep reinforcement learning. Procedia CIRP2018; 72(1): 1264–1269.
41.
LiuCLChangCCTsengCJ. Actor-critic Deep Reinforcement Learning for solving job shop scheduling problems. IEEE Access2020; 8: 71752–71762.
42.
LuoSZhangLFanY. Dynamic multi-objective scheduling for flexible job shop by deep reinforcement learning. Comput Ind Eng2021; 159: 107489.
43.
MokhtariHHasaniA. An energy-efficient multi-objective optimization for flexible job-shop scheduling problem. Comput Chem Eng2017; 104: 339–352.
44.
FanJShenWGaoL, et al. A hybrid Jaya algorithm for solving flexible job shop scheduling problem considering multiple critical paths. J Manuf Syst2021; 60: 298–311.
45.
LiKDengQZhangL, et al. An effective MCTS-based algorithm for minimizing makespan in dynamic flexible job shop scheduling problem. Comput Ind Eng2021; 155: 107211.
Al-ShayeaAFararahENasrEA, et al. Model for integrating production scheduling and maintenance planning of flow shop production system. IEEE Access2020; 8: 208826–208835.
48.
JongWRChenHTLinYH, et al. The multi-layered job-shop automatic scheduling system of mould manufacturing for Industry 3.5. Comput Ind Eng2020; 149: 106797.
49.
SongWChenXLiQ, et al. Flexible job shop scheduling via graph neural network and deep reinforcement learning. IEEE Trans Ind Inform2022; 1–11. DOI: 10.1109/TII.2022.3189725.
50.
CecilJAlbuhamoodSCecil-XavierA, et al. An advanced cyber physical framework for micro devices assembly. IEEE Trans Syst Man Cybern Syst2019; 49(1): 92–106.
