Aiming at the Batch Scheduling Problem for Multi-Production Line Hybrid Flow Shop (BSP-MPLHFS) with batch constraints and functional differences among production lines, a hierarchical iterative optimization method integrating an Improved Genetic Algorithm (IGA) and Double Deep Q-Learning with Prioritized Sampling (PS-DDQN) is proposed. In the upper layer, IGA endowed with strong global search capability is employed to generate batching schemes, while non-uniform crossover and mutation operators are adopted to enhance its search ability and prevent it from falling into local optima. In the lower layer, the PS-DDQN model is invoked to provide production scheduling results for all sub-batches generated by the upper layer algorithm and provides it with fitness values simultaneously. The sub-batch scheduling problem handled by the PS-DDQN model is transformed into a Markov Decision Process (MDP). Seven features are designed to characterize the shop floor state, and 11 scheduling rules are constructed as the action set. To further improve learning efficiency and model stability, a prioritized sampling mechanism is introduced to dynamically adjust sampling probabilities based on the importance of samples in the experience replay pool. Finally, alternating iterations are performed to optimize both the batching schemes and scheduling results. Experimental validation is conducted using real-world data from the finishing shop of automobile wheel hub axle tubes in a manufacturing enterprise. The applicability and superiority of the proposed algorithm are demonstrated across four dimensions: batching method verification, performance experiment analysis, framework experiment validation, and solution efficiency analysis.
FangQChenYZhuangW, et al. Multi-line optimal scheduling research based on PSOGA hybrid optimization algorithm. Mod Manuf Eng2012; 5: 27–32. (in Chinese).
2.
YangZMaZWuS. Optimized flowshop scheduling of multiple production lines for precast production. Autom Constr2016; 72: 321–329.
3.
MaZYangZLiuS, et al. Optimized rescheduling of multiple production lines for flowshop production of reinforced precast concrete components. Autom Constr2018; 95: 86–97.
4.
ChenYZhangJLuJ. Modeling of scheduling for multiple assembly lines based on hybrid genetic-tabu search algorithm. J Zhejiang Univ Technol2013; 41(4): 355–359. (in Chinese).
5.
GuimarãesIFOuazeneYde SouzaMC, et al. Flowshop scheduling problem with parallel semi-lines and final synchronization operation. Comput Oper Res2019; 108: 121–133.
6.
BrammerJLutzBNeumannD. Permutation flow shop scheduling with multiple lines and demand plans using reinforcement learning. Eur J Oper Res2022; 299(1): 75–86.
7.
XuGGuanZPengK, et al. Collaborative scheduling of machining-assembly in complex multiple parallel production lines environment considering kitting constraints. Int J Ind Eng Comput2023; 14: 749–766.
8.
WangHZhaoDChenM, et al. Research on assignment optimization of PC components mixed-flow production based on MO-CGJAYA/D algorithm. Chin J Manag Sci2024; 32(7): 95–105. (in Chinese).
9.
LiCShenHLiL, et al. A batch splitting flexible job shop scheduling model for energy saving under alternative process plans. J Mech Eng2017; 53(5): 12–23. (in Chinese).
10.
ChenTLChengCYChouYH. Multi-objective genetic algorithm for energy-efficient hybrid flow shop scheduling with lot streaming. Ann Oper Res2020; 290: 813–836.
11.
ZhangWZhuangMTangH, et al. A hybrid particle swarm optimisation for flexible casting job shop scheduling problem with batch processing machine. IET Collab Intell Manuf2024; 6(4): e12117.
12.
TangHHuangJRenC, et al. Integrated scheduling of multi-objective lot-streaming hybrid flowshop with AGV based on deep reinforcement learning. Int J Prod Res2025; 63(4): 1275–1303.
13.
XueLZhaoSMahmoudiA, et al. Flexible job-shop scheduling problem with parallel batch machines based on an enhanced multi-population genetic algorithm. Complex Intell Syst2024; 10(3): 4083–4101.
14.
LiJGaoHShenN, et al. Research on batch scheduling of hybrid production line for machine tool precision spindle processing. J Mech Eng2021; 57(5): 185–195. (in Chinese).
15.
YanFChenHDingG, et al. Improved invasive weed algorithm based on double layer search framework for flexible batch scheduling problem. Comput Integr Manuf Syst2023; 29(2): 556. (in Chinese).
16.
ZahediZSalimAYusriskiR, et al. Optimization of an integrated batch production and maintenance scheduling on flow shop with two machines. Int J Ind Eng Comput2019; 10(2): 225–238.
17.
ZhuangHDengQLuoQ, et al. Modelling and optimization for integrated scheduling problem considering spare parts production, batch transportation and equipment operation. Expert Syst Appl2024; 252: 124194.
18.
BathrinathSSaravanasankarSMahapatraS, et al. An improved meta-heuristic approach for solving identical parallel processor scheduling problem. Proc Inst Mech Eng Part B: J Eng Manuf2016; 230(6): 1114–1126.
19.
HanYChenXXuM, et al. A multi-objective flexible job-shop cell scheduling problem with sequence-dependent family setup times and intercellular transportation by improved NSGA-II. Proc Inst Mech Eng Part B: J Eng Manuf2022; 236(5): 540–556.
20.
RamliANAb RashidMFF. A review of assembly line balancing optimisation with energy consideration using meta-heuristic algorithms. Proc Inst Mech Eng Part B: J Eng Manuf2022; 236(5): 475–485.
21.
ZhangZYinYFuY. An ensemble of brain storm optimization and Q-learning methods for distributed flexible job shop scheduling problems with distribution operations. Int J Gen Syst2024; 53(7–8): 863–897.
22.
ZhaoFLiuYXuT, et al. A reinforcement learning hyperheuristic algorithm for the distributed flowshops scheduling problem under consideration of emergency order insertion. Appl Soft Comput2024; 167: 112461.
23.
WangXLinZLeiR, et al. Brief review on applying reinforcement learning to job shop scheduling problems. J Syst Simul2022; 33(12): 2782–2791. (in Chinese).
24.
WangYF. Adaptive job shop scheduling strategy based on weighted Q-learning algorithm. J Intell Manuf2020; 31(2): 417–432.
25.
YangDShuXYuZ, et al. Dynamic flexible job shop scheduling based on deep reinforcement learning. Proc Inst Mech Eng, Part B: J Eng Manuf2025; 239(9): 1251–1264.
26.
KayhanBMYildizG. Reinforcement learning applications to machine scheduling problems: a comprehensive literature review. J Intell Manuf2023; 34(3): 905–929.
27.
ZhangXZhuGY. A literature review of reinforcement learning methods applied to job-shop scheduling problems. Comput Oper Res2025; 175: 106929.
28.
YuanEWangLChengS, et al. Solving flexible job shop scheduling problems via deep reinforcement learning. Expert Syst Appl2024; 245: 123019.
29.
YangBShiYWuZ. Q-learning based scheduling method for continuous pickling process of titanium strips. Proc Inst Mech Eng Part B: J Eng Manuf2025; 239(6–7): 810–820.
30.
BaoQZhengPDaiS. A digital twin-driven dynamic path planning approach for multiple automatic guided vehicles based on deep reinforcement learning. Proc Inst Mech Eng Part B: J Eng Manuf2024; 238(4): 488–499.
31.
HuHJiaXHeQ, et al. Deep reinforcement learning based AGVS real-time scheduling with mixed rule for flexible shop floor in industry 4.0. Comput Ind Eng2020; 149: 106749.
32.
QiaoDDuanLLiH, et al. Optimization of job shop scheduling problem based on deep reinforcement learning. Evol Intell2024; 17(1): 371–383.
33.
LuoS. Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning. Appl Soft Comput2020; 91: 106208.
34.
ZhaoYWangYTanY, et al. Dynamic jobshop scheduling algorithm based on deep Q network. IEEE Access2021; 9: 122995–123011.
35.
ChangXJiaXFuS, et al. Digital twin and deep reinforcement learning enabled real-time scheduling for complex product flexible shop-floor. Proc Inst Mech Eng Part B: J Eng Manuf2023; 237(8): 1254–1268.
36.
LiYGuWYuanM, et al. Real-time data-driven dynamic scheduling for flexible job shop with insufficient transportation resources using hybrid deep q network. Robot Comput Integr Manuf2022; 74: 102283.
37.
LuoSZhangLFanY. Dynamic multi-objective scheduling for flexible job shop by deep reinforcement learning. Comput Ind Eng2021; 159: 107489.
38.
ChangJYuDHuY, et al. Deep reinforcement learning for dynamic flexible job shop scheduling with random job arrival. Processes2022; 10(4): 760.
39.
LiuRPiplaniRToroC. A deep multi-agent reinforcement learning approach to solve dynamic job shop scheduling problem. Comput Oper Res2023; 159: 106294.
40.
LuSWangYKongM, et al. A double deep Q-network framework for a flexible job shop scheduling problem with dynamic job arrivals and urgent job insertions. Eng Appl Artif Intell2024; 133: 108487.
41.
YuanMLuSZhengL, et al. Distributed heterogeneous flexible job-shop scheduling problem considering automated guided vehicle transportation via improved deep q network. Swarm Evol Comput2025; 94: 101902.
42.
YuanMZhengLHuangH, et al. Research on flexible job shop scheduling problem with AGV using double DQN. J Intell Manuf2025; 36(1): 509–535.
43.
WangHSarkerBRLiJ, et al. Adaptive scheduling for assembly job shop with uncertain assembly times based on dual Q-learning. Int J Prod Res2021; 59(19): 5867–5883.
44.
YangSXuZWangJ. Intelligent decision-making of scheduling for dynamic permutation flowshop via deep reinforcement learning. Sensors2021; 21(3): 1019.
45.
MezianeMEATaghezoutN. A hybrid genetic algorithm with a neighborhood function for flexible job shop scheduling. Multiagent Grid Syst2018; 14(2): 161–175.
46.
SunYLiJFuX, et al. Application research based on improved genetic algorithm in cloud task scheduling. J Intell Fuzzy Syst2020; 38(1): 239–246.
47.
JingQWangQYuanB. Research on flexible job-shop scheduling based on improved genetic algorithm. Manuf Technol Mach Tool2024; 4: 167–172. (in Chinese).
48.
CaiJPengZDingS, et al. Problem-specific multi-objective invasive weed optimization algorithm for reconnaissance mission scheduling problem. Comput Ind Eng2021; 157: 107345.
49.
ShiueYRLeeKCSuCT. Real-time scheduling for a smart factory using a reinforcement learning approach. Comput Ind Eng2018; 125: 604–614.
50.
LiYCaoYGuoJ, et al. Flexible job shop scheduling considering peak power constraint. China Mech Eng2025; 36(2): 280–293. (in Chinese).
