CMDRL: A cavity-aware deep reinforcement learning framework with spatiotemporal attention for 3D bin Packing

Abstract

The three-dimensional bin packing problem (3D-BPP) is a classic optimization challenge in logistics. It aims to maximize space utilization and transportation efficiency by optimizing how items are arranged within containers. Conventional approaches typically rely on manually crafted heuristic rules; however, such rules often struggle to capture the rich three-dimensional spatial relationships between items and the container, leading to suboptimal packing solutions. Motivated by the success of deep reinforcement learning (DRL) in complex sequential decision-making, this paper proposes Cavity-Map-based Deep Reinforcement Learning (CMDRL) for 3D-BPP. First, we introduce a cavity-map representation of the packing state. By incorporating features such as the number of faces and proximity, the cavity map more precisely encodes 3D geometric relationships among items and between items and the container, addressing limitations in existing geometric representations. We then develop an enhanced spatiotemporal attention mechanism that dynamically fuses the temporal sequence of arriving items with the container’s evolving spatial layout, thereby improving both item selection and placement decisions. Experimental results demonstrate that our method consistently reduces gap ratios across diverse packing scenarios, while outperforming state-of-the-art DRL baselines in both efficiency and scalability. Ablation studies further confirm the contributions of the cavity map and the enhanced spatiotemporal attention mechanism, showing that their combination yields substantial improvements in packing performance. Overall, this work advances research on 3D packing and offers a practical solution for logistics and other complex spatial optimization problems.