Accurately predicting the future trajectories of other traffic participants represents a critical capability for autonomous vehicles. Current methods lack a robust perception model capable of handling complex scenarios. To this end, this paper introduces Transense, which employs the Transformer architecture to analyze agents’ lane selection intentions. Through iterative refinement, Transense effectively identifies pivotal factors influencing agents’ lane choices. Furthermore, a segmented refinement approach is applied in the decoder phase, pre-decoding guiding trajectories and innovatively integrating an attention mechanism to facilitate interaction between the guiding trajectories and scene information, thereby enabling trajectory correction. This methodology preserves temporal information while enhancing prediction accuracy. Experimental validation on the nuScenes and Argoverse datasets demonstrates that our method outperforms existing approaches.
YuanYWengXOuY, et al. Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting. In: Proceedings of the IEEE/CVF international conference on computer vision, Virtual Online, 2021, pp.9813–9823.
2.
SalzmannTIvanovicBChakravartyP, et al. Trajectron++: dynamically-feasible trajectory forecasting with heterogeneous data. In: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16, 2020, pp.683–700. Springer International Publishing.
3.
GaoJSunCZhaoH, et al. Vectornet: encoding HD maps and agent dynamics from vectorized representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 2020, pp.11525–11533.
4.
ZhaoHGaoJLanT, et al. Tnt: target-driven trajectory prediction. In: Conference on robot learning, PMLR, London, United Kingdom, 2021, pp.895–904.
5.
DeoNWolffEBeijbomO. Multimodal trajectory prediction conditioned on lane-graph traversals. In: Conference on Robot Learning, PMLR, Auckland, New Zealand, 2022, pp.203–212.
6.
KimBDParkSHLeeS, et al. Lapred: lane-aware prediction of multi-modal future trajectories of dynamic agents. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, WA, USA, 2021, pp.14636–14645.
7.
RenSHeKGirshickR, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence2017; 39: 1137–1149.
8.
CaesarHBankitiVLangAH, et al. Nuscenes: a multimodal dataset for autonomous driving. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, WA, USA, 2020, pp.11621–11631.
9.
ChangMFLambertJSangkloyP, et al. Argoverse: 3D tracking and forecasting with rich maps. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach, CA, USA, 2019, pp.8748–8757.
10.
Phan-MinhTGrigoreECBoultonFA, et al. Covernet: multimodal behavior prediction using trajectory sets. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, WA, USA, 2020, pp.14074–14083.
11.
HuangZMoXLvC. Recoat: a deep learning-based framework for multi-modal motion prediction in autonomous driving application. In: 2022 IEEE 25th international conference on intelligent transportation systems (ITSC), Macau, China, 2022, pp.988–993. IEEE.
12.
CasasSLuoWUrtasunR.Intentnet: learning to predict intention from raw sensor data. In: Conference on Robot Learning, PMLR, Zürich, Switzerland, 2018, pp.947–956.
13.
ZengWLiangMLiaoR, et al. Lanercnn: distributed representations for graph-centric motion forecasting. In: 2021 IEEE/RSJ international conference on intelligent robots and systems (IROS), Prague, Czech Republic, 2021, pp.532–539. IEEE.
14.
HuangZMoXLvC. Multi-modal motion prediction with transformer-based neural network for autonomous driving. In: 2022 international conference on robotics and automation (ICRA), Philadelphia, PA, USA, 2022, pp.2605–2611. IEEE.
15.
GillesTSabatiniSTsishkouD, et al. Gohome: graph-oriented heatmap output for future motion estimation. In: 2022 international conference on robotics and automation (ICRA), Philadelphia, PA, USA, 2022, pp.9107–9114. IEEE.
16.
MoXHuangZXingY, et al. Multi-agent trajectory prediction with heterogeneous edge-enhanced graph attention network. IEEE Trans Intell Transp Syst2022; 23(7): 9554–9567.
17.
NgiamJCaineBVasudevanV, et al. Scene transformer: a unified architecture for predicting multiple agent trajectories. In: International Conference on Learning Representations, Virtual Online, 2022, pp. 23950–23974.
18.
LiuYZhangJFangL, et al. Multimodal motion prediction with stacked transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp.7577–7586.
19.
AlahiAGoelKRamanathanV, et al. Social LSTM: human trajectory prediction in crowded spaces. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, 2016, pp.961–971. DOI:10.1109/CVPR.2016.110.
20.
GuptaAJohnsonJFei-FeiL, et al. Social GAN: socially acceptable trajectories with generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 2018, pp.2255–2264.
21.
JiaXSunLTomizukaM, et al. Ide-net: interactive driving event and pattern extraction from human data. IEEE Robot Autom Lett2021; 6(2): 3065–3072.
22.
DaFZhangY. Path-aware graph attention for HD maps in motion prediction. In: 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA, 2022, pp.6430–6436. IEEE.
23.
LiangM, et al. Learning lane graph representations for motion forecasting. In: Computer vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. Springer International Publishing, 2020, pp.541–556.
24.
VaswaniAShazeerNParmarN, et al. Attention is all you need. Adv Neural Inf Process Syst2017; 30: 5998–6008.
25.
TangXKanMShanS, et al. Hpnet: dynamic trajectory forecasting with historical prediction attention. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, Washington, USA, 2024, pp.15261–15270.
26.
JiangZMaYShiB, et al. Social NSTransformers: low-quality pedestrian trajectory prediction. IEEE Transact Artif Intell2024; 5(11): 5575–5588.
27.
JiaXWuPChenL, et al. HDGT: heterogeneous driving graph transformer for multi-agent trajectory prediction via scene encoding. IEEE Trans Pattern Anal Mach Intell2023; 45: 13860–13875.
28.
SadeghianAKosarajuVSadeghianA, et al. Sophie: an attentive GAN for predicting paths compliant to social and physical constraints. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach, CA, USA, 2019, pp.1349–1358.
29.
CasasSGulinoCSuoS, et al. Implicit latent variable model for scene-consistent motion forecasting. In: Computer vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIII, Vol. 16. Springer International Publishing, 2020, pp.624–641.
30.
MercatJGillesTEl ZoghbyN, et al. Multi-head attention for multi-modal joint vehicle motion forecasting. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 2020, pp.9638–9644. IEEE.
31.
ZhouZWangJLiYH, et al. Query-centric trajectory prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 2023, pp.17863–17873.
32.
ShiSJiangLDaiD, et al. Motion transformer with global intention localization and local movement refinement. Adv Neural Inf Process Syst2022; 35: 6531–6543.
33.
WangXSuTDaF, et al. ProphNet: Efficient agent-centric motion forecasting with anchor-informed proposals. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Vancouver, BC, Canada, 2023, pp.21995–22003.
34.
ZhouZYeLWangJ, et al. HiVT: hierarchical vector transformer for multi-agent motion prediction. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2022, pp.8813–8823.
35.
GillesTSabatiniSTsishkouD, et al. THOMAS: trajectory heatmap output with learned multi-agent sampling. In: Proceedings of the International Conference on Learning Representations (ICLR), Virtual Online, 2022, pp.16005–16022.
36.
VeličkovićPCucurullGCasanovaA, et al. Graph attention networks. In Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, Canada, 2018, pp. 2920–2931.
37.
FantiniDGSilvaRNSiqueiraMBB, et al. Wind speed short-term prediction using recurrent neural network GRU model and stationary wavelet transform GRU hybrid model. Energy Convers Manag2024; 308: 118333.
38.
YaoYLuoZLiS, et al. Recurrent mvsnet for high-resolution multi-view stereo depth inference. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach, CA, USA, 2019.
39.
KipfTNWellingM.Semi-supervised classification with graph convolutional networks. In Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France, 2017, pp. 2713–2726.
40.
LiuJJHouQChengMM, et al. A simple pooling-based design for real-time salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach, CA, USA, 2019, pp. 3917–3926.
41.
LeeSPurushwalkam Shiva PrakashSCogswellM, et al. Stochastic multiple choice learning for training diverse deep ensembles. Adv Neural Inf Process Syst2016; 29: 2119–2127.
42.
WangMZhuXYuC, et al. Ganet: goal area network for motion forecasting. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK, 2023, pp.1609–1615. IEEE.
43.
GirgisRGolemoFCodevillaF, et al. Latent variable sequential set transformers for joint multi-agent motion prediction. In Proceedings of the International Conference on Learning Representations (ICLR), Virtual Online, 2022, pp. 24292–24318.
