The realization of universal robots is an ultimate goal of researchers. However, a key hurdle in achieving this goal lies in the robots’ ability to manipulate objects in their unstructured environments according to different tasks. The learning-based approach is considered an effective way to address generalization. The impressive performance of foundation models in the fields of computer vision and natural language suggests the potential of embedding foundation models into manipulation tasks as a viable path toward achieving general manipulation capability. However, we believe achieving general manipulation capability requires an overarching framework akin to auto driving. This framework should encompass multiple functional modules, with different foundation models assuming distinct roles in facilitating general manipulation capability. This survey focuses on the contributions of foundation models to robot learning for manipulation. We propose a comprehensive framework and detail how foundation models can address challenges in each module of the framework. What’s more, we examine current approaches, outline challenges, suggest future research directions, and identify potential risks associated with integrating foundation models into this domain.
AusserlechnerPHabergerDThalhammerS, et al. (2024) Zs6D: zero-shot 6D object pose estimation using vision transformers. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). Yokohama, Japan, 13–17 May 2024, IEEE, pp. 463–469.
6.
AustinJOdenaANyeM, et al. (2021) Program synthesis with large language models. arXiv preprint arXiv:2108.07732.
7.
BahlSMendoncaRChenL, et al. (2023) Affordances from human videos as a versatile representation for robotics. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023. pp. 13778–13790.
8.
BaiYWongLTwanT (2024) Survey on fundamental deep learning 3D reconstruction techniques. arXiv preprint arXiv:2407.08137.
9.
BainMNagraniAVarolG, et al. (2021) Frozen in time: a joint video and image encoder for end-to-end retrieval. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October, 2021. pp. 1728–1738.
10.
BelkhaleSDingTXiaoT, et al. (2024) RT-H: action hierarchies using language. arXiv preprint arXiv:2403.01823.
11.
BharadhwajHVakilJSharmaM, et al. (2023) Roboagent: generalization and efficiency in robot manipulation via semantic augmentations and action chunking. arXiv preprint arXiv:2309.01918.
12.
BhatejaCGuoDGhoshD, et al. (2023) Robotic offline RL from internet videos via value-function pre-training. arXiv preprint arXiv:2309.13041.
13.
BicchiA (2000) Hands for dexterous manipulation and robust grasping: a difficult road toward simplicity. IEEE Transactions on Robotics and Automation16(6): 652–662.
14.
BillardAKragicD (2019) Trends and challenges in robot manipulation. Science364(6446): eaat8414.
15.
BjorckJCastañedaFCherniadevN, et al. (2025) GR00t N1: an open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734.
16.
BlackKNakamotoMAtreyaP, et al. (2023) Zero-shot robotic manipulation with pretrained image-editing diffusion models. arXiv preprint arXiv:2310.10639.
17.
BlackKBrownNDriessD, et al. (2024) π_0: a vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164.
18.
BlattmannADockhornTKulalS, et al. (2023) Stable video diffusion: scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127.
19.
BohgJHausmanKSankaranB, et al. (2017) Interactive perception: leveraging action in perception and perception in action. IEEE Transactions on Robotics33(6): 1273–1291.
20.
BousmalisKTrigeorgisGSilbermanN, et al. (2016) Domain separation networks. Advances in Neural Information Processing Systems29: 343–351.
21.
BousmalisKVezzaniGRaoD, et al. (2023) Robocat: a self-improving foundation agent for robotic manipulation. arXiv preprint arXiv:2306.11706.
22.
BrohanABrownNCarbajalJ, et al. (2022) RT-1: robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817.
23.
BrohanABrownNCarbajalJ, et al. (2023) RT-2: vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818.
24.
BrooksTHolynskiAEfrosAA (2023) InstructPix2Pix: learning to follow image editing instructions. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023. pp. 18392–18402.
BrownTMannBRyderN, et al. (2020) Language models are few-shot learners. Advances in Neural Information Processing Systems33: 1877–1901.
27.
BuQCaiJChenL, et al. (2025) AgiBot world colosseo: a large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669.
28.
BuckerAFigueredoLHaddadinS, et al. (2023) Latte: language trajectory transformer. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), London, United Kingdom, 29 May–02 June, 2023, IEEE, pp. 7287–7294.
29.
BurdeVBenbihiABurgetP, et al. (2024) Comparative evaluation of 3D reconstruction methods for object pose estimation. arXiv preprint arXiv:2408.08234.
30.
ButimeJGutierrezICorzoLG, et al. (2006) 3D reconstruction methods, a survey. In: Proceedings of the First International Conference on Computer Vision Theory and Applications. INSTICC, pp. 457–463.
31.
CaiJHeYYuanW, et al. (2024) OV9D: open-vocabulary category-level 9D object pose and size estimation. arXiv preprint arXiv:2403.12396.
32.
CalliBSinghABruceJ, et al. (2017) Yale-CMU-berkeley dataset for robotic manipulation research. The International Journal of Robotics Research36(3): 261–268.
33.
CaraffaABoscainiDHamzaA, et al. (2024) Freeze: training-free zero-shot 6D pose estimation with geometric and vision foundation models. In: European Conference on Computer Vision. Springer, pp. 414–431.
34.
CaronMTouvronHMisraI, et al. (2021) Emerging properties in self-supervised vision transformers. In: 2021 Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October, 2021. pp. 9650–9660.
35.
CassinisRTampaliniF (2007) Amirolos an active marker internet-based robot localization system. Robotics and Autonomous Systems55(4): 306–315.
36.
ChabraRLenssenJEIlgE, et al. (2020) Deep local shapes: learning local SDF priors for detailed 3D reconstruction. In: Computer Vision–ECCV 2020: 16Th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16. Springer, pp. 608–625.
ChangAXFunkhouserTGuibasL, et al. (2015) Shapenet: an information-rich 3D model repository. arXiv preprint arXiv:1512.03012.
39.
CheangCLChenGJingY, et al. (2024) Gr-2: a generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06158.
40.
ChebotarYVuongQHausmanK, et al. (2023) Q-transformer: scalable offline reinforcement learning via autoregressive Q-functions. In: Conference on Robot Learning. PMLR, pp. 3909–3928.
41.
ChenJTHuangCM (2023) Forgetful large language models: lessons learned from using LLMs in robot programming. arXiv preprint arXiv:2310.06646.
42.
ChenCCulbertsonPLepertM, et al. (2021a) Trajectotree: trajectory optimization meets tree search for planning multi-contact dexterous manipulation. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 27 September–01 October, 2021, IEEE, pp. 8262–8268.
43.
ChenMTworekJJunH, et al. (2021b) Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
44.
ChenBXiaFIchterB, et al. (2023a) Open-vocabulary queryable scene representations for real world planning. In:2023 IEEE International Conference on Robotics and Automation (ICRA), London, United Kingdom, 29 May–02 June, 2023, IEEE, pp. 11509–11522.
45.
ChenXDjolongaJPadlewskiP, et al. (2023b) PaLI-X: on scaling up a multilingual vision and language model. arXiv preprint arXiv:2305.18565.
46.
ChenZKiamiSGuptaA, et al. (2023c) GenAug: retargeting behaviors to unseen situations via generative augmentation. arXiv preprint arXiv:2302.06671.
47.
ChenZZhangSLuoS, et al. (2023d) Tacchi: a pluggable and low computational cost elastomer deformation simulator for optical tactile sensors. IEEE Robotics and Automation Letters8(3): 1239–1246.
48.
ChenBChenZChenX, et al. (2024a) Teleoperation of an anthropomorphic robot hand with a metamorphic palm and tunable-stiffness soft fingers. Soft Robotics.
49.
ChenBXuZKirmaniS, et al. (2024b) SpatialVLM: endowing vision-language models with spatial reasoning capabilities. In: 2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June, 2024, pp. 14455–14465.
50.
ChenZWalsmanAMemmelM, et al. (2024c) URDformer: a pipeline for constructing articulated simulation environments from real-world images. arXiv preprint arXiv:2405.11656.
51.
ChengHKSchwingAG (2022) Xmem: long-term video object segmentation with an atkinson-shiffrin memory model. In: European Conference on Computer Vision. Springer, pp. 640–658.
52.
ChengXPatilSTemelZ, et al. (2023) Enhancing dexterity in robotic manipulation via hierarchical contact exploration. IEEE Robotics and Automation Letters9(1): 390–397.
53.
ChengXLiJYangS, et al. (2024) Open-television: teleoperation with immersive active visual feedback. arXiv preprint arXiv:2407.01512.
54.
ChiCFengSDuY, et al. (2023) Diffusion policy: visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04137.
55.
ChiCXuZPanC, et al. (2024) Universal manipulation interface: in-the-wild robot teaching without in-the-wild robots. arXiv preprint arXiv:2402.10329.
56.
CoumansEBaiY (2016) Pybullet, a python module for physics simulation for games. Robotics and Machine Learning.
57.
CrucianiSSmithCKragicD, et al. (2018) Dexterous manipulation graphs. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 01–05 October, 2018, IEEE, pp. 2040–2047.
58.
CuiYNiekumSGuptaA, et al. (2022) Can foundation models perform zero-shot task specification for robot manipulation? In: Learning for Dynamics and Control Conference. PMLR, pp. 893–905.
59.
CuiYKaramchetiSPalletiR, et al. (2023) No, to the right: online language corrections for robotic manipulation via shared autonomy. In: Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction,Stockholm, Sweden, 13-16 March, 2023, pp. 93–101.
60.
DaiQZhangJLiQ, et al. (2022) Domain randomization-enhanced depth simulation and restoration for perceiving and grasping specular and transparent objects. In: European Conference on Computer Vision. Springer, pp. 374–391.
61.
DaiQZhuYGengY, et al. (2023) GraspNeRF: multiview-based 6-DoF grasp detection for transparent and specular objects using generalizable NeRF. In:2023 IEEE International Conference on Robotics and Automation (ICRA), London, United Kingdom, 29 May–02 June, 2023, IEEE, pp. 1757–1763.
62.
DaiTWongJJiangY, et al. (2024) Automated creation of digital cousins for robust policy learning. arXiv preprint arXiv:2410.07408.
63.
DamenDDoughtyHFarinellaGM, et al. (2022) Rescaling egocentric vision: collection, pipeline and challenges for epic-kitchens-100. International Journal of Computer Vision130: 1–23.
64.
DarvishKPencoLRamosJ, et al. (2023) Teleoperation of humanoid robots: a survey. IEEE Transactions on Robotics39(3): 1706–1727.
De PaceFGorjupGBaiH, et al. (2021) Leveraging enhanced virtual reality methods and environments for efficient, intuitive, and immersive teleoperation of robots. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi'an, China, 30 May–05 June, 2021, IEEE, pp. 12967–12973.
67.
DehghaniMDjolongaJMustafaB, et al. (2023) Scaling vision transformers to 22 billion parameters. In: International Conference on Machine Learning. PMLR, pp. 7480–7512.
68.
DeitkeMVanderBiltEHerrastiA, et al. (2022) ProcTHOR: large-scale embodied ai using procedural generation. Advances in Neural Information Processing Systems35: 5982–5994.
69.
DeitkeMSchwenkDSalvadorJ, et al. (2023) Objaverse: a universe of annotated 3D objects. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June, 2023. pp. 13142–13153.
70.
DelPretoJLiuCLuoY, et al. (2022) Actionsense: a multimodal dataset and recording framework for human activities using wearable sensors in a kitchen environment. Advances in Neural Information Processing Systems35: 13800–13813.
71.
DengJDongWSocherR, et al. (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June, 2009, IEEE, pp. 248–255.
72.
DengXMousavianAXiangY, et al. (2021) PoseRBPF: a Rao–blackwellized particle filter for 6-D object pose tracking. IEEE Transactions on Robotics37(5): 1328–1342.
73.
DevlinJChangMWLeeK, et al. (2018) BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
74.
Di PaloNByravanAHasencleverL, et al. (2023) Towards a unified agent with foundation models. arXiv preprint arXiv:2307.09668.
75.
DingYZhangXAmiriS, et al. (2022) Robot task planning and situation handling in open worlds. arXiv preprint arXiv:2210.01287.
76.
DingYZhangXPaxtonC, et al. (2023) Task and motion planning with large language models for object rearrangement. arXiv preprint arXiv:2303.06247.
77.
DriessDXiaFSajjadiMS, et al. (2023) PaLM-E: an embodied multimodal language model. arXiv preprint arXiv:2303.03378.
78.
DuYYangSDaiB, et al. (2024) Learning universal policies via text-guided video generation. Advances in Neural Information Processing Systems36: 1–17.
79.
DuanJPumacayWKumarN, et al. (2024) Aha: a vision-language-model for detecting and reasoning over failures in robotic manipulation. arXiv preprint arXiv:2410.00371.
80.
EllisKWongLNyeM, et al. (2023) Dreamcoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences381(2251): 20220050.
81.
EysenbachBAsawaSChaudhariS, et al. (2020) Off-dynamics reinforcement learning: training for transfer with domain classifiers. arXiv preprint arXiv:2006.13916.
82.
EzeCCrickC (2024) Learning by watching: a review of video-based learning approaches for robot manipulation. arXiv preprint arXiv:2402.07127.
83.
FangBSunFLiuH, et al. (2017a) A novel data glove using inertial and magnetic sensors for motion capture and robotic arm-hand teleoperation. Industrial Robot: An International Journal44(2): 155–165.
84.
FangBSunFLiuH, et al. (2017b) Robotic teleoperation systems using a wearable multimodal fusion device. International Journal of Advanced Robotic Systems14(4): 1729881417717057.
85.
FangBMaXWangJ, et al. (2020a) Vision-based posture-consistent teleoperation of robotic arm using multi-stage deep neural network. Robotics and Autonomous Systems131: 103592.
86.
FangHSWangCGouM, et al. (2020b) Graspnet-1billion: a large-scale benchmark for general object grasping. In: 2020 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020, pp. 11444–11453.
87.
FangHFangHSXuS, et al. (2022) TransCG: a large-scale real-world dataset for transparent object depth completion and a grasping baseline. IEEE Robotics and Automation Letters7(3): 7383–7390.
88.
FangHFangHSWangY, et al. (2024) Airexo: low-cost exoskeletons for learning whole-arm manipulation in the wild. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). Yokohama, Japan, 13–17 May 2024, IEEE, pp. 15031–15038.
FirooziRTuckerJTianS, et al. (2023) Foundation models in robotics: applications, challenges, and the future. The International Journal of Robotics Research44: 701.
92.
FuKPengJHeQ, et al. (2021) Single image 3D object reconstruction based on deep learning: a review. Multimedia Tools and Applications80(1): 463–498.
93.
FuZZhaoQWuQ, et al. (2024) Humanplus: humanoid shadowing and imitation from humans. arXiv preprint arXiv:2406.10454.
94.
GaoJSarkarBXiaF, et al. (2023) Physically grounded vision-language models for robotic manipulation.
95.
GaoPGengSZhangR, et al. (2024) Clip-adapter: better vision-language models with feature adapters. International Journal of Computer Vision132(2): 581–595.
96.
GeYMacalusoALiLE, et al. (2023) Policy adaptation from foundation model feedback. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June, 2023. pp. 19059–19069.
97.
GengHXuHZhaoC, et al. (2023a) GAPartNet: cross-category domain-generalizable object perception and manipulation via generalizable and actionable parts. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June, 2023. pp. 7081–7091.
98.
GengYAnBGengH, et al. (2023b) RLAfford: end-to-end affordance learning for robotic manipulation. In:2023 IEEE International Conference on Robotics and Automation (ICRA), London, United Kingdom, 29 May–02 June, 2023, IEEE, pp. 5880–5886.
99.
GervetTXianZGkanatsiosN, et al. (2023) Act3D: 3D feature field transformers for multi-task robotic manipulation. In: 7th Annual Conference on Robot Learning, Atlanta, USA, 2023.
100.
GibsonJJ (2014) The Ecological Approach to Visual Perception: Classic Edition. Psychology press.
101.
GraumanKWestburyAByrneE, et al. (2022) Ego4D: around the world in 3,000 hours of egocentric video. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022. pp. 18995–19012.
102.
GraumanKWestburyATorresaniL, et al. (2024) Ego-Exo4D: understanding skilled human activity from first-and third-person perspectives. In: 2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June, 2024, pp. 19383–19400.
103.
GuXLinTYKuoW, et al. (2021) Open-vocabulary object detection via vision and language knowledge distillation. arXiv preprint arXiv:2104.13921.
104.
GuJXiangFLiX, et al. (2023) Maniskill2: a unified benchmark for generalizable manipulation skills. arXiv preprint arXiv:2302.04659.
105.
GuanJHaoYWuQ, et al. (2024) A survey of 6DoF object pose estimation methods for different application scenarios. Sensors24(4): 1076.
106.
GuoHWuFQinY, et al. (2023) Recent trends in task and motion planning for robotics: a survey. ACM Computing Surveys55(13s): 1–36.
107.
GuptaAYuJZhaoTZ, et al. (2021) Reset-free reinforcement learning via multi-task learning: learning dexterous manipulation behaviors without human intervention. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi'an, China, 30 May–05 June, 2021, IEEE, pp. 6664–6671.
108.
GuzeyIDaiYSavvaG, et al. (2024) Bridging the human to robot dexterity gap through object-oriented rewards. arXiv preprint arXiv:2410.23289.
109.
HaHSongS (2022) Semantic abstraction: open-world 3D scene understanding from 2D vision-language models. arXiv preprint arXiv:2207.11514.
110.
HaHFlorencePSongS (2023) Scaling up and distilling down: language-guided robot skill acquisition. In: Conference on Robot Learning. PMLR, pp. 3766–3777.
111.
HanKWangYChenH, et al. (2022) A survey on vision transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence45(1): 87–110.
112.
HaqueATancikMEfrosAA, et al. (2023) Instruct-NeRF2NeRF: editing 3D scenes with instructions. In: 2023 Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October, 2023, pp. 19740–19750.
113.
HavilandJCorkeP (2021) Neo: a novel expeditious optimisation algorithm for reactive motion control of manipulators. IEEE Robotics and Automation Letters6(2): 1043–1050.
114.
HeKChenXXieS, et al. (2022) Masked autoencoders are scalable vision learners. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022. pp. 16000–16009.
115.
HejnaJMirchandaniSBalakrishnaA, et al. (2025) Robot data curation with mutual information estimators. arXiv preprint arXiv:2502.08623.
116.
HerzogARaoKHausmanK, et al. (2023) Deep rl at scale: sorting waste in office buildings with a fleet of mobile manipulators.
117.
HoganFRRodriguezA (2020) Feedback control of the pusher-slider system: a story of hybrid and underactuated contact dynamicsAlgorithmic Foundations of Robotics XII: Proceedings of the Twelfth Workshop on the Algorithmic Foundations of Robotics. Springer, 800–815.
118.
HongYZhenHChenP, et al. (2023) 3d-llm: injecting the 3d world into large language models. Advances in Neural Information Processing Systems36: 20482–20494.
119.
HowellTGileadiNTunyasuvunakoolS, et al. (2022) Predictive sampling: real-time behaviour synthesis with mujoco. arXiv preprint arXiv:2212.00541.
120.
HuYLinFZhangT, et al. (2023a) Look before you leap: unveiling the power of gpt-4v in robotic vision-language planning. arXiv preprint arXiv:2311.17842.
121.
HuYXieQJainV, et al. (2023b) Toward general-purpose robots via foundation models: a survey and meta-analysis. arXiv preprint arXiv:2312.08782.
122.
HuYYangJChenL, et al. (2023c) Planning-oriented autonomous driving. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June, 2023. pp. 17853–17862.
123.
HuangEChengXMasonMT (2021) Efficient contact mode enumeration in 3D. In: Algorithmic Foundations of Robotics XIV: Proceedings of the Fourteenth Workshop on the Algorithmic Foundations of Robotics 14. Springer, pp. 485–501.
124.
HuangWAbbeelPPathakD, et al. (2022) Language models as zero-shot planners: extracting actionable knowledge for embodied agents. In: International Conference on Machine Learning. PMLR, pp. 9118–9147.
125.
HuangJYongSMaX, et al. (2023a) An embodied generalist agent in 3D world. arXiv preprint arXiv:2311.12871.
126.
HuangSJiangZDongH, et al. (2023b) Instruct2ACT: mapping multi-modality instructions to robotic actions with large language model. arXiv preprint arXiv:2305.11176.
127.
HuangWWangCZhangR, et al. (2023c) VoxPoser: composable 3D value maps for robotic manipulation with language models. arXiv preprint arXiv:2307.05973.
128.
HuangWXiaFShahD, et al. (2023d) Grounded decoding: guiding text generation with grounded models for robot control. arXiv preprint arXiv:2303.00855.
129.
HuangWWangCLiY, et al. (2024a) ReKep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652.
130.
HuangZWenYWangZ, et al. (2024b) Surface reconstruction from point clouds: a survey and a benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence.
131.
IssacJWüthrichMCifuentesCG, et al. (2016) Depth-based object tracking using a robust gaussian filter. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16–21 May, 2016, IEEE, 608–615.
132.
IyerAPengZDaiY, et al. (2024) Open teach: a versatile teleoperation system for robotic manipulation. arXiv preprint arXiv:2403.07870.
133.
JaegleAGimenoFBrockA, et al. (2021) Perceiver: general perception with iterative attention. In: International Conference on Machine Learning. PMLR, pp. 4651–4664.
134.
JainVAttarianMJoshiNJ, et al. (2024) Vid2Robot: end-to-end video-conditioned policy learning with cross-attention transformers. arXiv preprint arXiv:2403.12943.
135.
JamesSMaZArrojoDR, et al. (2020) RLBench: the robot learning benchmark & learning environment. IEEE Robotics and Automation Letters5(2): 3019–3026.
136.
JangEIrpanAKhansariM, et al. (2022) BC-Z: zero-shot task generalization with robotic imitation learning. In: Conference on Robot Learning. PMLR, pp. 991–1002.
137.
JansenPA (2020) Visually-grounded planning without vision: language models infer detailed plans from high-level instructions. arXiv preprint arXiv:2009.14259.
138.
JatavallabhulaKMKuwajerwalaAGuQ, et al. (2023) Conceptfusion: open-set multimodal 3D mapping. arXiv preprint arXiv:2302.07241.
139.
JiangZZhuYSvetlikM, et al. (2021) Synergies between affordance and geometry: 6-DoF grasp detection via implicit representations. arXiv preprint arXiv:2104.01542.
140.
JiangZHsuCCZhuY (2022) Ditto: building digital twins of articulated objects from interaction. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18-24 June 2022. pp. 5616–5626.
141.
JiangYGuptaAZhangZ, et al. (2023) VIMA: robot manipulation with multimodal prompts.
142.
JiangYWangCZhangR, et al. (2024a) TRANSIC: sim-to-real policy transfer by learning from online correction. arXiv preprint arXiv:2405.10315.
143.
JiangYYuMZhuX, et al. (2024b) Contact-implicit model predictive control for dexterous in-hand manipulation: a long-horizon and robust approach. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Abu Dhabi, United Arab Emirates, 14–18 October, 2024, IEEE, pp. 5260–5266.
144.
JiangZXieYLinK, et al. (2024c) DexMimicGen: automated data generation for bimanual dexterous manipulation via imitation learning. arXiv preprint arXiv:2410.24185.
145.
JinW (2024) Complementarity-free multi-contact modeling and optimization for dexterous manipulation. arXiv preprint arXiv:2408.07855.
146.
JinYLiDYongA, et al. (2024) RobotGPT: robot manipulation learning from chatgpt. IEEE Robotics and Automation Letters9: 2543.
147.
JingYZhuXLiuX, et al. (2023) Exploring visual pre-training for robot manipulation: datasets, models and methods. arXiv preprint arXiv:2308.03620.
148.
JoublinFCeravolaASmirnovP, et al. (2023) Copal: corrective planning of robot actions with large language models. arXiv preprint arXiv:2310.07263.
149.
JumperJEvansRPritzelA, et al. (2021) Highly accurate protein structure prediction with alphafold. Nature596(7873): 583–589.
150.
KannanSSVenkateshVLMinBC (2023) Smart-LLM: smart multi-agent robot task planning using large language models. arXiv preprint arXiv:2309.10062.
151.
KapelyukhIVosyliusVJohnsE (2023) DALL-E-Bot: introducing web-scale diffusion models to robotics. IEEE Robotics and Automation Letters8(7): 3956–3963.
152.
KaramchetiSNairSChenAS, et al. (2023) Language-driven representation learning for robotics. arXiv preprint arXiv:2302.12766.
153.
KarashchukPRuppKLDickinsonES, et al. (2021) Anipose: a toolkit for robust markerless 3D pose estimation. Cell Reports36(13): 109730.
154.
KareerSPatelDPunamiyaR, et al. (2024) Egomimic: scaling imitation learning via egocentric video. arXiv preprint arXiv:2410.24221.
155.
KeTWGkanatsiosNFragkiadakiK (2024) 3d diffuser actor: policy diffusion with 3d scene representations. arXiv preprint arXiv:2402.10885.
156.
KerblBKopanasGLeimkühlerT, et al. (2023) 3D gaussian splatting for real-time radiance field rendering. https://arxiv.org/abs/2308.04079
157.
KerrJKimCMGoldbergK, et al. (2023) LERF: language embedded radiance fields. In: 2023 Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October, 2023, pp. 19729–19739.
158.
KhadivarFBillardA (2023) Adaptive fingers coordination for robust grasp and in-hand manipulation under disturbances and unknown dynamics. IEEE Transactions on Robotics39(5): 3350–3367.
159.
KhanSNaseerMHayatM, et al. (2022) Transformers in vision: a survey. ACM Computing Surveys54(10s): 1–41.
160.
KhanMAKenneyMPainterJ, et al. (2023) Natural language robot programming: NLP integrated with autonomous robotic grasping. arXiv preprint arXiv:2304.02993.
161.
KhandelwalAWeihsLMottaghiR, et al. (2022) Simple but effective: clip embeddings for embodied ai. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022. pp. 14829–14838.
162.
KhannaMMaoYJiangH, et al. (2024) Habitat synthetic scenes dataset (HSSD-200): an analysis of 3D scene scale and realism tradeoffs for objectgoal navigation. In: 2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June, 2024, pp. 16384–16393.
163.
KhazatskyAPertschKNairS, et al. (2024) DROID: a large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945.
164.
KimGKangDKimJH, et al. (2023) Contact-implicit MPS: controlling diverse quadruped motions without pre-planned contact modes or trajectories. arXiv preprint arXiv:2312.08961: 29–84.
165.
KimMJPertschKKaramchetiS, et al. (2024) Openvla: an open-source vision-language-action model. arXiv preprint arXiv:2406.09246.
KirillovAMintunERaviN, et al. (2023) Segment anything. arXiv preprint arXiv:2304.02643.
168.
KleebergerKBormannRKrausW, et al. (2020) A survey on learning-based robotic grasping. Current Robotics Reports1: 239–249.
169.
KoberJBagnellJAPetersJ (2013) Reinforcement learning in robotics: a survey. The International Journal of Robotics Research32(11): 1238–1274.
170.
KokicMStorkJAHausteinJA, et al. (2017) Affordance detection for task-specific grasping using deep learning. In: 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), Birmingham, UK, 15–17 November, 2017, IEEE, pp. 91–98.
171.
KristinssonKDumontGA (1992) System identification and control using genetic algorithms. IEEE Transactions on Systems, Man, and Cybernetics22(5): 1033–1046.
172.
KroemerONiekumSKonidarisG (2021) A review of robot learning for manipulation: challenges, representations, and algorithms. Journal of Machine Learning Research22(1): 1395–1476.
173.
KulhanekJSattlerT (2024) NeRFbaselines: consistent and reproducible evaluation of novel view synthesis methods. arXiv preprint arXiv:2406.17345.
LabbéYManuelliLMousavianA, et al. (2022) MegaPose: 6D pose estimation of novel objects via render & compare. arXiv preprint arXiv:2212.06870.
176.
Le Cleac’hSHowellTAYangS, et al. (2024) Fast contact-implicit model predictive control. IEEE Transactions on Robotics40: 1617–1629.
177.
Le LidecQJalletWMontautL, et al. (2024) Contact models in robotics: a comparative analysis. IEEE Transactions on Robotics.
178.
LeeKHXiaoTLiA, et al. (2023) PI-QT-Opt: predictive information improves multi-task robotic reinforcement learning at scale. In: Conference on Robot Learning. PMLR, pp. 1696–1707.
LiYWangGJiX, et al. (2018) Deepim: deep iterative matching for 6D pose estimation. In: Proceedings of the European Conference on Computer Vision (ECCV),Munich, Germany, 8-14 September, 2018, pp. 683–698.
181.
LiSMaXLiangH, et al. (2019) Vision-based teleoperation of shadow dexterous hand using end-to-end deep neural network. In: 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May, 2019, IEEE, pp. 416–422.
182.
LiSJiangJRuppelP, et al. (2020) A mobile robot hand-arm teleoperation system by vision and IMU. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 24 October 2020–24 January 2021, IEEE, pp. 10900–10906.
183.
LiCXiaFMartín-MartínR, et al. (2021) iGibson 2.0: object-centric simulation for robot learning of everyday household tasks. arXiv preprint arXiv:2108.03272.
184.
LiJGaoWWuY, et al. (2022) High-quality indoor scene 3D reconstruction with RGB-D cameras: a brief review. Computational Visual Media8(3): 369–393.
185.
LiCZhangRWongJ, et al. (2023a) Behavior-1K: a benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In: Conference on Robot Learning. PMLR, pp. 80–93.
186.
LiFVutukurSRYuH, et al. (2023b) NeRF-pose: a first-reconstruct-then-regress approach for weakly-supervised 6D object pose estimation. In: 2023 Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October, 2023, pp. 2123–2133.
187.
LiJGaoQJohnstonM, et al. (2023c) Mastering robot manipulation with multimodal prompts through pretraining and multi-task fine-tuning. arXiv preprint arXiv:2310.09676.
188.
LiXLiuMZhangH, et al. (2023d) Vision-language foundation models as effective robot imitators. arXiv preprint arXiv:2311.01378.
189.
LiangJHuangWXiaF, et al. (2023) Code as policies: language model programs for embodied control. In:2023 IEEE International Conference on Robotics and Automation (ICRA), London, United Kingdom, 29 May–02 June, 2023, IEEE, pp. 9493–9500.
190.
LiaoGZhouKBaoZ, et al. (2024) OV-NeRF: open-vocabulary neural radiance fields with vision and language foundation models for 3d semantic understanding. IEEE Transactions on Circuits and Systems for Video Technology34: 12923.
191.
LinYTremblayJTyreeS, et al. (2022) Keypoint-based category-level object pose tracking from an rgb sequence with uncertainty estimation. In: 2022 International Conference on Robotics and Automation (ICRA),Philadelphia, PA, USA, 23-27 May, 2022, IEEE, pp. 1258–1264.
192.
LinKAgiaCMigimatsuT, et al. (2023) Text2Motion: from natural language instructions to feasible plans. arXiv preprint arXiv:2303.12153.
193.
LinFHuYShengP, et al. (2024a) Data scaling laws in imitation learning for robotic manipulation. arXiv preprint arXiv:2410.18647.
194.
LinJLiuLLuD, et al. (2024b) SAM-6D: segment anything model meets zero-shot 6D object pose estimation. In: 2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June, 2024, pp. 27906–27916.
195.
LiuSZhangXZhangZ, et al. (2021) Editing conditional radiance fields. In: 2023 Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October, 2023, pp. 5773–5783.
196.
LiuHNasirianySZhangL, et al. (2022a) Robot learning on the job: human-in-the-loop autonomy and learning during deployment. The International Journal of Robotics Research44: 1727.
197.
LiuLXuWFuH, et al. (2022b) AKB-48: a real-world articulated object knowledge base. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022. pp. 14809–14818.
198.
LiuYHuangBZhuZ, et al. (2024e) Learning world models with identifiable factorization. Advances in Neural Information Processing Systems36: 1–34.
199.
LiuBJiangYZhangX, et al. (2023a) LLM+ P: empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477.
200.
LiuJMahdavi-AmiriASavvaM (2023b) Paris: part-level reconstruction and motion analysis for articulated objects. In: 2023 Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October, 2023, pp. 352–363.
201.
LiuKZhanFZhangJ, et al. (2023c) Weakly supervised 3D open-vocabulary segmentation. Advances in Neural Information Processing Systems36: 53433–53456.
202.
LiuMZhuYCaiH, et al. (2023d) PartSLIP: low-shot part segmentation for 3D point clouds via pretrained image-language models. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June, 2023. pp. 21736–21746.
203.
LiuRWuRVan HoorickB, et al. (2023e) Zero-1-to-3: zero-shot one image to 3D object. In: 2023 Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October, 2023, pp. 9298–9309.
204.
LiuSZengZRenT, et al. (2023f) Grounding DINO: marrying DINO with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499.
205.
LiuIArthurCHeS, et al. (2024b) VoxAct-B: voxel-based acting and stabilizing policy for bimanual manipulation. arXiv preprint arXiv:2407.04152.
206.
LiuJSunWYangH, et al. (2024c) Deep learning-based object pose estimation: a comprehensive survey. arXiv preprint arXiv:2405.07801.
207.
LiuSWuLLiB, et al. (2024d) RDT-1B: a diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07864.
208.
LiuBZhuYGaoC, et al. (2024a) Libero: benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems36: 1–16.
209.
LuGZhangSWangZ, et al. (2024) Manigaussian: dynamic gaussian splatting for multi-task robotic manipulation. In: European Conference on Computer Vision. Springer, pp. 349–366.
210.
LuoJHuZXuC, et al. (2024) SERL: a software suite for sample-efficient robotic reinforcement learning. arXiv preprint arXiv:2401.16013.
211.
LynchCWahidATompsonJ, et al. (2023) Interactive language: talking to robots in real time. IEEE Robotics and Automation Letters.
212.
LyuJBaiCYangJ, et al. (2024) Cross-domain policy adaptation by capturing representation mismatch. arXiv preprint arXiv:2405.15369.
213.
MaYJSodhaniSJayaramanD, et al. (2022) VIP: towards universal visual reward and representation via value-implicit pre-training. arXiv preprint arXiv:2210.00030.
214.
MaYJKumarVZhangA, et al. (2023a) LIV: language-image representations and rewards for robotic control. In: International Conference on Machine Learning. PMLR, 23301–23320.
215.
MaYJLiangWWangG, et al. (2023b) Eureka: human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931.
216.
MaYJLiangWWangHJ, et al. (2024) Dreureka: language model guided sim-to-real transfer. arXiv preprint arXiv:2406.01967.
217.
MajumdarAYadavKArnaudS, et al. (2023) Where are we in the search for an artificial visual cortex for embodied intelligence?arXiv preprint arXiv:2303.18240.
218.
MakenPGuptaA (2023) 2d-to-3d: a review for computational 3d image reconstruction from x-ray images. Archives of Computational Methods in Engineering30(1): 85–114.
219.
MandiZBharadhwajHMoensV, et al. (2022) Cacti: a framework for scalable multi-task multi-scene visual imitation learning. arXiv preprint arXiv:2212.05711.
220.
MandiZWengYBauerD, et al. (2024) Real2code: reconstruct articulated objects via code generation. arXiv preprint arXiv:2406.08474.
221.
MandlekarANasirianySWenB, et al. (2023) Mimicgen: a data generation system for scalable robot learning using human demonstrations. arXiv preprint arXiv:2310.17596.
222.
MatasJJamesSDavisonAJ (2018) Sim-to-real reinforcement learning for deformable object manipulationConference on Robot Learning. PMLR, 734–743.
223.
McCarthyRTanDCSchmidtD, et al. (2024) Towards generalist robot learning from internet video: a survey. arXiv preprint arXiv:2404.19664.
224.
MeesOHermannLRosete-BeasE, et al. (2022) Calvin: a benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks. IEEE Robotics and Automation Letters7(3): 7327–7334.
225.
MendoncaRBahlSPathakD (2023) Structured world models from human videos. arXiv preprint arXiv:2308.10901.
226.
MiechAZhukovDAlayracJB, et al. (2019) Howto100m: learning a text-video embedding by watching hundred million narrated video clipsProceedings of the IEEE/CVF International Conference on Computer Vision, 2630–2640.
227.
MildenhallBSrinivasanPPTancikM, et al. (2021) Nerf: representing scenes as neural radiance fields for view synthesis. Communications of the ACM65(1): 99–106.
228.
MindererMGritsenkoAStoneA, et al. (2022) Simple open-vocabulary object detectionEuropean Conference on Computer Vision. Springer, 728–755.
229.
MirjaliliRKrawezMSilenziS, et al. (2023) Lan-grasp: using large language models for semantic object grasping. arXiv preprint arXiv:2310.05239.
230.
MoYZhangHKongT (2023) Towards open-world interactive disambiguation for robotic grasping. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), London, United Kingdom, 29 May–02 June, 2023, IEEE, pp. 8061–8067.
231.
MoonSSonHHurD, et al. (2024) Genflow: generalizable recurrent flow for 6d pose refinement of novel objects. In: 2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June, 2024, pp. 10039–10049.
NagraniASeoPHSeyboldB, et al. (2022) Learning audio-video modalities from image captions. In: European Conference on Computer Vision. Springer, pp. 407–426.
234.
NairSRajeswaranAKumarV, et al. (2022) R3M: a universal visual representation for robot manipulation. arXiv preprint arXiv:2203.12601.
235.
NakanishiJItaderaSAoyamaT, et al. (2020) Towards the development of an intuitive teleoperation system for human support robot using a vr device. Advanced Robotics34(19): 1239–1253.
236.
NanKXieRZhouP, et al. (2024) OpenVid-1M: a large-scale high-quality dataset for text-to-video generation. arXiv preprint arXiv:2407.02371.
237.
NasirianySMaddukuriAZhangL, et al. (2024) Robocasa: large-scale simulation of everyday tasks for generalist robots. arXiv preprint arXiv:2406.02523.
NguyenTVuMNHuangB, et al. (2023) Language-conditioned affordance-pose detection in 3D point clouds. arXiv preprint arXiv:2309.10911.
240.
NguyenVNGroueixTSalzmannM, et al. (2024) GigaPose: fast and robust novel object pose estimation via one correspondence. In: 2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June, 2024, pp. 9903–9913.
241.
OkamuraAMSmabyNCutkoskyMR (2000) An overview of dexterous manipulation. In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), San Francisco, CA, USA, 24–28 April, 2000, IEEE, Vol. 1, 255–262.
242.
ÖnolAÖLongPPadırT (2019) Contact-implicit trajectory optimization based on a variable smooth contact model and successive convexification. In: 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May, 2019, IEEE, pp. 2447–2453.
243.
PadalkarAPooleyAJainA, et al. (2023) Open X-embodiment: robotic learning datasets and RT-X models. arXiv preprint arXiv:2310.08864.
244.
PangT (2023) Planning, Sensing, and Control for Contact-Rich Robotic Manipulation With Quasi-Static Contact Models. Massachusetts Institute of Technology.
245.
PangTTedrakeR (2021) A convex quasistatic time-stepping scheme for rigid multibody systems with contact and friction. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi'an, China, 30 May–05 June, 2021, IEEE, pp. 6614–6620.
246.
PangTSuhHTYangL, et al. (2023) Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models. IEEE Transactions on Robotics39: 4691.
247.
PatelDEghbalzadehHKamraN, et al. (2023) Pretrained language models as visual planners for human assistance. arXiv preprint arXiv:2304.09179.
248.
PavlakosGShanDRadosavovicI, et al. (2024) Reconstructing hands in 3D with transformers. In: 2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June, 2024, pp. 9826–9836.
249.
PertschKStachowiczKIchterB, et al. (2025) Fast: efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747.
250.
PoiesiFBoscainiD (2022) Learning general and distinctive 3D local deep descriptors for point cloud registration. IEEE Transactions on Pattern Analysis and Machine Intelligence45(3): 3979–3985.
251.
PuigXRaKBobenM, et al. (2018) Virtualhome: simulating household activities via programs. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition,CVPR 2018, Salt Lake City, UT, USA, June, 18-22, 2018, pp. 8494–8502.
252.
QiCRYiLSuH, et al. (2017) Pointnet++: deep hierarchical feature learning on point sets in a metric space. Advances in Neural Information Processing Systems30: 5099–5108.
253.
QianGLiYPengH, et al. (2022) Pointnext: revisiting pointnet++ with improved training and scaling strategies. Advances in Neural Information Processing Systems35: 23192–23204.
254.
QinYWuYHLiuS, et al. (2022) Dexmv: imitation learning for dexterous manipulation from human videos. In: European Conference on Computer Vision. Springer, pp. 570–587.
255.
RadfordAKimJWHallacyC, et al. (2021) Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning. PMLR, pp. 8748–8763.
256.
RadosavovicIXiaoTJamesS, et al. (2023) Real-world robot learning with masked visual pre-training. In: Conference on Robot Learning. PMLR, pp. 416–426.
257.
RameshAPavlovMGohG, et al. (2021) Zero-shot text-to-image generation. In: International Conference on Machine Learning. PMLR, pp. 8821–8831.
258.
RamosFPossasRCFoxD (2019) Bayessim: adaptive domain randomization via probabilistic inference for robotics simulators. arXiv preprint arXiv:1906.01728.
259.
RanaKLeeRPershouseD, et al. (2025) IMLE policy: fast and sample efficient visuomotor policy learning via implicit maximum likelihood estimation. arXiv preprint arXiv:2502.12371.
260.
ReedSZolnaKParisottoE, et al. (2022) A generalist agent. arXiv preprint arXiv:2205.06175.
261.
RenAZDixitABodrovaA, et al. (2023a) Robots that ask for help: uncertainty alignment for large language model planners. arXiv preprint arXiv:2307.01928.
262.
RenAZGovilBYangTY, et al. (2023b) Leveraging language for accelerated learning of tool manipulation. In: Conference on Robot Learning. PMLR, pp. 1531–1541.
263.
RohmerESinghSPFreeseM (2013) V-REP: a versatile and scalable robot simulation framework. In: Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, November 2013,IEEE, pp. 1321–1326.
264.
RombachRBlattmannALorenzD, et al. (2022) High-resolution image synthesis with latent diffusion models. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022. pp. 10684–10695.
265.
RuizNLiYJampaniV, et al. (2023) Dreambooth: fine tuning text-to-image diffusion models for subject-driven generation. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June, 2023. pp. 22500–22510.
266.
RusuAAVečeríkMRothörlT, et al. (2017) Sim-to-real robot learning from pixels with progressive nets. In: Conference on Robot Learning. PMLR, pp. 262–270.
267.
SajjanSMooreMPanM, et al(2020) Clear Grasp: 3D shape estimation of transparent objects for manipulation. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, May 31 - June 15 2020,IEEE, 3634–3642.
268.
SchwarzKLiaoYNiemeyerM, et al. (2020) Graf: generative radiance fields for 3d-aware image synthesis. Advances in Neural Information Processing Systems33: 20154–20166.
269.
SekerMYKroemerO (2024) Estimating material properties of interacting objects using sum-gp-ucb. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). Yokohama, Japan, 13–17 May, 2024, IEEE, pp. 16684–16690.
270.
SermanetPDingTZhaoJ, et al. (2023) RoboCQA: multimodal long-horizon reasoning for robotics. arXiv preprint arXiv:2311.00899.
271.
SermanetPDingTZhaoJ, et al. (2024) Robovqa: multimodal long-horizon reasoning for robotics. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). Yokohama, Japan, 13-17 May 2024, IEEE, pp. 645–652.
272.
ShafiullahNMMPaxtonCPintoL, et al. (2022) Clip-fields: weakly supervised semantic fields for robotic memory. arXiv preprint arXiv:2210.05663.
ShanDGengJShuM, et al. (2020) Understanding human hands in contact at internet scale. In: 2020 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June, 2020, pp. 9869–9878.
275.
ShawKBahlSPathakD (2023) Videodex: learning dexterity from internet videos. In: Conference on Robot Learning. PMLR, pp. 654–665.
ShiJJinYLiD, et al. (2024) ASGrasp: generalizable transparent object reconstruction and grasping from rgb-d active stereo camera. arXiv preprint arXiv:2405.05648.
278.
ShiraiYZhaoTSuhH, et al. (2024) Is linear feedback on smoothed dynamics sufficient for stabilizing contact-rich plans?arXiv preprint arXiv:2411.06542.
279.
ShridharMManuelliLFoxD (2021) Cliport: what and where pathways for robotic manipulation.
280.
ShridharMManuelliLFoxD (2023) Perceiver-actor: a multi-task transformer for robotic manipulation. In: Conference on Robot Learning. PMLR, pp. 785–799.
281.
ShvetsovaNKuklevaAHongX, et al. (2025) Howtocaption: prompting llms to transform video annotations at scale. In: European Conference on Computer Vision. Springer, pp. 1–18.
282.
SilvaAJRamirezOADVegaVP, et al. (2009) Phantom omni haptic device: kinematic and manipulability. In: Proceedings of the 2009 Electronics, Robotics and Automotive Mechanics Conference (CERMA), Cuernavaca, Mexico, 25 September2009, IEEE, pp. 193–198.
283.
SilverTHariprasadVShuttleworthRS, et al. (2022) PDDL planning with pretrained large language models. In: NeurIPS 2022 Foundation Models for Decision Making Workshop, New Orleans, LA, USA, 3 December 2022.
284.
SinghIBlukisVMousavianA, et al. (2023) ProgPrompt: generating situated robot task plans using large language models. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), London, United Kingdom, 29 May–02 June, 2023, IEEE, pp. 11523–11530.
285.
SmithLDhawanNZhangM, et al. (2019) AVIDL: learning multi-stage tasks via pixel-level translation of human videos. arXiv preprint arXiv:1912.04443.
286.
SongCHWuJWashingtonC, et al. (2023) LLM-planner: few-shot grounded planning for embodied agents with large language models. In: 2023 Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October, 2023, pp. 2998–3009.
287.
StansfieldSA (1991) Robotic grasping of unknown objects: a knowledge-based approach. The International Journal of Robotics Research10(4): 314–326.
288.
StellaFDella SantinaCHughesJ (2023) How can LLMs transform the robotic design process?Nature Machine Intelligence5: 1–4.
289.
StoiberMSundermeyerMTriebelR (2022) Iterative corresponding geometry: fusing region and depth for highly efficient 3D tracking of textureless objects. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022. pp. 6855–6865.
290.
StoneAXiaoTLuY, et al. (2023) Open-world object manipulation using pre-trained vision-language models. arXiv preprint arXiv:2303.00905.
291.
SuhHJTPangTTedrakeR (2022) Bundled gradients through contact via randomized smoothing. IEEE Robotics and Automation Letters7(2): 4000–4007.
292.
SuhHTSimchowitzMPangT, et al. (2023) How does noising data affect learned contact dynamics?
293.
SunCSunMChenHT (2022) Direct voxel grid optimization: super-fast convergence for radiance fields reconstruction. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022. pp. 5459–5469.
294.
SundaresanPAntonovaRBohglJ (2022) DiffCloud: real-to-sim from point clouds with differentiable simulation and rendering of deformable objects. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),Kyoto, Japan, 23-27 October, 2022, IEEE, pp. 10828–10835.
295.
TanCSunFKongT, et al. (2018) A survey on deep transfer learning. In: Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part III 27. Springer, pp. 270–279.
296.
TangCHuangDGeW, et al. (2023) Graspgpt: leveraging semantic knowledge from a large language model for task-oriented grasping. IEEE Robotics and Automation Letters.
297.
TangYHuangWWangY, et al. (2025) UAD: unsupervised affordance distillation for generalization in robotic manipulation. arXiv preprint arXiv:2506.09284.
298.
TatiyaGFrancisJWuHH, et al. (2023) MOSAIC: learning unified multi-sensory object property representations for robot perception. arXiv preprint arXiv:2309.08508.
299.
TeamOMGhoshDWalkeH, et al. (2024) Octo: an open-source generalist robot policy. arXiv preprint arXiv:2405.12213.
TodorovE (2014) Convex and analytically-invertible dynamics with contacts and constraints: theory and implementation in mujoco. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May - 5 Jun, 2014, IEEE, pp. 6054–6061.
302.
TodorovEErezTTassaY (2012) Mujoco: a physics engine for model-based control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal, 07-12 October 2012,IEEE, pp. 5026–5033.
303.
TorneMSimeonovALiZ, et al. (2024) Reconciling reality through simulation: a real-to-sim-to-real approach for robust manipulation. arXiv preprint arXiv:2403.03949.
304.
TrivediDZhangJSunSH, et al. (2021) Learning to synthesize programs as interpretable and generalizable policies. Advances in Neural Information Processing Systems34: 25146–25163.
305.
ValmeekamKMarquezMOlmoA, et al. (2023) Planbench: an extensible benchmark for evaluating large language models on planning and reasoning about change. In: Thirty-Seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, New Orleans, LA, USA, 10-16 December, 2023.
306.
Van Den OordAVinyalsOKavukcuogluK (2017) Neural discrete representation learning. Advances in Neural Information Processing Systems30: 6306–6315.
307.
VempralaSBonattiRBuckerA, et al. (2023) ChatGPT for robotics: design principles and model abilities. Microsoft Autonomous Systems and Robotics Research2: 20.
308.
WangHSridharSHuangJ, et al. (2019) Normalized object coordinate space for category-level 6D object pose and size estimation. In: 2020 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020, pp. 2642–2651.
309.
WangRDuSSYangL, et al. (2020b) Is long horizon RL more difficult than short horizon RL?Advances in Neural Information Processing Systems33: 9075–9085.
310.
WangCChaiMHeM, et al. (2022a) Clip-NeRF: text-and-image driven manipulation of neural radiance fields. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022. pp. 3835–3844.
311.
WangC.JiK.GengJ., et al. (2024) Imperative learning: A self-supervised neuro-symbolic learning framework for robot autonomy. The International Journal of Robotics Research02783649251353181.
312.
WangDKohlerCZhuX, et al. (2022b) BulletArm: an open-source robotic manipulation benchmark and learning framework. In: The International Symposium of Robotics Research. Springer, pp. 335–350.
313.
WangCFanLSunJ, et al. (2023a) Mimicplay: long-horizon imitation learning by watching human play. arXiv preprint arXiv:2302.12422.
314.
WangFChenZWangG, et al. (2023b) Masked space-time hash encoding for efficient dynamic scene reconstruction. Advances in Neural Information Processing Systems36: 70497–70510.
315.
WangLLingYYuanZ, et al. (2023c) GenSim: generating robotic simulation tasks via large language models. arXiv preprint arXiv:2310.01361.
316.
WangYHeYLiY, et al. (2023d) InternVid: a large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942.
317.
WangYXianZChenF, et al. (2023e) Robogen: towards unleashing infinite data for automated robot learning via generative simulation. arXiv preprint arXiv:2311.01455.
318.
WangBZhangJDongS, et al. (2024a) VLM see, robot do: human demo video to robot action plan via vision language model. arXiv preprint arXiv:2410.08792.
319.
WangCMartín-MartínRXuD, et al. (2020a) 6-PACK: category-level 6D pose tracker with anchor-based keypoints. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 31 May - 31 August 2020,IEEE, pp. 10059–10066.
320.
WangCShiHWangW, et al. (2024b) DexCap: scalable and portable mocap data collection system for dexterous manipulation. arXiv preprint arXiv:2403.07788.
321.
WangLChenXZhaoJ, et al. (2024c) Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers. arXiv preprint arXiv:2409.20537.
322.
WangZLorraineJWangY, et al. (2024d) LLaMA-mesh: unifying 3D mesh generation with language models. arXiv preprint arXiv:2411.09595.
323.
WenBBekrisK (2021) Bundletrack: 6D pose tracking for novel objects without instance or category-level 3d models. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 27 September–01 October, 2021, IEEE, pp. 8067–8074.
324.
WenCZhangYLiZ, et al. (2019) Pixel2Mesh++: multi-view 3D mesh generation via deformation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, South Korea,Oct. 27- Nov. 22019, pp. 1042–1051.
325.
WenBMitashCRenB, et al. (2020) se (3)-tracknet: data-driven 6d pose tracking by calibrating image residuals in synthetic domains. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 24 October 2020–24 January 2021, IEEE, pp. 10367–10373.
326.
WenBTremblayJBlukisV, et al. (2023a) BundleSDF: neural 6-DoF tracking and 3D reconstruction of unknown objects. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023. pp. 606–617.
327.
WenCLinXSoJ, et al. (2023b) Any-point trajectory modeling for policy learning. arXiv preprint arXiv:2401.00025.
328.
WenBYangWKautzJ, et al. (2024a) Foundationpose: unified 6D pose estimation and tracking of novel objects. In: 2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June, 2024, pp. 17868–17879.
329.
WenJZhuYLiJ, et al. (2024b) TinyVLA: towards fast, data-efficient vision-language-action models for robotic manipulation. arXiv preprint arXiv:2409.12514.
WuRZhaoYMoK, et al. (2021) Vat-mart: learning visual action trajectory proposals for manipulating 3d articulated objects. arXiv preprint arXiv:2106.14440.
332.
WuHJingYCheangC, et al. (2023a) Unleashing large-scale video generative pre-training for visual robot manipulation. arXiv preprint arXiv:2312.13139.
333.
WuJAntonovaRKanA, et al. (2023b) TidyBot: personalized robot assistance with large language models. arXiv preprint arXiv:2305.05658.
334.
WuPShentuYYiZ, et al. (2023c) GELLO: a general, low-cost, and intuitive teleoperation framework for robot manipulators. arXiv preprint arXiv:2309.13037.
335.
XianZGkanatsiosNGervetT, et al. (2023) ChainedDiffuser: unifying trajectory diffusion and keypose prediction for robotic manipulation. In: 7th Annual Conference on Robot Learning,Atlanta, Georgia, USA, 6-9 November 2023.
336.
XiangFQinYMoK, et al. (2020) SAPIEN: a simulated part-based interactive environment. In: 2020 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June, 2020, pp. 11097–11107.
337.
XiangJLvZXuS, et al. (2024) Structured 3D latents for scalable and versatile 3D generation. arXiv preprint arXiv:2412.01506.
338.
XiaoTChanHSermanetP, et al. (2022a) Robotic skill acquisition via instruction augmentation with vision-language models. arXiv preprint arXiv:2211.11736.
339.
XiaoTRadosavovicIDarrellT, et al. (2022b) Masked visual pre-training for motor control. arXiv preprint arXiv:2203.06173.
340.
XiaoXLiuJWangZ, et al. (2023) Robot learning in the era of foundation models: a survey. arXiv preprint arXiv:2311.14379.
341.
XieTZhaoSWuCH, et al. (2023a) Text2Reward: automated dense reward function generation for reinforcement learning. arXiv preprint arXiv:2309.11489.
342.
XieYYuCZhuT, et al. (2023b) Translating natural language to planning goals with large-language models. arXiv preprint arXiv:2302.05128.
343.
XiongHLiQChenYC, et al. (2021) Learning by watching: physical imitation of manipulation skills from human videos. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 27 September–01 October, 2021, IEEE, pp. 7827–7834.
344.
XuZWuJZengA, et al. (2019) DensePhysnet: learning dense physical object representations via multi-step dynamic interactions. arXiv preprint arXiv:1906.03853.
345.
XuKZhaoSZhouZ, et al. (2023a) A joint modeling of vision-language-action for target-oriented grasping in clutter. arXiv preprint arXiv:2302.12610.
346.
XuMHuangPYuW, et al. (2023b) Creative robot tool use with large language models. arXiv preprint arXiv:2310.13065.
347.
XuZGaoCLiuZ, et al. (2024) Manifoundation model for general-purpose robotic manipulation of contact synthesis with arbitrary objects and robots. arXiv preprint arXiv:2405.06964.
348.
XueHHangTZengY, et al. (2022) Advancing high-resolution video-language representation with large-scale video transcriptions. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022. pp. 5036–5045.
349.
XueLGaoMXingC, et al. (2023a) ULIP: learning a unified representation of language, images, and point clouds for 3D understanding. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June, 2023. pp. 1179–1189.
350.
XueLYuNZhangS, et al. (2023b) ULIP-2: towards scalable multimodal pre-training for 3D understanding. arXiv preprint arXiv:2305.08275.
351.
YanSZhuTWangZ, et al. (2022) VideoCoCa: video-text modeling with zero-shot transfer from contrastive captioners. arXiv preprint arXiv:2212.04979.
YangTJingYWuH, et al. (2023c) Moma-force: visual-force imitation for real-world mobile manipulation. In: 2023 IEEE/RSJ international conference on intelligent robots and systems (IROS), Detroit, MI, USA, 01–05 October, 2023, IEEE, pp. 6847–6852.
356.
YangZRamanSSShahA, et al. (2023d) Plug in the safety chip: enforcing constraints for LLM-driven robot agents. arXiv preprint arXiv:2309.09919.
357.
YangJMarkMSVuB, et al. (2024a) Robot fine-tuning made easy: pre-training rewards and policies for autonomous real-world reinforcement learning. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). Yokohama, Japan, 13–17 May, 2024, IEEE, pp. 4804–4811.
358.
YangLKangBHuangZ, et al. (2024b) Depth anything v2. Advances in Neural Information Processing Systems37: 21875–21911.
359.
YangYJiaBZhiP, et al. (2024c) PhyScene: physically interactable 3D scene synthesis for embodied ai. In: 2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June, 2024, pp. 16262–16272.
360.
YaoLHanJWenY, et al. (2022a) DetCLIP: dictionary-enriched visual-concept paralleled pre-training for open-world detection. Advances in Neural Information Processing Systems35: 9125–9138.
361.
YaoSZhaoJYuD, et al. (2022b) React: synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.
362.
YeWZhangYWangM, et al. (2023a) Foundation reinforcement learning: towards embodied generalist agents with foundation prior assistance. arXiv preprint arXiv:2310.02635.
363.
YeYLiXGuptaA, et al. (2023b) Affordance diffusion: synthesizing hand-object interactions. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June, 2023. pp. 22479–22489.
364.
YeSJangJJeonB, et al. (2024) Latent action pretraining from videos. arXiv preprint arXiv:2410.11758.
365.
YoshidaTMasumoriAIkegamiT (2025) From text to motion: grounding GPT-4 in a humanoid robot “Alter3”. Frontiers in Robotics and AI12: 1581110.
366.
YuCWangP (2022) Dexterous manipulation for multi-fingered robotic hands with reinforcement learning: a review. Frontiers in Neurorobotics16: 861825.
367.
YuXTangLRaoY, et al. (2022) Point-BERT: pre-training 3D point cloud transformers with masked point modeling. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022. pp. 19313–19322.
368.
YuTXiaoTStoneA, et al. (2023) Scaling robot learning with semantically imagined experience. arXiv preprint arXiv:2302.11550.
369.
YuanCWenCZhangT, et al. (2024) General flow as foundation affordance for scalable robot learning. arXiv preprint arXiv:2401.11439.
370.
ZareianARosaKDHuDH, et al. (2021) Open-vocabulary object detection using captions. In: 2021 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June, 2021, pp. 14393–14402.
371.
ZarrinRSJitoshoRYamaneK (2023) Hybrid learning-and model-based planning and control of in-hand manipulation. In: 2023 IEEE/RSJ international conference on intelligent robots and systems (IROS), Detroit, MI, USA, 01–05 October, 2023, IEEE, pp. 8720–8726.
372.
ZeYLiuYShiR, et al. (2024a) H-index: visual reinforcement learning with hand-informed representations for dexterous manipulation. Advances in Neural Information Processing Systems36: 1–16.
373.
ZeYYanGWuYH, et al. (2023) GNFactor: multi-task real robot learning with generalizable neural feature fields. In: Conference on Robot Learning. PMLR, pp. 284–301.
374.
ZeYZhangGZhangK, et al. (2024b) 3D diffusion policy: generalizable visuomotor policy learning via simple 3D representations. In: ICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, Yokohama, Japan, 17th May, 2024.
375.
ZellersRLuXHesselJ, et al. (2021) MERLOT: multimodal neural script knowledge models. Advances in Neural Information Processing Systems34: 23634–23651.
376.
ZhaiXKolesnikovAHoulsbyN, et al. (2022) Scaling vision transformers. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022. pp. 12104–12113.
377.
ZhangKLucetESandrettoJAD, et al. (2022a) Task and motion planning methods: applications and limitations. In: ICINCO 2022-19th International Conference on Informatics in Control, Automation and Robotics. SCITEPRESS-Science and Technology Publications, 476–483.
378.
ZhangRGuoZZhangW, et al. (2022b) PointCLIP: point cloud understanding by clip. In: 2022 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June, 2022, pp. 8552–8562.
379.
ZhangCHanDQiaoY, et al. (2023a) Faster segment anything: towards lightweight sam for mobile applications. arXiv preprint arXiv:2306.14289.
380.
ZhangHDuWShanJ, et al. (2023b) Building cooperative embodied agents modularly with large language models. arXiv preprint arXiv:2307.02485.
381.
ZhangJZhangJPertschK, et al. (2023c) Bootstrap your own skills: learning to solve new tasks with large language model guidance. arXiv preprint arXiv:2310.10021.
382.
ZhangXKunduAFunkhouserT, et al. (2023d) Nerflets: local radiance fields for efficient structure-aware 3D scene representation from 2D supervision. In: 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023, pp. 8274–8284.
383.
ZhangZZhangLWangZ, et al. (2023e) Part-level scene reconstruction affords robot interaction. In: 2023 IEEE/RSJ international conference on intelligent robots and systems (IROS), Detroit, MI, USA, 01–05 October, 2023, IEEE, pp. 11178–11185.
384.
ZhangJGuoYChenX, et al. (2024a) HiRT: enhancing robotic control with hierarchical robot transformers. arXiv preprint arXiv:2410.05273.
385.
ZhangJHuangJJinS, et al. (2024b) Vision-language models for vision tasks: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence.
386.
ZhangXQiuWLiYC, et al. (2024c) Debiased offline representation learning for fast online adaptation in non-stationary dynamics. arXiv preprint arXiv:2402.11317.
387.
ZhaoWQueraltaJPWesterlundT (2020) Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In: 2020 IEEE symposium series on computational intelligence (SSCI), Canberra, ACT, Australia, 01–04 December, 2020, IEEE, pp. 737–744.
388.
ZhaoTZKumarVLevineS, et al. (2023a) Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705.
389.
ZhaoXDingWAnY, et al. (2023b) Fast segment anything. arXiv preprint arXiv:2306.12156.
390.
ZhaoXLiMWeberC, et al. (2023c) Chat with the environment: interactive multimodal perception using large language models. arXiv preprint arXiv:2303.08268.
391.
ZhaoZChengSDingY, et al. (2024) A Survey of Optimization-based Task and Motion Planning: From Classical to Learning Approaches. IEEE/ASME Transactions on Mechatronics.
392.
ZhenHQiuXChenP, et al. (2024) 3D-VLA: a 3D vision-language-action generative world model. arXiv preprint arXiv:2403.09631.
393.
ZhiSLaidlowTLeuteneggerS, et al. (2021) In-place scene labelling and understanding with implicit scene representation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October, 2021, pp. 15838–15847.
394.
ZhouJWeiCWangH, et al. (2021) iBOT: image BERT pre-training with online tokenizer. arXiv preprint arXiv:2111.07832.
395.
ZhouKYangJLoyCC, et al. (2022) Learning to prompt for vision-language models. International Journal of Computer Vision130(9): 2337–2348.
396.
ZhouHYaoXMengY, et al. (2023) Language-conditioned learning for robotic manipulation: a survey. arXiv preprint arXiv:2312.10807.
397.
ZhouLWuGZuoY, et al. (2024a) A comprehensive review of vision-based 3D reconstruction methods. Sensors24(7): 2314.
398.
ZhouZAtreyaPLeeA, et al. (2024b) Autonomous improvement of instruction following skills via foundation models. arXiv preprint arXiv:2407.20635.
399.
ZhuYWongJMandlekarA, et al. (2020) Robosuite: a modular simulation framework and benchmark for robot learning. arXiv preprint arXiv:2009.12293.
400.
ZhuHMeduriARighettiL (2023a) Efficient object manipulation planning with monte carlo tree search. In: 2023 IEEE/RSJ international conference on intelligent robots and systems (IROS), Detroit, MI, USA, 01–05 October 2023, IEEE, pp. 10628–10635.
401.
ZhuYJoshiAStoneP, et al. (2023b) VIOLA: imitation learning for vision-based manipulation with object proposal priors. In: Conference on Robot Learning. PMLR, pp. 1199–1210.
402.
ZhuHZhaoTNiX, et al. (2024) Should we learn contact-rich manipulation policies from sampling-based planners?arXiv preprint arXiv:2412.09743.
403.
ZhuangJWangCLinL, et al. (2023) Dreameditor: text-driven 3D scene editing with neural fields. In: SIGGRAPH Asia 2023 Conference Papers, Sydney, NSW, Australia, 12-15 December, 2023, pp. 1–10.
404.
ZookASunFYSpjutJ, et al. (2024) GRS: generating robotic simulation tasks from real-world images. arXiv preprint arXiv:2410.15536.
405.
ZuoXSamangoueiPZhouY, et al. (2025) FMGS: foundation model embedded 3D gaussian splatting for holistic 3D scene understanding. International Journal of Computer Vision133(2): 611–627.
