The hard problems of robotics occur at the interface of an artificial agent and its surrounding physical world: perception and action. These are often regarded as tasks requiring tacit, unformalized and possibly unformalizable knowledge, making ontology engineering of apparently little value for robotics. We argue instead that an important role has appeared for ontology in current developments in robotics. Recent AI techniques such as foundation models have revealed the potential of explicit knowledge to be useful at all parts of a robotic system, but these techniques will require assistance from formal, verifiable methods to produce trustworthy systems. Further, robotics can contribute to the development of ontology engineering. As robotic agents become more autonomously capable, questions about the meaning of agency and responsibility become increasingly pressing, and will require careful consensus building across a variety of stakeholder groups. Finally, robotics may be a challenging test ground for the philosophical assumptions underlying foundational ontologies, and can thus assist the ontology engineering community in better understanding the consequences of ontological modeling decisions more generally.
AghzalM.PlakuE.SteinG. J.YaoZ. (2025). A survey on large language models for automated planning. https://arxiv.org/abs/2502.12435
2.
AthalyeA.KumarN.SilverT.LiangY.Lozano-PérezT.KaelblingL. P. (2024). Predicate invention from pixels via pretrained vision-language models. https://arxiv.org/abs/2501.00296.
3.
BassiounyA.ElsayedA.FalomirZ.PobilA. (2025). UJI-butler: A symbolic/non-symbolic robotic system that learns through multi-modal interaction. International Journal of Social Robotics, 0, 1–21. https://doi.org/10.1007/s12369-025-01234-5
4.
BatemanJ.FarrarS. (2005). Modelling models of robot navigation using formal spatial ontology. In C. Freksa, M. Knauff, B. Krieg-Brückner, B. Nebel, & T. Barkowsky (Eds.), Proceedings of the 4th international conference on spatial cognition 2004. Lecture Notes in Computer Science (LNCS) (pp. 366–389). Springer. https://doi.org/10.1007/978-3-540-32255-9_21
5.
BiccheriL.BorgoS.FerrarioR. (2023). On the relation of instrumental dependence. In Formal ontology in information systems (pp. 47–61). IOS Press.
6.
BlackK.BrownN.DriessD.EsmailA.EquiM.FinnC.FusaiN.GroomL.HausmanK.IchterB.JakubczakS.JonesT.KeL.LevineS.Li-BellA.MothukuriM.NairS.PertschK.ShiL. X.…ZhilinskyU. (2024). : A vision-language-action flow model for general robot control. Arxiv preprint, https://arxiv.org/abs/2410.24164.
7.
BorgoS. (2020). Ontological challenges to cohabitation with self-taught robots. Semantic Web, 11(1), 161–167.
8.
BorgoS.FerrarioR.GangemiA.GuarinoN.MasoloC.PorelloD.SanfilippoE. M.VieuL.BorgoS.GaltonA.KutzO. (2022). DOLCE: A descriptive ontology for linguistic and cognitive engineering. Applied Ontology, 17(1), 45–69. https://doi.org/10.3233/AO-210259
9.
BorgoS.GaltonA.KutzO. (2022). Foundational ontologies in action. Understanding foundational ontology through examples. Applied Ontology, 17(1), 1–16.
10.
ChenL.LeiY.JinS.ZhangY.ZhangL. (2024). RLingua: Improving reinforcement learning sample efficiency in robotic manipulations with large language models. IEEE Robotics and Automation Letters, 9, 1–8. https://doi.org/10.1109/LRA.2024.3400189
11.
CurtisA.KumarN.CaoJ.Lozano-PérezT.KaelblingL. P. (2024). Trust the PRoC3S: Solving long-horizon robotics problems with LLMs and constraint satisfaction. https://arxiv.org/abs/2406.05572
12.
de la RosaT.GopalakrishnanS.PozancoA.ZengZ.BorrajoD. (2024). TRIP-PAL: Travel planning with guarantees by combining large language models and automated planners. https://arxiv.org/abs/2406.10196
13.
DevarakondaV. N.KaypakA. U.YuanS.KrishnamurthyP.FangY.KhorramiF. (2024). MultiTalk: Introspective and extrospective dialogue for human–environment–LLM alignment. https://arxiv.org/abs/2409.16455
14.
DreyfusH. L. (1972). What computers can’t do. Harper & Row.
15.
DreyfusH. L. (1992). What computers still can’t do: A critique of artificial reason. The MIT Press.
16.
DuY.WatkinsO.WangZ.ColasC.DarrellT.AbbeelP.GuptaA.AndreasJ. (2023). Guiding pretraining in reinforcement learning with large language models. https://arxiv.org/abs/2302.06692
17.
DumontierM.HoehndorfR. (2010). Realism for scientific ontologies. In Proceedings of the 6th international conference of formal ontology in information systems (FOIS 2010) (pp. 387–399). IOS Press.
18.
DziriN.LuX.SclarM.LiX. L.JiangL.LinB. Y.WelleckS.WestP.BhagavatulaC.BrasR. L.HwangJ. D.SanyalS.RenX.EttingerA.HarchaouiZ.ChoiY. (2023). Faith and fate: Limits of transformers on compositionality. In Proceedings of the 37th conference on Neural Information Processing Systems (NeurIPS). Curran Associates Inc. https://openreview.net/forum?id=Fkckkr3ya8
19.
FarrellR.WareS. G. (2024). Large language models as narrative planning search guides. IEEE Transactions on Games, 0, 1–10. https://doi.org/10.1109/TG.2024.3487416
20.
FouilhéG.EiflerR.ThiébauxS.AsherN. (2025). Conversational goal-conflict explanations in planning via multi-agent LLMs. In Proceedings of the AAAI workshop LM4Plan. OpenReview.
21.
GrenonP.SmithB.GoldbergL. (2004). Biodynamic ontology: Applying BFO in the biomedical domain. In D. M. Pisanelli (Ed.), Special issue Ontologies in Medicine of the journal of Studies in Health Technology and Informatics (Vol. 102, pp. 20–38). IOS Press. https://doi.org/10.3233/978-1-60750-945-5-20
22.
GuanL.ZhouY.LiuD.ZhaY.AmorH. B.KambhampatiS. (2024). Task success is not enough: Investigating the use of video-language models as behavior critics for catching undesirable agent behaviors. In Proceedings of the first Conference on Language Modelling (COLM). OpenReview. https://openreview.net/forum?id=otKo4zFKmH
HoppeF.IlievskiF.KaloJ.-C. (2025). Investigating the robustness of deductive reasoning with large language models. https://arxiv.org/abs/2502.04352
26.
Izquierdo-BadiolaS.CanalG.RizzoC.AlenyàG. (2024). PlanCollabNL: Leveraging large language models for adaptive plan generation in human–robot collaboration. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (pp. 17344–17350). IEEE Xplore.
27.
JamesW. (2014). Pragmatism: A new name for some old ways of thinking. Cambridge Library Collection - Philosophy. Cambridge University Press. Reprint; first published in 1907. https://doi.org/10.1017/cbo9781107360471
28.
KambhampatiS. (2024). Can large language models reason and plan?Annals of the New York Academy of Sciences, 1534(1), 15–18. Publisher Copyright: 2024 The New York Academy of Sciences. https://doi.org/10.1111/nyas.15125
29.
KambhampatiS.ValmeekamK.GuanL.VermaM.StechlyK.BhambriS.SaldytL.MurthyA. (2024a). LLMs can’t plan, but can help planning in LLM-modulo frameworks. https://arxiv.org/abs/2402.01817
30.
KambhampatiS.ValmeekamK.GuanL.VermaM.StechlyK.BhambriS.SaldytL. P.MurthyA. B. (2024b). Position: LLMs can’t plan, but can help planning in LLM-modulo frameworks. In R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, & F. Berkenkamp (Eds.), Proceedings of the 41st international conference on machine learning. Volume 235 of Proceedings of Machine Learning Research (pp. 22895–22907). PMLR. https://proceedings.mlr.press/v235/kambhampati24a.html
31.
KeetC. M.KhanZ. (2022). Foundational ontologies: From theory to practice and back. Journal of Knowledge Structures and Systems, 3(1), 67–71.
32.
KhalidiM. A. (2010). Interactive kinds. The British Journal for the Philosophy of Science, 61(2), 335–360. https://doi.org/10.1093/bjps/axp042
33.
KhanZ.KeetC. M. (2012). ONSET: Automated foundational ontology selection and explanation. In A. ten Teije, J. Völker, S. Handschuh, H. Stuckenschmidt, M. d’Acquin, A. Nikolov, N. Aussenac-Gilles, & N. Hernandez (Eds.), Knowledge engineering and knowledge management (pp. 237–251). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-33876-2_22
34.
KümpelM.MuellerC. A.BeetzM. (2021). Semantic digital twins for retail logistics. In M. Freitag, H. Kotzab, & N. Megow (Eds.), Dynamics in logistics: Twenty-five years of interdisciplinary logistics research in Bremen, Germany (pp. 129–153). Springer International Publishing.
35.
MaR.LuijkxJ.AjanovićZ.KoberJ. (2024). ExploRLLM: Guiding exploration in reinforcement learning with large language models. In proceedings of the Data generation for robotics workshop at RSS. OpenReview. https://openreview.net/forum?id=NT2zgVXHn5
36.
MaY. J.LiangW.WangG.HuangD. -A.BastaniO.JayaramanD.ZhuY.FanL.AnandkumarA. (2024). Eureka: Human-level reward design via coding large language models. In proceedings of the 12th International Conference on Learning Representations (ICLR). OpenReview. https://openreview.net/forum?id=IEduRUO55F
MizoguchiR.BorgoS. (2021). The role of the systemic view in foundational ontologies. In Proceedings of the 5th workshop on Foundational Ontology. CEUR-WS.
40.
MoorJ. (2006). The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21(4), 18–21. https://doi.org/10.1109/MIS.2006.80
41.
MoravecH. (1988). Mind children: The future of robot and human intelligence. Harvard University Press.
42.
NeuhausF. (2023). Ontologies in the era of large language models—A perspective. Applied Ontology, 18, 399–407.
43.
NguyenG. H.BeßlerD.StelterS.PomarlanM.BeetzM. (2024). Translating universal scene descriptions into knowledge graphs for robotic environment. In Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA) (pp. 9389–9395). IEEE Xplore. https://doi.org/10.1109/ICRA57147.2024.10611691
44.
Olivares-AlarcosA.AndriellaA.FoixS.AlenyàG. (2023). Robot explanatory narratives of collaborative and adaptive experiences. In 2023 IEEE international conference on robotics and automation (ICRA) (pp. 11964–11971).
45.
Olivares-AlarcosA.BeßlerD.KhamisA.GoncalvesP.HabibM. K.Bermejo-AlonsoJ.BarretoM.DiabM.RosellJ.QuintasJ.OlszweskaJ.NakawalaH.PignatonE.GyrardA.BorgoS.AlenyaG.BeetzM.LiH. (2019). A review and comparison of ontology-based approaches to robot autonomy. The Knowledge Engineering Review, 34, 1–29. https://doi.org/10.1017/S0269888919000237
46.
Olivares-AlarcosA.FoixS.AlenyàG. (2023). Time-to-contact for robot safety stop in close collaborative tasks. Control, Robotics and Sensors (pp. 87–104). Institution of Engineering and Technology.
47.
OltramariA. (2019). Artificial intelligence within the bounds of ontological reason. In S. Borgo, R. Ferrario, & C. Masolo (Eds.), Ontology makes sense (pp. 37–48). IOS Press.
48.
OswaldJ.SrinivasK.KokelH.LeeJ.KatzM.SohrabiS. (2024). Large language models as planning domain generators. In Proceedings of the thirty-fourth international conference on automated planning and scheduling, ICAPS ’24. AAAI Press.
PanS.LuoL.WangY.ChenC.WangJ.WuX. (2024). Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering, 36(7), 3580–3599.
51.
PolanyiM. (1966). The tacit dimension. Anchor.
52.
PorzelR.MalakaR. (2004). A task-based approach for ontology evaluation. In Proceedings of the Workshop on Ontology Learning and Population at the 16th European Conference on Artificial Intelligence (ECAI).
53.
SmithB.CeustersW. (2010). Ontological realism: A methodology for coordinated evolution of scientific ontologies. Applied Ontology, 5(3-4), 139–188. https://doi.org/10.3233/AO-2010-0079
54.
TongE.OpipariA.LewisS.ZengZ.JenkinsO. C. (2024). OVAL-prompt: Open-vocabulary affordance localization for robot manipulation through LLM affordance-grounding. https://arxiv.org/abs/2404.11000
55.
ValmeekamK.MarquezM.SreedharanS.KambhampatiS. (2023). On the planning abilities of large language models—A critical investigation. In Thirty-seventh conference on neural information processing systems. OpenReview. https://openreview.net/forum?id=X6dEqXIsEW
56.
ValmeekamK.StechlyK.GundawarA.KambhampatiS. (2024). Planning in strawberry fields: Evaluating and improving the planning and scheduling capabilities of LRM o1. https://arxiv.org/abs/2410.02162
57.
ValmeekamK.StechlyK.KambhampatiS. (2024). LLMs still can’t plan; can LRMs? A preliminary evaluation of OpenAI’s o1 on planbench. https://arxiv.org/abs/2409.13373
58.
XieT.ZhaoS.WuC. H.LiuY.LuoQ.ZhongV.YangY.YuT. (2024). Text2Reward: Reward shaping with language models for reinforcement learning. In Proceedings of the 37th conference on Neural Information Processing Systems (NeurIPS). Curran Associates Inc. https://openreview.net/forum?id=tUM39YTRxH
59.
ZhengZ.XieZ.WangZ.HooiB. (2025). Monte Carlo tree search for comprehensive exploration in LLM-based automatic heuristic design. https://arxiv.org/abs/2501.08603
60.
ZitkovichB.YuT.XuS.XuP.XiaoT.XiaF.WuJ.WohlhartP.WelkerS.WahidA.VuongQ.VanhouckeV.TranH.SoricutR.SinghA.SinghJ.SermanetP.SanketiP. R.SalazarG.…HanK. (2023). RT-2: Vision-language-action models transfer web knowledge to robotic control. In J. Tan, M. Toussaint, & K. Darvish (Eds.), Proceedings of the 7th conference on robot learning. Proceedings of Machine Learning Research (Vol. 229, pp. 2165–2183). PMLR.
