Natural language control of UAVs using large language models with robust semantic interpretation and logic-guided task planning

Abstract

The operational complexity of Unmanned Aerial Vehicles (UAVs) presents a significant barrier to their widespread adoption. This paper introduces a comprehensive end-to-end framework that enables intuitive UAV control through natural language, aiming to lower the operational threshold for non-expert users. The proposed system leverages a Large Language Model (LLM) to parse complex, multi-step commands. The primary contribution lies in the middleware for semantic interpretation and task execution logic, rather than in new control law developments. Distinct from prior works that focus on isolated command translation or lack systematic verification, this paper introduces three novel components: (1) a Robust Semantic Interpretation Engine that resolves mathematical expressions and ambiguous semantics; (2) a Formalized Sequential Task Executor with a finite-state machine and dual-condition completion verification; and (3) an end-to-end integration framework that bridges LLM-based planning with standard low-level PID control. All validations are performed in a numerical simulation environment without physical flight tests or hardware-in-the-loop experiments. The framework’s algorithmic feasibility and logic integrity are demonstrated through rigorous numerical simulations under idealized conditions (no wind disturbances, perfect state feedback). These results confirm the effectiveness of the proposed middleware and task scheduling logic, while real-world deployment would require additional validation against environmental uncertainties and perception noise.