A graphics-based digital twin (GBDT) framework for accurate UAV localization in GPS-denied environments

Abstract

Autonomous navigation of Unmanned Aerial Vehicles (UAVs) is crucial for effective assessment of large-scale civil infrastructure, as manual UAV control is time consuming and prone to mishaps. Autonomous navigation outdoors typically employs GPS signals to enable accurate localization and reduce long-term drifts. However, in many civil engineering applications, GPS signals are either poor or unavailable due to interference and/or multi-path effects. Current approaches for UAV localization in GPS-denied environments, track geometric features such as corners; however, these natural features can be misrepresented due to the presence of occlusions and/or image processing errors, resulting in significant localization errors that can grow with time. AprilTags can reduce localization errors, but installation on large-scale civil infrastructure can be challenging. Therefore, this paper proposes a framework that leverages the wealth of visual and geometric information encoded in a Graphics-Based Digital Twin (GBDT) of a target infrastructure to provide accurate localization of a UAV. The GBDT is comprised of a computer graphics model that faithfully represents a target structure’s geometry, structural features, and visual textures. The visual details of the GBDT are exploited to design an object recognition algorithm that detects GBDTtags, which are distinctive objects (or a collection of components) on the target structure in advance; GBDTtags provide functionality similar to AprilTags. When the camera attached to a UAV detects one or more GBDTtags, the vertices of the GBDTtags are mapped from the image plane into the GBDT coordinate system, allowing for the UAV to be localized. The framework, termed herein as GBDTpose, is first validated numerically using Blender, Gazebo, and Mavros software-in-the-loop (SITL). Subsequently, field validation is carried out using the Kavita and Lalit Bahl Smart Bridge at the University of Illinois Urbana-Champaign (UIUC). Results show that localization in GPS-denied environments can be achieved with 5-50 cm accuracy without the need for physical markers being placed on the structure.