Research on the visual/inertial integrated carrier landing guidance algorithm

Abstract

In this article, we describe a visual/inertial integrated carrier landing guidance algorithm for aircraft carrier landing operations. The airborne vision system and the inertial navigation system are employed as sensors to obtain feature points of carrier runway as well as the aircraft inertial measurements. In this algorithm, visual and inertial information are integrated, and the Pose from Orthographic Projection and Scaling with Iterations algorithm is utilized to estimate aircraft position and carrier motion. A simulation was constructed validating this algorithm, and the results of this simulation show satisfactory accuracy as well as high efficiency in carrier landing guidance.

Keywords

Carrier landing guidance vision inertial POSIT

Introduction

The carrier-borne aircraft is one of the most important constituents of the aircraft carrier. However, the complex landing environment can potentially lead to a landing failure because of a small landing area, carrier motion, and occasional conditions of low visibility.¹ Aircraft carrier landing is one of the most difficult operational challenges in the world.

For a safe landing operation, various kinds of landing guidance systems are developed for guiding aircraft to follow accurate slope–slide path and reach touchdown position, such as a shipboard tracking radar system^2,3 and Shipboard Relative Global Positioning System.^4,5 These guidance systems can meet the typical carrier landing requirements but lack guidance capacity when wireless communication cannot function.

With the development of photoelectric technology, the airborne vision-guided technology, which can be considered as an autonomous navigation technology, has attracted attention in aircraft landing research. Sharp et al.⁶ and Saripalli et al.⁷ used specially designed planar features to land a helicopter. In the work of Gui et al.,⁸ maximum between-class variance algorithm, region growing algorithm, and negative Laplacian of Gaussian operator are used to detect and track centers of the infrared lamps in the images. In a previous study,⁹ the runway boundary lines were extracted using Hough transforms, and the lateral offset of the unmanned aerial vehicle (UAV) was estimated with respect to the runway centerline. All these methods can be applied to landing on an aircraft carrier. In the work of Yakimenko et al.,¹⁰ three infrared points were used to calculate the relative position and orientation of a UAV with respect to a ship. A three-dimensional (3-D) model-based tracker was used to estimate the camera poses, and their detection targets are a warped patch of a reference image, three visual features including two white triangles on the runway and a white line along the stern.^11,12 However, the methods mentioned above cannot estimate the ship motion, and their results can potentially be improved for greater accuracy. Ding et al.¹³ estimated the ship motion using Newton’s iterative algorithm, but there is still room for improvement in terms of accuracy and efficiency.

This article presents a visual/inertial integrated carrier landing guidance algorithm that requires less execution time as well as eliminates the need for Jacobian determinant calculation in Newton’s iterative algorithm. In this algorithm, the Pose from Orthographic Projection and Scaling with Iterations (POSIT) algorithm is utilized to estimate the relative position and attitude between the aircraft and the carrier from visual information, and the carrier motion can be decoupled by using the aircraft attitude information from the airborne inertial navigation system (INS). Then, the changing position of the landing point caused by carrier motion can be estimated, and the optimal trajectory can be planned according to the height of the landing point. The position of the aircraft deviating from the optimal trajectory is calculated and delivered to the aircraft control system. A simulation is constructed to validate the performance of the algorithm.

This article is organized as follows. An overview of the visual/inertial integrated landing guidance system, the mathematical model, and the details of the landing guidance algorithm are presented in the “Materials and methods” section, followed by simulation and comparison of the results in the “Simulation” section, and finally, concluding remarks in the “Conclusion” section.

Materials and methods

System overview

We consider an infrared system as the airborne vision system. The infrared system is installed in front of the aircraft, and four infrared cooperated targets are set on the deck for carrier identification and location. Other common vision systems can also apply the method discussed in this article for the pose estimation after completing the recognition of characteristic points.

The schematic of system is shown in Figure 1. The four red points represent the four cooperated targets that are set on the deck for visual detection.

Figure 1.

The vision system and the cooperated targets.

The vision system gauges the image of the runway containing cooperated targets and utilizes image recognition methods such as the maximum between-class variance algorithm,⁸ edge extraction based on Canny operator, and contour-based moment algorithm to recognize the image coordinates of the targets’ centers. There are one-to-one relationships between the two-dimensional (2-D) infrared image coordinates on the vision system and the real 3-D position of infrared cooperated targets on the carrier. Through this relationship, the POSIT algorithm calculates the relative pose between the aircraft and the carrier. By integrating the aircraft attitude measurements from the INS, the carrier motion can be decoupled from the relative pose information and then utilized to calculate a reference flight path. All landing guidance information is delivered to the flight control system (FCS) for controlling the aircraft.

The complete block diagram for the carrier landing guidance system is shown in Figure 2.

Figure 2.

The visual/inertial integrated carrier landing guidance system block diagram.

Coordinate system

The aircraft carrier is assumed to be at sailing straight with typical roll, pitch, and heave dynamics, which have considerable influence on the position of landing point. The aircraft is gliding to the expected landing point with a horizontal angle of 3.5°.

The frames involved are shown in Figure 3. All coordinate systems are right-handed. The frame F_O is defined as the navigation coordinate, where the y-axis coincides with the runway centerline and the z-axis points upward. The aircraft body frame F_B and the carrier body frame F_S are designed with the z-axis oriented upward and the y-axis pointing to the head. The carrier runway frame F_R corresponds to the expected landing point on the carrier deck, translated by the offset vector ${}_{R}^{S}T$ and rotated around its z-axis expressed as matrix ${}_{R}^{S}R$ from the carrier frame F_S. The camera frame F_C corresponds to the origin of the camera, translated by the offset vector ${}_{C}^{B}T$ and rotated around its x-axis expressed as the matrix ${}_{B}^{C}R$ from the aircraft body frame.

Figure 3.

Coordinate system.

The pose of the aircraft and the aircraft carrier in F_O are, respectively, defined as

{}^{O}p_{B} = {({}^{O}x_{B}, {}^{O}y_{B}, {}^{O}z_{B})}^{T}, {}_{B}^{O}Φ = {({}^{O}θ_{B}, {}^{O}γ_{B}, {}^{O}ψ_{B})}^{T}

and

{}^{O}p_{S} = {({}^{O}x_{S}, {}^{O}y_{S}, {}^{O}z_{S})}^{T}, {}_{S}^{O}Φ = {({}^{O}θ_{S}, {}^{O}γ_{S}, {}^{O}ψ_{S})}^{T}

where ${}^{O}p_{B}$ and ${}^{O}p_{S}$ represent the relative positions; ${}_{B}^{O}Φ$ and ${}_{S}^{O}Φ$ are Euler angles for relative attitudes; ${}^{O}θ_{B}$ , ${}^{O}γ_{B}$ , and ${}^{O}ψ_{B}$ are the pitch, roll, and heading angles of the aircraft in the navigation frame, respectively; and ${}^{O}θ_{S}$ , ${}^{O}γ_{S}$ , and ${}^{O}ψ_{S}$ are the pitch, roll, and heading angles of the aircraft carrier in the navigation frame, respectively.

The coordinates of the four infrared cooperated targets are expressed as ${}^{R}p_{i g} = {(x_{i r}, y_{i r}, z_{i r})}^{T}$ $(i = 1, 2, 3, 4)$ in F_R. Thus, those coordinates in different frames can be calculated as follows

{}^{S}p_{i g} = {}_{R}^{S}R \cdot {}^{R}p_{i g} + {}_{R}^{S}T

{}^{O}p_{i g} = {}_{S}^{O}R \cdot {}^{S}p_{i g} + {}_{S}^{O}T = R_{z} (- {}^{O}ψ_{S}) \cdot R_{x} (- {}^{O}θ_{S}) \cdot R_{y} (- {}^{O}γ_{S}) \cdot {}^{S}p_{i g} + {}^{O}p_{S}

{}^{B}p_{i g} = {}_{O}^{B}R \cdot ({}^{O}p_{i g} - {}_{B}^{O}T) = R_{y} ({}^{O}γ_{B}) \cdot R_{x} ({}^{O}θ_{B}) \cdot R_{z} ({}^{O}ψ_{B}) \cdot ({}^{O}p_{i g} - {}^{O}p_{B})

{}^{C}p_{i g} = {(x_{i c}, y_{i c}, z_{i c})}^{T} = {}_{B}^{C}R \cdot ({}^{B}p_{i g} + {}_{C}^{B}T)

where ${}^{S}p_{i g}$ , ${}^{O}p_{i g}$ , ${}^{B}p_{i g}$ , and ${}^{C}p_{i g}$ are the coordinates of the targets in the carrier body frame, navigation frame, aircraft body frame, and camera frame, respectively. R_x(*), R_y(*), and R_z(*) are the functions to calculate the rotation matrix that helps rotate a vector around the x-axis, y-axis, and z-axis with an angle of *. For example, equation (4) can be expanded as follows

[\begin{matrix} x_{i o} \\ y_{i o} \\ z_{i o} \end{matrix}] = [\begin{matrix} cos {}^{O}ψ_{S} & - sin {}^{O}ψ_{S} & 0 \\ sin {}^{O}ψ_{S} & cos {}^{O}ψ_{S} & 0 \\ 0 & 0 & 1 \end{matrix}] \cdot [\begin{matrix} 1 & 0 & 0 \\ 0 & cos {}^{O}θ_{S} & - sin {}^{O}θ_{S} \\ 0 & sin {}^{O}θ_{S} & cos {}^{O}θ_{S} \end{matrix}] \cdot

[\begin{matrix} cos {}^{O}γ_{S} & 0 & sin {}^{O}γ_{S} \\ 0 & 1 & 0 \\ - sin {}^{O}γ_{S} & 0 & cos {}^{O}γ_{S} \end{matrix}] \cdot [\begin{matrix} x_{i s} \\ y_{i s} \\ z_{i s} \end{matrix}] + [\begin{matrix} {}^{O}x_{S} \\ {}^{O}y_{S} \\ {}^{O}z_{S} \end{matrix}]

According to the imaging principle in the study by Trawny et al.,¹⁴ the conversion relation between the coordinates ${}^{C}p_{i g} = {(x_{i c}, y_{i c}, z_{i c})}^{T}$ in F_C and their image coordinates ${(u_{i}, v_{i})}^{T}$ can be obtained

[\begin{matrix} u_{i} \\ v_{i} \end{matrix}] = \frac{1}{z_{i c}} [\begin{matrix} f & 0 \\ 0 & f \end{matrix}] [\begin{matrix} x_{i c} \\ y_{i c} \end{matrix}] + [\begin{matrix} c_{u} \\ c_{v} \end{matrix}]

where f is the focal length of the camera and ${(c_{u}, c_{v})}^{T}$ are the principal point coordinates. These variables are obtained by camera calibration.

Algorithm

The POSIT algorithm can estimate the object pose in a camera frame from a single image if four or more correspondences between 2-D points and 3-D points are given. The algorithm consists of two steps. First, the scaled orthographic projection is used to approximate the perspective projection and get the rotation matrix and translation vector from a set of linear equations. Second, the new scale factor is iteratively calculated for each point, and the first step is repeated using the new points instead of the original ones until the set threshold is met.^15
–17

Figure 4 shows the perspective projection and scaled orthogonal projection. The target frame F_D is parallel with the runway frame F_R and its origin point is the first cooperated target p_0g. The points denoted by P_ig are the perspective projection points on the camera focal plane. P′_ig points are called scaled orthogonal projections.

Figure 4.

Perspective projection and scaled orthogonal projection.

The coordinates of the points denoted by p_ig in F_D and F_R have the following relation

{}^{D}p_{i g} = {(x_{i d}, y_{i d}, z_{i d})}^{T} = {}_{R}^{D}R \cdot ({}^{R}p_{i g} + {}_{D}^{R}T) = {}^{R}p_{i g} + {}_{D}^{R}T

where ${}_{R}^{D}R = I_{3 \times 3}$ and ${}_{D}^{R}T = {}^{R}p_{0 g}$ .

The image coordinates can be expressed as

[\begin{matrix} w_{i} u ′_{i} \\ w_{i} v ′_{i} \\ w_{i} \end{matrix}] = [\begin{matrix} f & 0 & 0 \\ 0 & f & 0 \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} x_{i c} \\ y_{i c} \\ z_{i c} \end{matrix}] = [\begin{matrix} f & 0 & 0 \\ 0 & f & 0 \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} \begin{matrix} R_{1} \\ \begin{matrix} R_{2} \\ R_{3} \\ 0 \end{matrix} \end{matrix} & \begin{matrix} T_{x} \\ \begin{matrix} T_{y} \\ T_{z} \\ 1 \end{matrix} \end{matrix} \end{matrix}] [\begin{matrix} x_{i d} \\ y_{i d} \\ \begin{matrix} z_{i d} \\ 1 \end{matrix} \end{matrix}] = [\begin{matrix} f R_{1} & f T_{x} \\ f R_{2} & f T_{y} \\ R_{3} & T_{z} \end{matrix}] [\begin{matrix} {}^{D}p_{i g} \\ 1 \end{matrix}]

where ${}_{D}^{C}T = {}_{R}^{C}T + {}_{R}^{C}R \cdot {}_{D}^{R}T = {[\begin{matrix} T_{x} & T_{y} & T_{z} \end{matrix}]}^{T}$ , ${}_{D}^{C}R = {}_{R}^{C}R \cdot {}_{D}^{R}R = {}_{R}^{C}R = [\begin{matrix} R_{1}; & R_{2}; & R_{3}; \end{matrix}]$ , and ${({u^{'}}_{i}, {v^{'}}_{i})}^{T} = {(u_{i}, v_{i})}^{T} - {(c_{u}, c_{v})}^{T}$ .

By introducing a range scaling factor s = f/T_z, equation (9) can be rewritten as

[\begin{matrix} w_{i} u ′_{i} \\ w_{i} v ′_{i} \end{matrix}] = [\begin{matrix} s R_{1} & s T_{x} \\ s R_{2} & s T_{y} \end{matrix}] [\begin{matrix} {}^{D}p_{i g} \\ 1 \end{matrix}], w_{i} = R_{3} \cdot {}^{D}p_{i g} / T_{z} + 1, (i = 1, 2, 3)

Considering $s T_{x} = f T_{x} / T_{z} = u ′_{0}$ and $s T_{y} = f T_{y} / T_{z} = v ′_{0}$ , where ${({u^{'}}_{0}, {v^{'}}_{0})}^{T}$ is the image coordinate of the origin of the target coordinate, equation (10) can be expressed as

{\begin{matrix} {}^{D}p {_{i g}}^{T} \cdot s R_{1}^{T} = w_{i} {u^{'}}_{i} - u ′_{0} \\ {}^{D}p {_{i g}}^{T} \cdot s R_{2}^{T} = w_{i} {v^{'}}_{i} - v ′_{0} \end{matrix}, w_{i} = R_{3} \cdot {}^{D}p_{i g} / T_{z} + 1, (i = 1, 2, 3)

The following linear equations result

{\begin{matrix} A \cdot s R_{1}^{T} = x' \\ A \cdot s R_{2}^{T} = y' \end{matrix}, x^{'} = [\begin{matrix} w_{1} {u^{'}}_{1} - u ′_{0} \\ w_{2} {u^{'}}_{2} - u ′_{0} \\ w_{3} {u^{'}}_{3} - u ′_{0} \end{matrix}], y^{'} = [\begin{matrix} w_{1} {v^{'}}_{1} - v ′_{0} \\ w_{2} {v^{'}}_{2} - v ′_{0} \\ w_{3} {v^{'}}_{3} - v ′_{0} \end{matrix}], A = {[\begin{matrix} x_{i r} - x_{0 r} \\ y_{i r} - y_{0 r} \\ z_{i r} - z_{0 r} \end{matrix}]}_{3 \times 3}

Coinciding with scaled orthogonal projection, the initial values of perspective projection $w_{i} (i = 1, 2, 3)$ are all equal to one. Then sR₁^T and sR₂^T can be calculated using the linear equation (12).

However, if $z_{i r} = 0 (i = 1, 2, 3)$ , the third elements of sR₁^T and sR₂^T cannot be calculated. Let ${(I_{1}, I_{2}, I_{3})}^{T}$ and ${(J_{1}, J_{2}, J_{3})}^{T}$ represent the elements of sR₁^T and sR₂^T, respectively. ${(I_{1}, I_{2})}^{T}$ and ${(J_{1}, J_{2})}^{T}$ have been solved by the above equation. Then, the third elements I₃ and J₃ satisfy the following equation

{\begin{matrix} I_{3} \cdot J_{3} = - I_{1} J_{2} + I_{2} J_{1} \\ I_{3}^{2} - J_{3}^{2} = J_{1}^{2} + J_{2}^{2} - I_{1}^{2} - I_{2}^{2} \end{matrix}

Two solutions of I₃ and J₃ can be obtained by solving this equation. According to the characteristics of the rotation matrix

{}_{R}^{C}R = R_{y} (γ_{r_c}) \cdot R_{x} (θ_{r_c}) \cdot R_{z} (ψ_{r_c}) = [\begin{matrix} C_{γ} & 0 & - S_{γ} \\ 0 & 1 & 0 \\ S_{γ} & 0 & C_{γ} \end{matrix}] [\begin{matrix} 1 & 0 & 0 \\ 0 & C_{θ} & S_{θ} \\ 0 & - S_{θ} & C_{θ} \end{matrix}] [\begin{matrix} C_{ψ} & S_{ψ} & 0 \\ - S_{ψ} & C_{ψ} & 0 \\ 0 & 0 & 1 \end{matrix}]

where $θ_{r_c}$ , $γ_{r_c}$ , and $ψ_{r_c}$ , respectively, represent the rotation angles from F_R to F_C, C_γ is short for $cos (γ_{r_c})$ , and S_γ for $sin (γ_{r_c})$ , and so on.

In equation (14), $J_{3} = s \cdot sin (θ_{r_c})$ . Note that $s = f / T_{z} > 0$ and $- π < θ_{r_{c}} < 0$ in the landing environment, which means J₃ < 0, and the unique solution of I₃ and J₃ can be obtained. Now the elements of sR₁^T and sR₂^T are all worked out.

Because both R₁^T and R₂^T are unit vectors, s, ${}_{R}^{C}R$ , and ${}_{R}^{C}T$ can be solved by the following equations

R_{1}^{T} = \frac{(s R_{1}^{T})}{| s R_{1}^{T} |}, R_{2}^{T} = \frac{(s R_{2}^{T})}{| s R_{2}^{T} |}, R_{3}^{T} = R_{1}^{T} \times R_{2}^{T}

s = \sqrt{| s R_{1}^{T} | \cdot | s R_{2}^{T} |}

T_{x} = (s T_{x}) / s, T_{y} = (s T_{y}) / s, T_{z} = f / s

As the pose estimation in the first step has been obtained, better estimations of w_i can be obtained using equation (10). Then, by substituting the new w_i estimations into the linear equation (12), we can get more accurate results. This iterative process will end when $| w_{i (k)} - w_{i (k - 1)} |$ is less than the threshold. Normalize the rotation matrix again to get the results ${}_{D}^{C}R$ and ${}_{D}^{C}T$ . Then, ${}_{R}^{C}R = {}_{D}^{C}R$ and ${}_{R}^{C}T = {}_{D}^{C}T - {}_{R}^{C}R \cdot {}_{D}^{R}T$ .

In accordance with the formulas (3) to (6), the following can be derived

R_{z} (ψ_{B}) \cdot {}_{S}^{R}R \cdot R_{y} (γ_{S}) \cdot R_{x} (θ_{S}) = {[{}_{B}^{C}R \cdot R_{y} (γ_{B}) \cdot R_{x} (θ_{B})]}^{- 1} \cdot {}_{R}^{C}R \cdot {}_{S}^{R}R

{}_{B}^{O}T - {}_{S}^{R}R \cdot {}_{S}^{O}T = {}_{S}^{O}R \cdot {}_{R}^{S}T - {({}_{B}^{C}R \cdot {}_{O}^{B}R)}^{- 1} \cdot {}_{R}^{C}T - {}_{S}^{R}R \cdot {}_{R}^{S}T + {}_{P}^{O}R \cdot {}_{C}^{B}T

The right-hand side of equation (18) is formed by known values, so ψ_B, γ_S, θ_S, and the entire rotation matrix can be calculated. Assume that ${}^{O}x_{S} = {}^{O}y_{S} = 0$ , then through equation (19), ${[\begin{matrix} {}^{O}x_{B} & {}^{O}y_{B} & {}^{O}z_{B} - {}^{O}z_{S} \end{matrix}]}^{T}$ can be obtained. The entire process of applying the algorithm is shown in Figure 5. $θ_{B_INS}$ and $γ_{B_INS}$ represent the pitch and roll angle from the airborne INS, respectively. $z_{B_Ra}$ represents the height of the aircraft from the radio altimeter.

Figure 5.

The schematic of the algorithm application.

Control law

The optimal linear quadratic regulator (LQR) is employed in this study. LQR is a common method to analyze multiple-input multiple-output FCS in modern control theory.¹⁸

The reference trajectory can be calculated according to the aircraft carrier motion (γ_S, θ_S, and z_S). Then, the off-glide-path lateral distance d_x, vertical distance d_z (both refer to modified reference trajectory), and velocity V are obtained as inputs to the control law. The control objective is to regulate these variables to a value of zero for keeping the aircraft on the glide path. Therefore, the vertical state vector x_v and the lateral state vector x_l can be expressed as follows

\begin{matrix} x_{v} = [V; α; θ; q; d_{z};] \\ x_{l} = [β; ϕ; p; r; d_{x};] \end{matrix}

where α, β, θ, ϕ, q, p, and r are the attack angle, sideslip angle, pitch angle, bank angle, pitch rate, roll rate, and yaw rate, respectively. States of the aircraft are observable by using aircraft sensors.

The control vector is defined as

u = [u_{v}; u_{l}] = [δ_{t}; δ_{e}; δ_{r}; δ_{a};]

where δ_t, δ_e, δ_a, and δ_r are throttle, elevator, aileron, and rudder deflections, respectively. In the control vector, the throttle and elevator deflections are used for aircraft vertical control, and rudder and aileron deflections are used for aircraft longitudinal control. State feedback is utilized and the control inputs are computed as follows

[\begin{matrix} u_{v} \\ u_{l} \end{matrix}] = - [\begin{matrix} K_{v} & 0_{2 \times 7} \\ 0_{2 \times 7} & K_{l} \end{matrix}] [\begin{matrix} {\tilde{x}}_{v} \\ {\tilde{x}}_{l} \end{matrix}]

where K_v and K_l are state feedback gains. The optimal LQR method is used to compute the state feedback gains with the weighting matrix defined by Brian and Frank¹⁸

\begin{matrix} Q = diag (1 / {\tilde{x}}_{i_max}^{2}) \\ R = diag (1 / {\tilde{u}}_{i_max}^{2}) \end{matrix}

Simulation

A simulation is discussed for validating the performance of the visual/inertial integrated landing guidance algorithm. In this experiment, the aircraft carrier is assumed to move at a speed of 20 knots (about 10.3 m s⁻¹), with governed by classic carrier dynamics.^19

–22 The simulation starts when the aircraft is 1500 m away from the aircraft carrier. The aircraft model in this control law simulation is assumed to be F-14. The carrier model is assumed to be the American Nimitz-class aircraft carrier, and the coordinates of the cooperated targets in the carrier runway frame are (15, −65, 0), (−15, −65, 0), (15, 175, 0), and (−15, 175, 0), respectively. Sea-state level 3 in the study by Sweger¹⁹ is chosen as the experimental condition. The root mean square (RMS) amplitudes of the heave, roll, and pitch are 0.91 m, 0.32°, and 1.08°, respectively. The air wake is considered using the model in the American Air Force manned aircraft flight quality specifications MIL-F-8785C.

The resolution and field of view of the vision system are 768 × 576 and 8° × 6°, respectively, whereas the output frequency is 25 Hz. The focal length calibration errors of the vision system are less than 0.2%. The target coordinate extracting errors are less than two pixels, and they decrease as the distance between the aircraft and the carrier decreases. The INS is assumed to have initial errors in attitude, velocity, and position measurements, and the drift error of the gyroscopes is 0.01° h⁻¹, whereas the accuracy of the accelerometers is 50 µg.

Guidance performance

The guidance simulation involves estimating the carrier motion as well as the relative position between the aircraft and the carrier. Figure 6(a) and (b) shows the aircraft carrier motion estimation errors and standard deviations. Figure 7(a) and (b) shows the aircraft motion estimation errors and standard deviations. The horizontal axes in these figures are the real longitudinal distances between the aircraft and the carrier.

Figure 6.

(a) The aircraft carrier motion estimation errors in one landing phase, including the roll, pitch, and heave and (b) the standard deviations of the aircraft carrier motion estimation errors calculated from 20-time carrier landing.

Figure 7.

(a) The aircraft motion estimation error, including the relative heading angle, the relative lateral position, longitudinal position, and height and (b) the standard deviations of the aircraft motion estimation errors calculated from 20-time carrier landing.

As shown in Figures 6 and 7, the initial estimation errors exist because of the initial attitude error from the INS, the focal length error, and so on. As the aircraft approaches the carrier, the estimation errors reduce in value. When the distance between the aircraft and the carrier is less than 400 m, the aircraft position estimation errors converge to 0.5 m in lateral distance, 0.8 m in vertical distance, 2.5 m in longitudinal distance, and 0.1° in the heading angle estimation. The carrier motion angle estimation error is less than 0.2°, whereas the heave estimation error is less than 0.8 m.

Several algorithms that can be utilized in vision-based carrier landing guidance system have been researched. For comparison, Newton’s iterative algorithm, an algorithm from École Polytechnique Fédérale de Lausanne (EPFL algorithm), and the planar homography algorithm have also been studied. The details of the other three algorithms can be found in the works of Ding et al.,¹³ Hajri,¹⁷ and Moreno-Noguer et al.²² and Yang et al.,²³ respectively.

Table 1 lists the data obtained by simulating the carrier landing 20 times and presents the max and standard deviation of estimation errors, and the max and average values of execution time when the longitudinal distance between the aircraft and the carrier is 400 m. As presented in Table 1, the estimation error of the algorithms based on EPFL and planar homography is too large to meet the landing requirements, so the standard deviations of the estimation errors of those two algorithms are not given.

Table 1.

Comparison of the estimation results using the four algorithms.

		Algorithm based on POSIT	Newton’s iterative algorithm	Algorithm based on EPFL	Algorithm based on planar homography
Execution time (ms)	Max	0.28	0.45	1.57	1.60
Execution time (ms)	Avr	0.24	0.29	1.11	1.13
Carrier roll (°)	Max	0.312	0.332	1.050	1.050
Carrier roll (°)	Std	0.119	0.141	—	—
Carrier pitch (°)	Max	0.068	0.090	0.335	0.335
Carrier pitch (°)	Std	0.031	0.034	—	—
Lateral distance (m)	Max	0.25	0.22	208.05	102.66
Lateral distance (m)	Std	0.15	0.13	—	—
Longitudinal distance (m)	Max	1.30	1.23	28.92	30.37
Longitudinal distance (m)	Std	0.62	0.65	—	—
Relative height (m)	Max	0.35	0.35	1.71	1.39
Relative height (m)	Std	0.09	0.09	—	—
Relative heading (°)	Max	0.050	0.043	15.477	15.477
Relative heading (°)	Std	0.025	0.021	—	—

POSIT: Pose from Orthographic Projection and Scaling with Iterations; EPFL: École Polytechnique Fédérale de Lausanne; Max: maximum; Avr: average; Std: standard deviation.

The algorithm proposed in this article has advantages in terms of the execution time and comparative accuracy when compared with Newton’s iterative algorithm. Considering the difficulty of the engineering application, execution time, and estimation accuracy, the algorithm in this article is slightly better than Newton’s iterative algorithm.

Control law

The simulation for the control law validation is discussed subsequently. Figure 8(a) shows the aircraft lateral position ${}^{S}x_{B}$ in the carrier body frame F_S and the off-glide-path lateral distance d_x, respectively, when the aircraft is landing on the aircraft carrier using the control law. Figure 8(b) shows the aircraft height ${}^{S}z_{B}$ and the off-glide-path vertical distance d_z. The results of the aircraft lateral and vertical control response can converge to 0.5 m in terms of position errors.

Figure 8.

(a) The aircraft lateral position in the carrier body frame and the off-glide-path lateral distance and (b) the aircraft height in the carrier body frame and the off-glide-path vertical distance.

Conclusion

In this article, a new visual/inertial integrated carrier landing guidance algorithm based on POSIT was designed to calculate the aircraft pose and carrier motion in the carrier landing operation. The simulation performed using this algorithm was discussed for validation, and the results show that this algorithm performs precisely with respect to calculating landing estimates and satisfactory control response. On comparing simulation results across algorithms, it is obvious that this algorithm has unique advantages in terms of the execution time, comparative accuracy compared with Newton’s iterative algorithm, and much better performance than that of the algorithms based on EPFL and planar homography. These results can be applied in carrier landing operations.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Yue Meng

References

Crassidis

Mook

Mcgrath

. Automatic carrier landing system utilizing aircraft sensors. J Guid Control Dyn 1993; 16(5): 914–921.

Zhimin

Guanxin

. Modeling and simulation on error sources of the designating system for fully automatic carrier landing. Res J Appl Sci Eng Technol 2013; 5(11): 3117–3123.

Wenling

Wenhai

Gaofeng

. Dissimilar redundancy structure design for carrier landing guidance computer and reliability analysis. Lecture Notes Electr Eng 2014; 297(2): 379–385.

Sousa

Wellons

Colby

. Test results of an F/A-18 automatic carrier landing using shipboard relative global positioning system. Patuxent River, Maryland: Naval Air Warfare Center Aircraft Division. Rep. NAWCADPAX/RTR-2003/122, 2003.

Schug

Aksteter

Huff

. Guidance and control for shipboard automatic landing using GPS. In: Proceedings of ION 57th annual meeting/CIGTF 20th biennial guidance, test symposium, Albuquerque, NM, 11–13 June 2001, pp. 852–862.

Sharp

Shakernia

Sastry

. A vision system for landing an unmanned aerial vehicle. In: Proceedings of international conference on robotics and automation, Seoul, Korea, 21–26 May 2001, Vol. 2, pp. 1720–1727. IEEE.

Saripalli

Montgomery

Sukhatme

. Visually guided landing of an unmanned aerial vehicle. IEEE Trans Robot Autom 2003; 19(3): 371–381.

Gui

Guo

Zhang

. Airborne vision-based navigation method for UAV accuracy landing using infrared lamps. J Intell Robot Syst 2013; 72(2): 197–218.

Anitha

Kumar

. Vision based autonomous landing of an unmanned aerial vehicle. Procedia Eng 2012; 38: 2250–2256.

10.

Yakimenko

Kaminer

Lentz

. Unmanned aircraft navigation for shipboard landing using infrared vision. IEEE Trans Aerosp Electron Syst 2002; 38(4): 1181–1200.

11.

Coutard

Chaumette

. Visual detection and 3D model-based tracking for landing on an aircraft carrier. In: Proceedings of international conference on robotics and automation, ICRA, Shanghai, China, 9–13 May 2011, pp. 1746–1751. IEEE.

12.

Coutard

Chaumette

Pflimlin

. Automatic landing on aircraft carrier by visual servoing. In: Proceedings of international conference on intelligent robots and systems, IROS, San Francisco, CA, USA, 25–30 September 2011, pp. 2843–2848. IEEE.

13.

Ding

Meng

. FLIR/INS/RA integrated landing guidance for landing on aircraft carrier. Int J Adv Robot Syst 2015; 12(5): 1–9.

14.

Trawny

Mourikis

Roumeliotis

. Vision-aided inertial navigation for pin-point landing using observations of mapped landmarks. J Field Robot 2007; 24(5): 357–378.

15.

Dementhon

Davis

. Model-based object pose in 25 lines of code. Int J Comput Vis 1995; 1 5(1–2): 123–141.

16.

Oberkampf

DeMenthon

Davis

. Iterative pose estimation using coplanar feature points. Computer Vision and Image Understanding 1996; 63(3): 495–511.

17.

Hajri

. UAV to UAV target detection and pose estimation. Doctoral dissertation, Naval Postgraduate School, Monterey, California, 2012.

18.

Brian

Frank

. Aircraft control and simulation. Hoboken, New Jersey, USA: John Wiley & Sons Inc, 1992.

19.

Sweger

. Design Specifications development for unmanned aircraft carrier landings: A simulation approach. U.S.N.A, Trident Scholar project report. Report no. 316, 1–71, 2003. The Rubicon Research Repository.

20.

Haitao

Xinmin

Wenchao

. Study of disturbances model on carrier-based aircraft landing process. Appl Mech Mater 2013; 321: 824–828.

21.

Deyi

Jianjun

. Carrier rocking motion model based on stress analysis. Appl Mech Mater 2014; 667: 430–436.

22.

Moreno-Noguer

Lepetit

Fua

. Accurate non-iterative O(n) solution to the PnP problem. In: Proceedings of 11th international conference on computer vision, Rio de Janeiro, Brazil, 14–21 October 2007. ICCV, pp. 1–8. IEEE.

23.

Yang

Qixin

Charles

. Pose estimation based on four coplanar point correspondences. In: Proceedings of the FSKD 2009 sixth international conference on fuzzy systems and knowledge discovery, Tianjin, China, 14–16 August 2009, Vol. 5, pp. 410–414. IEEE.