Abstract
The adaptive extended set-membership filter (AESMF) algorithm for robots' online modelling is today proposed for use in this field. Compared to the traditional ESMF, this novel filter method improves estimation accuracy under variable boundaries of unknown but bounded (UBB) process noise, which is often caused by the uncertainties of robotic dynamics. However, the applicability and stability of the AESMF method have not been tested in detail or demonstrated for real robotic systems. In this research, AESMF is applied for the actuator fault detections of a rotor-craft unmanned air vehicle (RUAV). The stability of AESMF is firstly analysed using mathematics and actuator healthy coefficients (AHC) are introduced for building the actuator failure model of RUAVs. AESMF is employed for the online boundary estimation of flight states and AHC parameters for fault tolerance control. Based on the proposed AESMF actuator fault estimation, flight experiments are conducted using a ServoHeli-40 RUAV platform and the flight results are compared with traditional ESMF and the adaptive extended Kalman filter (AEKF) in order to demonstrate its effectiveness, as well as for suggesting improvements for the actuator failure detection of RUAVs.
Introduction
Fault detection (FD) techniques have been widely adopted by many applications to detect faults in sensors and actuators, for example, in the process industry [1], for unmanned ground vehicles (UGV) [2] and fixed-wing aircrafts [3]. The structure of the fault tolerant controller, for example, the H∞ controller [3, 4] and adaptive controller [5, 6] can be reconstructed to maintain the best control performance when a fault in systems is detected by FD techniques [7]. With the development of redundancy sensor technology, FD-based tolerance control of non-backup actuator failure is becoming ever more important for robotic systems such as RUAVs. However, few FD methods for RUAVs have been proposed [8, 9] for tolerance control use and actuator fault modelling, such as jammed collective pitch surface, dead engine, and pitch surface effectiveness loss, remain unsolved for RUAV systems because of uncertainty regarding dynamics.
Due to the difficulty of obtaining a high fidelity full envelope model for fault detection, the multi-mode modelling technique has been proposed for rotor- aircraft such as tilt-rotor aircraft XV-15 [11] and the helicopters BO-105 [12], UH-60 [13], R-50 [14] and X-Cell [15]. A mode-dependent actuator failure model, which is simplified according to a specific flight mode such as hovering, cruising, taking off and landing, can be used for the corresponding flight mode. However, the mode-dependent actuator failure model still suffers from at least one problem: difficulties in accommodating the mode transition dynamics (uncertainty dynamics) of RUAVs, which creates a time-varying process noise and additionally, cannot be described by certain prior knowledge.
Recently, improvements in the sequence estimation method have made actuator fault estimation for RUAVs more feasible [16]. Among the statistical system estimation methods, the Kalman filter is the best known [17, 18, 19]. This filter is also known as a state estimator that utilizes the statistical knowledge of both the system measurement noise and the process noise (e.g., white noise) and achieves optimal estimation by optimizing the minimum function of the estimation residuals. Furthermore, its algorithm representation includes predictions and updates, which is convenient in online applications. Hence, this method is widely used and has been developed to address nonlinear system estimation, thus extending its application field, for example, the extended Kalman filter (EKF) [20], the unscented Kalman filter (UKF) [21] and the derivative-free nonlinear Kalman filter [22].
However, these Kalman filters require statistical knowledge of the system measurement noise and of the process noise in general, or else it assumes that those noises meet some distributions in order to solve optimal problems in real applications such as RUAVs' fault detection [23, 24]. However, in reality, the system and environment noise characteristics are typically quite complicated, time varying and difficult to accurately measure. In particular, for RUAV systems, mechanical manufacturing accuracy, sensory noise and air flow disturbances will result in uncertain statistical noises. The mismatching of a noise's statistical characteristics will cause filter bias. Moreover, due to the Kalman filter's noise sensitivity, this bias is amplified and causes estimator instability. Therefore, many adaptive schemes, e.g., the adaptive unscented Kalman filter (AUKF) [25] have been introduced to Kalman estimators in order to achieve online noise estimation of the system's process noise distribution, as well as to compensate for filtering faults. However, statistical dependency and sensitivity of the Kalman estimator limit its system applications.
In most real systems, although noise's statistical characteristics are unknown, they can be assumed as an unknown-but-bounded (UBB) noise. Due to the listed defects of Kalman filtering, researchers have proposed a set-membership filter (SMF). Based on the UBB assumption, this filter provides a feasible state boundary by calculating the available set members. Thus, the estimation result is a viable state set rather than a single value. This viable state set describes possible ranges of the estimated parameters and makes sure that the real value falls within this set. Scholte and Campbell [26] expanded this ellipsoid algorithm into a nonlinear system and proposed an extended set-member filter (Extended SMF (ESMF)). This follows the same mechanism for Kalman filtering by linearizing nonlinear systems. However, the difference is that ESMF uses Taylor expansions rather than nonlinear system linearizations. This method changes inaccurate linearization into process noise in order to accomplish nonlinear system estimation. Yet to solve its numerical instability, poor real-time performance and difficulties in the tuning of the filter parameters, Zhou proposed a UD factorization-based extended set-membership filter [27]. This method represents and updates the envelope matrices using UD factorization. The method combines measurements in an updating strategy to improve the algorithm's stability and reduces its complexity, and has been used for online modelling of RUAVs by the authors in real aggressive flight control experiments [28].
However, the described method – the ESMF method – still requires the priori boundary information of the system noise for a RUAV's actuator estimations. Specifically, this algorithm needs to know the uncertain boundaries of the process and measurement noise. Generally, measurement noise depends on sensory accuracy. If the sensors work under normal conditions, the uncertain boundary does not drift. Therefore, measurement of the uncertain boundary from the sensor can be decided by analysing the sensor data; on the other hand, process noise arises from the mismatch between the system state space model and the system's real dynamics, such as mismatched parameters for RUAVs. Therefore, it is necessary to manually tune the boundary parameters, since the priori information of uncertain boundaries are unknown. Meanwhile, as prior knowledge is limited, manually setting the process noise's uncertain boundary may not include, or may well exceed the real process noise boundary. This will lead to the ESMF's estimated uncertain boundary being too large to deviate and will cause the filter to diverge during estimation of robots' UBB process noise, such as in the case of an RUAV's actuator fault. To solve this problem, the authors have proposed an AESMF [29] to address the process noise uncertain boundary issue for online robots' dynamics estimation. This method applies gradient descent optimization rules and conducts online calculation to obtain the minimal ellipsoid boundaries of the prediction error, which also ensures that the filter's health-index satisfies the working conditions for uncertainty dynamics. However, the stability condition and applicability of the AESMF are not verified by theory or real experiments, which limit its applications for robots' online modelling.
In this paper, firstly, the AHCs are introduced for building the model of RUAVs' actuator failures. Then, the AESMF is reviewed and applied to estimate both the states and the AHCs parameters in real time, and stability is demonstrated by a theorem. Finally, real flight experiments with the ServoHeli-40 RUAV test-bed are conducted and performance comparisons for fault modelling, among them AESMF, AUKF and AEKF, are discussed in order to verify their applicability for RUAVs' actuator fault detections.

Body diagram of RUAV wit with respect to body coordination system
RUAV Dynamics Modelling
RUAV dynamics obey the Newton-Euler equation for rigid body in translational and rotational motion. Here, we considered a typical rigid RUAV in/near hover flight and the dynamic equation is conveniently described with respect to the body's coordination system, written is as:
where m ∊ R is the mass of the helicopter, VB ∊ R3×1 is the velocity vector for body frame, Ω
B
∊ R3×1 is the rotary speed vector for body frame, I ∊ R3×3 is inertia matrix,
The free body diagram of the RUAV with respect to the body's coordination system is as shown in Figure 1. By employing the lumped-parameter approach [30], which considers the RUAV as the composition made up of the main rotor, tail rotor, fuselage, horizontal stabilizer and vertical stabilizer, these components can be considered as the sources of forces and moments. The extern al force and moment, in hovering flight mode, can be written as:
where RB ∊ R3×3 is the transition matrix between navigation frame and body frame, and the subscripts 'F' refers to the forces and moments in the helicopter's body frame. The forces XT and ZT, which are actuated by the tail rotor along the X-axis and Z-axis, are ignored as XT, ZT ≈ 0 for normal helicopters in (2). The forces XV and YV, which are actuated by the tail stabilizer along the X-axis and Y-axis, can be ignored as XV, YV ≈ 0 in hovering flight mode [31].
The forces and torques generated by the main rotor are controlled by TM, a1 and b1. The tail rotor is considered a source of pure lateral force YT and anti-torque QT, which are controlled by TT. Thus, the forces and moments can be expressed as [31]:
where TM, TT are the force of the main rotor and tail rotor, a1, b1 are longitudinal and lateral flapping angle, QM, QT are the moments caused by the main rotor and tail rotor. Based on reference [32], Ti and Qi, where i={T,M}, can be calculated as follows:
where
where R1T is the radius of the tail rotor, R1M is the radius of the main rotor, R0T is the inner radius of the tail rotor, R0M is the inner radius of the main rotor, Ω T is the rotation speed of the tail rotor, Ω M is the rotation speed of the main rotor, bM is the number of the main rotor's blade, bT is the number of the tail rotor's blade, cd is the thrust coefficient of the main rotor, aT is the gradient of lift curve for the tail rotor, aM is the gradient of the lift curve for the main rotor, ρ is the density of air, cT is the width of the tail rotor, cM is the width of the main rotor, θcM is the collective pitch angle of the main rotor, θcT is the collective pitch angle of the tail rotor and Vc is the vertical speed of RUAV. All of the above parameters, which can be measured or identified, are shown in Table 1.
The identified model parameters
Let control output Uout = (θcM θcT a1 b1), then we can obtain a RUAV dynamics model based on (1) ∼ (4), which has the following state-space structure:
where x = (VBT Ω BT ) T is the state vector, y is the measurement vector and f(x, Uout) is a non-linear mapping function.
The actuator failures of the RUAV include the control surface jam, control surface bias and partial loss in the actuator effectiveness [33]. Define Uin as the inputs (desired surface position) and Uout as the outputs (real control surface position) of the actuators. The fault tolerant architecture then assumes the following actuator model:
where
and where γi and δi are the proportional effectiveness and failure bias of the i-th actuator's AHCs with i = {1, 2, 3, 4} and the range of Uin is defined as in [0,1]. Definition (6) means that control surface position bias is Δ f and (1 1 1 1) T −Γ f loss occurs in actuator effectiveness. For example, if γ2 = 0, this means the tail rotor is jammed at position δ2; if δ2 = 0, this means the tail rotor experiences 1-δ2 loss in effectiveness.
The AHC expression (6) aims to make the proportional effectiveness and failure bias of actuators parametric, so that a fault model can be built using online estimation [34]. The estimation model will be built in the following part.
To estimate the AHCs' parameters, we built an extended dynamics model for the joint estimation of states and parameters of RUAVs. To do this, AHCs and state vectors were concatenated into a single augmented state vector at discrete time k:
Thus, estimation was done recursively by writing the dynamics (1 ∼ 6) in discrete equations for the joint state as the following non-linear structure:
where
The AHCs' joint estimation has advantages over some existing modelling techniques, where only proportional loss in effectiveness has been considered. However, the distribution of
Hence, in the flowing parts, the AESMF method will be reviewed first and its stability demonstrated by a theorem. Following on, the AESMF will be used to estimate the AHC actuator fault model (8) and compare it to traditional ESMF and EKF via experiments.
AESMF Algorithm Review
The nonlinear system considered here is expressed as:
where xk ∊ Rn and yk+1 ∊ Rm are the state and measurement vectors, respectively, f() and h() are nonlinear C2 functions, wk ∊ Rn and vk ∊ Rm are the process and measurement noise vectors, respectively, and they meet the necessary conditions:
where E(a, P) stands for an ellipsoid set as:
where a is the centre, x is any point within the ellipsoid and P is a positive-definite matrix.
The initial estimation ellipsoid is defined as
where
Set
Intial conditions:
Construction of the noise boundaries at times k+1:
where XR2 is the interval of the Lagrange remainder of f(xk), Time update at time k+1:
where Measure update at time k+1:
where Process noise boundaries update at time k+1:
where tr{•} is the track of a matrix and
where, i = 1, 2, ‥, n and j = 1, 2, ‥, n. ΔT is the sampling time, and ηk ∊ R, which is selected by hand, is the parameter for converge rate.
Since this adaptive strategy for process noise boundary is based on gradient descent (GD), according to step 5), this type of AESMF is referred to as GD-ESMF in following experiments. For the details about the GD-ESMF algorithm, please refer to [29].
Existing Initial process noise ellipsoid satisfies:
then, AESMF is stable and there is:
proof:
Let
Make the Lyapunov function as:
According to the proof of reference theorem [26], the following inequalities exist:
where
and qi is the boundary of process noise
According to condition 2), there is:
Because
According to condition 2), as
Consider
Hence, GD rules (39) are stable and:
Which means
In the following parts, a simulation and flight experiments are both conducted to compare performance among AESMF, AUKF and AEKF for actuator fault estimations.
Simulations and Flight Experiments
To test the performance of the proposed AESMF (10∼16) for actuator fault estimation based on AHCs (5 and 6), firstly, a Servoheli-40 RUAV test-bed is introduced and the parameters of model (5) are identified, based on the collected flight data from the ServoHeli-40 by remote control. Then, based on the AHC model (6), the estimation methods such as AESMF, AUKF and AEKF are all applied for estimation. The results are compared in both the simulation and real flight experiments.
Flight Test Platform
All flight tests were conducted on the Servoheli-40 RUAV, which was developed in our laboratory. It is equipped with a 3-axis gyro, a 3-axis accelerometer, a compass and a GPS. The sensory data can be sampled and stored onto an SD card through an onboard DSP. More details of this experimental platform can be found in [35]. The measurement accuracy of attitude is 0.1 degrees and the measurement accuracy of velocity is 0.01 m/s; measurement accuracy of the position is 0.1 m and the measurement accuracy of acceleration is 0.01 m/s2. Figure 2 shows the test-bed RUAV. The parameters of model (8) in hovering mode are identified by the frequency response method [36] and are listed in Table 1. The matching accuracy is shown in Figure 3. The following experiments were conducted based on the there identified dynamics model.

SERVOHELI-40 small-size RUAV

The results of the model accuracy test
The proposed failure estimation scheme was firstly tested using the ServoHeli-40 mathematical model, which was identified with the real flight data from the hovering experiments, as above. Here, we combined the fault detection algorithm with a feedback linearization control scheme to perform the flights in our hardware-in-the-loop (HIL) environment [37], as shown in Figure 4; this was conducted due to the danger of actuator faults in real flight.

Hill environment for RUAV simulations
AUKF-based, the traditional ESMF-based and proposed AESMF-based failure estimation schemes were tested together under coupled actuator failures as:
This means that the roll control surface was 60% loss and its position bias was 15%, and the tail rotor control surface was jammed at a 15% position. Initial process noise ellipsoidal boundary Q0 was set larger than the real boundary as:
To test the advantage of ESMF-based methods in fault parameters' boundary estimation, compared to AUKF (3σ bound), the real process noise wk was generated as coloured ellipsoidal noise, which often happens in real practice as:
Figure 5 shows the AHC-bound estimation comparison among AUKF, ESMF and proposed GD-ESMF. It can be seen that AUKF's AHC boundaries is unable to cover the real fault parameters, because of the coloured process noise. AUKF requires some distribution noise; therefore, estimation is biased under coloured ellipsoidal noise. ESMF and the proposed GD-ESMF method for AHCs boundaries both can cover true value during estimation; however, the true values are not at the middle of the estimated ellipsoidal bound for ESMF, because of the biased noise. The proposed GD-ESMF obtains the best estimated boundaries for AHCs, because GD strategy regulates the ellipsoidal boundaries to real boundaries and the true values are almost at the middle of its ellipsoidal boundaries.

AUKF, ESMF, GD-ESMF and AHC estimation performance comparisons under coloured noise
The above is shown by taking a simulation where the proposed GD-ESMF-based actuator fault modelling for RUAVs has the best estimation performance of uncertainty boundaries and comparing it with AUKF and ESMF. However, because of its lowest computation cost among the three methods, adaptive KF-based estimators are often used for fault detection, although its performance is worse than GD-ESMF under the uncertainty dynamics in the above simulation. However, the adaptive KF-based method cannot work as a proper estimator for the fault model based control of RUAVs; this conclusion will be verified by a real flight test in the following part.
In this part, the AESMF is compared with an adaptive KF-based estimator for the online fault model based control of RUAVs in flight experiments. The computational burden of AUKF is large, such as the root mean square matrix, which cannot be realized efficiently in the embedded flight controller under C++ programing. Thus, AEKF was instead adopted for the comparisons with proposed GD-ESMF in real flight experiments in this part. To test the AEKF and proposed GD-ESMF estimation for actuator fault, the following experiment was designed using a ServoHeli-40 RUAV platform.
As shown in Figure 6, the flight system's structure includes an onboard part and a ground part. The onboard part includes the flight controller, fault generator and steering gears, and the ground part includes the remote controller for the pilot and the ground station. In the following experiments, a remote controller is used to have the ServoHeli-40 take on/off. There is a button on the remote controller for the pilot that can change the flight mode between manual control and autonomous control. For flight safety, the pilot can change the flight mode into manual mode at any time during the flight. The ground station can transmit the actuator fault parameters in model (6) to the onboard fault generator, which actuates the steering gears to simulate the actuators' faults, based on the parameters in the following tests. The electronic devices, such as the flight controller, onboard fault generator and ground station, were all developed by the authors for experimental use. All of the hardware and software were also created by the authors, so that the model and control algorithm could be freely embedded by C++ programing. The details of the flight control system can be referred to in [35].

The system's structure for the flight test
In autonomous control mode, the onboard flight controller can track the reference position and velocity of the helicopter based on the PI controller. The attitude stabilization loop is designed as H∞ controller, based on the identified model for the fault detection performance test. The performance of the tested controller in non-fault flight is shown in Figure 7, which has good position and velocity tracking performance in autonomous hovering mode and can be used for the fault detection flight test. The detail test procedure was designed as follows:
when the ServoHeli-40 is hovering, the actuator fault is simulated by the software of the embedded fault generator as:
Based on the identified model in Part 4.1, AEKF and GD-ESMF were designed based on model (8). Based on the online estimated boundaries of AHCs by AEKF (3σ-boundary) and linearization of (8) at hovering mode, a standard H∞ controller [38] for hovering control was designed. Based on the online estimated boundaries of AHCs by GD-ESMF, the same H∞ controller for hovering control was designed as in (3). The attitude rates and estimated boundaries were both recorded at 50HZ and analysed to demonstrate the advantage of the proposed GD-ESMF for actuator fault AHC estimation.

The performance of the flight controller in hovering mode: (a) velocity tracking; (b) position tracking
According to Figure 8, the designed H∞ controller worked well prior to a fault occurring; all the attitude rates were below 0.1 rad/s. However, after the simulated actuator fault happened (after a sampling time of 21500), the AEKF-based AHCs' boundary estimations were biased and could not cover the true values (see Figure 9); therefore, the H∞ controller obtained bad boundaries and rendered the RUAV unstable in hovering mode. It can clearly be seen from Figure 8 (a) that the attitude rates were all above 0.3 rad/s and fluctuating; at sampling time point 23000, the pilot changed the flight mode into remote control by hand to stop the simulation actuators faulting in the flight controller and to avoid the danger of crashing.

Tolerance control performance when actuator fault occurs

AEKF and GD-ESMF AHC parameter estimation performance when actuator fault occurs
On the other hand, for the proposed GD-ESMF estimation method for actuator faults, the estimated boundaries were able to cover the true value of AHCs; therefore, the designed H∞ controller was able to continuously keep the RUAV stable in hovering mode. This result shows that GD-ESMF for online fault modelling is more effective for actuator fault tolerance control when used in RUAVs, compared with the AEKF- based estimation method; thus, AEKF cannot be used under this flight mode.
The adaptive strategy of AEKF is the same with AUKF and only the precision of estimated mean value is lower than AUKF, due to linearization by the first order. The precision of estimated uncertainty boundaries is almost the same for AEKF and AUKF. Hence, compared with AEKF and AUKF, the above results were sufficient to verify that the AESMF has the best performance of computation efficiency and the best precision of estimated uncertainty boundaries for real applications, due to its adaptive strategy.
This paper proposed an AESMF application for actuator fault detection in RUAVs with uncertainty dynamics. AESMF's stability was firstly demonstrated mathematically and applied for on-line actuator failure modelling, based on AHCs. Compared with traditional ESMF, AUKF and AEKF, the results of flight experiments showed that the proposed scheme can automatically compensate for failures and effectively track the boundaries of fault parameters under UBB process noise, which is important for the fault model base control of RUAVs. The applicability of the proposed actuator fault detection algorithm was also verified in this flight test.
Footnotes
6.
This research was supported by The National Natural Science Foundation of China under Grant: 61203334 and 61433016.
