Sage Journals: Discover world-class research

Abstract

Multicopters are the most popular rotary type of unmanned aerial vehicles. They are a type of helicopter with three or more, usually fixed-pitch, propellers that lift and control the platform by individually changing their rotational velocities. The main advantages of a multicopter are its compactness, robustness, and low cost to build and repair. However, currently no published research determines objectively, quantitatively, and experimentally, the maneuverability and agility of multicopters. Numerous maneuverability and agility metrics, together with detailed test procedures and minimum requirements, exist for manned aircraft. Nevertheless, some of these are not directly applicable to small-size unmanned aircraft. A new test procedure, derived from manned aircraft industry practices and research, based on a simple open-loop step input maneuver, was developed. It experimentally determines nine maneuverability and agility metrics using only onboard flight controller logs. The test procedure is validated using two different multicopters.

Keywords

Rotary unmanned aerial vehicle maneuverability agility quantification

Introduction

In recent years, Unmanned Aerial Vehicles (UAVs) are emerging which must perform increasingly complex missions in urban environments. As some UAV types, such as multicopters, are inherently unstable, low-level stability control modules were developed. Over the last few years, these control modules evolved into fully autonomous flight controllers.¹ Rotary (or rotorcraft) UAVs are of interest for various missions such as short-range parcel/medicine transportation, close-range infrastructure inspection or, just for fun, drone racing.² Most UAV control research is focused on optimal control, obstacle detection, and autonomous navigation,^3,4 yet very little is focused on quantifying the inherent high maneuverability and agility of small UAVs, more specifically multicopters (or multirotors). As multicopters evolve both to extremely large and small sizes, retaining high agility is important. The higher the maneuverability and agility, the better the multicopter can counter turbulence and gusts reducing drift from its desired position which was the main motivation for this work. This becomes especially important for close-range inspections in confined areas. An objective and experimental test procedure is needed to objectively quantify a small UAV’s maneuverability and agility. In addition, this procedure enables comparing new prototypes or configurations to existing small UAVs to evaluate differences in maneuverability and agility.

Literature only provides metrics and experimental test procedures for large manned aircraft. The size and thus higher inherent agility of small rotary UAVs make it difficult to use the same metrics and methods directly. For this reason, research was conducted to find a simple experimental flight test procedure and corresponding set of objective maneuverability and agility metrics that can be extracted purely from flight controller logs to avoid the need for expensive external measurement systems. This method and its metrics are specifically suited for small (<25 kg) rotary UAS but are potentially applicable to fixed-wing UAS as well. The novelty of this paper lies in this new and simple experimental test procedure, its set of metrics, and its focus and validation on small rotary UAS. Its main advantage is that any UAV with a typical flight controller that can log its (raw) inertial measurement unit (IMU) data fast (≥50 Hz), can use this method to easily determine its maneuverability and agility. Its main disadvantage is that the quality of the IMU determines the accuracy of the agility results as no external positioning system is used, although for comparing UAVs with similar IMUs, this disadvantage diminishes greatly. In addition, the focus here is more on finding a representative set of metrics with good indicative quantification of the UAV’s maneuverability and agility rather than on absolute accuracy and calculating all possible metrics.

The next section recalls the definitions of maneuverability and agility together with other research from which the selected metrics are chosen and listed in “Selected maneuverability and agility metrics” section. “The experimental test procedure and setup” section describes the flight test procedure while in “Results and discussion” section the results are discussed. The final section states the conclusions and future work.

Related work

Maneuverability is defined as the ability to change the aircraft flight path by application of forces from the rotors or other control devices.⁵ In other words, maneuverability is the measure of the maximum achievable time rate of change of the velocity vector at any point in the flight envelope.⁶ Selected metrics (“Selected maneuverability and agility metrics” section) employed in this paper are control power (CP), attitude quickness, peak angular rate, and peak angular acceleration. Agility is defined as how quickly the aircraft flight path can be changed.⁵ In other words, agility is the measure of the maximum achievable time rate of change of the acceleration vector at any point in the flight envelope.⁶ Selected metrics (“Selected maneuverability and agility metrics” section) employed in this paper are time-to-peak angular acceleration, time to achieve 20° attitude angle, attitude change over first 0.2 or 1 s, and bandwidth based on 10–90% rise time.

The literature review below focuses on finding a simple experimental test procedure and corresponding objective quantitative maneuverability and agility metrics representative of the vehicle’s performance. Many more procedures and metrics exist in literature, some more accurate, yet they are more difficult and/or time consuming.

Racing drones and new UAV configurations and/or prototypes must trade off increased endurance or heavier payload capabilities against maneuverability and agility. Mehmood et al.⁷ investigated the influence of tilting all propellers on the agility of a hexarotor by comparing peak accelerations along the six degrees of freedom (“Selected maneuverability and agility metrics” section). Karimi et al.⁸ varied a set of metrics over the entire flight envelope of a fixed-wing UAV to find feasible combinations and, from this, find their influence on the minimum time required to perform pull-up and turn maneuvers. Campbell⁹ developed a method to quantify the maneuverability and gust tolerance of MAVs using reachability and disturbance sensitivity sets requiring system identification. The literature discussed above uses models, system identification, and simulation to identify a limited number of maneuverability and/or agility metrics. It does not provide a comprehensive set of metrics nor any experimental testing procedure. From these studies, only peak acceleration was retained as a maneuverability metric (“Selected maneuverability and agility metrics” section). As literature focused on maneuverability and agility of UAVs is scarce, manned aircraft literature was reviewed to find and/or derive potentially suitable metrics and methods for experimentally quantifying the maneuverability and agility of a small (rotary) UAV.

Most manned helicopter maneuverability and agility requirements and metrics were set up by the U.S. Armed Forces for the development and evaluation of new helicopters before the 1990s. Handling qualities of aircraft express the ease of flying for the pilot. Over the last two decades, the Aeronautical Design Standard 33 (ADS-33), the current version being E,¹⁰ is considered the most comprehensive set of both objective and subjective maneuverability, agility, and handing qualities requirements available. However, several (groups of) metrics are not directly applicable as most UAVs fly with the low-level stabilization controller active for safety and controllability reasons. Most UAVs are also controlled by an RC transmitter using “finger-controlled” sticks instead of a center or sidestick, rendering control force requirements invalid. ADS-33E also employs mission task elements (MTEs) that receive handling qualities ratings (HQRs) from test pilots based on the subjective Cooper–Harper scale, thus requiring multiple test pilots to perform these maneuvers. HQRs are less important for UAVs using low-level stabilization controllers as the flying workload is already greatly reduced. Thus, from ADS-33E, the most interesting metrics that were retained here are CP and attitude quickness, Q (“Selected maneuverability and agility metrics” section). Floyd et al.¹¹ performed one of the few experimental maneuverability and agility analyses of a manned helicopter and evaluated the following metrics: attitude change over 1 s, ${Δ α}_{1 s}$ ; peak angular acceleration, ${\ddot{α}}_{p e a k};$ time-to-peak angular acceleration, $T_{{\ddot{α}}_{p e a k}};$ peak angular rate (=CP); and time-to-peak angular rate, ${T_{\dot{α}}}_{p e a k}$ . All are retained here (“Selected maneuverability and agility metrics” section). Liefer et al.¹² and Paranjape and Ananthkrishnan¹³ investigated suitable agility metrics for fixed-wing fighters and found among others: time to catch a certain angle (stabilized); peak pitch/roll rates (=CP); and time to reach a certain angle (unstabilized), T_xx°. The latter two are retained here (“Selected maneuverability and agility metrics” section). Yilmaz and Pavel¹⁴ and Padfield¹⁵ investigated bandwidth and phase delays, based on ADS-33 criteria, as they are, and rightly so, considered primary precision task quality metrics. During our research a specific flight test procedure was developed, several flight tests were performed, and the bandwidths and phase lags were extracted through frequency response techniques for pitch, roll, and yaw, however, these were only partially successful. The main reason is that as the flights are performed manually, it is very difficult for the pilot to physically perform these small high frequency inputs, a common problem also reported by test pilots.¹⁵ Given the much higher peak angular rates of small UAVs compared to large manned helicopters, this becomes even more problematic. Automated control inputs would definitely work here but are outside the scope of this paper. Instead, the bandwidth was calculated based on the 10–90% rise time of the UAV’s attitude change (“Selected maneuverability and agility metrics” section).

Selected maneuverability and agility metrics

From literature and flight test experience, nine maneuverability and agility metrics were selected that can be numerically determined in a relatively simple and objective manner using only the flight controller logs. During the flight tests the low-level stabilization is active for safety reasons and therefore the UAV has an attitude command type of response for roll and pitch, while yaw has a rate command type.

Control power: CP [°/s]

CP corresponds to the peak angular rate $({\dot{α}}_{p e a k} = [p_{p e a k}, q_{p e a k}, r_{p e a k}])$ that the UAV can achieve about one of its axes (roll, pitch, and yaw, respectively).^7,10–12 If it is not possible to obtain it directly in a safe manner, it can be extrapolated from a moderate pilot step input as in equation (1):

C P = \frac{{\dot{α}}_{p e a k}}{% i n p u t_{p i l o t}}

(1)

The method is somewhat less robust as it is sensitive to the pilot’s aggressiveness and total input. Typical pilot input is around 40%. The pilot’s % input can be determined by finding the relationship between the pilot’s control stick travel and the desired attitude/rate sent to the stabilization low-level controller which is logged by the autopilot. The achievable attitude change from trim (for attitude types of response, i.e. roll and pitch) or angular rate (for rate types of response, i.e. yaw) for ADS-33E Level 1 aggressive agility must be larger than roll attitude Δϕ_peak ≥ 60°, pitch attitude Δθ_peak ≥ 30°, and yaw rate r_peak ≥ 60°/s. The specified attitudes or rates shall be achieved in each axis while limiting excursions in the other axes with the appropriate control inputs.

Attitude quickness: Q (1/s)

The attitude quickness responds to the ratio of peak angular rate $({\dot{α}}_{p e a k} = [p_{p e a k}, q_{p e a k}, r_{p e a k}])$ to change in attitude $(Δ α = [Δ ϕ, Δ θ, Δ ψ])$ for roll, pitch, and yaw, respectively, as in equation (2) 10:

Q = \frac{{\dot{α}}_{p e a k}}{Δ α}

(2)

The test is performed by the pilot giving a pulse control input to achieve a certain angle (5°–60°). As the low-level stabilization is active, an open-loop step input is equivalent here. The required attitude changes are performed as rapidly as possible from one steady attitude to another without significant reversals in the sign of the pilot’s control input relative to the trim position. ADS-33E requires an attitude change between 5° in pitch (10° in roll) and the limits of the operational flight envelope or 30° in pitch (60° in roll and yaw), whichever is less. The minimum quickness levels for attitude quickness are shown in Figure 1. Attitude quickness is considered more robust than CP as it measures the actual peak rate and considers if the rate was achieved for a low or high change in attitude angle.

Figure 1.

Attitude quickness requirements ADS-33E.¹⁰ADS-33E: Aeronautical Design Standard 33.

Time-to-peak angular rate: $T_{{\dot{α}}_{peak}} (s)$

The time-to-peak angular rate is the time between pilot (step) input and UAV peak angular rate as measure of response time.¹¹ The lower the time, the faster the UAV responds to pilot inputs. This metric is not employed by ADS-33E, so no reference value is available.

Peak acceleration: ${\ddot{α}}_{peak} (\circ / s^{2})$

The peak acceleration ${\ddot{α}}_{peak} = [{\dot{p}}_{p e a k}, {\dot{q}}_{p e a k}, {\dot{r}}_{p e a k}]$ is the peak initial angular acceleration of the aircraft following a step input command, measured in °/s² This metric is not employed by ADS-33E, so no reference value is available.⁷

Time-to-peak angular acceleration: $T_{{\dot{α}}_{peak}} (s)$

The time-to-peak angular acceleration is the time between pilot (step) input and UAV peak angular acceleration, i.e. a measure of response time.¹¹ The lower the time the faster the UAV responds to pilot inputs. This metric is not employed by ADS-33E, so no reference value is available.

Attitude change over 1 (first) second: ${Δα}_{1s} (\circ)$

This metric, $Δ α_{1s} = [Δ θ_{1 s}, Δ ϕ_{1 s}, Δ ψ_{1 s}]$ , is the attitude change achieved 1 s after the start of the pilot step input.¹¹ This metric is not employed by ADS-33E, so no reference value is available.

Attitude change over first 0.2 s: ${Δα}_{0.2s} (\circ)$

This is a new metric derived from the attitude change over 1 s. It is the attitude change achieved 0.2 s after pilot step input: $Δ α_{0.2s} = [Δ θ_{0.2s}, Δ ϕ_{0.2s}, Δ ψ_{0.2s}]$ . The reason for creating this new metric is that the pitch and roll rates are so high (see “Results and discussion” section) that measuring them over 1 s would mean that the multicopter would have tilted more than 200° in that period, which is possible but impractical and dangerous to perform as it would be flying upside down at the end of the maneuver.

Time to achieve xx° attitude angle: T_xx° (s)

Measures the time required for an aircraft to pitch (roll, yaw) through xx° starting from zero initial angle. The input commanded is full stick/pedals. The pilot does not have to stop at this attitude angle; he should just apply max input. Usually a change in attitude angle of 90° is performed, but this can be less (45°), whatever is feasible.^12,13 This metric is not in ADS-33E, so no reference value is available.

Bandwidth based on rise time 10–90%: $ω_{B W}$ (Hz)

Frequency response function analysis of flight tests in which the pilot must do sinusoidal inputs with increasing frequency is very hard to do manually by the pilot. As an alternative, one can estimate a representative bandwidth based on the 10–90% rise time of the UAV’s measured attitude change. The formula assumes a second-order behavior and a damping ratio of 0.7 in equation (3).¹⁶ This metric is not in ADS-33E, so no reference value is available.

ω_{B W} = \frac{0.35}{T_{90 %} - T_{10 %}}

(3)

The experimental test procedure and setup

The experimental test procedure is straightforward. The test starts from stabilized hover at low safe altitude. The pilot commands a step input of 50–100% of total stick travel and holds this input until the desired attitude (rol, pitch) or rate (yaw) is attained for 1 s. This should be done for roll, pitch, and yaw individually. Each test should be performed at least three times to ensure reliable data as human pilots are not capable of commanding the exact same input every time. If the rotary UAV is asymmetrical, it is advised to perform the maneuver in both directions (i.e. left and right) to individually determine the maneuverability and agility. However, for most rotary UAVs this is not the case and performing the test in one direction is sufficient as validated in flight tests. It is recommended to determine each agility metric based on the median of the test results of each axis. It is important that the pilot practices the application of as much as possible identical inputs both in aggression and precision to obtain high-quality and conclusive results. Good metrics to evaluate and compare the quality of the pilot inputs for each set of maneuvers are CP, peak angular acceleration, and time-to-peak angular acceleration. Depending on whether the tests are performed indoor or outdoor, the input can be more or less aggressive as one might have limited flight area. Outdoor tests should preferably be performed in zero wind conditions to negate external aerodynamic effects and at temperatures between 15° and 25° for nominal battery capacity and functionality.

After the flight tests, the flight logs were downloaded and converted into Matlab data files. A custom Matlab script was used to display the relevant data for (manually) extracting the required data points (see Figure 2). An Excel file contained several tables in which these data points were filled to calculate the final agility metrics. An example flight test data log of one of the UAVs used to validate this procedure together with the Matlab and Excel data processing files are freely distributed to the community (https://lirias.kuleuven.be/handle/123456789/570034). Ample documentation is given in these files to facilitate ease of use.

Figure 2.

Example log data of the proposed experimental test procedure. Data displayed is of roll step input with pilot desired roll attitude (green) and measured roll attitude (blue).

The test procedure was validated using two very different multicopter UAVs: a conventional hexacopter DJI Flamewheel F550 (Figure 3) and a custom compound multicopter prototype (Figure 4).¹⁷ Their specifications are given in Table 1. These two vehicles were chosen for their difference in configuration, weight, inertia, and moment arms between propellers. The compound configuration uses large propellers that rotate at constant speed to lift a significant portion of the weight, while the small propellers perform all control similar to a quadrotor. These control rotors are tilted (10°) to improve the yaw response due to the high directional inertia of the long design. The DJI F550 was flown in plus configuration: one arm front and back, two arms left and right. Both vehicles are controlled by a 3DR PixHawk flight controller running stock ArduPilot (ArduCopter 3.4) firmware.¹⁸ The flight controller logger was set to include IMU and attitude logging @50 Hz. This fast logging is essential as it directly affects the accuracy of the derived agility metrics. The user controlled the UAV through the PixHawk by a Futaba RC transmitter. Roll and pitch were attitude commanded, yaw was rate commanded. The low-level stabilization and altitude hold mode were active.

Figure 3.

DJI Flamewheel F550 hexarotor.

Figure 4.

Custom compound multicopter for narrow corridors.

Table 1.

Specifications test vehicles.

Parameter	Units	DJI F550 (Plus-config.)	Compound
Mass	g	1800	2772
Longitudinal distance between extremities	mm	800	1405
Lateral distance between extremities	mm	710	642
Longitudinal moment arm distance between propellers	mm	550	1132
Lateral moment arm distance between propellers	mm	550	1132
# Propellers × propeller diameter × propeller pitch	mm	6 × 203 × 114(6 × 8 × 4.5 inch)	2 × 406 × 140 (2× 16× 5.5 in.)
# Propellers × propeller diameter × propeller pitch	mm	6 × 203 × 114(6 × 8 × 4.5 inch)	4 × 239 × 127 (4× 9.4× 5.0 in.)
Speed controllers	–	6 × DJI 30A Opto	2 × Castle Creations Talon 35A
Speed controllers	–	6 × DJI 30A Opto	4 × DJI E310 20A
Motors	–	6 × DJI 2212 920 kV	2 × Tiger Motors MT2814-11 710 kV
Motors	–	6 × DJI 2212 920 kV	4 × DJI 2312 960 kV
Battery (Lithium Polymer)	–	Kypom 4S 4500 mA h 35C	Kypom 4S 4500 mA h 35C
Battery (Lithium Polymer)	–	Kypom 4S 4500 mA h 35C	ThunderPower G8 ProLite 3S 3300 mA h 25C

The flight test validation was performed as follows. The pilot commanded a step input corresponding to a desired attitude of ±30° (66% pilot input) for roll/pitch and 150°/s (60% pilot input) for yaw. As the low-level rate controllers were active, this resulted in the vehicle rolling/pitching toward 30° and stabilizing there. The yaw test required the pilot to stop the step yaw input after the vehicle had achieved its peak yaw rate, usually 180° was sufficient. Without prior knowledge to the flight tests validation campaign, 360° was chosen to be safe. The larger attitude difference for yaw is necessary as multicopters are much slower in yaw compared to pitch and roll and thus require more time to reach their peak yaw rate. The time to achieve xx° attitude (unstabilized) was calculated for 20° attitude change at which the vehicle was still rolling/pitching/yawing at full rate.

Results and discussion

The maneuverability and agility metrics for both test vehicles are shown in Table 2. Each metric has an arrow symbol next to its name to indicate it needs to be maximized (upward arrow) or minimized (downward arrow) to achieve the highest maneuverability and agility. Based on the comparison of the average relative standard deviations of each flight test data series, the test procedure accuracy threshold is estimated to be 10%. The two test vehicles are compared relative to each other in Table 3 including the accuracy threshold such that when the result is smaller than the threshold, a zero value is noted. The comparative percentages show how the compound multicopter maneuverability compares to the F550 multicopter. In addition, if the value has brackets () around it, one should interpret it as the compound multicopter performing worse than the F550 multicopter. So, by simply counting the number of bracketed values per axis one can summarily determine whether the compound is better than the F550 or not. Note that the large set of metrics in both tables is not (and will not) always (be) unanimously in favor of one vehicle as each metric measures a certain aspect of the maneuver. It is important to understand these aspects before interpreting the results. For instance, although the compound UAV overall has higher maneuverability and agility in roll, its bandwidth is slightly lower. Hence it is expected to have a little more difficulty in keeping drift minimal, yet this remains acceptable. The pitch maneuverability and agility of the compound UAV is significantly worse than the F550s, which is due to its much higher inertia in pitch direction even though its moment arm, the distance between front and aft propellers, is much longer. The compound’s yaw maneuverability and agility is slightly better than the F550’s despite higher yaw inertia as the compound UAV’s control propellers are slightly tilted to overcome this.

Table 2.

Maneuverability and agility results of test vehicles.

Metric			DJI F550			Compound
Name	Symbol	Unit	Roll	Pitch	Yaw	Roll	Pitch	Yaw
Control power ↑	CP	°/s	230	260	189	278	167	201
Attitude quickness ↑	Q	1/s	5.27	5.54	0.64	5.29	3.59	0.55
Time-to-peak angular rate ↓	$T_{{\dot{α}}_{peak}}$	s	0.22	0.25	1.81	0.24	0.29	1.40
Peak acceleration ↑	${\ddot{α}}_{p e a k}$	°/s²	2.59	3.09	2.49	3.35	1.84	2.72
Time-to-peak angular acceleration ↓	$T_{{\ddot{α}}_{peak}}$	s	0.34	0.36	1.82	0.23	0.28	1.48
Attitude change over 1 s ↑	$Δ α_{1 s}$	°	NA	NA	58.7	NA	NA	60.0
Attitude change over 0.2 s ↑	$Δ α_{0.2 s}$	°	10.66	9.14	0.38	12.65	5.48	0.79
Time to achieve 20° attitude angle ↓	$T_{20^{\circ}}$	s	0.28	0.27	0.63	0.24	0.34	0.65
Bandwidth ↑	$ω_{B W}$	Hz	1.85	1.88	0.24	1.61	1.19	0.21

Table 3.

Maneuverability and agility comparison of test vehicles including accuracy threshold (%). If the value has brackets () around it, one should interpret it as the compound multicopter performing worse than the F550 multicopter.

Metric		Compound w.r.t. F550
Name	Symbol	Roll	Pitch	Yaw
Control power ↑	CP	+21	(−36)	0
Attitude quickness ↑	Q	0	(−35)	(−13)
Time-to-peak angular rate ↓	$T_{{\dot{α}}_{peak}}$	0	(+17)	−23
Peak acceleration ↑	${\ddot{α}}_{p e a k}$	+29	(−41)	0
Time-to-peak angular acceleration ↓	$T_{{\ddot{α}}_{peak}}$	−30	−21	−19
Attitude change over 1 s ↑	$Δ α_{1 s}$	NA	NA	0
Attitude change over 0.2 s ↑	$Δ α_{0.2 s}$	+19	(−40)	+105
Time to achieve 20° attitude angle ↓	$T_{20^{\circ}}$	−13	(+27)	0
Bandwidth ↑	$ω_{B W}$	(−13)	(−37)	(−11)

Although the data processing tools and results allow calculating nine metrics, some conclusions can be drawn with respect to their relative importance. Data show the peak angular acceleration ${\ddot{α}}_{p e a k}$ is highly correlated to the CP for all axes. Measuring one of them should be sufficient and CP has, especially to a pilot, more physical/practical meaning. The peak angular acceleration ${\ddot{α}}_{p e a k}$ and time-to-peak angular acceleration $T_{{\ddot{α}}_{peak}}$ do prove to be well suited quality metrics to ensure the pilot input’s aggressiveness, and its precision, is as identical as possible for each test per axis. Attitude change over 0.2 s, $Δ α_{0.2 s}$ , is best suited for pitch and roll axis as they have very high CP and, for safety, only limited allowed attitude change. The yaw axis does not have this limitation and hence the attitude change over 1 s, $Δ α_{1 s}$ , is best suited. As the data logging happens at 50 Hz, time stamps have an inaccuracy of 0.02 s. Especially for the metrics $T_{20^{\circ}}$ , $T_{{\dot{α}}_{peak}}$ and $T_{{\ddot{α}}_{peak}}$ , which have values for both UAVs in the order of 0.20–0.30 s, this inaccuracy can have a significant effect on finding clear conclusions when comparing two UAVs. Therefore, these metrics should be less prominently used for small UAVs. Summarizing, the best subset of metrics for maneuverability and agility analysis consists of only five metrics: CP (or ${\ddot{α}}_{p e a k}$ ), Q, $Δ α_{0.2 s}$ for roll and pitch, $Δ α_{1 s}$ for yaw, and $ω_{B W}$ .

Overall, both vehicles perform well above the ADS-33E Level 1 requirements. To give an indication of how these results compare with manned helicopters: for aggressive agility helicopters ADS requires a minimum CP in hover (rate commanded) of 30°/s for pitch, 50°/s roll, and 60°/s yaw.¹¹ The F550 CPs are larger by a factor of three to eight, which is primarily due to the physical scale difference as the ADS-33E Level 1 requirements apply to large manned helicopters. Smaller aircraft have less inertia and thus higher maneuverability and agility. Equivalent Level 1 requirements for small rotary UAV in outdoor conditions are yet unknown. They depend on certain conditions such as vehicle mass, size, aerodynamic drag, propulsive power, wind, and turbulence levels, not to mention the application. Outdoor flight in confined spaces requires minimal drift while keeping a camera fixed on a certain location requires maximal stabilization. The focus of this paper is to establish a proper test procedure and in addition give some reference values where available from literature. The F550’s maneuverability and agility metrics can be a good initial reference as it is widely accepted to be a good multicopter for outdoor flight in fair conditions. Establishing new appropriate maneuverability and agility requirements is paramount, yet outside the scope of this paper.

Twice, a choice was made to deviate from the ADS-33E specifications. First, the roll and pitch maneuver was chosen to be performed with an attitude change of ±30°, which, for roll, is smaller than required by ADS-33E (60°). The choice was made for safety reasons due to the low flying altitude. Second, the yaw maneuver was performed for a 360° turn instead of a 60° attitude change or less required by ADS-33E for attitude quickness Q due to impracticality. The peak rate would not be achieved within the 60° maneuver, thus affecting other metrics such as CP and time-to-peak angular rate. In addition, the focus was on using a single maneuver for all metrics and the choice was made to deviate for Q_yaw. A 180° yaw attitude change is recommended for future experiments as the desired peak rate is achieved well before completing the 360° turn, used here, to have the purest results. Testing a UAV with the procedures described here and the calculation tools provided takes around five days’ work: two flight test days and three data analysis days.

Conclusions and future work

In this paper, we proposed and experimentally validated a simple test procedure, derived from manned aircraft industry practices and research, that is based on a simple manual open-loop step input maneuver. Using only onboard flight controller logs, nine maneuverability and agility metrics are determined for each axis. The test procedure was validated using two very different multicopter UAVs: a conventional hexacopter and a custom compound multicopter prototype. The accuracy of the results is estimated to be around 10%. The best subset of metrics for maneuverability and agility analysis are CP (or peak angular rate), attitude quickness, attitude change over 0.2 s for roll and pitch, attitude change over 1 s for yaw, and the bandwidth based on the 10–90% rise time of the measured attitude change. Where possible, the specifications and requirements of Aeronautical Design Standard (ADS)-33E were employed to follow the manned helicopter industry’s standards as, currently, there are no unmanned aviation standards. Overall, both vehicles perform well above the ADS-33E Level 1 requirements. However, the Level 1 requirements apply to large manned helicopters. For small rotary UAVs, the minimum requirements are unknown and establishing standards is important future work. Future work also includes determining the maneuverability and agility of conventional RC helicopters as their configuration may have even higher inherent maneuverability and agility, not to mention the ability to perform inverted flight. Another interesting path is expanding the research to handling quality studies including MTEs and Cooper–Harper rating by various test pilots. First-Person-View manual flight control, used in drone racing, larger UAVs, and manned multicopters could both benefit greatly from these results.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Appendix

References

Meier

Honegger

and Pollefeys

PX4: a node-based multithreaded open source robotics framework for deeply embedded platforms. In: IEEE International conference on robotics and automation (ICRA), Seattle, Washington, USA, 26th to 30th of May 2015: 6235–6240.

Thiels

Aho

Zietlow

et al . Use of unmanned aerial vehicles for medical product transport. Air Med J 2015; 34: 104–108.

Jenie

van Kampen

de Visser

et al . Three-dimensional velocity obstacle method for uncoordinated avoidance maneuvers of unmanned aerial vehicles. J Guid Control Dyn 2016; 39: 2312–2323.

Hosen

Helgesen

Fusini

et al . Vision-aided nonlinear observer for fixed-wing unmanned aerial vehicle navigation. J Guid Control Dyn 2016; 39: 1777–1789.

Lawrence

Coming

and Wharburton

Helicopter maneuverability and agility design sensitivity and air combat maneuver data correlation study. USAAVSCOM TR 91-D-3, 1991.

Whalley

MS.

Development and evaluation of an inverse solution technique for studying helicopter maneuverability and agility. NASA TM 102889, USAAVSCOM TR 90-A-008, 1991.

Mehmood

Nakamura

and Johnson

EN.

A maneuverability analysis of a novel hexarotor UAV concept. In: IEEE international conference on unmanned aircraft systems (ICUAS), Washington, DC, USA, 2016: 437–446.

Karimi

Pourtakdoust

and Nobahari

Trim and maneuverability analysis of a UAV using a new constrained PSO approach. J Aerosp Sci Technol 2011; 8: 45–56.

Campbell

A comparative framework for maneuverability and gust tolerance of aerial microsystems. MSc Dissertation, Dept. of Aerospace Engineering, University of Maryland, Maryland, 2012.

10.

AVSCOM. Aeronautical Design Standard-33-E-PRF: performance specification, handling qualities requirements for military rotorcraft, 2000.

11.

Floyd

Raymond

and Stewart

Army preliminary evaluation YAH-1R improved cobra agility and maneuverability helicopter. US Army AD-A018 420, 1975.

12.

Liefer

Valasek

Eggold

et al . Fighter agility metrics, research and test. AIAA J Aircraft 1992; 29: 452–457.

13.

Paranjape

and Ananthkrishnan

Combat aircraft agility metrics – a review. J Aerosp Sci Technol 2006; 58: 1–12.

14.

Yilmaz

and Pavel

Exploring the effects of rotor design parameters on helicopter handling qualities. In: CEAS air and space conference, Manchester, UK, 2009, ISBN 9781857682137.

15.

Padfield

GD.

Helicopter flight dynamics. 2nd ed. Wiley-Blackwell, New Jersey, USA, 2007, Chaps. 6, 7.

16.

Dorf

and Bishop

Modern control systems. 11th ed. Pearson Education, London, UK, 2008, Chap. 5.

17.

Verbeke

Hulens

Ramon

et al . The design and construction of a high endurance hexacopter suited for narrow corridors. In: IEEE international conference on unmanned aircraft systems (ICUAS), Orlando, FL, USA, 27–30 May, 2014: 543–551 .

18.

ArduPilot. Ardupilot autopilot suite, 2009, http://ardupilot.org/ardupilot/index.html

Experimental maneuverability and agility quantification for rotary unmanned aerial vehicle