The Autonomous Air Combat technique has been a lasting research topic for decades. However, no complete solutions seem to have appeared because of the highly dynamic and complex nature of the Autonomous Air Combat problem. In devising the Autonomous Air Combat solutions, we follow similar methodologies in the robotics community, and divide the overall scheme into two folds: the perception of other (enemy/friendly) aircraft, and the guidance/control for own aircraft. While the perception in the first fold serves as a foundation, this paper is mainly focused on the second one. Based on our survey, a review of own aircraft guidance/control in the (primarily one-to-one) Autonomous Air Combat solutions is presented. We divide different Autonomous Air Combat solutions into three groups, i.e. mathematics-based, knowledge-encoded, and learning-driven. In each group, we present the representative methods first; problem definition, solution, and a brief overview of the historical development are illustrated. We also comment on both weakness and strengths for each group/method. We point out certain technical paths/challenges that need to be addressed in the future Autonomous Air Combat development, i.e. to abstract and emulate the human pilot experiences, and to develop the online learning capabilites. Inspired by the state-of-art techniques in other similar fields (robotics, autonomous driving), we also propose potential solutions, i.e. traditional approaches enhanced by the novel data-driven technique. Via this paper, we hope to deliver an in-depth analysis of past experiences and potential challenges/solutions for the Autonomous Air Combat technique. We also advocate referring to the approaches/techniques that are utilized in other similar fields in devising the Autonomous Air Combat solutions.
Carpenter G and Falco M. Analysis of aircraft evasion strategies in air-to-air missile effectiveness models (no. RE-506). Grumman Aerospace Corp. Bethpage NY Research Report, August 1975.
2.
Lum DN. A methodology for modeling radar detection range in air combat simulation. Master Thesis, Air Force Institute of Technology, AFIT/GST/ENS/89M-12, 1989.
3.
Huynh HT, Costes Ph and Aumasson C. Numerical optimization of air combat maneuvers, In: Proceedings of The AIAA Guidance, Navigation and Control Conference, Monterey, CA, August 1987. AIAA 87-2392.
4.
JarmarkB. A missile duel between two aircraft. J Guid Control Dynam1985; 8: 508–513.
5.
DavidovitzAShinarJ. Two-target game model of an air combat with fire-and-forget all-aspect missiles. J Optimiz Theory App1989; 63: 133–165.
6.
Kaplan JA, Chappell AR, and McManus JW. The analysis of a generic air-to-air missile simulation model. Hampton, VA: NASA-TM-109057, 1994.
7.
RaivioT. Capture set computation of an optimally guided missile. J Guid Control Dynam2001; 24: 1167–1175.
8.
EhtamoHRaivioT. On applied nonlinear and bilevel programming or pursuit-evasion games. J Optimiz Theory Appl2001; 108: 65–96.
9.
Moritz K, Polis R, Well KH. Pursuit-evasion in medium-range air-combat scenarios. Comput Math Appl 1987; 13: 167–180.
10.
ShinarJDavidovitzA. Eccentric two-target model for qualitative air combat game analysis. J Guid Control Dynam1985; 8: 325–331.
11.
HeibstWBKrogullB. Design for air combat. J Aircr1973; 10: 247–253.
Taylor RT. Simulated combat for advanced technology assessments utilizing the adaptive maneuvering logic concepts. NASA Order no. L-24468C. 1987, Coastal Dynamics Technical Report.
15.
Ashworth BR and Kahlbaum Jr. WM. Description and performance of the langley differential maneuvering simulator. NASA TN-D7304. Washington, DC: NASA, 1973.
16.
Hankins III WW. Interactive computerized air combat opponent. NASA-19760022220. Washington, DC: NASA, 1976.
17.
Hankins III WW. Computer-automated opponent for manned air-to-air combat simulations. NASA TP-1518. Washington, DC: NASA, 1979.
18.
Kelly MJ, Lee Wooldridge A, Hennessy RT, et al. Air combat maneuvering performance measurement. Proc Hum Factors Ergon Soc Annu Meet 1979; 23: 324–328.
19.
Goodrich KH. Development of a six degree-of-freedom simulation environment for tactical guidance research and evaluation. PhD Dissertation, George Washington University, 1991.
20.
GoodrichKHMcManusJWChappellAR. Batch simulation environment for integrated batch and piloted air combat analysis. J Aircr1995; 32: 1263–1268.
21.
DongYHuangJAiJ. Visual perception-based target aircraft movement prediction for autonomous air combat. J Aircr2014; 52: 538–552.
22.
DongYZhangYAiJ. Full-altitude attitude angles envelope and model predictive control-based attitude angles protection for civil aircraft. Aerosp Sci Technol2015; 55: 292–306.
23.
Dong Y, Zhang L and Ai J. Attitude protection for side stick-operated aircraft-the classic approach. In: Proceedings of 29th congress of the international council of the aeronautical sciences, St. Petersburg, Russia, September, 2013.
24.
YouDShimDH. Design of an aerial combat guidance law using virtual pursuit point concept. Proc IMechE, Part G: J Aerospace Engineering2015; 229: 792–813.
25.
Byrnes MW. Nightfall: Machine autonomy in air-to-air combat. Air Space Power J 2014; 48–75.
26.
Stillion J. Trends in air-to-air combat: Implications for future air superiority, center for strategic and budgetary assessments, Washington DC, USA: Center for Strategic and Budgetary Assessments. https://csbaonline.org/uploads/documents/Air-to-Air-Report-.pdf (2015).
27.
Toubman A, Roessingh JJ, Spronck P, et al., Rapid adaptation of air combat behavior, In: Proceedings of the 22nd European conference on artificial intelligence, Hague, Holland, the Netherlands 29 August–2 September 2016, pp.1791–1796.
28.
Tiron R. Can UAVs dogfight? Arlington, VA: Association for Unmanned Vehicle Systems International: Unmanned Systems, vol. 24, 2006, pp.39–42.
29.
DongYCamciEKayacanE. Faster rrt-based nonholonomic path planning in 2d building environments using skeleton-constrained path biasing. J Intell Rob Syst2018; 89: 387–401.
30.
Dong Y and Zhang Y. Application of RRT algorithm to unmanned ground vehicle motion planning and obstacle avoidance. In: Proceedings of international conference on intelligent unmanned systems, Bali, Indonesia, 26–29 August 2015.
31.
Dong Y, Fu C and Kayacan E. RRT-based 3D path planning for formation landing of quadrotor UAVs. In: Proceedings of 14th international conference on control, automation, robotics and vision (ICARCV), Phuket, Thailand, 13–15 November 2016, pp. 1–6. IEEE.
32.
Zhang L, Liu Z, Dong Y, et al. Automatic patrol trajectory control of UAV in a forest surveillance and fires detection mission. In: Proceedings of 2018 aviation technology, integration, and operations conference, Atlanta, GA, USA, 25–29 June 2018. AIAA 2018-3260.
33.
DongYAiJ. Trial input method and own-aircraft state prediction in autonomous air combat. J Aircr2012; 49: 947–954.
34.
Dong Y and Ai J. Maneuvering strategy and own aircraft movement prediction in trial input method—low angle of attack. In: Proceedings of AIAA Infotech@ Aerospace, Garden Grove, CA, USA, 19–21 June 2012. AIAA 2012-2594.
35.
Dong Y, Yang X and Jianliang AI. Research on enemy aircraft bank angle estimation for autonomous air combat. In: Proceedings of the 29th congress of the international council of the aeronautical sciences, St. Petersburg, Russia, 2014.
36.
EndsleyMR. A survey of situation awareness requirements in air-to-air combat fighters. The Int J Aviat Psychol1993; 3: 157–168.
37.
Noonan CA. Sensor data fusion for air to air situation awareness beyond visual range. London, UK: British Aerospace Defense Ltd., Multi-Sensor Multi-Target Data Fusion, Tracking and Identification Techniques for Guidance and Control Applications, 1996, pp.268–283.
38.
Wu F, Gopalarathnam A, and Kim S. Post-stall aerodynamic modeling and gain-scheduled control design. In: Proceedings of the AIAA guidance, navigation, and control conference and exhibit, San Francisco, CA, August 2005. AIAA 2005-6169.
39.
Leatham A and Linch U. Two numerical methods to solve realistic air-to-air combat differential games In: Proceedings of the 12th AIAA aerospace sciences meeting, Washington, DC, January 1974. AIAA TP 74-22.
40.
BreitnerMH. The genesis of differential games in light of Isaacs’s contributions. J Optimiz Theory Appl2005; 124: 523–559.
41.
IsaacsRP. Games of pursuit, paper P-257, Satan Monica, CA: Rand Corporation, 1951.
42.
IsaacsRP. Differential games, I: introduction, research memorandum RM-1391, Santa Monica, CA: Rand Corporation, 1954.
43.
IsaacsRP. Differential games, II: the definition and formulation, research memorandum RM-1399, Santa Monica, CA: Rand Corporation, 1954.
44.
IsaacsRP. Differential games, III: the basic principles of the solution process, research memorandum RM-1411, Santa Monica, CA: Rand Corporation, 1954.
45.
IsaacsRP. Differential games, IV: mainly examples, research memorandum RM-1486, Santa Monica, CA: Rand Corporation, 1955.
46.
Isaacs R. Differential games: a mathematical theory with applications to warfare and pursuit, control and optimization. North Chelmsford, MA: Courier Corporation, republished in 1999.
47.
IsaacsR. Differential games: their scope, nature, and future. J Optimiz Theory Appl1969; 3: 283–295.
48.
Breakwell JV and Merz AW. Toward a complete solution of the homicidal chauffeur game. In: Proceedings of the first international conference on the theory and application of differential games. University of Massachusetts, Amherst, MA, USA, 29 September–1 October 1969, pp.1–5.
49.
Merz AW. The homicidal chauffeur – a differential game. PhD Dissertation, Stanford University, 1971.
50.
MerzAW. The homicidal chauffeur. AIAA J1974; 12: 259–260.
51.
WongRE. Some aerospace differential games. J Spacecr Rockets1967; 4: 1460–1465.
52.
Lynch UH. Differential game barriers and their application in air-to-air combat. PhD Dissertation, Air Force Institute, AFIT-DS/MC/73-1, 1973.
53.
MeierL. A new technique for solving pursuit-evasion differential games. IEEE Trans Autom Control1969; 14: 352–359.
54.
MerzAW. The game of two identical cars. J Optimiz Theory Appl1972; 9: 324–343.
55.
Mitchell I. Games of two identical vehicles. Technical Report SUDAAR 740, Department of Aeronautics and Astronautics, Stanford University, 2001.
56.
BreakwellJV. Minimum required capture radius in a coplanar model of the aerial combat problem. AIAA J1977; 15: 1089–1094.
57.
BretonMSzajowskiK. Advances in dynamic games: theory, applications, and numerical methods for differential and stochastic games (vol. 11), Berlin, Germany: Springer Science & Business Media, 2011.
58.
LewinJBreakwellJV. The surveillance-evasion game of degree. J Optimiz Theory Appl1975; 16: 339–353.
59.
LewinJOlsderGJ. Conic surveillance evasion. J Optimiz Theory Appl1979; 27: 107–125.
60.
Lewin J. Decoy in pursuit-evasion games. PhD Dissertation, Stanford University, 1973.
61.
Cardaliaguet P, Quincampoix M and Saint-Pierre P. Set-valued numerical analysis for optimal control and differential games, in Stochastic and differential games. Boston, MA: Birkhäuser Publishing, 1999, pp.177–247.
62.
BerkovitzLDFlemingWH. On differential games with integral payoff, Santa Monica, CA: Rand Corporation, 1955.
63.
Ryll-Nardzewski C. A theory of pursuit and evasion. In: Melvin D, Lloyd SS and Albert TW (eds) Advances in game theory. Princeton, New Jersey: Princeton University Press, 1962, pp.113–126.
64.
Berkovitz LD. A variational approach to differential games. In: Advances in game theory. vol 52, 1964, pp.127–174.
65.
Othling Jr. WL. Application of differential game theory to pursuit-evasion problems of two aircraft. PhD Dissertation, DS/MC/67-1, Air Force Institute, 1970.
66.
Merz AW. A differential game solution to the coplanar tail-chase aerial combat problem. Report, NASA-CR-137809, 1976.
67.
MerzAW. Coplanar tail-chase aerial combat as a differential game. AIAA J1977; 15: 1419–1423.
68.
Merz AW. Application of differential game theory to role-determination in aerial combat. Report, NASA-CR-137713, 1975.
69.
MerzAW. To pursue or to evade – that is the question. J Guid Control Dynam1985; 8: 161–166.
70.
Falco M and Cohen V. Strategy synthesis in aerial dogfight game models. In: Proceedings of the 11th AIAA aerospace science meeting, Washington, DC, January 1973. AIAA 73-233.
71.
Ciletti M, Meier L, and Salmon D. Application of reachable sets techniques to air combat analysis. In: Proceedings of the 11th AIAA aerospace science meeting, Washington, DC, January 1973. AIAA 73-232.
72.
OlsterGJBreakwellJV. Role determination in an aerial dogfight. Int J Game Theory1974; 3: 47–66.
73.
PengWYVincenTL. Some aspects of aerial combat. AIAA J1975; 13: 7–11.
74.
KelleyHLeftonL. A preference-ordered discrete-gaming approach to air-combat analysis. IEEE Trans Autom Control1978; 23: 642–645.
75.
Baron S, Chu KC, Ho YC, et al. A new approach to aerial combat games. Report, NASA CR-1626, 1970.
76.
Ho Y, Bryson A and Baron S. Differential games and optimal pursuit-evasion strategies. IEEE Trans Autom Control 1965; 10.
77.
Williamson Noble SMD. Toward a differential game solution to a practical two-aircraft pursuit-evasion problem in three-dimensional space. Master Thesis, Air Force Institute, GA/MC/71-5, 1971.
78.
JarmarkBSAMerzAWBreakwellJV. The variable-speed tail-chase aerial combat problem. J Guid Control Dynam1981; 4: 323–328.
79.
GuelmanMShinarJGreenA. Qualitative study of a planar pursuit evasion game in the atmosphere. J Guid Control Dynam1990; 13: 1136–1142.
80.
He F and Yao Y. Maneuver decision-making on air-to-air combat via hybrid control, In: Proceeding of the 2010 IEEE aerospace conference, Big Sky Montana, USA, 6–13 March 2010.
81.
RobertsDAMontgomeryRC. Development and application of a gradient method for solving differential games, NASA-TN-D-6502, Washington, DC: NASA, 1971.
82.
Roberts DA. The application of gradient methods to deferential games. Master Thesis, George Washington University, 1971.
83.
TaylorLW. Application of the epsilon technique to a realistic optimal pursuit-evasion problem. J Optimiz Theory Appl1975; 15: 685–702.
84.
Lin H. Contributions to the theory and applications of pursuit-evasion games (No. UCLA-70-22), California University Los Angeles, 1970.
85.
Leatham AL. A neighboring optimal method to solve differential games involving realistic aerodynamic vehicle models. Wright-Patterson Air Force Base, OH: Air Force Flight Dynamics Laboratory, TM-73-56-FXG, 1973.
86.
Anderson GM and Othling WL. A near optimal closed-loop control law for a class of aircraft/aircraft pursuit-evasion differential games. In: Proceedings of the IEEE conference on decision and control, Miami Beach, FL, USA, 15–17 December 1971, vol. 10, pp.449–450.
87.
AndersonG. A real-time closed-loop solution method for a class of nonlinear differential games. IEEE Trans Autom Control1972; 17: 576–577.
88.
Anderson G. Minmax range/barrier trajectories and their application to pursuit-evasion combat problems. In: Proceedings of the IEEE conference on decision and control, Del Webb's Towne House Phoenix, AZ, USA, 20–22 November 1974, vol. 13, pp.679–680.
89.
AndersonGM. A near-optimal closed-loop solution method for nonsingular zero-sum differential games. J Optimiz Theory Appl1974; 13: 303–318.
90.
AndersonGM. A transition matrix method for generating near-optimal closed-loop solutions to non-linear zero-sum differential games. Int J Syst Sci1976; 7: 529–543.
91.
Järmark B. Near-optimal closed-loop strategy for aerial combat games, Royal Institute of Technology, Department of Automatic Control, 1976.
92.
AndersonGM. Feedback control for a pursuing spacecraft using differential dynamic programming. AIAA J1977; 15: 1084–1088.
93.
JarmarkBHillbergC. Pursuit-evasion between two realistic aircraft. J Guid Control Dynam1984; 7: 690–694.
94.
Sun W. Pursuit-evasion and differential games. PhD Dissertation, Georgia Institute of Technology, 2017.
95.
JacobsonDH. New second-order and first-order algorithms for determining optimal control: a differential dynamic programming approach. J Optimiz Theory Appl1968; 2: 411–440.
96.
TabakD. Numerical solutions of differential game problems. Int J Syst Sci1975; 6: 591–599.
97.
JarmarkABerntS. Convergence control in differential dynamic programing applied to air-to-air combat. AIAA J1976; 14: 118–121.
98.
Jarmark B. On convergence control in differential dynamic programming applied to realistic aircraft and differential game problems. In: Proceedings of the IEEE conference on decision and control, IEEE, New Orleans, LA, USA, 7–9 December 1977, pp.471–479.
99.
LeondesCTPaineG. Extensions in quasilinearization techniques for optimal control. J Optimiz Theory Appl1968; 2: 316–330.
100.
LeondesCTPaineG. Computational results for extensions in quasilinearization techniques for optimal control. J Optimiz Theory Appl1968; 2: 395–410.
101.
Graham RG, Leondes CT, Stear EB, et al. Quasilinearization solutions of differential games and evaluation of suboptimal strategies (UCLA-ENG-71-28). PhD Thesis, California University Los Angeles, 1971.
102.
AlyGMChanWC. Application of a modified quasilinearization technique to totally singular optimal control problems. Int J Control1973; 17: 809–815.
103.
YeoBPWaldronKJGohBS. Optimal initial choice of multipliers in the quasilinearization method for optimal control problems with bounded controls. Int J Control1974; 20: 17–33.
104.
YeoBP. Use of quasi-linearization in the computation of optimal singular controls. AIAA J1975; 13: 102–105.
105.
Farber N and Shinar J. An approximate feedback solution of a variable speed and nonlinear pursuit-evasion game between two airplanes in a horizontal plane. In: Proceedings of the 6th AIAA atmospheric flight mechanics conference, Danvers, MA, USA, 11 August–13 August 1980. AIAA 1980-1597.
106.
Shinar J. Solution techniques for realistic pursuit-evasion games[R]. Technion-Israel Inst Of Tech Haifa, Dept of Aeronautical Engineering, 1980.
107.
FarberNShinarJ. Approximate solution of singularly perturbed nonlinear pursuit-evasion games. J Optimiz Theory Appl1980; 32: 39–73.
108.
Shinar J. Validation of zero-order feedback strategies for medium range air-to-air interception in a horizontal plane, NASA TM-84237. Washington, DC: NASA.
109.
ShinarJFarberN. Horizontal variable-speed interception game solved by forced singular perturbation technique. J Optimiz Theory Appl1984; 42: 603–636.
110.
ArdemaMDRajanN. An approach to three-dimensional aircraft pursuit-evasion. Comput Math Appl1987; 13: 97–110.
111.
HorieKConwayBA. Optimal fighter pursuit-evasion maneuvers found via two-sided optimization. J Guid Control Dynam2006; 29: 105–112.
112.
LionsP. Generalized solutions of Hamilton-Jacobi equations (vol, 69). London, UK: London Pitman, 1982.
113.
EvansLCSouganidisPE. Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs Equations, Madison, WI: Wisconsin University–Madison Mathematics Research Center, 1983.
114.
CrandallMGLionsP. Viscosity solutions of Hamilton-Jacobi equations. Trans Am Math Soc1983; 277: 1–42.
115.
LionsPSouganidisPE. Differential games, optimal control and directional derivatives of viscosity solutions of Bellman’s and Isaacs’ equations. SIAM J Control Optim1985; 23: 566–583.
116.
BardiMFalconeMSoraviaPNumerical methods for pursuit-evasion games via viscosity solutions. In: BardiMRaghavanTESParthasarathyT (eds). Stochastic and differential games: theory and numerical methods, Boston, MA: Birkhäuser Boston, 1999, pp. 105–175.
117.
BernhardPierre. Pursuit-evasion games and zero-sum two-person Differential games. Encyclopedia of systems and control, Berlin, Germany: Springer-Verlag, 2015, pp. 8–8.
118.
Raivio T, Ehtamo H and Hämäläinen RP. A decomposition method for a class of pursuit-evasion games. In: Proceedings of the 7th international symposium on differential games and applications (vol. 2), Shonan Village Center, Kanagawa, Japan, 16–18 December 1996, pp.784–795.
119.
RaivioTEhtamoH. Visual aircraft identification as a pursuit-evasion game. J Guid Control Dynam2000; 23: 701–708.
120.
Grimm W and Well KH. Modelling air combat as differential game recent approaches and future requirements. In: Hämäläinen RP and Ehtamo HK (eds) Differential games—developments in modelling and computation. Berlin, Germany: Springer, 1991, pp.1–13.
121.
ShinarJDavidovitzA. A two-target game analysis in line-of-sight coordinates. Comput Math Appl1987; 13: 123–140.
122.
HeymannMArdemaMDRajanN. A formulation and analysis of combat games, NASA-TP-2487, Washington, DC: NASA, 1985.
123.
BlaquièreAGérardFLeitmannG. Quantitative and qualitative games1969; vol. 58Cambridge, MA: Academic Press.
124.
GetzWMLeitmannG. Qualitative differential games with two targets. J Math Anal Appl1979; 68: 421–430.
125.
GetzWMPachterM. Two-target pursuit-evasion differential games in the plane. J Optimiz Theory Appl1981; 34: 383–403.
126.
GetzWMPachterM. Capturability in a two-target game of two cars. J Guid Control Dynam1981; 4: 15–21.
127.
ShinarJDavidovitzA. Unified approach for two-target game analysis. IFAC Proc Vol1987; 20: 65–71.
128.
SkowronskiJMStonierRJ. The barrier in a pursuit-evasion game with two targets. Comput Math Appl1987; 13: 37–45.
129.
ArdemaMDHeymannMRajanN. Combat games. J Optimiz Theory Appl1985; 46: 391–398.
130.
ArdemaAHeymannMRajanN. Analysis of a combat problem: the turret game. J Optimiz Theory Appl1987; 54: 23–42.
131.
Heymann M, Rajan N and Ardema M. On optimal strategies in event-constrained differential games[C]//Decision and Control. In: 24th IEEE Conference on. IEEE (vol. 24), Bonaventure Hotel & Spa Ft. Lauderdale, FL, USA, 11–13 December 1985, pp.1115–1118.
132.
YavinY. Stochastic two-target pursuit-evasion differential games in the plane. J Optimiz Theory Appl1988; 56: 325–343.
133.
JärmarkB. Two aggressive aircraft in a realistic short-range combat as a differential game study. Comput Math Appl1989; 18: 101–105.
134.
NeumanF. On the approximate solution of complex combat games. J Guid Control Dynam1990; 13: 128–136.
135.
RutowskiES. Energy approach to the general aircraft performance problem. J Aeronaut Sci1954; 21: 187–195.
136.
EdelbaumTNKelleyHJ. Energy climbs, energy turns, and asymptotic expansions. J Aircr1970; 7: 93–95.
Rajan N. The extremal trajectory map – a new representation of combat capability. In: 5th atmospheric flight mechanics conference for future space systems, Boulder, CO, USA, 6–8 August 1979, p.1622.
147.
RajanNPrasadURRaoNJ. Pursuit-evasion of two aircraft in a horizontal plane. J Guid Control Dynam1980; 3: 261–267.
148.
PrasadURRajanNRaoNJ. Planar pursuit-evasion with variable speeds, part 1, extremal trajectory maps. J Optimiz Theory Appl1981; 33: 401–418.
149.
RajanNPrasadURRaoNJ. Planar pursuit-evasion with variable speeds, part 2, barrier sections. J Optimiz Theory Appl1981; 33: 419–432.
150.
RajanNArdemaMD. Optimal feedback strategies for pursuit-evasion and interception in a plane, NASA TM-84311, Washington, DC: NASA, 1983.
151.
RajanNArdemaMD. Computation of optimal feedback strategies for interception in a horizontal plane. J Guid Control Dynam1984; 7: 627–629.
152.
RajanNArdemaMD. Barriers and dispersal surfaces in minimum-time interception. J Optimiz Theory Appl1984; 42: 201–228.
153.
Prasad UR and Rajan N. Aircraft pursuit-evasion problems with variable speeds. In: Pursuit-evasion differential games, Yavin Y, Pachter M and Rodin EY (eds) 1987, pp.111–121.
154.
Ure NK and Inalhan G. Autonomous control of unmanned combat air vehicles. IEEE Control Systems Magazine, 2012, 1066(033X/12).
155.
Karli M, Efe MÖ and Sever H. Close range one to one air combat maneuvering for autonomous UAV. In: 8th Ankara international aerospace conference, METU, Ankara, Turkey, 10–12 September 2015, pp.10–12.
156.
Sprinkle J, Eklund JM, Kim HJ, et al. Encoding aerial pursuit/evasion games with fixed wing aircraft into a nonlinear model predictive tracking controller. In: 43rd IEEE conference on decision and control (CDC), The Atlantis, Paradise Islands, The Bahamas, 14–17 December 2004 (vol. 3), pp.2609–2614. New York, NY: IEEE.
157.
Eklund JM, Sprinkle J and Sastry SS. Implementing and testing a nonlinear model predictive tracking controller for aerial pursuit/evasion games on a fixed wing aircraft. In: Proceedings of the American control conference, Balakrishnan Portand Hilton and Towers, Portland, OR, USA, 8–10 June 2005 (vol. 3), p.1509.
158.
ShawRL. Fighter combat, Annapolis, MD: Naval Institute Press, 1985.
159.
Yiqun D, Jun F, Bin Y, et al. Position and heading angle control of an unmanned quadrotor helicopter using LQR method. In: 34th Chinese Control Conference (CCC), Hangzhou, China, 2015. New York, NY: IEEE.
160.
Dong Y, Liu Z, Yu B, et al. Application of model predictive control to position and height limitation of a quadrotor unmanned aerial vehicle. In: ASME 2015 international design engineering technical conferences and computers and information in engineering conference, Boston, MA, USA, 2–5 August 2015. New York, NY: American Society of Mechanical Engineers.
161.
Ghamry KA, Dong Y, Kamel MA, et al. Real-time autonomous take-off, tracking and landing of UAV on a moving UGV platform. In: 2016 24th Mediterranean conference on control and automation (MED), Athens, Greece, 21–24 June 2016. New York, NY: IEEE.
162.
Dong Y, Zhang Y and Ai J. Experimental test of artificial potential field-based automobiles automated perpendicular parking. Int J Veh Technol 2016.
163.
DongYZhangYAiJ. Experimental test of unmanned ground vehicle delivering goods using RRT path planning algorithm. Unmanned Syst2017; 5: 45–57.
164.
Baron S and Kleinman DL. A Markovian approach to dynamic games of combat In: 1971 IEEE conference on decision and control, Miami Beach, FL, 1971, pp.464–469. New York, NY: IEEE.
165.
BaronSKleinmanDLSerbinS. A study of the Markov game approach to tactical maneuvering problems, NASA CR-1979, Washington, DC: NASA, 1972.
166.
KushnerHKleinmanA. Numerical methods for the solution of the degenerate nonlinear elliptic equations arising in optimal stochastic control theory. IEEE Trans Autom Control1968; 13: 344–353.
167.
Oberle RA. An air combat maneuver conversion model. Arlington, VA: Center for Naval Analyses Operations Evaluation Group, CRC-274, November 1974.
168.
FeiginPDPinkasOShinarJ. A simple Markov model for the analysis of multiple air combat. Nav Res Logist Q1984; 31: 413–429.
169.
Reuss GC. Analysis of aircraft combat sustainability using a Markov chain. Master Thesis, AD-A201 667, Naval Postgraduate School, Monterey, CA, 1988.
170.
Tunay I, Goodwin J and Mukai H. Markov Chain Combat Models for Attrition Prediction and Mission Control. Washington DC, USA: Department of Systems Science and Mathematics, Washingron University. www.ssm.wustl.edu/Publications/ilker.pdf (2000).
171.
ShengdeJ, et al.A Continuous-time Markov Decision Process Based Method on Pursuit-Evasion Problem. IFAC Proc Vol2014; 47: 620–625.
172.
JiaSWangXShenL. A continuous-time Markov decision process-based method with application in a pursuit-evasion example. IEEE Trans Syst Man Cybern Syst2016; 46: 1215–1225.
173.
Feng C and Yao P. On close-range air combat based on hidden Markov model. In: 2016 IEEE Chinese Guidance, Navigation and Control Conference (CGNCC), Nanjing, China, 12–14 August 2016, pp.687–694. New York, NY: IEEE.
174.
ChangqiangHKangshengDHanqiaoH, et al.Autonomous air combat maneuver decision using Bayesian inference and moving horizon optimization. J Syst Eng Electron2018; 29: 86–97.
175.
VirtanenKRaivioTHämäläinenRP. Decision theoretical approach to pilot simulation. J Aircr1999; 36: 632–641.
176.
Raivio T, Hamalainen R and Virtanen K. Modeling pilot’s sequential maneuvering decisions by a multistage influence diagram. In: AIAA guidance, navigation, and control conference and exhibit, Montreal, Canada, 6–9 August 2001.
177.
Virtanen K, Raivio T and Hämäläinen RP. An influence diagram approach to one-on-one air combat. In: Proceedings of the 10th international symposium on differential games and applications, St. Petersburg, Russia, 2002.
178.
VirtanenKRaivioTHamalainenRP. Modeling pilot’s sequential maneuvering decisions by a multistage influence diagram. J Guid Control Dynam2004; 27: 665–677.
179.
Virtanen K, Karelahti J, Raivio T, et al. A multistage influence diagram game for maneuvering decisions in air combat. In: Proceedings of the 11th international symposium on differential games and applications, Tucson, Arizona, 18–21 December 2004.
180.
VirtanenK. Optimal pilot decisions and flight trajectories in air combat, Espoo, Finland: Helsinki University of Technology, 2005.
181.
VirtanenKKarelahtiJRaivioT. Modeling air combat by a moving horizon influence diagram game. J Guid Control Dynam2006; 29: 1080–1091.
182.
KarelahtiJ. Modeling and on-line solution of air combat optimization problems and games, Espoo, Finland: Helsinki University of Technology, 2008.
183.
RichaletJRaultATestudJL, et al.Model predictive heuristic control. Autom1978; 14: 413–428.
184.
Shinar J and Glizer VY. Application of the receding horizon strategy to singularly perturbed pursuit-evasion problems. In: 1992 American Control Conference, Chicago, IL, 24–26 June 1992, pp.3247–3248. New York, NY: IEEE.
185.
ShinarJGlizerVJ. Application of receding horizon control strategy to pursuit-evasion problems. Optim Contr Appl Met1995; 16: 127–141.
186.
CruzJBSimaanMAGacicA, et al.Game-theoretic modeling and control of a military air operation. IEEE Trans Aerosp Electron Syst2001; 37: 1393–1405.
187.
CruzJBSimaanMAGacicA, et al.Moving horizon Nash strategies for a military air operation. IEEE Trans Aerosp Electron Syst2002; 38: 989–999.
188.
Janisch BS. Receding horizon control for uncertain pursuit-evasion games. Master Thesis, Texas A&M University, 2016.
189.
McGrew J and How JP. Air combat strategy using approximate dynamic programming. In: Proceedings of the AIAA guidance, navigation and control conference and exhibit, Honolulu, HI, 2008. AIAA 2008-6796.
190.
McGrew JS. Real-time maneuvering decisions for autonomous air combat. Master Thesis, Massachusetts Institute of Technology, 2008.
191.
McGrewJSHowJBushL, et al.Air-combat strategy using approximate dynamic programming. J Guid Control Dynam2010; 33: 1641–1654.
192.
Ma Y, Ma X and Song X. A case study on air combat decision using approximated dynamic programming. Math Prob Eng. Article ID 183401, 2014.
193.
Burgin GH and Owens AJ. An adaptive maneuvering logic computer program for the simulation of one-to-one air-to-air combat (vol. 2). Program description, NASA CR 2583. Washington, DC: NASA, 1975.
194.
Burgin GH and Owens AJ. An adaptive maneuvering logic computer program for the simulation of one-to-one air-to-air combat (vol. 1). General description, NASA CR 2582. Washington, DC: NASA, 1975.
195.
BurginGHEgglestonDM. Design of an all-attitude flight control system to execute commanded bank angles and angles of attack, NASA CR 145004, Washington, DC: NASA, 1976.
196.
BurginGH. Improvements to the adaptive maneuvering logic program, NASA CR 3985, Washington, DC: NASA, 1986.
197.
BurginGHSidorLB. Rule-based air combat simulation, NASA CR-4160, Washington, DC: NASA, 1988.
198.
Burgin G, Williams W and Sidor L. The Adaptive Maneuvering Logic program in support of the pilot’s associate program – a heuristic approach to missile evasion. In: 24th aerospace sciences meeting, Reno, NV, USA, 6–9 January 1986.
199.
Goodrich K and Mcmanus J. Development of a tactical guidance research and evaluation system (TGRES). In: Flight simulation technologies conference and exhibit, Boston, MA, USA, 14–16 August 1989, p.3312.
200.
Goodrich K and Mcmanus J. An integrated environment for tactical guidance research and evaluation. In: Orbital debris conference: technical issues and future directions, Baltimore, MD, USA, 16–19 April 1990, p.1287.
201.
GoodrichKH. A high-fidelity, six-degree-of-freedom batch simulation environment for tactical guidance research and evaluation, NASA TM-4440, Washington, DC: NASA, 1993.
202.
KleinVRatvaskyTRCobleighBR. Aerodynamic parameters of High-Angle-of attack Research Vehicle (HARV) estimated from flight data, NASA TM-102692, Washington, DC: NASA, 1990.
203.
MorelliEA. F-18 High Alpha Research Vehicle (HARV) parameter identification flight test maneuvers for optimal input design validation and lateral control effectiveness, NASA CR-198248, Washington, DC: NASA, 1995.
204.
MessinaMD. Using the HARV simulation aerodynamic model to determine forebody strake aerodynamic coefficients from flight data, NASA-CR-198247, Washington, DC: NASA, 1995.
205.
MessinaMDStricklandMEHofflerKD, et al.Simulation model of the F/A-18 high angle-of-attack research vehicle utilized for the design of advanced control laws, NASA-TM-110216, Washington, DC: NASA, 1996.
206.
MorelliEA. Piloted parameter identification flight test maneuvers for closed loop modeling of the F-18 high alpha research vehicle (HARV), NASA-CR-198268, Washington, DC: NASA, 1996.
207.
MorelliEA. Flight test maneuvers for closed loop lateral-directional modeling of the F-18 High Alpha Research Vehicle (HARV) using forebody strakes, NASA-CR-198270, Washington, DC: NASA, 1996.
208.
BattersonJGMorelliEA. Parameter identification flight test maneuvers for closed loop modeling of the F-18 High Alpha Research Vehicle (HARV), NASA-CR-198269, Washington, DC: NASA, 1996.
209.
IliffKWWangKC. Flight-determined subsonic longitudinal stability and control derivatives of the F-18 High Angle of Attack Research Vehicle (HARV) with thrust vectoring, NASA/TP-97-206539, Washington, DC: NASA, 1997.
210.
Goodrich KH, McManus JW and Chappell AR. A high-fidelity batch simulation environment for integrated batch and piloted air combat simulation analysis, AIAA Paper 92-4145, 1992.
211.
McManus JW. A concurrent distributed system for aircraft tactical decision generation. In: 9th IEEE/AIAA/NASA conference on digital avionics systems, Virginia Beach, VA, USA, 15–18 October 1990, pp.505–512. New York, NY: IEEE.
212.
ChappellARMcManusJWGoodrichKH. Trial maneuver generation and selection in the PALADIN tactical decision generation system, NASA-TM-107722, Washington, DC: NASA, 1992.
213.
McManus JW, Chappell AR and Arbuckle PD. Situation assessment in the Paladin tactical decision generation system, 1992. Document ID: 19920018649.
214.
Dong Y. Deep Learning-Based Opponent Aircraft Attitude Detection in Autonomous Air Combat. Journal of Aerospace Information Systems 2019; 16: 162–167.
215.
LewisMSAikenEW. Piloted simulation of one-on-one helicopter air combat at NOE flight levels, NASA-TM-86686, Washington, DC: NASA, 1985.
216.
Austin F, Carbone G, Falco M, et al. Automated maneuvering decisions for air-to-air combat. In: Guidance, navigation and control conference, Portland, OR, USA, 20–22 August 1990, p.2393.
217.
AustinFCarboneGHinzH, et al.Game theory for automated maneuvering during air-to-air combat. J Guid Control Dynam1990; 13: 1143–1149.
218.
KatzA. Tree lookahead in air combat. J Aircr1994; 31: 970–973.
219.
Park H, Yoo D and Tahk M. Optimal WVR air-to-air combat maneuver generation algorithm using scoring function based on security strategy. In: 2014 Asia-Pacific International Symposium on Aerospace Technology, Shanghai, China, 24–26 September 2014.
220.
Park H, Yoo D and Tahk M. Generation of the optimal air-to-air combat maneuver using scoring function matrix. In: 2014 KSAS spring conference, Wonju, Korea, 17–18 April 2014.
221.
Park H, Lee B, Yoo D, et al. Differential game based 3-dimensional air combat maneuver generation using scoring function matrix. In: 2015 KSAS spring conference, Jeongseon, Korea, 22–24 April 2015.
222.
ParkHLeeB-YTahkM-J, et al.Differential game based air combat maneuver generation using scoring function matrix. Int J Aeronaut Space Sci2016; 17: 204–213.
223.
Lee BY, Han S, Park HJ, et al. One-versus-one air-to-air combat maneuver generation based on the differential game. In: Proceedings of the 2016 international council of the aeronautical sciences, Daejeon, Korea, 25–30 Sep, 2016.
224.
Lee BY, HanS, Park H, et al. One-versus-one air combat algorithm considering direction of the lift vector. In: 2017 25th Mediterranean conference on control and automation (MED), Valletta, Malta, 3–6 July 2017, pp.265–270. New York, NY: IEEE.
225.
Rodin EY, Lirov Y, Mittnik S, et al. Pursuit and evasion by artificial intelligence. In: Second international symposium on differential games application, Williamsburg, VA, USA, 21–22 August 1986.
226.
Rodin EY, Lirov Y, Mittnik S, et al. Artificial intelligence in air combat games. In: Yavin Y, Pachter M and Rodin EY (eds) Pursuit evasion differential games, Elsevier Ltd., 1987, pp.261–274.
227.
LirovYRodinEYMcElhaneyBG, et al.Artificial intelligence modelling of control systems. Simulation1988; 50: 12–24.
228.
LirovYRodinEYGhoshBK. Automated learning by tactical decision systems in air combat. Comput Math Appl1989; 18: 151–160.
229.
RodinEY. Semantic control theory. Appl Math Lett1988; 1: 73–78.
230.
Rodin EY, Geist D and Lirov Y. Flight and fire control knowledge representation. In: Proceedings of the 28th IEEE conference on decision and control, Tampa, FL, 1989. New York, NY: IEEE.
231.
RodinEYGeistD. Flight and fire control with logic programming. Comput Math Appl1990; 20: 15–27.
232.
Lazarus E. The application of value-driven decision-making in air combat simulation. In: 1997 IEEE international conference on systems, man, and cybernetics. Computational cybernetics and simulation, Orlando, FL, 1997 (vol. 3), pp.2302–2307. New York, NY: IEEE.
233.
Kerchner RM, Hughes RG and Lee A. TAC BRAWLER – an application of engagement simulation modeling to simulator visual system display requirements for air combat maneuvering. In: Proceedings with 2nd Symposium on Aviation Psychology, Columbus, OH, USA, 25–28 April 1983. NASA.
234.
Buschor DC. Sensitivity analysis of Brawler pilot skill levels. Master Thesis, Air Force Institute of Technology, AFIT/GOA/ENS/98M-01, 1998.
235.
Mayo BR, Paciencia TJ and Croghan DP. BRAWLER to CFAM: incorporating stochastic engagement-level data in deterministic campaign models. In: Proceedings of the 2016 Winter Simulation Conference, Arlington, VA, USA, 11–14 December 2016, pp.484–489. New York, NY: IEEE Press.
Bent N. The helicopter air-to-air value-driven engagement model (HAVDEM)-Prototype development. In: 47th, AHS, Annual Forum, Phoenix, AZ, USA, 6–8 May 1991, pp.369–377.
238.
Bent NE. The helicopter air-to-air value-driven engagement model (HAVDEM). In: 19th ICAS, Congress, Anaheim, CA, 1994, pp.2181–2189.
239.
Wooldridge L, Kelly MJ, Obermayer RW, et al. Air combat maneuvering performance measurement state space analysis. Thousand Oaks, CA: Vreuls Research Corp, 1982.
240.
ÇelikHOktayTTürkmenİ. Model predictive kontrol and robustness test of the unmanned aerial vehicle (ZANKA-I) in various turbulence. J Aeronaut Space Technol2016; 9: 31–42.
241.
WangXYadavVBalakrishnanSN. Cooperative UAV formation flying with obstacle/collision avoidance. IEEE Trans Control Syst Technol2007; 15: 672–679.
242.
LaValle SM. Rapidly-exploring random trees: a new tool for path planning. Technical Report, Computer Science Department, Iowa State University (TR 98-11), 1998.
243.
Denny J, Greco E, Thomas S, et al. MARRT: medial axis biased rapidly-exploring random trees. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), Hongkong, China, 31 May–5 June 2014.
244.
Gammell JD, Srinivasa SS and Barfoot TD. Informed RRT*: optimal sampling-based path planning focused via direct sampling of an admissible ellipsoidal heuristic. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA, 14–18 September 2014.
245.
Luders B, Kothari M and How J. Chance constrained RRT for probabilistic robustness to environmental uncertainty. In: AIAA guidance, navigation, and control conference, Toronto, Ontario, Canada, 2–5 August 2010, p.8160.
Karli M, Efe MÖ and Sever H. Extracting basic fighter maneuvers from actual flight data. Lecture Notes on Information Theory, vol. 5, No. 1, pp.1–6, June 2017.
248.
Wang W, Xi J and Zhao D. Driving style analysis using primitive driving patterns with Bayesian nonparametric approaches. IEEE Trans Intell Transp Syst, 2018, pp.1–13.
249.
Silver D, Huang A, Maddison CJ, et al., Mastering the game of Go with deep neural networks and tree search. Nature 2016; 529: 484.
250.
Turkmen I and Celik H. Incorporation of neural network to HPMHT for tracking multiple targets. Elektron Elektrotech 2015;21: 3–6.
251.
Dong Y. Implementing Deep Learning for comprehensive aircraft icing and actuator/sensor fault detection/identification[J]. Engineering Applications of Artificial Intelligence 2019, 83: 28–44.
252.
McMahon DC. A neural network trained to select aircraft maneuvers during air combat: a comparison of network and rule-based performance. In: 1990 IJCNN international joint conference on neural networks, San Diego, CA, 1990, pp.107–112. New York, NY: IEEE.
253.
RodinEYAminSM. Maneuver prediction in air combat via artificial neural networks. Comput Math Appl1992; 24: 95–112.
254.
Teng TH, Tan AH, Tan YS, et al. Self-organizing neural networks for learning air combat maneuvers. In: The 2012 international joint conference on neural networks (IJCNN), Brisbane, QLD, 2012, pp.1–8. New York, NY: IEEE.
255.
Tan AH. FALCON: a fusion architecture for learning, cognition, and navigation. In: 2004 IEEE international joint conference on neural networks, Budapest, Hungary, 2004 (vol. 4), pp.3297–3302. New York, NY: IEEE.
256.
CarpenterGAGrossbergS. A massively parallel architecture for a self-organizing neural pattern recognition machine. Comp Vis Graph Image Process1987; 37: 54–115.
257.
Poropudas J and Virtanen K. Analyzing air combat simulation results with dynamic Bayesian networks. In: 2007 winter simulation conference, Washington, DC, 2007, pp.1370–1377. New York, NY: IEEE.
258.
Du P and Liu H. Study on air combat tactics decision-making based on Bayesian networks. In: 2010 2nd IEEE international conference on information management and engineering (ICIME), Chengdu, China, 2010, pp.252–256. New York, NY: IEEE.
259.
AndersenSKOlesenKGJensenFV, et al.HUGIN – a shell for building Bayesian belief universes for expert systems. IJCAI1989; 89: 1080–1085.
260.
Akbari S and Menhaj MB. A fuzzy guidance law for modeling offensive air-to-air combat maneuver. In: Joint 9th IFSA world congress and 20th NAFIPS international conference, Vancouver, BC, 2001, pp.3027–3031. New York, NY: IEEE.
Akabari S, Menhaj MB and Nikravesh SK. Computational intelligence, theory and applications: International Conference 8th Fuzzy Days, Dortmund, Germany, 29 September–1 October 2004, Springer-Verlag New York Inc., p. 171.
RaoDBalas-TimarD. A Neuro-Fuzzy hybridization approach to model the pilot agent in air warfare simulation systems. J Battlef Technol2014; 17: 23–23.
265.
Karli M, Efe MÖ and Sever H. Air combat learning from F-16 flight information. In: 2017 IEEE international conference on fuzzy systems (FUZZ-IEEE), Naples, Italy, 2017, pp.1–6. New York, NY: IEEE.
266.
Ernest N, Cohen K, Schumacher C, et al. Learning of intelligent controllers for autonomous unmanned combat aerial vehicles by genetic cascading fuzzy methods. SAE Technical Paper, 2014.
267.
ErnestNCohenKKivelevitchE, et al.Genetic fuzzy trees and their application towards autonomous training and control of a squadron of unmanned combat aerial vehicles. Unmanned Syst2015; 3: 185–204.
268.
ErnestNCarrollDSchumacherC, et al.Genetic fuzzy based artificial intelligence for unmanned combat aerial vehicle control in simulated air combat missions. J Def Manag2016; 6: 144–144.
269.
Ernest N, Carroll D, Bogart N, et al. Perspectives on genetic fuzzy based artificial intelligence for cooperative control of unmanned fighter aircraft. In: Proceedings of the 2017 world congress on unmanned systems engineering, Cambridge, UK, 27–28 July 2017.
270.
SuttonRSBartoAG. Reinforcement learning: an introduction, Cambridge, MA: MIT press, 2018.
271.
An X, Yingxin K, Lei Y, et al. Engagement maneuvering strategy of air combat based on fuzzy markov game theory. In: 2011 IEEE 2nd international conference on computing, control and industrial engineering (CCIE), Wuhan, China, 2011 (vol. 2), pp.126–129. New York, NY: IEEE.
272.
Liu P and Ma Y. A deep reinforcement learning based intelligent decision method for UCAV air combat. In: Asian Simulation Conference, Singapore, 2017, pp.274–286. Berlin, Germany: Springer.
273.
ZhouYMaYSongX, et al.Hierarchical fuzzy ART for Q-learning and its application in air combat simulation. Int J Model Simul Sci Comput2017; 8: 1750052–1750052.
274.
SilverDSchrittwieserJSimonyanK, et al.Mastering the game of Go without human knowledge. Nature2017; 550: 354–354.
275.
Legrand M. Deep reinforcement learning for autonomous vehicle control among human drivers. Master Dissertation, Vrije Universiteit Brussel, 2017.
276.
MarinaLSanduA. Deep reinforcement learning for autonomous vehicles-state of the art. Bulletin of the Transilvania University of Brasov. Eng Sci. Series I,2017; 10: 195–202.
277.
Wang S, Jia D and Weng X. Deep Reinforcement Learning for Autonomous Driving. arXiv preprint arXiv:1811.11329, 2018.
278.
Chen, J., B. Yuan and M. Tomizuka. Model-free deep reinforcement learning for urban autonomous driving. arXiv preprint arXiv:1904.09503, 2019.
279.
KoberJBagnellJAPetersJ. Reinforcement learning in robotics: a survey. Int J Robot Res2013; 32: 1238–1274.
280.
Pham HX, La HM, Feil-Seifer D, et al. Autonomous UAV navigation using reinforcement learning. arXiv preprint arXiv:1801.05086, 2018.
281.
Gu S, Holly E, Lillicrap T, et al. Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In: 2017 IEEE international conference on robotics and automation (ICRA), Singapore, 29 May–3 June 2017. New York, NY: IEEE.
282.
Singh A, Lary Y, Hartikainen K, et al. End-to-end robotic reinforcement learning without reward engineering. arXiv preprint arXiv:1904.07854, 2019.
