Due to the consideration of control performance and the uncertainty of the dynamic characteristics of nonlinear systems, designing the auxiliary signal for active fault diagnosis presents significant challenges. This paper presents a novel data-driven approach for auxiliary signal design in the active fault diagnosis of nonlinear systems while ensuring guaranteed control performance. Specifically, we introduce a double actor-critic network to generate tracking and diagnostic signals, respectively. Subsequently, a two-objective optimization method based on deep reinforcement learning is proposed to address the tradeoff between tracking performance and fault diagnosis. Finally, the effectiveness of this method is verified through a cart-pole system with stochastic noise.
AndjelkovicICampbellSL (2011) Direct optimization determination of auxiliary test signals for linear problems with model uncertainty. In: IEEE conference on decision and control, Orlando, FL, 12–15 December, pp. 909–914. New York: IEEE.
2.
BartoAGSuttonRSAndersonCW (1983) Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems Man & Cybernetics13(5): 834–846.
3.
BlankeMKinnaertMLunzeJ, et al. (2006) Diagnosis and Fault-Tolerant Control. Berlin: Springer.
4.
ChiangLHRussellELBraatzRD (2001) Fault Detection and Diagnosis in Industrial Systems. London: Springer.
5.
GuoYHeX (2024) Active diagnosis of incipient actuator faults for stochastic systems. IEEE Transactions on Industrial Electronics71(1): 996–1005.
6.
HeirungTANMesbahA (2019) Input design for active fault diagnosis. Annual Reviews in Control47: 35–50.
7.
HellmanMERavivJ (1970) Probability of error, equivocation, and the Chernoff bound. IEEE Transactions on Information Theory16(4): 368–372.
8.
JiaFCaoFLyuG, et al. (2023) A novel framework of cooperative design: Bringing active fault diagnosis into fault-tolerant control. IEEE Transactions on Cybernetics53(5): 3301–3310.
9.
LeeYLLiangDWangWJ (2022) Optimal online liveness fault detection for multilayer cloud computing systems. IEEE Transactions on Dependable and Secure Computing19(5): 3464–3477.
10.
LiJCaoXChenR, et al. (2023) Graph neural network architecture search for rotating machinery fault diagnosis based on reinforcement learning. Mechanical System and Signal Processing202: 1107001.
11.
LiJLiuSReviriegoP, et al. (2020) Scheme for periodical concurrent fault detection in parallel CRC circuits. IET Computers & Digital Techniques14(2): 80–85.
12.
LiuZGaoHYuX, et al. (2024) B-spline wavelet neural-network-based adaptive control for linear-motor-driven systems via a novel gradient descent algorithm. IEEE Transactions on Industrial Electronics71(2): 1896–1905.
13.
NikoukhahR (1998) Guaranteed active failure detection and isolation for linear dynamical systems. Automatica34(11): 1345–1358.
14.
SchulmanJLevineSMoritzP, et al. (2015) Trust region policy optimization. Computer Science1: 1889–1897.
15.
SchulmanJWolskiFDhariwalP, et al. (2017) Proximal policy optimization algorithms. arXiv. Epub ahead of print 28 August. DOI: 10.48550/arXiv.1707.06347.
16.
SchölkopfBPlattJHofmannT (2007) A Kernel Method for the Two-Sample-Problem. (Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference). Cambridge, MA: MIT Press.
17.
ScottJRCampbellSL (2014) Auxiliary signal design for failure detection in high index differential-algebraic equations. Numerical Algebra, Control & Optimization4(2): 151–179.
18.
SilverDLeverGHeessN, et al. (2014) Deterministic policy gradient algorithms. International Conference on Machine Learning32: 387–395.
19.
SongXSunPSongS, et al. (2023) Quantized neural adaptive finite-time preassigned performance control for interconnected nonlinear systems. Neural Computing & Applications35(21): 15429–15446.
20.
SuttonRSMcAllesterDSinghS, et al. (1999) Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems12: 1057–1063.
21.
TabatabaeipourSM (2015) Active fault detection and isolation of discrete-time linear time-varying systems: a set-membership approach. International Journal of Systems Science46(11): 1917–1933.
22.
TaqviSAAZabiriHTufaLD, et al. (2021) A review on data-driven learning approaches for fault detection and diagnosis in chemical processes. ChemBioEng Reviews8(3): 239–259.
23.
TawarmalaniMSahinidisNV (2002) Convexification and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming, vol. 65. London: Springer.
24.
Van HasseltHGuezASilverD (2016) Deep reinforcement learning with double Q-learning. Thirtieth AAAI Conference on Artificial Intelligence2016: 2094–2100.
25.
VenkatasubramanianVRengaswamyRKavuriSN (2003a) A review of process fault detection and diagnosis. Part II: Qualitative models and search strategies. Computers & Chemical Engineering: An International Journal of Computer Applications in Chemical Engineering27(3): 313–326.
26.
VenkatasubramanianVRengaswamyRKavuriSN, et al. (2003b) A review of process fault detection and diagnosis. Part III: Process history based methods. Computers & Chemical Engineering: An International Journal of Computer Applications in Chemical Engineering27(3): 327–346.
27.
VenkatasubramanianVRengaswamyRYinK, et al. (2003c) A review of process fault detection and diagnosis. Part I: Quantitative model-based methods. Computers & Chemical Engineering: An International Journal of Computer Applications in Chemical Engineering27(3): 293–311.
28.
WangRZhuangZTaoH, et al. (2023a) Q-learning based fault estimation and fault tolerant iterative learning control for MIMO systems. ISA Transactions142: 123–135.
29.
WangXWangSLiangX, et al. (2024) Deep reinforcement learning: A survey. IEEE Transactions on Neural Networks and Learning Systems35(4): 5064–5078.
30.
WangZSchaulTHesselM, et al. (2016) Dueling network architectures for deep reinforcement learning. International Conference on Machine Learning48: 1995–2003.
31.
WangZZhangQTangL, et al. (2023b) Transfer reinforcement learning method with multi-label learning for compound fault recognition. Advanced Engineering Informatics55: 101818.
XuF (2021) Observer-based asymptotic active fault diagnosis: A two-layer optimization framework. Automatica128: 109558.
34.
XuF (2024) A new input design framework for asymptotic active fault diagnosis with application to integrated diagnosis and control. Automatica162: 111519.
35.
YanZXuFTanJ, et al. (2023) Reinforcement learning-based integrated active fault diagnosis and tracking control. ISA Transactions132: 364–376.
36.
YanZXuY (2019) Data-driven load frequency control for stochastic power systems: A deep reinforcement learning method with continuous action search. IEEE Transactions on Power Systems34(2): 1653–1656.
37.
ZhangYChadliMXiangZ (2024a) Prescribed-time adaptive fuzzy optimal control for nonlinear systems. IEEE Transactions on Fuzzy Systems32(4): 2403–2412.
38.
ZhangYXiangZ (2025) Prescribed-time optimal control for a class of switched nonlinear systems. IEEE Transactions on Automation Science and Engineering22: 3033–3043.
39.
ZhangYYanXZouW, et al. (2024b) Fuzzy optimal tracking control for autonomous surface vehicles with prescribed-time convergence analysis. IEEE Transactions on Fuzzy Systems32(11): 6523–6533.
40.
ZhaoZWangZZouL, et al. (2024) Zonotopic non-fragile set-membership fusion estimation for nonlinear systems under sensor resolution effects: Boundedness and monotonicity. Information Fusion105: 102232.
41.
ZhaoZWangZZouL, et al. (2025) Zonotope-based distributed set-membership fusion estimation for artificial neural networks under the dynamic event-triggered mechanism. IEEE Transactions on Neural Networks and Learning Systems36(1): 1637–1650.
42.
ZhuYZhaoSZhangY, et al. (2024) A review of statistical-based fault detection and diagnosis with probabilistic models. Symmetry-Basel16(4): 455.
