A hierarchical hybrid thermal control strategy for electric vehicle motors based on ADRC and TD3

Abstract

Electric vehicle motors operating under high power density generate substantial heat, posing significant challenges to temperature regulation accuracy and cooling energy efficiency. Conventional thermal management strategies often struggle to achieve robust performance under highly dynamic and uncertain operating conditions. This paper proposes a hybrid thermal control strategy integrating active disturbance rejection control (ADRC) and Twin Delayed Deep Deterministic Policy Gradient (TD3) within a hierarchical architecture. A multi-heat-source coupled motor thermal model is established to capture complex thermal dynamics of stator, rotor, and cooling subsystems. The lower-layer ADRC ensures fast temperature tracking and disturbance suppression through an extended state observer, while the upper-layer TD3 optimizes cooling energy consumption by learning long-term policies. An adaptive coordination mechanism balances real-time regulation and energy efficiency optimization. Hardware-in-the-loop experiments are conducted on a 150 kW permanent magnet synchronous motor under standard driving cycles and extreme conditions. Results demonstrate that the proposed strategy reduces temperature control RMSE to 1.47 °C and cooling energy consumption by 31.9% compared with conventional PID control, while maintaining strong robustness under ±30% parameter perturbations. These findings indicate that the ADRC–TD3 hybrid strategy provides an effective solution for intelligent thermal management of electric vehicle motors.

Introduction

With the intensifying global energy crisis and worsening environmental pollution, electric vehicles, as an important representative of new energy vehicles, are experiencing unprecedented rapid development. The motor system, as the core power unit of electric vehicles, directly affects the vehicle’s power performance and service life through its operating efficiency and reliability. However, motors generate substantial heat during high power density operation, and severe temperature fluctuations not only reduce motor efficiency but may also lead to serious problems such as permanent magnet demagnetization and insulation material aging, making motor thermal management a key technical bottleneck constraining electric vehicle performance improvement. Traditional thermal management strategies often struggle to achieve a good balance between temperature control accuracy and energy consumption optimization, urgently requiring the development of more intelligent and efficient control methods to cope with complex and variable operating conditions.

In recent years, domestic and international scholars have conducted extensive research work in the field of electric vehicle motor thermal management. In terms of traditional control methods, active disturbance rejection control (ADRC) has been widely applied in motor control due to its strong robustness and anti-interference capability.^1,2 The ADRC framework proposed by Zhang et al. can effectively suppress torque ripple in permanent magnet synchronous motors, achieving active compensation for internal and external disturbances. ³ Meanwhile, the rapid development of reinforcement learning technology has brought new solutions to thermal management control. Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, as a representative algorithm of deep reinforcement learning, has demonstrated excellent performance in electric vehicle energy management.^4,5 Wang et al. utilized model predictive control to achieve energy-efficient optimal operation of thermal management systems, validating the potential of intelligent control methods in reducing system energy consumption. ⁶ The combination of deep reinforcement learning and fuzzy logic control further enhanced the adaptability and robustness of control strategies.^7,8 In the field of permanent magnet synchronous motor control, the TD3 algorithm has been successfully applied to the optimal control of wind turbine generators, demonstrating good dynamic response characteristics. ⁹ Tang and Zhang combined fuzzy control with the TD3 algorithm to achieve coordinated control of electro-hydraulic composite braking for electric vehicles. ¹⁰ These research achievements provide important references for intelligent motor thermal management control, but existing methods still have problems such as slow response speed and large overshoot when handling strongly coupled and nonlinear thermal dynamic characteristics. Despite these advances, current thermal management methods still exhibit notable limitations when applied to electric vehicle motors. Purely model-based controllers often suffer from performance degradation under strong disturbances and parameter variations, while reinforcement learning-based approaches alone typically struggle with slow transient response and safety-critical constraint handling. In particular, achieving fast temperature regulation while maintaining long-term energy efficiency remains a challenging issue for systems with strongly coupled and nonlinear thermal dynamics.

To address these challenges, this paper proposes a hybrid thermal control strategy that integrates active disturbance rejection control and the Twin Delayed Deep Deterministic Policy Gradient algorithm within a hierarchical control framework. In the proposed architecture, ADRC serves as the lower-layer controller to ensure fast temperature tracking and disturbance rejection through an extended state observer, while TD3 operates at the upper layer to optimize cooling energy consumption by learning long-term control policies. The main contributions of this work are threefold. First, a motor thermal dynamic model considering multi-heat-source coupling effects is established to accurately characterize the thermal behavior of key motor components. Second, an adaptive temperature control scheme based on ADRC is designed to enhance robustness against uncertainties and disturbances. Third, a dual-layer ADRC–TD3 collaborative control strategy is developed to achieve coordinated optimization of temperature regulation performance and cooling energy efficiency. The effectiveness of the proposed method is validated on a hardware-in-the-loop experimental platform under both standard driving cycles and extreme operating conditions, providing practical insights for the intelligent design of electric vehicle motor thermal management systems.

The contributions of this study are summarized as follows:

1. A multi-heat-source coupled thermal modeling framework is developed for electric vehicle motors, capturing the strongly coupled and nonlinear thermal dynamics of key components under high power density and dynamic operating conditions.

2. A robust lower-layer temperature control strategy based on active disturbance rejection control is designed, in which an extended state observer enables real-time disturbance estimation and compensation, significantly improving transient response and robustness.

3. A dual-layer collaborative control architecture that integrates ADRC with the Twin Delayed Deep Deterministic Policy Gradient algorithm is proposed, allowing fast temperature regulation and long-term cooling energy optimization to be achieved simultaneously.

4. The proposed method is experimentally validated on a hardware-in-the-loop platform, demonstrating superior performance in temperature control accuracy, energy efficiency, dynamic response, and robustness compared with conventional control strategies.

Methods

Electric vehicle motor thermal management system modeling

Accurate modeling of the thermal management system of a permanent magnet synchronous motor requires comprehensive consideration of heat source distribution, heat transfer paths, and cooling mechanisms. The heat during motor operation mainly comes from stator copper loss, iron core loss, and rotor eddy current loss, and the spatial distribution and time-varying characteristics of these heat sources determine the dynamic evolution of the motor temperature field of the temperature field. ¹¹

In this study, a hybrid modeling approach combining distributed-parameter and lumped-parameter methods is adopted. The distribution characteristics of the motor’s internal temperature field are obtained through three-dimensional finite element analysis, while the thermal dynamic behavior of key components is described using an equivalent thermal network. The heat conduction equation can be expressed as:

$$\rho c_p\frac{\partial T}{\partial t}=\nabla ·(k\nabla T)+q_{\mathrm{gen}}-q_{\mathrm{cool}}$$

(1)

where $\rho$ denotes the material density (kg/m³), $c_p$ is the specific heat capacity (J/(kg K)), $T$ represents the temperature (K), $t$ is the time (s), $k$ is the thermal conductivity (W/(m K)), $q_{\mathrm{gen}}$ is the internal heat generation rate (W/m³), and $q_{\mathrm{cool}}$ denotes the cooling heat flux density (W/m³).

As shown in Figure 1, the simplified lumped-parameter model divides the motor into five key thermal nodes: stator winding, stator core, rotor magnet, rotor core, and housing. The temperature dynamics of each node can be described as:

$$C_i\frac{d{T}_i}{\mathrm{dt}}=P_{\mathrm{loss},i}(t)+\sum _{j=1}^n\frac{T_j-T_i}{R_{\mathrm{ij}}}-h_i{A}_i(T_i-T_{\mathrm{fluid}})$$

(2)

where $C_i$ denotes the thermal capacity of node $i$ (J/K), $P_{\mathrm{loss},i}$ is the power loss (W), $R_{\mathrm{ij}}$ is the thermal resistance (K/W), $h_i$ is the convective heat transfer coefficient (W/(m² K)), $A_i$ is the heat transfer area (m²), and $T_{\mathrm{fluid}}$ is the coolant temperature (K).

Figure 1.

Schematic diagram of motor thermal network model structure.

The dynamic characteristics of the cooling system are described through a fluid network model, considering the influence of coolant flow rate, radiator efficiency, and ambient temperature. ¹² The coolant temperature change rate is:

$$m_{\mathrm{fluid}}{c}_{p,\mathrm{fluid}}\frac{d{T}_{\mathrm{fluid}}}{\mathrm{dt}}=Q_{\mathrm{absorbed}}-Q_{\mathrm{radiator}}$$

(3)

where $m_{\mathrm{fluid}}$ is the coolant mass (kg), $c_{p,\mathrm{fluid}}$ is the coolant specific heat capacity (J/(kg K)), $Q_{\mathrm{absorbed}}$ is the heat absorbed from the motor (W), and $Q_{\mathrm{radiator}}$ is the heat dissipation from the radiator (W).

Active disturbance rejection controller (ADRC) design

The ADRC design is based on the principle of estimating and compensating for system uncertainties and external disturbances in a unified manner through an extended state observer (ESO). ¹³ An improved active disturbance rejection controller was designed according to the characteristics of the motor temperature control system.

The system model can be expressed in standard form as:

$$\overline{y}=f(y,\stackrel{·}{y},d,t)+b_{0}u$$

(4)

where $y$ denotes the system output (temperature deviation), $u$ is the control input (cooling power adjustment), $b_0$ is the control gain estimate, $f(·)$ denotes the total disturbance including model uncertainties and external perturbations, and $d$ is the external disturbance vector.

The ESO adopts a nonlinear structure to enhance disturbance estimation accuracy:

$$\{\begin{array}{cccc}e=z_1-y\hfill & z_1=z_2-{\beta }_1\mathrm{fal}(e,{\alpha }_1,{\delta }_1)\hfill & z_2=z_3-{\beta }_2\mathrm{fal}(e,{\alpha }_2,{\delta }_2)+b_0\hfill & u{z}_3=-{\beta }_3\mathrm{fal}(e,{\alpha }_3,{\delta }_3)\hfill \end{array}$$

(5)

where $z_1$ , $z_2$ , $z_3$ are the estimated values of the system state, its derivative, and the total disturbance, respectively, ${\beta }_i$ are the observer gains, and $\mathrm{fal}(·)$ is the nonlinear function:

$$\mathrm{fal}(e,\alpha ,\delta )=\{\begin{array}{ccc}e/{\delta }^{1-\alpha },\hfill & |e|\le \delta |e{|}^{\alpha }·\mathrm{sign}(e),\hfill & |e|>\delta \hfill \end{array}$$

(6)

where $\alpha \in (0,1)$ controls the degree of nonlinearity, and $\delta >0$ is the linear segment boundary. As shown in Figure 2, the nonlinear state error feedback (NLSEF) control law is expressed as:

$$u_0=k_p\mathrm{fal}(e_1,{\alpha }_p,{\delta }_p)+k_d\mathrm{fal}(e_2,{\alpha }_d,{\delta }_d)$$

(7)

where $e_1=\nu -z_1$ is the tracking error, $e_2=\overline{v}-z_2$ is the differential error, $v$ is the reference trajectory, and $k_p$ , $k_d$ are gain parameters. The final control input is given by:

$$u=\frac{u_0-z_3}{b_0}$$

(8)

Figure 2.

Active disturbance rejection controller structure block diagram.

Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm architecture

The TD3 algorithm mitigates value function overestimation in deep reinforcement learning through three key mechanisms: dual Q-networks, delayed policy updates, and target policy smoothing. ¹⁴ In this study, TD3 is employed as the upper-layer controller to optimize long-term cooling energy consumption in the motor thermal management system.

The state space $\mathcal{S}$ design comprehensively considers the observable variables of the system:

$$s_t=[T_{\mathrm{sw}},T_{\mathrm{sc}},T_{\mathrm{rm}},T_{\mathrm{rc}},T_f,P_m,{\omega }_m,T_{\mathrm{amb}},t_{\mathrm{pred}}{]}^T\in {\mathbb{R}}^9$$

(9)

where $T_{\mathrm{sw}}$ and $T_{\mathrm{sc}}$ are the stator winding and core temperatures, respectively, $T_{\mathrm{rm}}$ and $T_{\mathrm{rc}}$ are the rotor magnet and core temperatures, respectively, $T_f$ is the coolant temperature, $P_m$ is the motor power, ${\omega }_m$ is the rotational speed, $T_{\mathrm{amb}}$ is the ambient temperature, and $t_{\mathrm{pred}}$ corresponds to predicted future operating conditions.

The action space $a_t$ consists of three continuous control variables:

$$a_t=[u_{\mathrm{pump}},u_{\mathrm{valve}},u_{\mathrm{fan}}{]}^T\in [0,1{]}^3$$

(10)

where $u_{\mathrm{pump}}$ controls the pump flow rate (0–60 L/min), $u_{\mathrm{valve}}$ controls the valve opening (0%–100%), and $u_{\mathrm{fan}}$ controls the fan speed (0–2400 rpm).

The reward function design needs to balance multiple objectives ¹⁵ :

$$\begin{array}{c}\multicolumn{1}{c}{r_t=-w_1|T_t-T_{\mathrm{ref}}{|}^2-w_2{P}_{\mathrm{cool},t}}\\ \multicolumn{1}{c}{-w_{3}max(0,T_{\max ,t}-T_{\lim }{)}^2-w_4|\mathrm{\Delta }{u}_t{|}_2^2}\end{array}$$

(11)

where $w_i$ are the weight coefficients, $T_{\mathrm{ref}}$ is the target temperature, $P_{\mathrm{cool},t}$ is the cooling power consumption, $T_{\lim }$ is the safety limit, and $\mathrm{\Delta }{u}_t$ is the control increment.

As shown in Figure 3, the TD3 algorithm includes one Actor network $\mu (s|{\theta }^{\mu })$ and two Critic networks $Q_i(s,a|{\theta }^{Q_i})$ . The corresponding target networks are updated using a soft update mechanism:

$${\theta }^{\prime }\leftarrow \tau \theta +(1-\tau ){\theta }^{\prime }$$

(12)

where $\tau <<1$ is the soft update rate.

Figure 3.

TD3 algorithm network architecture.

ADRC–TD3 hybrid control strategy integration framework

The proposed hybrid control strategy integrates ADRC and TD3 within a hierarchical architecture to exploit their respective strengths. ¹⁶ In this structure, TD3 operates at the upper layer to perform long-term energy optimization and generate reference trajectories, while ADRC functions at the lower layer to ensure fast temperature tracking and disturbance rejection.

As illustrated in Figure 4, the core of the integration framework lies in the coordination mechanism between the two control layers. The TD3 module generates a temperature reference trajectory and a feedforward control signal:

$$[T_{\mathrm{ref}}^{*}(t),u_{\mathrm{ff}}(t)]={\pi }_{\mathrm{TD}3}(s_t,H_t)$$

(13)

where $H_t$ is the historical information set. The ADRC controller produces a compensatory control signal based on real-time error feedback:

$$u_{\mathrm{fb}}(t)=f_{\mathrm{ADRC}}(T_{\mathrm{ref}}^{*}(t)-T(t),\hat{f}(t))$$

(14)

where $\hat{f}(t)$ is the total disturbance estimated by ESO. The final control input is determined by combining the feedforward and feedback components:

$$u(t)=\alpha (t)·u_{\mathrm{ff}}(t)+[1-\alpha (t)]·u_{\mathrm{fb}}(t)$$

(15)

where $\alpha (t)\in [0,1]$ is an adaptive weighting coefficient, that is, dynamically adjusted according to the system state:

$$\alpha (t)=\frac{1}{1+e^{-k(|e(t)|-e_{\mathrm{th}})}}$$

(16)

where $e(t)$ is the temperature error, $e_{\mathrm{th}}$ is the switching threshold, and $k$ is the switching rate parameter.

Figure 4.

ADRC–TD3 hybrid control architecture.

Experimental platform construction and test environment configuration

The experimental platform was built based on a 150 kW permanent magnet synchronous motor, integrating a complete thermal management system and data acquisition system. ¹⁷ Figure 5 illustrates the hardware-in-the-loop (HIL) experimental configuration for electric vehicle motor thermal management.

Figure 5.

Hardware-in-the-loop experimental platform for electric vehicle motor thermal management.

As shown in Figure 5, the ADRC–TD3 control algorithm is executed on a dSPACE MicroAutoBox II real-time controller in a closed-loop configuration. The controller generates control commands for the motor driver and cooling actuators, while temperature, flow, and pressure measurements are fed back to the controller in real time. This HIL setup provides the experimental basis for evaluating the fast dynamic response and real-time feasibility reported in Table 1. All experimental results presented in this paper are obtained from the physical HIL bench rather than numerical simulation. The technical specifications of the key equipment are summarized in Table 2.

Table 1.

Dynamic response performance comparison under extreme conditions.

Condition type	Performance indicator	PID	MPC	ADRC only	TD3 only	ADRC–TD3
Rapid acceleration (0–100 km/h)	Response time (s)	5.1	3.8	3.2	4.3	2.8
	Temperature rise rate (°C/s)	4.1	3.2	2.8	3.5	2.5
	Maximum temperature (°C)	97.2	93.5	92.1	94.3	91.3
Continuous hill climbing (30%/5 min)	Average temperature (°C)	89.2	86.8	85.6	87.3	84.1
	Temperature fluctuation (°C)	±5.2	±3.6	±2.8	±3.9	±2.0
	Adjustment count	47	32	28	35	23
Rapid deceleration (100–0 km/h)	Temperature drop rate (°C/s)	1.8	2.3	2.7	2.1	3
	Recovery time (s)	68	52	43	58	37
	Overcooling amplitude (°C)	8.3	5.6	3.2	6.1	2.4

Table 2.

Main equipment parameters of experimental platform.

Equipment name	Model specification	Technical parameters	Accuracy grade
Permanent magnet synchronous motor	PMSM-150	150 kW/380 V/3000 rpm	–
Temperature sensor	PT100-A grade	−50 to 200 °C	±0.15 °C
Flow meter	LWGY-25	0.5–10 m3/h	±0.5%
Variable frequency water pump	WLB-50	0–60 L/min	–
Real-time controller	dSPACE MAB-II	900 MHz/1 GB RAM	–
Power analyzer	WT3000E	0.01% rdg	±0.02%

Test environment configuration includes standard and extreme operating conditions. Standard conditions use NEDC, WLTC, CLTC cycles; extreme conditions include rapid acceleration (0–100 km/h), continuous hill climbing (30% gradient), and high temperature environment (45 °C).^18,19 The test condition design referenced the latest domestic and international energy management strategy research results.^20,21 The energy management of hybrid electric vehicles involves coordinated control of multiple energy storage systems, and this paper drew on relevant control architecture design experience.^22,23 Energy management methods based on real-time model predictive control provided theoretical support for online optimization of the TD3 algorithm. ²⁴

Algorithm implementation process and pseudocode

The ADRC–TD3 hybrid control algorithm is implemented using a modular design to facilitate debugging, parameter tuning, and future extensibility. ²⁵ The overall execution flow of the hybrid control strategy is summarized in Table 3.

Table 3.

ADRC–TD3 hybrid control main program pseudocode.

Line	Code content
1	Initialize: ADRC_params, TD3_networks, Replay_buffer
2	Load: Pre-trained TD3 models
3	while system_running do
4	Read: T_current, P_motor, $\omega$ _motor, T_ambient
5	s_t ← Construct_State (sensor_data)
6	if training_mode then
7	a_t ← TD3_Actor (s_t) + exploration_noise
8	else
9	a_t ← TD3_Actor (s_t)
10	end if
11	T_ref, u_ff ← Decode_Action (a_t)
12	e_T ← T_ref – T_current
13	if
14	u_fb ← ADRC_Controller (e_T, ESO_states)
15	u_final ← 0.3 × u_ff + 0.7 × u_fb
16	else
17	u_final ← 0.8 × u_ff + 0.2 × ADRC_Controller (e_T)
18	end if
19	Apply_Control (u_final)
20	Store_Transition (s_t, a_t, r_t, s_{t + 1})
21	end while

Table 3 presents the main program structure of the ADRC–TD3 hybrid controller. During operation, sensor measurements are first collected to construct the system state vector.^26,27 The TD3 actor network generates supervisory control actions, which are decoded into a temperature reference trajectory and feedforward control signals. ²⁸ The ADRC controller then computes feedback compensation based on the temperature tracking error and the estimated disturbances from the extended state observer (ESO). ²⁹ The final control command is obtained through adaptive weighting between feedforward and feedback components and is applied to the motor thermal management system. The TD3 network training is performed using offline batch updates, with network parameters updated once every 100 control cycles. This training schedule reduces online computational burden while maintaining policy learning stability.

Performance evaluation index and comparison benchmark setting

A comprehensive performance evaluation system is constructed from four dimensions: control accuracy, energy efficiency, dynamic response, and robustness. Temperature control accuracy is evaluated using the following metrics:

$$\mathrm{RMSE}=\sqrt{\frac{1}{N}\sum _{i=1}^N{(T_i-T_{\mathrm{ref},i})}^2}$$

(17)

$$\mathrm{MAE}=\frac{1}{N}\sum _{i=1}^N|T_i-T_{\mathrm{ref},i}|$$

(18)

$$\mathrm{IAE}={\int }_0^{T_{\mathrm{end}}}|e(t)|\mathrm{dt}$$

(19)

Energy efficiency indicators are defined as:

$${\eta }_{thermal}=\frac{Q_{removed}}{W_{cooling}}\times 100\%$$

(20)

$$\mathrm{COP}=\frac{Q_{\mathrm{cooling}}}{P_{\mathrm{input}}}$$

(21)

where $Q_{\mathrm{removed}}$ denotes the removed heat, $W_{\mathrm{cooling}}$ is the cooling system energy consumption, $Q_{\mathrm{cooling}}$ is the cooling capacity, and $P_{\mathrm{input}}$ is the input power.

Dynamic response indicators include rise time $t_r$ , settling time $t_s$ , overshoot $M_p$ , and steady-state error $e_{\mathrm{ss}}$ .

Table 4 lists the configuration parameters of comparison benchmark algorithms.

Table 4.

Comparison algorithm parameter configuration.

Algorithm type	Key parameter settings	Sampling period	Prediction horizon
PID	Kp = 2.5, Ki = 0.8, Kd = 0.3	10 ms	–
MPC	N = 20, Nu = 5, Q = diag (10, 1), R = 0.1	100 ms	2 s
Fuzzy control	7 × 7 rule base, Mamdani inference	50 ms	–
ADRC only	ω o  = 30, ω c  = 10, b0 = 0.5	10 ms	–
TD3 only	lr = 3e-4, γ = 0.99, batch = 128	100 ms	–

Robustness testing is evaluated by introducing parameter perturbations and external disturbances. The parameter perturbation range is set to ±30% of the nominal value, and external disturbances include step load changes, random noise interference, and sensor faults. Performance degradation rate is defined as:

$$\mathrm{\Delta }P=\frac{P_{nominal}-P_{perturbed}}{P_{nominal}}\times 100\%$$

(22)

where $P_{\mathrm{nominal}}$ and $P_{\mathrm{perturbed}}$ are the performance indicators under nominal and perturbed conditions, respectively.

The construction of the performance evaluation system drew on the evaluation method of adaptive equivalent ratio model predictive control. ³⁰ The application of machine learning methods in hybrid electric vehicle energy management provided reference benchmarks for algorithm performance comparison. ³¹ Control strategy optimization based on energy flow experiments validated the rationality of the experimental scheme in this paper. ³² Reinforcement learning control research on battery and cabin thermal management provided technical reference for system integration testing. ³³

For the PID baseline, the gains $K_p$ , $K_i$ , and $K_d$ are tuned on a held-out validation set using a grid search to minimize the temperature tracking RMSE while constraining overshoot and steady-state error. The search ranges and final selected values are reported in Table 4 for reproducibility. For ADRC, the observer bandwidth ${\omega }_o$ , controller bandwidth ${\omega }_c$ , and the estimated control gain $b_o$ are selected following the bandwidth-parameterization principle and then fine-tuned on the validation set under the same objective. For TD3, standard hyperparameters are used and tuned on the validation set, including learning rate, discount factor, batch size, target smoothing noise, and delayed policy update interval. All hyperparameter ranges and final settings are explicitly reported to ensure reproducibility.

Results

Temperature control accuracy analysis

Motor temperature control accuracy directly affects system safety and reliability. This section systematically evaluates the temperature regulation performance of the ADRC–TD3 hybrid control strategy under different operating conditions based on multiple experimental trials. The target temperature was set to 85 °C with an ambient temperature of 25 °C. Temperature responses of key components, including the stator winding and rotor magnet, were measured and analyzed.

As shown in Figure 6, during the 1180 s NEDC driving cycle test, the ADRC–TD3 strategy achieved superior temperature tracking performance. The maximum stator temperature deviation during load transitions was limited to 2.3 °C, representing a 73.6% reduction compared with the 8.7 °C deviation observed under conventional PID control. In addition, the settling time required to reach steady-state temperature was reduced from 48 s (PID) to 23 s, corresponding to a 52.1% improvement in response speed.

Figure 6.

Stator temperature tracking curve under NEDC condition.

Table 5 details the temperature control accuracy statistics for different parts.

Table 5.

Temperature control accuracy comparison for key parts (NEDC condition).

Monitoring position	Control method	Average temperature (°C)	RMSE (°C)	Maximum deviation (°C)	Overshoot (%)	Steady-state error (°C)
Stator winding	PID	85.8	3.76	8.7	10.2	±1.8
	MPC	85.3	2.45	5.4	6.4	±1.2
	ADRC–TD3	85.1	1.38	2.3	2.7	±0.5
Rotor magnet	PID	88.2	4.23	9.8	11.5	±2.1
	MPC	87.4	2.86	6.2	7.3	±1.4
	ADRC–TD3	86.8	1.67	3.1	3.6	±0.7
Stator core	PID	83.6	3.12	7.3	8.6	±1.5
	MPC	83.2	2.08	4.6	5.4	±1.0
	ADRC–TD3	83	1.15	2	2.4	±0.4

As shown in Table 5, the consistent reduction in RMSE, maximum deviation, and overshoot across all monitoring positions indicates that the proposed ADRC–TD3 strategy improves both transient and steady-state temperature regulation. This improvement can be mainly attributed to the complementary roles of the two control layers. The ADRC component effectively compensates for unmodeled thermal dynamics and external disturbances, thereby reducing peak deviations during load changes, while the TD3-based high-level controller adjusts cooling intensity to avoid aggressive control actions that typically lead to overshoot. As a result, the hybrid strategy achieves not only lower tracking error but also smoother temperature evolution, which is particularly important for preventing thermal fatigue in motor components.

As shown in Figure 7, under the more dynamic WLTC condition, the temperature control error exhibits normal distribution characteristics. The error distribution of the ADRC–TD3 strategy is more concentrated, with 95% of sampling points having temperature errors within ±2 °C, while the error distribution range of PID control reaches ±5 °C. The error standard deviation decreased from 3.2 °C for PID to 1.1 °C, indicating a significant improvement in control stability. The more concentrated error distribution under the WLTC condition further suggests that the ADRC–TD3 strategy provides improved control consistency under highly dynamic driving scenarios. The reduction in error variance implies that the controller can maintain stable performance despite frequent operating point transitions, which is difficult to achieve with fixed-parameter PID control. This statistical characteristic highlights the robustness advantage of the proposed hybrid control architecture.

Figure 7.

Temperature control error distribution under WLTC condition.

Energy consumption optimization performance evaluation

Cooling system energy consumption is an important factor affecting overall vehicle range. This section evaluates the energy consumption optimization effect of the hybrid control strategy from three dimensions: power consumption, energy efficiency, and energy saving rate.

As shown in Figure 8, under the standard driving cycle, the average cooling power of the ADRC–TD3 strategy was 1.85 kW, representing a 32.0% reduction compared with 2.72 kW under PID control and a 19.9% reduction compared with 2.31 kW under MPC. The peak cooling power decreased from 4.8 kW (PID) to 3.2 kW under the proposed strategy, thereby reducing instantaneous battery load during high-demand phases.

Figure 8.

Cooling system power consumption curve under typical conditions.

Table 6 summarizes the energy consumption statistics under different operating conditions. The ADRC–TD3 strategy achieved an energy saving rate exceeding 30% across all tested scenarios, while the cooling energy ratio (CER) remained below 3.5%. The reduction in both average and peak cooling power demonstrates that the proposed strategy avoids unnecessary overcooling, which is a common drawback of rule-based or purely feedback controllers. By jointly considering temperature regulation and energy consumption objectives, the TD3-based supervisory controller enables anticipative cooling actions, while ADRC ensures fast correction when thermal disturbances occur. This coordinated behavior explains why the energy saving rate consistently exceeds 30% across different operating conditions, rather than being limited to a specific driving cycle.

Table 6.

Cooling system energy consumption comparison analysis under different conditions.

		PID control	MPC	ADRC–TD3	Energy saving rate (%)
Test condition	Duration (min)	Energy consumption (kWh)	CER (%)	Energy consumption (kWh)	CER (%)
NEDC	20	0.91	4.6	0.77	3.9
WLTC	30	1.36	5.1	1.16	4.3
Urban condition	25	0.85	3.8	0.72	3.2
Highway condition	15	0.98	5.4	0.84	4.6
Combined condition	40	1.82	4.8	1.54	4.1

As shown in Figure 9, the hybrid control strategy achieves energy consumption optimization through intelligent coordination of the working states of the pump and fan. In the low load phase (0–300 s), the pump flow rate is maintained at a low flow rate of 18 L/min, and the fan power is only 120 W; in the high load phase (600–900 s), the system quickly responds by increasing the flow rate to 48 L/min and the fan power to 680 W; in the steady-state phase, unnecessary excessive cooling is avoided through precise control. The coordinated adjustment of pump flow rate and fan power illustrates how the hybrid controller dynamically allocates cooling resources based on real-time thermal demand. Such coordination reduces mechanical wear of auxiliary components and lowers instantaneous battery load, which contributes to improved system reliability and long-term energy efficiency.

Figure 9.

Coordinated optimization curve of pump flow rate and fan power.

Control response characteristics under dynamic conditions

The frequent operating condition transitions in actual electric vehicle operation place strict requirements on the dynamic response capability of the control system.

As shown in Figure 10, during the 0–100 km/h rapid acceleration test, motor power increased from 15 to 150 kW within 12 s. Under this condition, the ADRC–TD3 strategy limited the temperature rise rate to 2.5 °C/s, with a maximum temperature of 91.3 °C, remaining below the 95 °C safety threshold. In contrast, the PID controller exhibited a temperature rise rate of 4.1 °C/s and a maximum temperature of 97.2 °C, indicating potential overheating risk. The response time of the hybrid strategy was 2.8 s, which was ∼45% shorter than that of PID control.

Figure 10.

Temperature dynamic response under rapid acceleration condition.

Table 1 provides the dynamic response indicators under extreme conditions. Compared with ADRC-only and TD3-only strategies, the hybrid controller achieves the shortest response time and the lowest temperature rise rate under extreme conditions. This indicates that neither classical control nor reinforcement learning alone is sufficient to handle rapid power transitions effectively. The ADRC layer provides fast disturbance rejection during abrupt changes, while the TD3 layer mitigates delayed or excessive cooling responses, leading to improved safety margins under aggressive driving scenarios. Table 1 summarizes the dynamic response metrics under extreme operating conditions. Compared with ADRC-only and TD3-only strategies, the hybrid controller achieved both the shortest response time and the lowest temperature rise rate. These results suggest that neither classical feedback control nor reinforcement learning alone was sufficient to effectively manage rapid power transitions. The ADRC layer provided fast disturbance rejection during abrupt changes, while the TD3 layer mitigated delayed or excessive cooling actions, thereby improving safety margins under aggressive driving scenarios.

To evaluate the real-time capability of the proposed ADRC–TD3 hybrid control strategy, the computational latency and control update period were measured under hardware-in-the-loop conditions. The low-level ADRC controller operates at a fixed control period of 10 ms, while the high-level TD3 policy is updated at a lower frequency and only provides supervisory setpoints. As a result, the worst-case execution time of the entire control loop remains below 8.4 ms, ensuring that all control actions are completed within a single sampling period.

Compared with MPC and pure TD3 controllers, the proposed hierarchical strategy significantly reduces online computational burden by decoupling fast control from learning-based optimization. Although the overall computational complexity is higher than that of PID and ADRC-only methods, the measured CPU utilization remains below 12% on the target embedded platform, indicating that the proposed algorithm satisfies real-time requirements for vehicle-mounted applications.

As shown in Figure 11, during the continuous urban–suburban–highway transition test, the system experienced six major power step changes. The ADRC–TD3 strategy completed adaptive adjustment within ∼15 s after each transition, maintaining temperature fluctuations within ±2.2 °C. In contrast, traditional control methods exhibited temperature fluctuations exceeding ±5 °C and required more than 40 s to re-stabilize.

Figure 11.

Temperature control performance during continuous operating condition transitions.

Algorithm convergence and stability validation

Training efficiency and convergence behavior are critical considerations for the practical deployment of reinforcement learning algorithms in engineering applications. As shown in Figure 12, the cumulative reward of the TD3 algorithm increased progressively during training. After ∼8500 training episodes, the reward value converged from an initial value of −420 to around −128, with post-convergence fluctuations remaining within ±15. In comparison, the standard DDPG algorithm required ∼16,000 episodes to reach convergence. This indicates that the TD3 algorithm achieved ∼46.9% faster convergence under the same training conditions.

Figure 12.

TD3 algorithm training process convergence curve.

Table 7 presents the convergence performance under different initialization strategies. The results show that pre-training initialization significantly accelerated convergence, reducing the required training episodes by 52.7% and increasing the convergence success rate to 96%. The improved convergence behavior can be attributed to the structural advantages of TD3, including reduced overestimation bias and delayed policy updates, which enhance training stability. Furthermore, the results indicate that pre-training and transfer learning reduced the exploration burden, which is particularly important for practical deployment scenarios where extensive online training is infeasible.

Table 7.

TD3 algorithm convergence performance statistics.

Initialization method	Convergence episodes	Final reward value	Reward variance	Training time (h)	Success rate (%)
Random initialization	11,200	−145	42.3	20.5	85
Xavier initialization	8500	−128	28.6	15.6	92
Pre-training initialization	5300	−122	21.4	9.7	96
Transfer learning	3800	−125	24.8	7	94
Hybrid initialization	7200	−126	26.2	13.2	93

These results demonstrate that the proposed learning framework achieves stable convergence while maintaining practical training efficiency. As shown in Figure 13, the estimation errors of the extended state observer (ESO) for system states and total disturbances were maintained within 3% and 5%, respectively. Under a 20% parameter perturbation condition, the estimation accuracy remained within an acceptable range, confirming the robustness of the observer design. When the observer bandwidth increased from 10 to 50 rad/s, the estimation delay decreased from 15 to 3 ms; however, noise amplification effects became more pronounced, illustrating the trade-off between response speed and noise sensitivity.

Figure 13.

Extended state observer estimation accuracy.

As shown in Figure 13, the estimation errors of the extended state observer (ESO) for system states and total disturbances were maintained within 3% and 5%, respectively. Under a 20% parameter perturbation condition, the estimation accuracy remained within an acceptable range, confirming the robustness of the observer design. When the observer bandwidth increased from 10 to 50 rad/s, the estimation delay decreased from 15 to 3 ms; however, noise amplification effects became more pronounced, illustrating the trade-off between response speed and noise sensitivity.

Comparative analysis with traditional control strategies

A comprehensive comparative analysis was conducted to evaluate the overall advantages of the proposed hybrid control strategy. As shown in Figure 14, six performance dimensions were considered: control accuracy, energy saving performance, response speed, robustness, computational efficiency, and implementation complexity. The ADRC–TD3 strategy achieved high scores in control accuracy (9.2), energy saving performance (9.0), and robustness (8.9), resulting in an overall composite score of 8.53, which was higher than those of the comparison methods.

Figure 14.

Comprehensive performance radar chart of different control strategies.

Table 8 presents detailed quantitative comparisons of the performance indicators. The radar chart and composite evaluation results indicate that the ADRC–TD3 strategy achieved balanced performance across multiple dimensions, rather than optimizing a single metric at the expense of others. Although computational resource consumption was higher than that of purely classical controllers, the improvements in control accuracy, robustness, and energy efficiency suggest that the additional computational cost is acceptable for safety-critical electric vehicle applications.

Table 8.

Comprehensive performance comparison analysis of control strategies.

Performance dimension	Evaluation indicator	PID	MPC	Fuzzy control	ADRC only	TD3 only	ADRC–TD3
Control accuracy	RMSE (°C)	3.82	2.64	3.15	2.15	2.93	1.47
	Steady-state Error (%)	2.1	1.4	1.7	1.1	1.5	0.6
Energy saving effect	Average power (kW)	2.72	2.31	2.48	2.18	2.05	1.85
	CER (%)	5.4	4.6	4.9	4.3	4.1	3.7
Response speed	Rise time (s)	8.5	6.2	7.1	4.8	7.3	3.6
	Settling time (s)	48	38	42	31	45	23
Robustness	Parameter sensitivity	High	Medium	Medium	Low	Medium	Low
	Disturbance rejection (dB)	12	18	15	24	16	26
Computational complexity	CPU utilization (%)	3.2	12.5	8.6	5.8	15.3	11.2
	Memory usage (MB)	12	85	42	28	156	142

As shown in Figure 15, a cost–benefit analysis over a 5-year operational period was performed. Although the initial implementation cost of the ADRC–TD3 strategy increased by ∼1500 CNY, the energy savings and reduced maintenance costs allowed the additional investment to be recovered by the fourth year. Over 5 years, the total net benefit was estimated to reach ∼2850 CNY.

Figure 15.

Long-term operating cost–benefit analysis.

Robustness test results

System robustness is a critical factor in ensuring reliable practical deployment. As shown in Figure 16, during the motor parameter perturbation test, thermal capacity and thermal resistance parameters were varied within ±30% of their nominal values. Under these conditions, the performance degradation rate of the ADRC–TD3 strategy was 15.3%, whereas the degradation rate of the PID controller reached 48.7%. These results indicate that the hybrid strategy maintained stable performance despite significant model uncertainty. The robustness improvement can be attributed to the real-time disturbance estimation capability of the extended state observer (ESO) and the adaptive adjustment mechanism of the TD3 supervisory layer. Together, these mechanisms enhanced the controller’s ability to compensate for parameter deviations and external disturbances.

Figure 16.

Performance degradation curve under parameter perturbation.

Table 9 summarizes the robustness test results under various disturbance conditions. As shown in Table 9, the performance degradation rate of the ADRC–TD3 strategy was maintained within 20% across all tested disturbance scenarios, and the recovery time was consistently the shortest among the compared methods.

Table 9.

Robustness test data under different disturbance types.

Disturbance type	Disturbance intensity	PID performance degradation (%)	MPC performance degradation (%)	ADRC–TD3 performance degradation (%)	Recovery time (s)
Load mutation	±50 kW	42.5	28.3	12.6	4.2
Ambient temperature change	±20 °C	35.8	23.6	10.8	6.5
Coolant flow fluctuation	±30%	38.2	31.4	14.3	5.8
Sensor noise	SNR = 20 dB	28.6	18.5	8.7	2.3
Communication delay	100 ms	45.3	35.2	16.9	3.6
Actuator saturation	Sustained 10 s	52.1	38.7	18.5	7.2

As illustrated in Figure 17, during the temperature sensor fault simulation, when the primary sensor failed, the system switched to an ESO-based temperature estimation mode within 1.5 s, and the control performance degraded by only 8%. In the actuator fault scenario involving partial pump failure (flow rate reduced by 40%), the controller compensated by increasing fan speed and adjusting valve opening, while maintaining the temperature below the safety threshold. Overall, the robustness and fault-tolerance results indicate that the proposed hybrid control strategy maintained acceptable performance under a wide range of disturbances and fault conditions. The extended state observer enabled real-time disturbance reconstruction and state estimation, while the TD3 supervisory controller adaptively adjusted cooling commands to compensate for actuator and sensor limitations. This cooperative mechanism prevented abrupt performance collapse and supported graceful degradation, which is essential for reliability and safety in real-world electric vehicle operation.

Figure 17.

Fault-tolerant control performance under fault modes.

The experimental results demonstrate that the ADRC–TD3 hybrid control strategy not only improves quantitative performance metrics such as temperature tracking accuracy and energy consumption, but also enhances dynamic responsiveness, robustness, and fault tolerance. These analytical observations confirm that the proposed method offers a practical and reliable solution for electric vehicle motor thermal management under complex and uncertain operating conditions.

Discussion

The ADRC–TD3 hybrid control strategy proposed in this study demonstrates clear advantages in electric vehicle motor thermal management by combining the fast disturbance rejection capability of active disturbance rejection control with the long-term optimization ability of deep reinforcement learning. Compared with purely classical or purely learning-based controllers, the hierarchical architecture allows each control layer to focus on complementary objectives, improving overall stability and efficiency. In particular, this design reduces the reliance on an accurate thermal model while retaining a clear control structure, which is beneficial for engineering deployment where parameter drift and operating-condition variability are common.

From an engineering perspective, the superior temperature regulation accuracy and energy-saving performance can be attributed to the decoupling of fast thermal disturbances and long-term energy optimization. The ADRC layer effectively suppresses model uncertainties and external disturbances in real time, while the TD3-based high-level controller adjusts control policies based on global performance objectives. This cooperation enables stable performance under dynamic operating conditions, which is particularly important for real-world electric vehicle applications characterized by frequent load changes and environmental variations. Moreover, the hierarchical coordination helps mitigate the typical stability and safety concerns of end-to-end reinforcement learning controllers by constraining the learning component to a supervisory role, thereby improving controllability and interpretability in safety-critical thermal regulation tasks.

Although the proposed strategy exhibits higher computational complexity than traditional control methods, the hierarchical design and task scheduling mechanism ensure that real-time requirements are still satisfied in vehicle-mounted applications. This balance between control performance and computational cost indicates good practical feasibility. Nevertheless, the method currently focuses on motor-level thermal management, and interactions with other thermal subsystems are not explicitly considered. For example, thermal coupling among the motor, power electronics, and battery cooling loop may alter the optimal cooling allocation under real driving conditions, potentially affecting vehicle-level energy efficiency. In addition, the TD3 policy still requires representative training data to generalize across different ambient temperatures, aging states, and hardware configurations. This limitation motivates further investigation into integrated thermal management strategies at the vehicle level, as well as more sample-efficient and transferable learning schemes for reducing calibration and deployment costs.

Conclusion

This paper proposed a hierarchical hybrid thermal control strategy integrating active disturbance rejection control and the Twin Delayed Deep Deterministic Policy Gradient algorithm for electric vehicle motor thermal management. By combining fast disturbance rejection with long-term energy optimization, the proposed framework enables coordinated improvement in temperature regulation performance and cooling energy efficiency. First, hardware-in-the-loop experiments conducted on a 150 kW permanent magnet synchronous motor under standard driving cycles and extreme operating conditions verify that the proposed strategy achieves improved temperature tracking accuracy, reduced cooling energy consumption, fast dynamic response, and strong robustness compared with conventional control methods. These results demonstrate the effectiveness of integrating classical control theory with deep reinforcement learning for managing complex motor thermal dynamics. Second, despite the demonstrated performance advantages, several limitations remain. The deep reinforcement learning component relies on offline training with representative operating data, which may limit direct transferability to motors or cooling systems with significantly different thermal characteristics. In addition, although the hierarchical design alleviates part of the real-time burden, the overall computational complexity is still higher than that of purely classical control strategies. Finally, future work will focus on reducing computational complexity and training requirements through more efficient learning strategies, as well as extending the proposed framework to integrated vehicle-level thermal management systems involving multiple coupled thermal subsystems, such as motors, batteries, and power electronics.