Lithium-ion battery remaining useful life prediction based on grey support vector machines

Abstract

In this article, an improved grey prediction model is proposed to address low-accuracy prediction issue of grey forecasting model. The first step is using a trigonometric function to transform the original data sequence to smooth the data, which is called smoothness of grey prediction model, and then a grey support vector machine model by integrating the improved grey model with support vector machine is introduced. At the initial stage of the model, trigonometric functions and accumulation generation operation can be used to preprocess the data, which enhances the smoothness of the data and reduces the associated randomness. In addition, support vector machine is implemented to establish a prediction model for the pre-processed data and select the optimal model parameters via genetic algorithms. Finally, the data are restored through the ‘regressive generate’ operation to obtain the forecasting data. To prove that the grey support vector machine model is superior to the other models, the battery life data from the Center for Advanced Life Cycle Engineering are selected, and the presented model is used to predict the remaining useful life of the battery. The predicted result is compared to that of grey model and support vector machines. For a more intuitive comparison of the three models, this article quantifies the root mean square errors for these three different models in the case of different ratio of training samples and prediction samples. The results show that the effect of grey support vector machine model is optimal, and the corresponding root mean square error is only 3.18%.

Keywords

Grey forecasting model trigonometric function support vector machine genetic algorithms root mean square error

Introduction

Lithium-ion battery is an ideal battery in the 21st century. Along with the improvement of performance, the battery can be widely applied in many areas.¹ Thus, the issue of collecting the battery life information and its reliability becomes critical. To obtain the authentic life data of lithium-ion battery, advanced testing technique and assessment method are needed to study its remaining useful life (RUL).

Scholars have proposed some methods aimed at predicting the RUL of lithium-ion battery. Common predicting methods can be divided into two categories: establishing failure physical model and data-driven method. Compared to failure physical models, data-driven method does not require the information of a specific failure mechanism. Instead, given a specific training sample, it can excavate implicit information between input and output, which provides a fairly-well failure trend prediction.²

At this point, the usage of data-driven methods is convenient due to their ability to transform high-dimensional and noisy environmental data into low-dimensional information for diagnostics and, especially, for prognostic tasks.³ Based on Rakhmatov et al.’s⁴ model, Xu et al.⁵ proposed a better model for predicting battery life. But this method has a range limit, for which the results cannot lead to an ideal predicting accuracy. Shen et al.⁶ used an artificial neural network to predict the degradation trend of battery life. At first, the predicting data and test data show a good consistency, but it gets worse with a higher error later. W Gu et al.⁷ proposed a data-driven modelling approach, based on grey system theory, for lithium-ion battery accelerated life testing. The prediction accuracy of this approach is not high because it can only predict the trend roughly. Using single support vector machine (SVM), A Nuhic et al.⁸ realized the modelling and projection for battery life. This approach has a high accuracy because it searches for the global optimum, while the accuracy will be decreased for mutational data. Literatures^9,10 used the method of particle filtering to predict RUL. Because particle filtering method relies on physical or experience model to establish the equation of state and there exist noise, measurement error in practical application, the prediction results exist certain error. Based on SVM, literature¹¹ proposed a similar method of relevance vector machine (RVM), which is a kind of probability prediction method. Because of its advantages of super simple parameters and generalization ability, RVM has already been applied to the forecasting field. Yoshida et al.¹² investigated the capacity loss mechanism of large capacity lithium-ion cells for satellite application and developed a simple life estimation model to fit the capacity loss data. In their subsequent work,¹³ they revised their previously developed model by considering solid electrolyte interface growth blocking mechanism. The results showed that the revised model can be used to better fit 10-year long-term capacity loss data. Burgess¹⁴ divided the float service life of a battery into two phases. During the first phase, the capacity loss was small. The capacity loss increased once the second phase began. A Kalman filter was applied to estimate the remaining float service life of a valve-regulated lead acid battery once the second phase began. However, the battery capacity fade in the second phase was so short that an early failure alarm could not be triggered by this approach. Di Maio et al.¹⁵ combined an RVM and an exponential function to estimate the RUL of bearings. Based on a similar idea, Zio and Di Maio¹⁶ used an RVM to find the most representative relevance vectors to fit a crack growth model for predicting RUL.

Those single predicting methods proposed in the aforementioned literature have different drawbacks. In this article, we first improve the grey forecasting model through trigonometric function and integrate the SVM with improved grey model (GM). Then, a grey support vector machine (SGM-SVM) model is proposed. Before modelling, the data are pre-processed using trigonometric function and accumulated generating operation in smoothness of grey prediction model (SGM), to improve the associated smoothness and weaken the corresponding randomness. After prepossessing, an SVM model for the data is established. Moreover, genetic algorithm is applied to select the model parameters, and, at last, the data are restored to obtain the predicted RUL.

Grey forecasting theory

Grey system analytical method compares the similarity and difference among the development laws of system elements, that is, correlation degree analysis, and searches for the system change rule by handling original data. After weakening the randomness of original data, a differential equation model can be built based on the generated data to predict the growing trend and future status. The most frequently used grey forecasting model is the first-order GM. Figure 1 shows the steps of constructing a grey forecasting model.

Figure 1.

Steps of building GM.

Step 1. Set the original sequence as

X^{(0)} = {x^{(0)} (1), x^{(0)} (2), \dots, x^{(0)} (n)}

where $x^{(0)} (i) > 1, 2, \dots, n$ . Take accumulated generating operation for this data to generate the grey data and obtain equation (1)

X^{(1)} = {x^{(1)} (1), x^{(1)} (2), \dots, x^{(1)} (n)}

(1)

x^{(1)} (k) = \sum_{i = 1}^{k} x^{(0)} (i), k = 1, 2, \dots, n

(2)

Step 2. Build the first-order differential equation

\frac{d x^{(1)} (t)}{dt} + a x^{(1)} (t) = b

for the sequence $X^{(1)} = {x^{(1)} (1), x^{(1)} (2), \dots, x^{(1)} (n)}$ from Step 1, where a is the development coefficient and b is the grey actuating quantity.

Step 3. Solve for parameters a and b. Use least-square method to confirm the parameter list $φ = [a, b]^{T}$

φ = {[B^{T} B]}^{- 1} B^{T} Y

(3)

B = [\begin{matrix} - z^{(1)} (2) \\ - z^{(1)} (3) \\ ⋮ \\ - z^{(1)} (n) \end{matrix} \begin{matrix} 1 \\ 1 \\ ⋮ \\ 1 \end{matrix}]

(4)

z^{(1)} (k) = \frac{1}{2} [x^{(1)} (k) + x^{(1)} (k - 1)]

(5)

Y = {(x^{(0)} (2), x^{(0)} (3), \dots, x^{(0)} (n))}^{T}

(6)

Step 4. Under starting condition ${\hat{x}}^{(1)} (1) = x^{(1)} (1) = x^{(0)} (1)$ , the generated sequence model is given in equation (7)

{\hat{x}}^{(1)} (1) = (x^{(0)} (1) - \frac{\hat{b}}{\hat{a}}) e^{- \hat{a} (k - 1)} + \frac{\hat{b}}{\hat{a}}, k = 2, 3, \dots, n

(7)

Step 5. Under starting condition ${\hat{x}}^{(1)} (1) = x^{(1)} (1) = x^{(0)} (1)$ , we can get the original sequence through regressive operation for modelling forecasting sequence ${\hat{x}}^{(1)} (1)$ . Regressive operation is an operation that subtracts numbers in sequence before and after to get generated sequence. It is a restore operation of accumulation operation

{\hat{x}}^{(0)} (k) = {\hat{x}}^{(1)} (k) - {\hat{x}}^{(1)} (k - 1), k = 2, 3, \dots, n

(8)

That is, ${\hat{x}}^{(0)} (1) = x^{(0)} (1)$

{\hat{x}}^{(0)} (k) = (1 - e^{\hat{a}}) (x^{(0)} (1) - \frac{\hat{b}}{\hat{a}}) e^{- \hat{a} (k - 1)}, k = 2, 3, \dots, n

(9)

Note that if $k > n$ , the future forecasting value could be obtained.

Grey forecasting model has the characteristics of simple principles and convenience of calculations. But under restrictions of development coefficient, it can only forecast a rough trend of a long-term data set and the corresponding accuracy is not high, which makes its practical value low. So, the grey forecasting model has some theory drawbacks and many researchers try to optimize and improve it.

Literature^17–19 shows that the data smoothness is an important factor that can influence the accuracy of grey forecasting model. In other words, increasing the data smoothness can improve the model accuracy effectively. Particularly, trigonometric function is used to address the data smoothness. This article improves the grey forecasting model, based on the one shown in Figure 1, using trigonometric function $\sin (x) (π / 2 \leq x \leq π)$ to increase data smoothness. The improved grey forecasting model is called SGM. Basic steps are shown in Figure 2.

Figure 2.

Steps of building SGM.

To normalize the sequence

Y^{(0)} = {y^{(0)} (1), y^{(0)} (2), \dots, y^{(0)} (n)}

For $\forall i$ , processed $y^{(0)'} (i)$ are all within the range [0.5π, π]. Then, a new sequence is generated

X^{(0)} = {x^{(0)} (1), x^{(0)} (2), \dots, x^{(0)} (n)}

using trigonometric function

x^{(0)} (i) = \sin (y^{(0)'} (i)), i = 1, 2, \dots, n \cdot

Forecast the result based on one-step grey forecasting model and use ${y^{(0)}}^{'} (i) = π - \arcsin ({\hat{x}}^{(0)} (i))$ to restore data and standardize ${y^{(0)}}^{'} (i)$ to $y^{(1)} (i)$

Y^{(1)} = {y^{(1)} (1), y^{(1)} (2), \dots, y^{(1)} (n)}

Original sequence $Y^{(0)} = {y^{(0)} (1), y^{(0)} (2), \dots, y^{(0)} (n)}$ is processed by SGM to generate the sequence $Y^{(1)} = {y^{(1)} (1), y^{(1)} (2), \dots, y^{(1)} (n)}$ .

SVM

SVM includes two parts: support vector classification (SVC) and support vector regression (SVR). The SVR algorithm is mainly discussed in this article.²⁰

SVR is a function that uses limited training sample data ${y_{i}, m_{i}}$ , $i = 1, 2, \dots, l$ ( $y_{i} \in R^{n}$ is an n-dimensional sample input and $m_{i} \in R$ is a sample output). A mapping function f of input and output variable can be built by learning and training.^21,22

Under most circumstances, data for training sample have non-linear relationship. For non-linear cases, the idea of SVR is to transform space non-linear problems to linear problems in a space with high dimension through a non-linear mapping and replace the inner product of sample vector by kernel function $K (y_{i}, y_{j})$ . Hence, one can build the linear regression estimation in this characteristic space as

f (y) = (w \times f (y)) + b . F

(10)

In equation (10), the sample points $(y_{1}, m_{1}), (y_{2}, m_{2}), \dots, (y_{l}, m_{l})$ are independent and identically distributed, and b is the deviation value.

For an unknown value x outside the training sample collection, optimal solutions w and b could be found which make $| f (y) - (w \cdot y) - b | \leq ε$ . The problem is equivalent to solving the following optimization problem¹¹

min ϕ (y) = \frac{1}{2} ‖ w ‖^{2} = \frac{1}{2} (w \cdot w)

(11)

Subject to \begin{matrix} {\begin{matrix} m_{i} - ((w \cdot y_{i}) + b) \leq ε \\ ((w \cdot y_{i}) + b) - m_{i} \leq ε \end{matrix} & i = 1, 2, \dots, l \end{matrix}

(12)

In equation (12), $ε > 0$ is the insensitive loss function.

If $f (y)$ in the sample data problem cannot be estimated in accuracy $ε$ , while importing slack variables $ξ$ and $\overset{⌢}{ξ}$ , then problem can be transformed into

\min ϕ (y) = \frac{1}{2} {‖ w ‖}^{2} + C \sum_{i = 1}^{l} (ξ_{i} + {\overset{⌢}{ξ}}_{i})

(13)

Subject to \begin{matrix} {\begin{matrix} m_{i} - ((w \cdot y_{i}) + b) \leq ε + ξ_{i} \\ ((w \cdot y_{i}) + b) - m_{i} \leq ε + {\overset{⌢}{ξ}}_{i} \\ ξ_{i}, {\overset{⌢}{ξ}}_{i} \geq 0 \end{matrix} & i = 1, 2, \dots, l \end{matrix}

(14)

where $C > 0$ is the penalty factor used to control the samples whose errors are higher than $ε$ . The associated dual problem would be constructed by introducing Lagrangian multipliers

\begin{array}{l} \max Q (α_{i} - {\overset{⌢}{α}}_{i}) = - \frac{1}{2} \sum_{i, j = 1}^{l} (α_{i} - {\overset{⌢}{α}}_{i}) (α_{j} - {\overset{⌢}{α}}_{j}) (y_{i} \cdot y_{j}) \\ - ε \sum_{i = 1}^{l} (α_{i} + {\overset{⌢}{α}}_{i}) + \sum_{i = 1}^{l} y_{i} (α_{i} - {\overset{⌢}{α}}_{i}) \end{array}

(15)

Subject to {\begin{matrix} \sum_{i = 1}^{l} (α_{i} - {\overset{⌢}{α}}_{i}) = 0 \\ α_{i}, {\overset{⌢}{α}}_{i} \in [0, C] \end{matrix}

(16)

Then, SVR estimate function can be written as

f (y) = \sum_{i = 1}^{l} (α_{i} - {\overset{⌢}{α}}_{i}) K (y_{i}, y) + b

(17)

SGM-SVM

Establishment of model

Although the prediction accuracy of original grey forecasting model is low, the improved GM can use trigonometric functions and the accumulation generation operation to enhance the data smoothness and weaken the randomness. In this section, a new method called SGM-SVM that combines the improved GM and SVM is proposed. Specific steps are included in Figure 3.

Figure 3.

Steps of building SGM-SVM.

Selection of the model parameters

For the SGM-SVM, the most important question is how to select the model parameters.

Kernel function

The structure of the characteristic space depends on the type of kernel function, so the selection of kernel function directly influences the effect of transformation from a non-linear separable space to a linear separable space. Previous research^23,24 analyses and compares the SVM algorithms based on different kernel functions, making a conclusion that the accuracy of SVM algorithm based on Gaussian kernel (Radial Basis Function) is the best. The Gaussian kernel function has the advantage of fewer parameters and smaller prediction errors; thus, it is selected as the kernel function

K (y_{i}, y) = \exp (- \frac{{‖ y_{i} - y ‖}^{2}}{δ^{2}})

(18)

where $δ$ is the width of the function parameters.

$ε$ – the cost function parameter and penalty factor C

For the cost of function parameters and penalty factor, the existing literatures have not given more reasonable options, most of which present empirical value according to different data. Genetic algorithm is used for these two parameters to global search for better parameter values.

Battery RUL prediction model

This article chooses two sets of data provided by the Center for Advanced Life Cycle Engineering (CALCE) to forecast the battery RUL and selects the discharge capacity of the battery as the health indicator. To verify the prediction accuracy of the proposed SGM-SVM model, as for data 1, the top 200 data of battery discharge capacity is selected, in which the prior 120 data are training samples, and the rest 80 data are predicted samples, while as for data 2, the top 150 data of battery discharge capacity is selected, in which the prior 90 data are training samples, and the rest 60 data are predicted samples. Then, establish the prediction model. In particular, to prove the superiority of the proposed model, comparing to other models, the original GM and the single SVM are used to predict the battery RUL as well. Finally, a comparison is made among the results of the three different models.

Now, the original GM is used to forecast the battery RUL. Because grey prediction model is just a simple predictive model of first-order differential equation, its prediction accuracy is not very high and it can just predict rough trends. The results are shown in Figure 4.

Figure 4.

Prediction results of GM.

Then, a single SVM is used to realize a prediction. Due to the fact that SVM itself has a strong predicting ability, to demonstrate the effect of the proposed SGM-SVM, a further research should be conducted on the predicting results of a single SVM, including determination of the model parameters of SVM, selection of the historical data of different length from the first training data and establishment of forecasting model to predict the rest of the data. As for data 1, the parameters of model are $ε = 0.01$ , $C = 500$ and $δ^{2} = 2.25$ . Based on the theory of structure risk minimization, the error is minimized when the length of the historical data is 50 (error is 0.8572). Therefore, best historical data length is set as 50. As for data 2, the parameters of model are $ε = 0.01$ , $C = 426$ and $δ^{2} = 3.14$ and the best historical data length is 80 (error is 0.9782). Prediction results are presented in Figure 5.

Figure 5.

Prediction results of SVM.

The picture implies that the predicted results are close to the actual values. So, the SVM can predict the overall trend and has a strong predicting ability. However, the prediction at the inflection point is not ideal, and there are some errors and deviations between the actual discharge capability and predicted discharge capability. The predicated data curve is inconsistent with the actual data curve when they come across each other. Meanwhile, the trend of actual data curve and predicated data curve is not consistent with each other. Hence, trigonometric function and cumulative generation is used in the improved grey forecasting model to handle the original data. The processed data are shown in Figure 6. This operation can improve the regularity and smoothness at the inflection point and enhance the accuracy of model prediction. Prediction results of SGM-SVM are shown in Figure 7.

Figure 6.

Processed data by SGM.

Figure 7.

Prediction results of SGM-SVM.

From Figure 6, it could be concluded that the SGM-SVM can predict the battery discharge capacity accurately and has a higher accuracy than simple SVM model. We provide root mean square error (RMSE) of each model to compare the predicted results of these three models.

If the actual observed value is $y_{i}$ and the predicted value is ${\hat{y}}_{i}$ , the RMSE can be expressed as

RMSE = \sqrt{\frac{1}{N} \sum_{i = 1}^{N} {(\frac{y_{i} - {\hat{y}}_{i}}{y_{i}})}^{2}} \times 100 %

The RMSEs of the three models are presented in Table 1. The results show that the predicting accuracy of SGM-SVM is higher than that of GM and SVM models.

Table 1.

RMSE of three models.

Model	GM	SVM	SGM-SVM
RMSE (data 1, %)	11.35	6.72	3.18
RMSE (data 2, %)	25.16	9.34	5.27

GM: grey model; SVM: support vector machine; SGM-SVM: grey support vector machine; RMSE: root mean square error.

As for the prediction result, it is just for one case of the ratio of training samples and prediction samples. In order to prove whether the SGM-SVM is the best model for predicting, it is necessary to discuss the case of different ratio of training samples and prediction samples. The RMSEs of the different ratio of training samples and prediction samples are presented in Tables 2 and 3.

Table 2.

RMSE of different ratio of training samples and prediction samples (data 1).

Ratio of training and prediction sample number	RMSE (data 1, %)
Ratio of training and prediction sample number	GM	SVM	SGM-SVM
1.5	11.35	6.72	3.18
1.3	12.14	7.39	3.96
1.1	13.68	8.14	4.85
0.9	15.11	10.08	6.04
0.7	17.36	11.97	8.94
0.6	19.45	14.04	10.47
0.5	22.52	16.54	13.48
0.4	24.25	18.27	16.73

RMSE: root mean square error; GM: grey model; SVM: support vector machine; SGM-SVM: grey support vector machine.

Table 3.

RMSE of different ratio of training samples and prediction samples (data 2).

Ratio of training and prediction sample number	RMSE (data 2, %)
Ratio of training and prediction sample number	GM	SVM	SGM-SVM
1.5	25.11	9.34	5.27
1.3	27.16	10.77	6.34
1.1	28.38	12.98	6.99
0.9	29.95	15.34	8.24
0.7	32.52	16.49	10.72
0.6	35.25	18.93	12.91
0.5	39.07	21.30	14.35
0.4	42.48	23.55	16.06

RMSE: root mean square error; GM: grey model; SVM: support vector machine; SGM-SVM: grey support vector machine.

Analysing the table above, the results show that no matter what ratio of training samples and prediction samples number, the predicting accuracy of SGM-SVM is higher than that of GM and SVM models.

Conclusion

The original grey forecasting model is improved using trigonometric function. Then, an SGM-SVM model that integrates the improved GM with the SVM is proposed. Genetic algorithm is applied to search for optimal model parameters. Finally, RMSEs of three models are compared, and the results conclude that the accuracy of SGM-SVM is higher than that of GM and SVM models. This SGM-SVM provides an efficient way to estimate battery RUL.

Footnotes

Academic Editor: Joo Ho Choi

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Wang

Pan

The technology of lithium-ion batteries for spacecraft application. Aerospa 2000; 17: 54–59.

Zhang

Lee

A review on prognostics and health monitoring of Li-ion battery. J Power Source 2011; 196: 6007–6014.

Widodo

Shim

M-C

Caesarendra

. Intelligent prognostics for battery health monitoring based on sample entropy. Expert Syst Appl 2011; 38: 11763–11769.

Rakhmatov

Vrudhula

Wallach

DA.

A model for battery lifetime analysis for organizing applications on a pocket computer. IEEE Trans Very Large Scale Integr Syst 2003; 11: 1019–1030.

Chao

Lithium-ion battery lifespan prediction model. J Appl Sci 2006; 4: 008.

Shen

Chan

EWC

. A new battery available capacity indicator for electric vehicles using neural network. Energ Convers Manage 2002; 43: 817–826.

Sun

Wei

. A new method of accelerated life testing based on the grey system theory for a model-based lithium-ion battery life evaluation system. J Power Source 2014; 267: 366–379.

Nuhic

Terzimehic

Soczka-Guth

. Health diagnosis and remaining useful life prognostics of lithium-ion batteries using data-driven methods. J Power Source 2013; 239: 680–688.

Saha

Goebel

Modeling Li-ion battery capacity depletion in a particle filtering framework. In: Proceedings of the annual conference of the Prognostics and Health Management Society, San Diego, CA, September 2009, pp.1–10. IEEE.

10.

Saha

Goebel

Poll

. An integrated approach to battery health monitoring using Bayesian regression and state estimation. In: Proceedings of autotestcon, Big Sky, MT, March 2007, pp.646–653. IEEE.

11.

Zhou

Liu

Peng

. Combined sparse Bayesian learning strategy for remaining useful life forecasting of lithium-ion battery. In: 2012 second international conference on measurement, computer, communication and control (IMCCC), Harbin City, Heilongjiang, December 2012, pp.457–461. IEEE.

12.

Yoshida

Imamura

Inoue

. Capacity loss mechanism of space lithium-ion cells and its life estimation method. Electrochemistry 2003; 71: 1018–1024.

13.

Yoshida

Imamura

Inoue

. Verification of life estimation model for space lithium-ion cells. Electrochemistry 2010; 78: 482–488.

14.

Burgess

WL.

Valve regulated lead acid battery float service life estimation using a Kalman filter. J Power Source 2009; 191: 16–21.

15.

Di Maio

Tsui

Zio

. Combining relevance vector machines and exponential regression for bearing residual life estimation. Mech Syst Signal Process 2012; 31: 405–427.

16.

Zio

Di Maio

Fatigue crack growth estimation by relevance vector machine. Expert Syst Appl 2012; 39: 10681–10692.

17.

Meng

Study on generalization for grey forecasting model. Syst Eng Theory Pract 2002; 22: 138–141.

18.

Cui-feng

Wen-zhan

An approach of the grey modelling based on cotx transformation. Syst Eng 2005; 23: 110–114.

19.

Yao-jin

Chen-xi

. The research and application of grey forecasting model base on eponential & triangle function. Zhanzhou, China: Journal of Zhangzhou Normal University, 2009.

20.

Cheng

X-K.

Research and application of civil aviation passenger volume model based on discriminant analysis-SVR. J Sichuan Univ (Nat Sci Edn) 2008; 45: 527–531.

21.

Hongmin

Haiying

Jin

. Support vector machine regression algorithm and its application. Beijing Inst Petro-chem Tech 2010; 18: 62–66.

22.

Chang-hua

Jin-tao

Wei

. Reliability assessment of performance degradation using support vector machines. Syst Eng Electr 2009; 5: 1246–1249.

23.

Cristianini

Shawe-Taylor

An introduction to support vector machines and other kernel-based learning method. Cambridge: Cambridge University Press, 2000.

24.

Bing

Study of life prediction method based on support vector machine for lithium-ion battery. Wuhan, China: Huazhong University of Science and Technology, 2012.