Short-term power load forecasting based on support vector machine and particle swarm optimization

Abstract

In this work, we summarized the characteristics and influencing factors of load forecasting based on its application status. The common methods of the short-term load forecasting were analyzed to derive their advantages and disadvantages. According to the historical load and meteorological data in a certain region of Taizhou, Zhejiang Province, a least squares support vector machine model was used to discuss the influencing factors of forecasting. The regularity of the load change was concluded to correct the “abnormal data” in the historical load data, thus normalizing the relevant factors in load forecasting. The two parameters are as follows Gauss kernel function and Eigen parameter C in LSSVM had a significant impact on the model, which was still solved by empirical methods. Therefore, the particle swarm optimization was used to optimize the model parameters. Taking the error of test set as the basis of judgment, the optimization of model parameters was achieved to improve forecast accuracy. The practical examples showed that the method in the work had good convergence, forecast accuracy, and training speed.

Introduction

In this work, the short-term load is forecasted by the least squares support vector machine (LS-SVM) and improved particle swarm optimization (PSO) algorithms. In Shawe-Taylor and Cristianini,^{1, 2} PSO and chaos optimization algorithms are used to select the parameters of a support vector machine (SVM) model, respectively. After that, the two methods are combined to forecast the short-term power load. In Suykens and Vandewalle, ³ the short-term power load is forecasted by combining SVM based on PSO with the fuzzy reasoning. In Gong, ⁴ the short-term load forecasting, SVM regression, and sequential minimal optimization (SMO) theory are conducted with intensive research. Then, the short-term loads are forecasted by linear regression, SVM regression, and SMO. Based on the analysis of parameters and performances of SVM, the Grid-search method is introduced to the short-term load forecasting algorithm based on SVM, solving parameter selection problem of SVM.⁵ In Heng, ⁶ the input variables are pretreated by rough set theory to realize the optimal selection, reducing the dependence on experience and improving model adaptability. Combined with rough set theory, genetic algorithm (GA) is used to optimize the model parameters of LSSAM, establishing the short-term load forecasting model of LSSVM. In Yang and Cheng,⁷ the reasonable historical data are filtered by clustering to form training samples. Then, the forecast smoothness and error loss function are integrated to constitute the objective function of problem. LIBSVM is a set of support vector machine library developed by Professor Chih-Jen Lin in 2001. This library is fast in operation and can be used to classify and regression data conveniently. Because libSVM program is small, flexible to use, less input parameters, and is open source, easy to expand, so it has become the most widely used SVM Library in China. Using LIBSVM algorithm, large-scale optimization problem of SVM is transformed into secondary optimization problem with analytic solution. Based on the chaotic characteristics of load time series and LS-SVM, a short-term load forecasting model is established by combining phase space reconstruction theory of chaotic time series and regression theory of SVM.⁷ In literature,² PSO and chaos optimization are applied to select the parameters of the SVM model respectively. Finally, the two methods are combined to predict short-term power load. In literature, ⁸ the SVM and fuzzy inference of PSO are integrated for the short-term load forecasting. In literature, ⁹ the short-term load forecasting, support vector regression, and SMO theory are deeply studied, and linear regression, support vector regression, and SMO are used to predict the short-term load in three ways. ¹⁰ Based on the analysis of the SVM's parameter performance, the Grid-search method is introduced to the SVM-based short-term load forecasting algorithm, in order to solve the problem of parameter selection in SVM. In literature, ¹¹ the input variables are preprocessed by rough set theory, which realizes the optimal selection of input variables, reduces the dependence on experience in the process of establishing prediction models, and improves the adaptability of models. The GA is used to optimize the model parameters of the LS-SVM. A short-term load forecasting model of LS-SVM is established, which combines rough set theory and GA. Literature ¹² in filtering the historical data through clustering of the training sample, smoothness and error loss function prediction combined constitute the objective function of the problem, using LIBSVM algorithm for large-scale optimization problems SVM into two analytical solutions of optimization problems. ¹³ Based on the chaotic characteristics of load time series, combined with the theory of phase space reconstruction of chaotic time series and the regression theory of SVM, a short-term load forecasting model based on load chaos characteristics and LS-SVM is established.

After going through mass data, we obtain the following difficulties from the perspective of short-term load forecast.

Overall consideration of the factors affecting load forecast

The load forecast is to predict the future value of power load according to its past and present. However, the historical data of the load are lost or wrong due to measurement and human factors. It is unfavorable to grasp the changing trend of load, increasing the difficulty of load analysis. Therefore, it is necessary to finish the identification and correction of bad data before the training and forecast of the historical data.

2. Selection of load forecasting method

Correct selection of forecasting model is the most critical step in load forecasting. With the deepening of load forecasting technology research, various load forecasting methods come into being, with different research characteristics and working conditions. No method can be suitable for all situations. In order to improve the forecast accuracy, the appropriate forecasting method is selected according to the actual situation.

With the rapid development of science and technology, the research and application of artificial intelligence methods have great advantages and application potential. The main algorithms areLS-SVM(Least squares support vector machine ) ^14,15,16, neural network algorithm ¹⁷, fuzzy reasoning system ¹⁸, genetic algorithm, chaos theory ¹⁹. These methods and expert systems, Tabu search, mosquito search, simulated annealing, data analysis, adaptive, self-learning and other technologies closely combined, complementary prediction methods, collectively known as intelligent technology.(1) artificial neural network method. In 1991, Park. D. C. et al. first introduced artificial neural network into load forecasting, and then, the research on neural network load forecasting emerged in endlessly.²⁰ The advantage of neural network technology is that it can imitate the intelligent processing of human brain and has adaptive function to a large number of non-structural and non-accurate laws. The disadvantage is that the training process is slow, and it can not guarantee its convergence. At the same time, the structure of the neural network, the appropriate selection of input variables, the number of hidden layers and other issues need to be explored in practice.(2) fuzzy control method. Fuzzy prediction method only simulates the reasoning and judgment of experts, and does not need to establish an accurate mathematical model.²¹ Fuzzy theory is suitable for describing widely existing uncertainties, and it has powerful nonlinear mapping ability. It can uniformly approximate any nonlinear function defined on a compact density with arbitrary precision, and can extract their similarity from a large number of data. However, with the further study and application of fuzzy theory, fuzzy theory has exposed some shortcomings: the learning ability of fuzzy is weak; when the mapping area is not fine enough, the mapping output is rough.^22,23 (3) genetic algorithm. Genetic algorithm is a stochastic, iterative and evolutionary search method based on natural selection and population genetic mechanism. Genetic algorithm has the ability of global optimization. Generally, genetic algorithm is used to optimize ANN weights in order to overcome the shortcomings of BP algorithm in convergence performance and local minimum, and improve the prediction accuracy. (4) support vector machine. A new machine learning algorithm, Support Vector Machine (SVM), was proposed by Vapnik et al. of Bell Laboratory in 1995. Unlike the empirical risk minimization (EMR) induction principle that most machine learning methods are based on, it is based on structural risk minimization (SRM) and VC dimension theory, and achieves a good balance between model complexity and learning ability. Therefore, its generalization ability is much better than that of artificial neural network and fuzzy logic. SVM regression algorithm has the advantages of short convergence time, high prediction accuracy,²⁴ less adjustable parameters and easy structure determination, and it does not need too much prior information and use skills. Therefore, more and more attention has been paid to the application of SVM in the field of power load forecasting. SVM has broad application space and development prospects, and is considered as the best alternative to neural network method.

SVM regression principle

LS-SVM is firstly proposed by Suykens and Vandewalle.³ It is an extension of the standard SVM. Compared with other versions of SVM, the LS-SVM has fewer parameters to be selected. In addition, the equation constraints are used to replace the original inequality constraints, reducing some uncertainties. The loss function is directly defined as the sum of squares of errors to transform the optimized inequality constraints into equality constraints. Therefore, the quadratic programming problem is transformed into linear equations to reduce the computational complexity, accelerating the solution speed. The basic principle is as follows.

For the nonlinear load forecasting model

$$f(x)=(\omega ,\phi (x))+b$$

(1)

Given a set of data points: $(x_i,y_i),i=1,\dots ,l$ where $x_i\in R^d$ is the factor closely related to the forecast quantity, such as the historical load data and meteorological factor; d the dimension of selected input variable; $y_i\in R$ the expected value of forecast quantity; $l$ the total number of known data points; $\phi (x)$ the non-linear mapping from the input space to high-dimensional feature space. According to the principle of structural minimization, the LS-SVM optimization target can be expressed as equation (2).

$$\min \frac{1}{2}{\Vert \omega \Vert }^2+\frac{1}{2}\gamma {\displaystyle \sum _{i=1}^l{e}_i^2}$$

(2)

$$s.t.{\omega }^T\phi (x_i)+b+e_i=y_i,i=1,\dots ,l$$

where $e_i$ the error; $e\in R^{l\times 1}$ the error vector; $\gamma$ the regularization parameter which controls the degree of punishment for error. By introducing Lagrange multiplier $\lambda \in R^{l\times 1}$ , we transform equation (2) into

$$\begin{array}{l}\min J=\frac{1}{2}{\Vert \omega \Vert }^2+\frac{1}{2}\gamma {\displaystyle \sum _{i=1}^l{e}_i^2}\\ -{\displaystyle \sum _{i=1}^l{\lambda }_i({\omega }^T\phi (x_i)+b+e_i-y_i)}\end{array}$$

(3)

According to Karush-Kuhn-Tucker (KKT) condition

$$\{\begin{array}{l}\frac{\partial J}{\partial \omega }=0\to {\displaystyle \sum _{i=1}^l{\lambda }_i\phi (x_i)}\\ \frac{\partial J}{\partial b}=0\to {\displaystyle \sum _{i=1}^l{\lambda }_i=0}\\ \frac{\partial J}{\partial e_i}=0\to {\lambda }_i=\gamma e_i,i=1,2,\dots ,l\\ \frac{\partial J}{\partial {\lambda }_i}=0\to {\omega }^T\phi (x_i)+b+e_i-y_i=0,i=1,2,\dots ,l\end{array}$$

(4)

After the elimination of $\omega$ and $e$ , the solution of equation (4) is expressed as follows.

$$\begin{array}{l}\left|y(d,t)-\overline{y(t)}\right|>\theta ,\hspace{1em}y(d,t)\\ =\left\{\begin{array}{cc}\overline{y(t)}+\theta & y(d,t)>\overline{y(t)}\\ \overline{y(t)}-\theta & y(d,t)<\overline{y(t)}\end{array}\right\}\end{array}$$

(5)where $\overline{\lambda }=[{\lambda }_1,{\lambda }_2,\mathrm{\dots }{{\lambda }_l]}^T,\overline{I}=[1,1,\mathrm{\dots },{1]}^T$ is the $l\times 1$ -dimensional column vector. $Y={[y_1,y_2,\mathrm{\dots },y_l]}^T,\Omega \in R^{l\times l}$ , and ${\Omega }_{ij}=\phi {(x_i)}^T\phi (x_j)=K(x_i,x_j)$ . K is the kernel function satisfying Mercer condition ( $K(x_i,x_j)=\phi {(x_i)}^T\phi (x_j)$ ). The kernel function of original space is used to replace the dot product operation of high-dimensional feature space, thus simplifying the calculation. Therefore, the nonlinear forecasting model is expressed as equation (6).

$$y={\displaystyle \sum _{i=1}^l{\lambda }_{i}K(x_i,x)}+b$$

(6)where ${\lambda }_i$ and $b$ are obtained by solving equation (6); $K(\cdot ,\cdot )$ is the nonlinear mapping from input to high-dimensional feature space.

PSO theory

Principle of standard PSO

PSO is a swarm intelligence evolutionary computation technology based on iterative optimization. A swarm of random particles is initialized to find optimal solution by iterations. In each iteration process, the particle updates the velocity and the position in the next iteration by tracking individual extreme $P_{ibest}$ (the optimal solution found by the particle itself) and global extreme $g_{best}$ (the optimal solution found by the whole swarm). According to the two extremes, the particle determines its own flight speed and distance.

It is assumed that there are $m$ particles consisting of a swarm in a $d$ -dimensional search space. Wherein, the i-th particle is expressed as a $d$ -dimensional vector x_i ( $x_i=(x_{i1},x_{i2},\mathrm{\dots },x_{id}),i=1,2,\mathrm{\dots },m,$ ), i.e., the position of the i-th particle in the $d$ -dimensional search space is $x_i$ . As a $d$ -dimensional vector, the flight speed of the i-th particle $v_i=(v_{i1},v_{i2},\mathrm{\dots },v_{id})$ ; the i-th particle finds the optimal position $p_i=(p_{i1},p_{i2},\mathrm{\dots },p_{id})$ so far; the whole swarm finds the optimal position $p_g=(p_{g1},p_{g2},\mathrm{\dots },p_{gd})$ so far. ¹⁰

The standard PSO updates the speed and position of the particle by the following equations.

$$v_{id}=w{v}_{id}+c_1{r}_1(p_{id}-x_{id})+c_2{r}_2(p_{gd}-x_{id})$$

(7)

$$x_{id}=x_{id}+v_{id}$$

(8)where $w$ is the inertia weight coefficient; $c_1$ and $c_2$ are the non-negative constants—acceleration constants; $r_1$ and $r_2$ the random numbers between 0 and 1.

Improved PSO theory

In the work, we designed an improved PSO, which controls population characteristics by diversity metrics, to solve premature convergence of particle swarm. Specific implementation process includes the following two aspects.

Selection of initial particle swarm

The initial particle swarms are randomly selected. Ideally, the positions spread over the entire solution space to increase the probability of finding the global optimal solution. However, the initial particle swarm has limited particles and large solution space. If limited particles are not uniformly distributed in the whole solution space, then the possibility of local optimum will be increased.

The concept of average inter-particle distance is introduced and defined as equation (9).

$$D(t)=\frac{1}{mL}{\displaystyle \sum _{i=1}^m\sqrt{{\displaystyle \sum _{d=1}^n{(p_{id}-\overline{p_d})}^2}}}$$

(9)where $L$ the maximum diagonal length of search space; $n$ the dimension of solution space; $p_{id}$ the $d$ -dimensional coordinate value of the i-th particle position; $\overline{p_d}$ the mean value of $d$ -dimensional coordinate values of all the particle positions.

The average inter-particle distance indicates the dispersion degree of particles in the swarm. The smaller D(t) leads to the more concentrated swarm; the larger D(t) leads to the more dispersed swarm.

2. Judgment of premature convergence

In the whole iterative process of standard PSO, the particles approach to global history optimum solution. Standard PSO reaches fast and slow convergence rates at the initial and latter stages, respectively. If the local extreme point is encountered, the speeds of all particles will soon drop to zero. The swarm loses the ability to evolve, and then the algorithm gets into local optimal point because of premature convergence. For the particle, the position determines the fitness. Therefore, the current state of the swarm can be judged according to the overall change in the fitness of all particles. If the current fitness of the i-th particle is $f_i$ , the current average fitness of the swarm $\overline{f}$ , then the fitness variance of the swarm can be defined as

$${\sigma }^2={\displaystyle \sum _{i=1}^m{\left(\frac{f_i-\overline{f}}{f}\right)}^2}$$

(10)where $m$ is the number of particles in swarm; $f$ the normalized scaling factor which is used to restrict the size of ${\sigma }^2$ .

Fitness variance reflects the aggregation degree of particles in the swarm. The smaller ${\sigma }^2$ leads to the larger aggregation degree of the particles in the swarm; otherwise, the larger ${\sigma }^2$ leads to the smaller the aggregation degree. The larger number of iterations leads to the closer fitness of particles and smaller ${\sigma }^2$ in the swarm. When ${\sigma }^2<\beta$ ( $\beta$ is a given threshold), the algorithm will enter the later search phase. Here, the swarm easily falls into local optimum to generate premature convergence.

Load forecasting procedure

The concrete process of the improved PSO is as follows. In the premise of uniform initial swarm distribution, the basic operation of standard PSO is firstly implemented until the particles are in the precocious state. After that, the particle solution space is reallocated to guide particles to quickly jump out of local optimum, thus accelerating the convergence. The specific algorithm and flow chart are presented in Figure 1.

Figure 1.

The flow chart. PSD: particle swarm optimization; SVM: support vector machine.

1. The particle swarm is initialized according to the above method. The swarm size is set as m; the number of maximum evolutional generations as $T_{\max }$ ; the iteration termination threshold as $\epsilon$ ; the initial and final values of inertia weight are as $w_{\max }$ and $w_{\min }$ ; the acceleration constants as $c_1$ and $c_2$ .

2. The fitness values $f(x_i)$ of particles are calculated for comparison according to the current position. The current point of the i-th particle is set as the optimal position $p_{ibest}$ ; the best of all the particles as the optimal position $g_{best}$ of swarm.

3. We calculate the average inter-particle distance D(t) and fitness variance ${\sigma }^2$ of the swarm. If D(t)< $\alpha$ ; ${\sigma }^2<\beta$ ( $\alpha$ and $\beta$ are given thresholds), then the swarm will be precocious, and turn to Step (4); otherwise, turn to Step (5).

4. The particle swarm is re-initialized according to the method described above.

5. The velocity and position of each particle are updated according to equations (7) and (8) to produce new swarm X(t).

6. The fitness values of new positions of particles in X(t) are calculated to compare with the historical optimal positions of individuals and swarm, respectively. If the new position has better fitness value, then the historical optimal position will be replaced; otherwise, it will remain unchanged.

7. Check whether the end condition of optimization is satisfied (equal to $T_{\max }$ or less than $\epsilon$ ). If it is satisfied, then the optimization will be ended to derive the optimal solution; otherwise, let t=t+1, and continue the calculation and circulation.

Error Evaluation Index

Absolute Error (AE)

$$E_{AE}=\left|L-\widehat{L}\right|$$

(11)

Relative Error (RE)

$$E_{RE}=\frac{\left|L-\widehat{L}\right|}{L}\times 100\%$$

(12)

Mean Absolute Error (MAE)

$$E_{MAE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|L_i-{\widehat{L}}_i\right|}$$

(13)

Mean Absolute Percentage Error (MAPE)

$$E_{MAPE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|\frac{L_i-{\widehat{L}}_i}{L_i}\right|}\times 100\%$$

(14)

Mean-Square Error (MSE)

$$E_{MSE}=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left(\frac{L_i-{\widehat{L}}_i}{L_i}\right)}^{\text{2}}}}\times 100\%$$

(15)

where $L$ and $\widehat{L}$ are the actual and forecasted loads, respectively. $n$ is the number of load data. In power load forecasting, MAPE and MSE are often used as the evaluation standards of error.

Simulation and experiment

The simulation is implemented in MALAB2008a. Firstly, LS-SVM algorithm was used to predict the load data of certain region in Taizhou, Zhejiang Province in 2015. The predicted results are compared with the actual load data of the forecast day using MAE. Figure 2 shows the forecasted and actual load curves by LS-SVM. Multiple samples are forecasted to overcome contingent factors (see Table 1).

Figure 2.

Forecast results by LS-SVM algorithm (green: actual load; red: forecast load).

Table 1.

Forecast results of multiple samples by SVM.

SVM	C = 30	Theta = 2
Date	27 March	28 February	31 April	20 May
Error (%)	2.73	2.76	3.18	3.08
Mean error (%)	2.9375

SVM: support vector machine.

In Software MALAB2016a, the improved PSO was used to predict the critical parameters of the SVM model. The predicted results are compared with the actual load data by MAE. Similarly, different sample sets in certain region of Taizhou, Zhejiang in 2009 are forecasted and averaged to reduce the impact of contingent factors. Table 2 shows model parameter optimization results after the iteration calculation. Figure 2 shows the forecasted and actual load curves.

Table 2.

Forecast results of multiple samples by improved PSO.

Improved PSO
Date	27 March	28 February	31 March	20 May
C	0.1	150	2.6252	110.6786
Theta	0.7102	0.1728	8.5543	0.6184
Error (%)	1.15	1.22	1.69	1.22
Mean error (%) 1.3200

PSO: particle swarm optimization.

Table 2 and Figure 3 show that the improved PSO has better searching ability and precision. Within four forecast days, the MAEs of the forecast model have total average of 2.06% and the maximum error of less than 3.02%. Therefore, the algorithm is effective and feasible for short-term load forecasting. On 28 February, the error of forecasted value is large probably because of equipment maintenance and circuit breaker tripping. It is difficult to reflect the change only by the data.

Figure 3.

Forecasted and actual load curves by improved PSO (green: actual load; red: forecasted load).

In Figure 3, the curves marked by circle and triangle are actual and predicted loads. Comparative results show that the forecast errors of LS-SVM method are less than 4% in general working days or holidays. However, these forecast errors are still large. The prediction method has slightly larger forecast error in holidays than in working days, which basically accords with the reality. On 31 March and 20 May, the forecast values have large errors. This is probably because equipment maintenance and circuit breaker tripping result in large load fluctuations. It is difficult to reflect the change only by the data.

The precocious convergence judgment mechanism based on the population diversity information is guided by the particle location updating, reducing the randomness of the algorithm, making the improved new algorithm can jump out of the local most advantages, always keep the particle better dispersivity, and gradually search the better regions outside the current optimal region and reach the better area. Global optimization and global optimization ability are significantly enhanced without slowing down the convergence speed.

Theta is Gauss's normalization parameter that determines the width of the function around the center point. The kernel width coefficient reflects the correlation between the support vectors, which is related to the input space range of the learning samples. The larger the sample input space is, the greater the value is. The relationship between support vectors is relatively relaxed, the learning machine is relatively complex, and the generalization ability is not guaranteed. It is difficult to achieve enough accuracy in the regression model because of the large influence between the support vectors. The normal number of C to coordinate the two needs the testers' experience to determine which is difficult for testers and sometimes takes a long time.

Conclusions

As the main basis for development of power generation and transmission schemes, short-term load forecasting is an important daily work in dispatching operation department of power system. It has become one of the important contents of power system management modernization. In the work, the short-term load was forecasted by LS-SVM and improved PSO. SVM can better solve practical problems such as small sample, nonlinear, high dimension, and local minimum point based on profound theory. Consequently, the SVM model achieved the ideal effect in short-term load forecasting.