Sage Journals: Discover world-class research

Abstract

A novel hybrid particle swarm optimization variant is proposed, which combines particle swarm optimization with Gaussian mutation operation based on random strategy. It applies Gaussian mutation on the positions of some randomly selected particles to enhance the search accuracy and convergence speed of swarm. The proposed algorithm can retain the diversity of population and improve the ability of global search. A suite of benchmark test functions is employed to evaluate the performance of the proposed method. The results have been compared with three state-of-the-art particle swarm optimization variants. Experimental results show that the Gaussian mutation and the random select strategy help the proposed algorithm to achieve faster convergence rate and provide better solutions in most of the problems. Further, the new algorithm has been tested on the high-dimensional problems. The results show that the proposed algorithm is not sensitive to high-dimensional problems and can even give a better performance. Moreover, the sensitivity analysis of the parameters was carried out and the setting of the parameters was given.

Keywords

Particle swarm optimization stochastic mutation Gaussian distribution random selection strategy animal foraging behavior

Introduction

Particle swarm optimization (PSO) is a population-based stochastic optimization algorithm which is inspired by the social behaviors of animals like fish schooling and bird flocking.¹ Because of faster convergence rates and easy to implement, PSO has been used in many different industrial areas, such as power systems,² image process,³ optimizing machining process parameters,⁴ PID control,⁵ machine learning and pattern recognition,⁶ economic dispatch,⁷ and thermodynamics.^8,9 However, the premature convergence always is the main shortcoming of PSO. It can converge quickly in the early stage of the evolution process but easy to trap in the local optima in the later stage.

In order to deal with the premature convergence problem of PSO, different methods have been combined with PSO. Qu et al.¹⁰ integrated a novel local search technique with some existing PSO-based multimodal optimization algorithms to enhance their local search ability. Xi et al.¹¹ proposed a local search strategy to improve the performance of quantum-behaved PSO, the range of local search linearly decreases with time growing of the local search strategy. Liu et al.¹² proposed a hybrid PSO algorithm named HLPSO-GD, in which the genetic disturbance is used to cross the corresponding particle in the external archive, and generate new individuals which will improve the swarm ability to escape from the local optima. Ling et al.¹³ presented a hybrid PSO incorporated with wavelet-based mutation operation, which enhances the exploration ability of PSO. Krohling¹⁴ introduced a novel PSO algorithm combined with the Gaussian probability distribution, which named Gaussian Swarm, and the number of particles is the only parameter to be specified by the user. In low-dimensional search space, Gaussian swarm has shown good performance for solving multimodal optimization problems. For high-dimensional problems, Krohling¹⁵ incorporated the Gaussian and Cauchy jumps into the Gaussian swarm to improve the performance of it, the jump strategy is implemented as a mutation operator based on the Gaussian and Cauchy probability distribution. Then, Krohling and Mendel¹⁶ integrated the same jump strategy into Bare Bones PSO. Li et al.¹⁷ proposed a fast PSO, in which the Cauchy mutation and natural selection strategy are introduced into PSO. Wang et al.¹⁸ applied the Cauchy mutation as well, they disturbed the best particle of swarm with the Cauchy mutation, and the idea is that the mutated best particle could lead all the rest of particles to the better positions. Furthermore, Wang et al.¹⁹ introduced an adaptive Cauchy mutation into PSO. These works considerably improve the performance of PSO by using mutation. However, for complex problems, the efficiency and performance of the existing variant PSO algorithms need to be improved further.

This paper is organized as follows. The next section introduces to the standard PSO (SPSO). “PSO with Gaussian mutation” section presents the details of the hybrid PSO with Gaussian mutation. Experimental results and analysis are discussed in “Simulation results and analysis” section. Fifteen standard benchmark functions are given to evaluate the performance of the proposed method. Furthermore, the sensitivity analysis of the parameters is discussed in this section. A conclusion will be drawn in the final section.

An overview of SPSO

Assume $f (x) : x \to R, x \in R D$ is the function to be optimized. Each particle is considered as a potential solution in the D-dimensional solution space of the function. The position vector of particle i is represented as x_i= (x_i₁, x_i₂, … , x_iD), and its quality is measured by a predefined fitness function which is related to the function to be optimized. Similarly, v_i= (v_i₁, v_i₂, … , v_iD) is the velocity vector of particle i, which represents the displacement of particle. The population are initialized stochastically in the D-dimensional search space. In the process of evolution, the SPSO updates its velocity and position according to the following formulas²⁰

$v_{i}^{t + 1} = w v_{i}^{t} + r_{1} c_{1} (p_{i}^{t} - x_{i}^{t}) + r_{2} c_{2} (p_{g}^{t} - x_{i}^{t})$

(1)

$x_{i}^{t + 1} = x_{i}^{t} + v_{i}^{t + 1}$
(2)
where w is inertia weight which provides a balance between local exploitation and global exploration; c₁ and c₂ are two positive constants and called cognition coefficient and social coefficient, respectively; r₁ and r₂ are two random numbers following uniform distribution in the range [0,1] at t moment. $p_{i}^{t}$ represents the best previous position of the ith particle at t moment. $p_{g}^{t}$ denotes the best position found so far among all particles at t moment. Equation (1) is used to update the particle’s velocity and then the particle flies toward a new position according to equation (2).

Because of its fast convergence rate and ease of implementation, SPSO has been successfully applied to many real-world applications. However, the premature convergence problem has been identified as a major problem in PSO. PSO can find the global optimum quickly on simple and unimodal functions, but it is easy to fall into local optima with specific initial positions and velocities on multimodal functions.

PSO with Gaussian mutation

Stochastic mutation is a simple and efficient learning strategy, which has been studied and applied by many researchers. Wang et al.²¹ considered that there are three principles which should be taken into account to develop an efficient mutation operator, namely: (1) when to perform mutation?; (2) what to mutate?; (3) how to perform mutation?

Hybrid PSO with Gaussian mutation

By using the mutation idea, the proposed hybrid PSO was integrated with a randomized mutation. In this paper, there are m randomly chosen particles mutated in every iteration. The mutation, which is presented in equation (3), is performed on the current position x_i of the mutated particles
$x_{i}^{t + 1} = x_{i}^{t} + Rand \times Step$
(3)

Here, Rand is a random number that can be sampled from any probability distribution and Step presents the step size of mutation.

Inspired by the animal foraging behavior, mutations can be performed with a small step at most of the time. This can take advantage of the current search information of particles to improve its search speed without affecting the search accuracy. Then, the particles occasionally mutate by a long step, which can make the particle jump out of the local optimum. Thus, for the Rand, we use the Gaussian distribution as suggested in the literature,^15,22,23 which can produce a small number with large probability and a large number with small probability. The Step is set to X_max for the mutation can cover the search space with high probability. In general, the mutation of the proposed algorithm is shown in equation (4)
$x_{i}^{t + 1} = x_{i}^{t} + g_{i}^{t} \times X_{max}$
(4)
where $g_{i}^{t}$ ∼ N (μ, σ²), μ and σ² are the mean and variance of Gaussian distribution, respectively. μ is generally set to 0 and σ determines the probability of the different sampling values.

The proposed hybrid algorithm, which combined the PSO with Gaussian mutation in random select strategy, is called as rGMPSO. The process of the rGMPSO is summarized as follows. First, the parameter m, which denotes the number of mutated particles in every iteration, must be defined. Second, m particles are randomly chosen from the population. Finally, the selected m particles update its positions by equation (4), and the others update by equation (2).

The pseudo code of rGMPSO is shown as follows.
Algorithm 1: Pseudo code of rGMPS

1 t = 1;

2 Initialize X, V;

3 Evaluate swarm;

4 Update $p_{i}^{t}, p_{g}^{t}$ ;

5 m = 2; // m: the number of mutated particles;

6 while stop criterion is not satisfied do

7 t = t + 1;

8 Update V with Eq.(1);

9 Select mutated particles randomly;

10 for Every particle in the swarm do

11 if the particle is selected to mutate then

12 Mutate X with Eq.(4);

13 else

14 Update X with Eq.(2);

15 end

16 end

17 Evaluate swarm;

18 Update $p_{i}^{t}, p_{g}^{t}$ ;

19 end

20 Output the results;

Convergence analysis

Solis and Wets²⁴ gave the hypothesis and theorem proving of the convergence of stochastic search algorithm. The main assumptions and related theorems are as follows:

HYPOTHESIS 1 f (D(x, ξ)) ≤ f (x), and if ξ ∈ S,f (D(x, ξ)) ≤ f (ξ).

HYPOTHESIS 2 For any subset A of S with v(A) > 0, it has that
$Π_{k = 0}^{\infty} [1 - μ_{k} (A)] = 0$
where D is a map to generate the solution of the problem, f is the objective function, S ⊂ Rⁿ presents the solution space of problem, ξ is a random vector generate from distribution μ_k, v(A) denotes n-dimensional volume or Lebesgue measure of the set A, μ_k(A) is given by equation (5), it can be viewed as conditional probability measures
$μ_{k} (A) = P (x^{k} \in A | x^{0}, x^{1}, \dots, x^{(k_{- 1})})$
(5)

The hypothesis 1 request the series of objective function value is monotone decreasing sequence, i.e. ${f (x)^{k}}_{k = 0} \infty$ nonincreasing. And the hypothesis 2 request that sampling strategy given by μ_k cannot consistently ignore a positive volume subset of S, i.e. μ_k(A) = 1 when k → ∞, or supp(µ_k) ⊃ S.
THEOREM 1 Suppose f is a measurable function, S is a measurable subset of Rn and (H1) and (H2) are satisfied. Let ${x^{k}}_{k = 0} \infty$ be a sequence generated by the algorithm. Then
$lim_{k \to\infty} P (x^{k} \in R_{E, M}) = 1$
where P (x^k ∈ R_E,M) is the probability that at step k, the point x^k generated by the algorithm is in R_E,M.

Theorem 1 proves that an algorithm satisfies the two hypotheses and can converge in probability 1.

The rGMPSO algorithm is a random search algorithm, which combined the swarm intelligent with mutation. The series of solutions of rGMPSO is ${p_{g}^{t}}_{t = 0} \infty$ , where t is evolution iteration, $p_{g}^{t}$ is the best objective function so far at t iteration. From the principle of PSO, the series of objective function value of rGMPSO, i.e. ${p_{g}^{t}}_{t = 0} \infty$ , is nonincreasing, so it satisfied the hypothesis 1.

Because of the Gaussian mutation in every iteration, rGMPSO can be regarded as a random search algorithm. And the support set of Gaussian is R, the search space generated by equation (3) is a hypersphere with the center $x_{i}^{t}$ and radius r = ∞. Then, rGMPSO restrict the search space with equation (6)
$x_{id}^{t} = {UB x_{id}^{t} > UB LB x_{id}^{t} < LB x_{id}^{t} o . w . \forall x_{i} \in L$
(6)

Thus, the search space of rGMPSO, which denotes as L, satisfies equation (7)
$L = S$
(7)

In other words, the search space of rGMPSO covers the solution space of the problem, i.e. rGMPSO satisfies the hypothesis 2.

According to Theorem 1, rGMPSO converges to global optima in probability 1.

Simulation results and analysis

Benchmark functions

Fifteen benchmark functions are used to evaluate the performance of rGMPSO. These functions are divided into three groups. The first group contains functions f₁ to f₅ with fixed numbers of dimensions, which are chosen from Ali et al.²⁵

The second group is unimodal function including functions f₆ to f₁₀. The third group functions f₁₁ to f₁₅ are multimodal functions. The last two group functions are taken from the literature.^13,21,26 In general, multimodal functions, i.e. the third group functions, are often regarded as the most difficult in function optimization. Table 1 gives a brief description of the expression, search space, and minimum for all 15 functions.
Table 1.
Benchmark functions.

No. Expression Search space Minimum

f ₁ $0.25 x_{1}^{4} - 0.5 x_{1}^{2} + 0.1 x_{1} + 0.5 x_{2}^{2}$ [−10,10]² −0.3523

f2 $(| x_{1} | - 5)^{2} + (| x_{2} | - 5) 2$ [−10,10]² 0

f3 $x_{1}^{2} + 2 x_{2}^{2} - 0.3 cos (3 π x_{1}) cos (4 π x_{2}) + 0.3$ [−50,50]² 0

f4 $4 x_{1}^{2} - 2.1 x_{1}^{4} + \frac{1}{3} x_{1}^{6} + x_{1} x_{2} - 4 x_{2}^{2} + 4 x_{2}^{4}$ [−5,5]² −1.0316

f5 $- 0.1 \sum_{i = 1}^{2} cos (5 π x_{i}) + \sum_{i = 1}^{2} x_{i}^{2}$ [−1,1]² −0.2

f ₆ $\sum_{i = 1}^{D} x_{i}^{2}$ [−100,100]^D 0

f ₇ $10^{6} x_{1}^{2} + \sum_{i = 2}^{D} x_{i}^{2}$ [−100,100]^D 0

f ₈ $\sum_{i = 1}^{D} [\sum_{j = 1}^{i} x_{j}] 2$ [−100,100]^D 0

f ₉ $\sum_{i = 1}^{D - 1} [100 (x_{i}^{2} - x_{i + 1})^{2} + (x_{i} - 1)^{2}]$ [−50,50]^D 0

f ₁₀ $\sum_{i = 1}^{D} {ix}_{i}^{4} + random [0, 1)$ [−1.28,1.28]^D 0

f ₁₁ $\frac{1}{4000} \sum_{i = 1}^{D} x_{i}^{2} - Π_{i = 1}^{D} cos (\frac{x_{i}}{\sqrt{i}}) + 1$ [−600,600]^D 0

f ₁₂ $- 20 exp (- 0.02 \sqrt{\frac{1}{D} \sum_{i = 1}^{D} x_{i}^{2}}) - exp (\frac{1}{D} \sum_{i = 1}^{D} cos (2 π x_{i})) + 20 + e$ [−30,30]^D 0

f ₁₃ $\sum_{i = 1}^{D} [x_{i}^{2} - 10 cos (2 π x_{i}) + 10]$ [−5.12,5.12]^D 0

f ₁₄ $\sum_{i = 1}^{D - 1} (x_{i}^{2} + x_{i + 1}^{2}) 0.25$ [−100,100]^D 0

$\times [sin (50 (x_{i}^{2} + x_{i + 1}^{2})^{0.1}) + 1.0]$

f ₁₅ $- \sum_{i = 1}^{D} [x_{i} sin (\sqrt{| x_{i} |})]$ [−500,500]^D −418.9829D

Experimental settings

To examine the performance of rGMPSO, the following experimental settings are used. The functions f₁ to f₅ were run with two dimensions, and the functions f₆ to f₁₅ were run with 30 dimensions. For SPSO, the acceleration constants c₁ and c₂ were set to 2, the inertia weight w was linearly decreased from 0.9 to 0.4 as suggested by Shi and Eberhart.²⁷ For rGMPSO, c₁, c₂, and w were set to the same as SPSO, the number of mutated particles, i.e. m, was set to 2, and the $g_{i}^{t},$ is sample from standard normal distribution, i.e. μ = 0, σ = 1. For both algorithms, the population size was set to 20 and the maximum number of iterations was set to 100 × D. All experiments were run 50 times.

Numerical results

First, the results for the functions f₁ to f₅ are presented. Table 2 presents the mean best fitness and mean function evaluations (FEs) of rGMPSO and SPSO. From Table 2, one can observe that both rGMPSO and SPSO converge to the optimal solution for functions f₁, f₄, and f₅. In case of function f₂, SPSO obtains better mean best fitness, whereas in the case of function f₃, rGMPSO converges to the optimal solutions. (In python, a number is considered to be 0 while it is less than 1E-306). It may also be noted that rGMPSO obtains better mean FEs for all the five functions. Based on these observations, it may be inferred that SPSO provides good performance on the low-dimensional optimized problems. Compared with PSO, rGMPSO has faster convergence speed with the same search accuracy.
Table 2.
Mean results and standard deviations (values in parentheses) over 50 independent trials for functions f₁ to f₅.

Func. Best fitness
FEs

SPSO rGMPSO SPSO rGMPSO

f ₁ −0.3524 −0.3524 3834 3637

(3.02E-16) (3.38E-06) (138) (503)

f ₂ 5.04E-18 6.66E-06 3975 3752

(2.35E-17) (2.68E-05) (39) (378)

f ₃ 3.44E-17 0 569 89

(1.67E-16) (0) (1138) (34)

f ₄ −1.0316 −1.0316 3815 3761

(1.05E-15) (9.15E-06) (120) (316)

f ₅ −0.2 −0.2 573 93

(8.41E-17) (8.41E-17) (1041) (37)

FE: function evaluation; rGMPSO: PSO with Gaussian mutation in random select strategy; SPSO: standard PSO.

Next, the results of the five unimodal benchmark functions are given in Table 3. Table 3 shows the experimental results in terms of the mean best fitness and mean FEs for functions f₆ to f₁₀ in the 30-dimensional case. Functions f₆ to f₈, which are probably the most widely used benchmark functions, are always used to measure the convergence rate of algorithms. From Table 3, rGMPSO converges quickly to the optimal solutions of the three functions for its mean FEs are less than 700 which is 40–130 times faster than SPSO. That it is to say, rGMPSO has a great improvement over SPSO in terms of convergence rate. Function f₉ is a generalized Rosenbrock’s function, which is strongly nonseparable and the optimal solution is located in a very narrow ridge. It is commonly used to test the comprehensive property of optimized algorithms. Nevertheless, for function f₉, rGMPSO obtains better solution than SPSO, and the accuracy of the solution is within the acceptable error. From Figure 1(a), one can note that rGMPSO can continuously close to the optima in evolutions, and SPSO stagnate at about one-third of evolution progress.
Table 3.
Mean results and standard deviations (values in parentheses) over 50 independent trials for functions f₆ to f₁₀.

Func. Best fitness
FEs

SPSO rGMPSO SPSO rGMPSO

f ₆ 6.29 0 10,156 241

(12.49) (0) (17,013) (125)

f ₇ 9362.43 0 48,886 360

(10654.55) (0) (16,792) (173)

f ₈ 11678.37 0 58,006 674

(9672.04) (0) (4817) (463)

f ₉ 294.19 1.75E-05 48,504 53,957

(1417.79) (1.24E-04) (8121) (6848)

f ₁₀ 1.94 1.60E-04 49,641 34,899

(5.12) (1.26E-04) (8943) (16,976)

FE: function evaluation; rGMPSO: PSO with Gaussian mutation in random select strategy; SPSO: standard PSO.

Figure 1.
Convergence curves of functions (a) f₉ and (b) f₁₅.

Function f₁₀ is a quadratic function with random perturbations which increases the difficulty of searching. In case of function f₁₀, rGMPSO obtains the solutions better than SPSOs, which are quite close to the optimal solution for its small mean and standard deviation of best fitness. In conclusion, for unimodal benchmark functions, rGMPSO has much better performance than SPSO whether the accuracy of the solution or the convergence rate.

Next, the results of the five multimodal benchmark functions, which is considered difficult to optimize, are presented in Table 4. Encouragingly, the rGMPSO converges to global optima on all these five functions and has significant improvement over SPSO. For function f₁₂, because of the round-off error in computer, the mean best fitness of rGMPSO is not exactly equal to optima. Specially, the f₁₅ is the Schwefel’s problem 2.26, which has several deep local optima far from the global optimum, and the optimized algorithms are easy to be trapped. Figure 1(b) indicates that rGMPSO convergences fast than SPSO on f₁₅ functions. In short, compared with SPSO, the rGMPSO has a great improvement in both search accuracy and convergence rate, and it has perfect performance on both unimodal and multimodal benchmark functions.
Table 4.
Mean results and standard deviations (values in parentheses) over 50 independent trials for functions f₁₁ to f₁₅.

Func. Best fitness
FEs

SPSO rGMPSO SPSO rGMPSO

f ₁₁ 27.29 0 18,980 238

(41.77) (0) (22,709) (94)

f ₁₂ 7.80 4.44E-16 16,814 264

(7.43) (0) (19,474) (118)

f ₁₃ 70.87 0 42,401 271

(35.30) (0) (20,115) (114)

f ₁₄ 12.94 0 57,085 335

(14.09) (0) (11,099) (157)

f ₁₅ −8169.27 −12569.5 56,277 58,606

(1182.81) (5.08E-04) (8174) (1922)

FE: function evaluation; rGMPSO: PSO with Gaussian mutation in random select strategy; SPSO: standard PSO.

Table 5 summarized the success ratio (SR) of SPSO and rGMPSO on these benchmark functions. The SR is defined as equation (8)
$SR = \frac{n_{s}}{n_{t}}$
(8)
where n_s and n_t denote the number of runs that converge to the global optima and the total number of runs, respectively. From Table 5, it can conclude that the stability of SPSO is a little better than rGMPSO on the low-dimensional functions. For the high-dimensional functions, such as functions f₆ to f₁₅, the SR of SPSO decreases sharply and much worse than those of rGMPSO on these complicated functions.
Table 5.
Success ratio (SR) of the 15 benchmark functions (%).

Func. f ₁ f ₂ f ₃ f ₄ f ₅

SPSO 100 0 92 100 100

rGMPSO 96 0 100 90 100

Func. f ₆ f ₇ f ₈ f ₉ f ₁₀

SPSO 68 6 0 0 0

rGMPSO 100 100 100 0 0

Func. f ₁₁ f ₁₂ f ₁₃ f ₁₄ f ₁₅

SPSO 52 34 6 6 0

rGMPSO 100 100 100 100 100

rGMPSO: PSO with Gaussian mutation in random select strategy; SPSO: standard PSO.

In order to further demonstrate the performance of the rGMPSO, it has been compared with the HPSOWM,¹³ ATM-PSO,²¹ and CPSO.²⁸ The results are summarized in Table 6. The numerical results of ATM-PSO, HPSOWM, and CPSO are taken from the related articles. From Table 6, it can draw the conclusion that rGMPSO surpasses the others on the eight functions. These demonstrate the effectiveness of rGMPSO again.
Table 6.
Comparison between rGMPSO and the other PSO variants.

Func. rGMPSO ATM-PSO HPSOWM CPSO

f ₆ 0 8.90E-104 1.50E-08 9.48E-71

(0) (3.18E-103) (1.00E-08) (5.13E-70)

f ₈ 0 3.46E-10 – 8.10E-52

(0) (5.07E-10) – (1.24E-51)

f ₉ 1.75E-05 (1.24E-04) 15.13 (8.71E-01) 1.003 (2.16E-01) 1.01 (6.65E-01)

f ₁₀ 1.60E-04 (1.26E-04) 9.77E-03 (3.16E-03) 5.13E-03 (1.69E-03) 8.64E-03 (1.04E-02)

f ₁₁ 0 2.22E-02 1.00E-09 1.52E-02

(0) (2.03E-02) (1.00E-09) (2.22E-02)

f ₁₂ 4.44E-16 (0) 2.59E-14 (6.12E-15) 1.06E-05 (3.12E-06) 5.03E-15 (2.90E-15)

f ₁₃ 0 0 10.2854 5.00

(0) (0) (3.3522) (2.49E+01)

f ₁₅ −4158.99 – −3928.83 –

(D = 10) (953.3) – (176.65) –

f ₁₅ −12,569.5 – – −11,059.5

(D = 30) (5.08E-04) – – (1.42E + 03)

ATM-PSO: PSO with adaptive tabu and mutation; CPSO: Cellular PSO; HPSOWM: Hybrid PSO with wavelet mutation; rGMPSO: PSO with Gaussian mutation in random select strategy; SPSO: standard PSO.

To illustrate the performance of rGMPSO on the high-dimensional and complicated optimized problems, we conducted additional experiments on the 50-D and 100-D benchmark functions, where the max iterations of these experiments are set to 100 × D. The experimental results are tabulated in Table 7. As shown in Table 7, the search accuracy of rGMPSO on 50-D and 100-D functions is the same or better than 30-D; although the FEs increase with the dimension expansion, it has an approximate linear relationship with dimension. In all, the rGMPSO has the outstanding performance on high-dimensional benchmark functions.
Table 7.
Mean results for rGMPSO on f₆ to f₁₅ with 30-D, 50-D, 100-D.

Func. Best fitness
FEs

30-D 50-D 100-D 30-D 50-D 100-D

f ₆ 0 0 0 241 266 308

f ₇ 0 0 0 360 366 416

f ₈ 0 0 0 674 962 2192

f ₉ 1.75E-05 2.62E-06 7.55E-09 5.40E + 04 9.00E + 04 1.82E + 05

f ₁₀ 1.60E-04 8.74E-05 4.98E-05 3.49E + 04 6.75E + 04 1.29E + 05

f ₁₁ 0 0 0 238 282 299

f ₁₂ 4.44E-16 4.44E-16 4.44E-16 264 307 350

f ₁₃ 0 0 0 271 384 370

f ₁₄ 0 0 0 335 351 420

f ₁₅ −12,569.5 −20,944.4 −41,895.92 5.86E + 04 9.62E + 04 1.92E + 05

FE: function evaluation.

Sensitivity analysis

In rGMPSO, two parameters m and σ are introduced to adapt the mutation. m is the number of mutated particles, which controls the frequency of mutation. σ is the standard deviation of Gaussian distribution, which influence the step size of the mutated particles. The values of m are set to 2 and σ is set to 1 in the previous experiments, and the experimental results are stable of all benchmark functions.

Table 8 shows the experimental results of m ∈ {1, 2, 4, 10}. From Table 8, one can see that the parameter m has little effect on search accuracy of rGMPSO. For functions f₆ to f₁₄, rGMPSO can obtain the similar mean best fitness when m ∈ {1, 2, 4, 10}. When m = 1, the mean best fitness for function f₁₅ is not close to optima, this means that rGMPSO is easy to trap into local optima. From Table 8 it can also be seen that rGMPSO has faster convergence speed with larger m for functions f₆ to f₁₅. In general, rGMPSO has similar search accuracy with different m and has stronger ability of exploration and faster convergence speed with a larger m. Thus, from the experimental results, the m could be set to 10.
Table 8.
Mean results for rGMPSO on f₆ to f₁₅ with m ∈ {1, 2, 4, 10}.

Func. Best fitness
FEs

1 2 4 10 1 2 4 10

f ₆ 0 0 0 0 366 241 163 110

f ₇ 0 0 0 0 577 360 205 128

f ₈ 0 0 0 0 2264 674 439 142

f ₉ 8.79E-07 1.75E-05 1.34E-05 1.68E-04 56,213 53,957 46,776 34,695

f ₁₀ 1.37E-04 1.60E-04 1.86E-04 1.54E-04 42,018 34,899 33,175 33,788

f ₁₁ 0 0 0 0 392 238 164 114

f ₁₂ 4.44E-16 4.44E-16 4.44E-16 4.44E-16 543 264 194 134

f ₁₃ 0 0 0 0 480 271 178 126

f ₁₄ 0 0 0 0 585 335 199 111

f ₁₅ −12,474.7 −12,569.5 −12,569.5 −12,569.5 58,824 58,606 49,649 30,340

FE: function evaluation; rGMPSO: PSO with Gaussian mutation in random select strategy.

To show the influence of the parameter σ, an extra experiment was conducted. Table 9 summaries the results of σ ∈ {0.5, 1, 2, 4}. For functions f₆ to f₈ and f₁₁ to f₁₃, rGMPSO has the same mean best fitness when σ ∈ {0.5, 1, 2, 4}. For functions f₉, f₁₀, and f₁₅, the mean best fitness of rGMPSO improved when σ increases, excepting while σ = 2 for functions f₁₀ and f₁₅. For all benchmark functions, the mean FEs of rGMPSO decrease when σ increases. When σ = 0.5, for functions f₁₄ and f₁₅, the mean best fitness of rGMPSO is not equal to global optimum. It indicates that the rGMPSO has the probability to trap into premature when σ is too small. From the results of Table 9, we may safely draw the conclusion that a bigger mutated step can help the algorithm jump out of the local optima quickly. For rGMPSO, the parameter σ could be set to 4.
Table 9.
Mean results for rGMPSO on f₆ to f₁₅ with σ ∈ {0.5, 1, 2, 4}.

Func. Best fitness
FEs

0.5 1 2 4 0.5 1 2 4

f ₆ 0 0 0 0 910 241 130 115

f ₇ 0 0 0 0 1684 360 163 111

f ₈ 0 0 0 0 7940 674 300 216

f ₉ 1.29E-05 1.75E-05 4.50E-08 7.75E-09 55,121 53,957 53,382 53,785

f ₁₀ 2.19E-04 1.60E-04 9.71E-05 9.95E-05 43,426 34,899 35,962 34,091

f ₁₁ 0 0 0 0 924 238 132 106

f ₁₂ 4.44E-16 4.44E-16 4.44E-16 4.44E-16 1082 264 149 105

f ₁₃ 0 0 0 0 1869 271 159 112

f ₁₄ 3.06E-03 0 0 0 3538 335 146 118

f ₁₅ −11,922.6 −12,569.5 −12,567.1 −12,569.5 59,281 58,606 56,875 57,641

FE: function evaluation; rGMPSO: PSO with Gaussian mutation in random select strategy.

Conclusions

In this paper, we have proposed a hybrid PSO combined with the Gaussian mutation (the rGMPSO). Benefit from the mutation, the rGMPSO can balance the exploration and exploitation more effectively during the evolution and therefore it can achieve excellent performance on unimodal and multimodal problems. Simulation results have shown that the proposed rGMPSO is a better algorithm to solve complex optimization problems. Due to the properties of the Gaussian distribution, the solution stability and quality of the rGMPSO are improved. With the experiments on a suite of benchmark test functions, the rGMPSO gives better results than the methods of the ATM-PSO, the HPSOWM, and the CPSO. Furthermore, one of the main advantages of rGMPSO is easy to understand and to implement. In future, the extensions work can be the focus on the different distribution mutation and the selection of mutation particles.

No.	Expression	Search space	Minimum
f ₁	$0.25 x_{1}^{4} - 0.5 x_{1}^{2} + 0.1 x_{1} + 0.5 x_{2}^{2}$	[−10,10]²	−0.3523
f2	$(\| x_{1} \| - 5)^{2} + (\| x_{2} \| - 5) 2$	[−10,10]²	0
f3	$x_{1}^{2} + 2 x_{2}^{2} - 0.3 cos (3 π x_{1}) cos (4 π x_{2}) + 0.3$	[−50,50]²	0
f4	$4 x_{1}^{2} - 2.1 x_{1}^{4} + \frac{1}{3} x_{1}^{6} + x_{1} x_{2} - 4 x_{2}^{2} + 4 x_{2}^{4}$	[−5,5]²	−1.0316
f5	$- 0.1 \sum_{i = 1}^{2} cos (5 π x_{i}) + \sum_{i = 1}^{2} x_{i}^{2}$	[−1,1]²	−0.2
f ₆	$\sum_{i = 1}^{D} x_{i}^{2}$	[−100,100]^D	0
f ₇	$10^{6} x_{1}^{2} + \sum_{i = 2}^{D} x_{i}^{2}$	[−100,100]^D	0
f ₈	$\sum_{i = 1}^{D} [\sum_{j = 1}^{i} x_{j}] 2$	[−100,100]^D	0
f ₉	$\sum_{i = 1}^{D - 1} [100 (x_{i}^{2} - x_{i + 1})^{2} + (x_{i} - 1)^{2}]$	[−50,50]^D	0
f ₁₀	$\sum_{i = 1}^{D} {ix}_{i}^{4} + random [0, 1)$	[−1.28,1.28]^D	0
f ₁₁	$\frac{1}{4000} \sum_{i = 1}^{D} x_{i}^{2} - Π_{i = 1}^{D} cos (\frac{x_{i}}{\sqrt{i}}) + 1$	[−600,600]^D	0
f ₁₂	$- 20 exp (- 0.02 \sqrt{\frac{1}{D} \sum_{i = 1}^{D} x_{i}^{2}}) - exp (\frac{1}{D} \sum_{i = 1}^{D} cos (2 π x_{i})) + 20 + e$	[−30,30]^D	0
f ₁₃	$\sum_{i = 1}^{D} [x_{i}^{2} - 10 cos (2 π x_{i}) + 10]$	[−5.12,5.12]^D	0
f ₁₄	$\sum_{i = 1}^{D - 1} (x_{i}^{2} + x_{i + 1}^{2}) 0.25$	[−100,100]^D	0
	$\times [sin (50 (x_{i}^{2} + x_{i + 1}^{2})^{0.1}) + 1.0]$
f ₁₅	$- \sum_{i = 1}^{D} [x_{i} sin (\sqrt{\| x_{i} \|})]$	[−500,500]^D	−418.9829D

Func.	Best fitness	FEs
f ₁	−0.3524	−0.3524	3834	3637
	(3.02E-16)	(3.38E-06)	(138)	(503)
f ₂	5.04E-18	6.66E-06	3975	3752
	(2.35E-17)	(2.68E-05)	(39)	(378)
f ₃	3.44E-17	0	569	89
	(1.67E-16)	(0)	(1138)	(34)
f ₄	−1.0316	−1.0316	3815	3761
	(1.05E-15)	(9.15E-06)	(120)	(316)
f ₅	−0.2	−0.2	573	93
	(8.41E-17)	(8.41E-17)	(1041)	(37)

Func.	Best fitness	FEs
f ₆	6.29	0	10,156	241
	(12.49)	(0)	(17,013)	(125)
f ₇	9362.43	0	48,886	360
	(10654.55)	(0)	(16,792)	(173)
f ₈	11678.37	0	58,006	674
	(9672.04)	(0)	(4817)	(463)
f ₉	294.19	1.75E-05	48,504	53,957
	(1417.79)	(1.24E-04)	(8121)	(6848)
f ₁₀	1.94	1.60E-04	49,641	34,899
	(5.12)	(1.26E-04)	(8943)	(16,976)

Func.	Best fitness	FEs
f ₁₁	27.29	0	18,980	238
	(41.77)	(0)	(22,709)	(94)
f ₁₂	7.80	4.44E-16	16,814	264
	(7.43)	(0)	(19,474)	(118)
f ₁₃	70.87	0	42,401	271
	(35.30)	(0)	(20,115)	(114)
f ₁₄	12.94	0	57,085	335
	(14.09)	(0)	(11,099)	(157)
f ₁₅	−8169.27	−12569.5	56,277	58,606
	(1182.81)	(5.08E-04)	(8174)	(1922)

Func.	f ₁	f ₂	f ₃	f ₄	f ₅
SPSO	100	0	92	100	100
rGMPSO	96	0	100	90	100
Func.	f ₆	f ₇	f ₈	f ₉	f ₁₀
SPSO	68	6	0	0	0
rGMPSO	100	100	100	0	0
Func.	f ₁₁	f ₁₂	f ₁₃	f ₁₄	f ₁₅
SPSO	52	34	6	6	0
rGMPSO	100	100	100	100	100

Func.	rGMPSO	ATM-PSO	HPSOWM	CPSO
f ₆	0	8.90E-104	1.50E-08	9.48E-71
	(0)	(3.18E-103)	(1.00E-08)	(5.13E-70)
f ₈	0	3.46E-10	–	8.10E-52
	(0)	(5.07E-10)	–	(1.24E-51)
f ₉	1.75E-05 (1.24E-04)	15.13 (8.71E-01)	1.003 (2.16E-01)	1.01 (6.65E-01)
f ₁₀	1.60E-04 (1.26E-04)	9.77E-03 (3.16E-03)	5.13E-03 (1.69E-03)	8.64E-03 (1.04E-02)
f ₁₁	0	2.22E-02	1.00E-09	1.52E-02
	(0)	(2.03E-02)	(1.00E-09)	(2.22E-02)
f ₁₂	4.44E-16 (0)	2.59E-14 (6.12E-15)	1.06E-05 (3.12E-06)	5.03E-15 (2.90E-15)
f ₁₃	0	0	10.2854	5.00
	(0)	(0)	(3.3522)	(2.49E+01)
f ₁₅	−4158.99	–	−3928.83	–
(D = 10)	(953.3)	–	(176.65)	–
f ₁₅	−12,569.5	–	–	−11,059.5
(D = 30)	(5.08E-04)	–	–	(1.42E + 03)

Func.	Best fitness	FEs
f ₆	0	0	0	241	266	308
f ₇	0	0	0	360	366	416
f ₈	0	0	0	674	962	2192
f ₉	1.75E-05	2.62E-06	7.55E-09	5.40E + 04	9.00E + 04	1.82E + 05
f ₁₀	1.60E-04	8.74E-05	4.98E-05	3.49E + 04	6.75E + 04	1.29E + 05
f ₁₁	0	0	0	238	282	299
f ₁₂	4.44E-16	4.44E-16	4.44E-16	264	307	350
f ₁₃	0	0	0	271	384	370
f ₁₄	0	0	0	335	351	420
f ₁₅	−12,569.5	−20,944.4	−41,895.92	5.86E + 04	9.62E + 04	1.92E + 05

Func.	Best fitness	FEs
f ₆	0	0	0	0	366	241	163	110
f ₇	0	0	0	0	577	360	205	128
f ₈	0	0	0	0	2264	674	439	142
f ₉	8.79E-07	1.75E-05	1.34E-05	1.68E-04	56,213	53,957	46,776	34,695
f ₁₀	1.37E-04	1.60E-04	1.86E-04	1.54E-04	42,018	34,899	33,175	33,788
f ₁₁	0	0	0	0	392	238	164	114
f ₁₂	4.44E-16	4.44E-16	4.44E-16	4.44E-16	543	264	194	134
f ₁₃	0	0	0	0	480	271	178	126
f ₁₄	0	0	0	0	585	335	199	111
f ₁₅	−12,474.7	−12,569.5	−12,569.5	−12,569.5	58,824	58,606	49,649	30,340

Func.	Best fitness	FEs
f ₆	0	0	0	0	910	241	130	115
f ₇	0	0	0	0	1684	360	163	111
f ₈	0	0	0	0	7940	674	300	216
f ₉	1.29E-05	1.75E-05	4.50E-08	7.75E-09	55,121	53,957	53,382	53,785
f ₁₀	2.19E-04	1.60E-04	9.71E-05	9.95E-05	43,426	34,899	35,962	34,091
f ₁₁	0	0	0	0	924	238	132	106
f ₁₂	4.44E-16	4.44E-16	4.44E-16	4.44E-16	1082	264	149	105
f ₁₃	0	0	0	0	1869	271	159	112
f ₁₄	3.06E-03	0	0	0	3538	335	146	118
f ₁₅	−11,922.6	−12,569.5	−12,567.1	−12,569.5	59,281	58,606	56,875	57,641

Footnotes

Acknowledgments

The authors of the paper express great acknowledgment for these supports.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is supported by the National Natural Science Foundation of China (Grant No. 51685168,61300104).

References

1.
Kennedy J and Eberhart R. Particle swarm optimization. In: 1995 IEEE international conference on neural networks: proceedings, Perth, Australia, 27 November–1 December 1995, Vol. 4, pp.1942–1948. Piscataway, NJ: IEEE.

2.
AlRashidi MR El-Hawary ME . A survey of particle swarm optimization applications in electric power systems. IEEE Trans Evol Comput 2009; 13: 913–918.

3.
Luo X Wu X Zhang ZC . Image fusion of the feature level based on quantum-behaved particle swarm optimization algorithm. J Algorithms Comput Technol 2013; 7: 101–112.

4.
Yusup N Mohd Zain A Mohd Hashim SZ . Overview of PSO for optimizing process parameters of machining. Procedia Eng 2012; 29: 914–923.

5.
Stimac G Braut S Zigulic R . Comparative analysis of PSO algorithms for PID controller tuning. Chin J Mech Eng 2014; 27: 928–936.

6.
Kiranyaz S, Ince T and Gabbouj M. Multidimensional particle swarm optimization for machine learning and pattern recognition. Adaptation, learning, and optimization. Berlin,German: Springer, 2013.

7.
Mahor A Prasad V Rangnekar S . Economic dispatch using particle swarm optimization: a review. Renew Sustain Energy Rev 2009; 13: 2134–2141.

8.
Tian N Xu W Sun J et al. Estimation of heat flux in inverse heat conduction problems using quantum-behaved particle swarm optimization. J Algorithms Comput Technol 2010; 4: 25–46.

9.
Tian N Zhu L Lai C-H . Estimation of heat transfer coefficient in inverse heat conduction problem using quantum-behaved particle swarm optimization with Tikhonov regularization. J Algorithms Comput Technol 2014; 8: 233–248.

10.
Qu BY Liang JJ Suganthan PN . Niching particle swarm optimization with local search for multi-modal optimization. Inform Sci 2012; 197: 131–143.

11.
Xi M Wu X Sheng X et al. Improved quantum-behaved particle swarm optimization with local search strategy. J Algorithms Comput Technol 2016; 10: 1–10.

12.
Liu Y Niu B Luo Y . Hybrid learning particle swarm optimizer with genetic disturbance. Neurocomputing 2015; 151: 1237–1247.

13.
Ling S-H Iu HHC Chan KY et al. Hybrid particle swarm optimization with wavelet mutation and its industrial applications. IEEE Trans Syst Man Cybernet Part B Cybernet 2008; 38: 743–763.

14.
Krohling RA. Gaussian swarm: a novel particle swarm optimization algorithm. In: 2004 IEEE conference on cybernetics and intelligent systems, Singapore, 1–3 December 2004, Vol. 1, pp.372–376. Piscataway, NJ: IEEE.

15.
Krohling RA. Gaussian particle swarm with jumps. In: 2005 IEEE Congress on evolutionary computation, Edinburgh, UK, 2–5 Sepemebr 2005, Vol. 2, pp.1226–1231. Piscataway, NJ: IEEE.

16.
Krohling RA and Mendel E. Bare bones particle swarm optimization with Gaussian or Cauchy jumps. In: 2009 IEEE Congress on evolutionary computation, CEC’09, Trondheim, Norway, 18–21 May 2009, pp.3285–3291. Piscataway, NJ: IEEE.

17.
Li C Liu Y Zhou A et al. A fast particle swarm optimization algorithm with Cauchy mutation and natural selection strategy. In: Li Z (ed). Advances in computation and intelligence, volume 5821 of Theoretical computer science and general issues, Berlin, German: Springer, 2007, pp. 334–343.

18.
Wang H, Liu Y, Li C, et al. A hybrid particle swarm algorithm with Cauchy mutation. In: 2007 Swarm intelligence symposium, Honolulu, HI, USA, 1–5 April 2007, pp.356–360. Piscataway, NJ: IEEE.

19.
Wang H, Liu Y, Wu Z, et al. An improved particle swarm optimization with adaptive jumps. In: IEEE Congress on evolutionary computation, CEC 2008 (IEEE World Congress on computational intelligence), Hong Kong, China, 1–6 June 2008, pp.392–397. Piscataway, NJ: IEEE.

20.
Shi Y and Eberhart R. A modified particle swarm optimizer. In: 1998 IEEE international conference on evolutionary computation proceedings, 1998 (IEEE World Congress on computational intelligence), Anchorage, AK, USA, 4–9 May 1998, pp.69–73. Piscataway, NJ: IEEE.

21.
Wang Y-X Xiang Q-L Zhao Z-D . Particle swarm optimizer with adaptive tabu and mutation: A unified framework for efficient mutation operators. ACM Trans Autonom Adapt Syst 2010; 5: 1–27.

22.
Lee C-Y Yao X . Evolutionary programming using mutations based on the Levy probability distribution. IEEE Trans Evol Comput 2004; 8: 1–13.

23.
Yao X Liu Y Lin G . Evolutionary programming made faster. IEEE Trans Evol Comput 1999; 3: 82–102.

24.
Solis FJ Wets JB . Minimization by random search techniques. Math Oper Res 1981; 6: 19–30.

25.
Ali M Khompatraporn C Zabinsky ZB . A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. J Global Optim 2005; 31: 635–672.

26.
Leu M-S Yeh M-F . Grey particle swarm optimization. Appl Soft Comput 2012; 12: 2985–2996.

27.
Shi Y and Eberhart RC. Empirical study of particle swarm optimization. In: Proceedings of the 1999 Congress on evolutionary computation, CEC 99, Washington DC, USA, 6–9 July 1999, pp.32–49. Piscataway, NJ: IEEE.

28.
Yang S Liu H Liang G et al. Cellular particle swarm optimization. Inform Sci 2011; 181: 4460–4493.

Func.	Best fitness			FEs
Func.	30-D	50-D	100-D	30-D	50-D	100-D
f ₆	0	0	0	241	266	308
f ₇	0	0	0	360	366	416
f ₈	0	0	0	674	962	2192
f ₉	1.75E-05	2.62E-06	7.55E-09	5.40E + 04	9.00E + 04	1.82E + 05
f ₁₀	1.60E-04	8.74E-05	4.98E-05	3.49E + 04	6.75E + 04	1.29E + 05
f ₁₁	0	0	0	238	282	299
f ₁₂	4.44E-16	4.44E-16	4.44E-16	264	307	350
f ₁₃	0	0	0	271	384	370
f ₁₄	0	0	0	335	351	420
f ₁₅	−12,569.5	−20,944.4	−41,895.92	5.86E + 04	9.62E + 04	1.92E + 05

An effective hybrid particle swarm optimization with Gaussian mutation

Abstract

Keywords

Introduction

An overview of SPSO

PSO with Gaussian mutation

Hybrid PSO with Gaussian mutation

Algorithm 1: Pseudo code of rGMPS

Convergence analysis

Simulation results and analysis

Benchmark functions

Experimental settings

Numerical results

Sensitivity analysis

Conclusions

Footnotes

Acknowledgments

Declaration of conflicting interests

Funding

References