Abstract
Engineering optimization problems usually contain various constraints and mixed integer-discrete-continuous types of design variables. We propose an efficient particle swarm optimization (PSO) algorithm for such problems. First, we transform the constrained optimization problem into an unconstrained problem without introducing problem-dependent or user-defined parameters such as penalty factors or Lagrange multipliers (such parameters are usually required in general optimization algorithms). Then, we extend the above PSO method to handle integer, discrete, and continuous design variables in a simple manner with a high degree of precision. The proposed PSO scheme is fairly simple and therefore easy to implement. To demonstrate the effectiveness of our method, several mechanical design optimization problems are solved, and the numerical results are compared with results reported in the literature.
1. Introduction
Most engineering optimal design problems require optimization techniques that are capable of handling the problem of equality and inequality constraints. In recent years, the particle swarm optimization (PSO) algorithm [1], whose mechanism was inspired by the social and cooperative behavior of swarms such as a flock of birds or a group of people, has been applied to various optimization problems with some promising results. It is a population-based search algorithm where a randomly initialized particle of the swarm updates its state (position) in a search space based on the information of its own and other agents' historical behavior until an optimal state has been reached (see, e.g., [2–4] and the references therein). However, the original PSO is naturally a stochastic unconstrained optimization method, and thus it cannot be directly applied to constrained optimization problems which is also the case for most swarm-inspired metaheuristic search technologies. This has motivated many researchers to investigate techniques for handling constrained optimization problems, and some methods have been proposed for example, (i) the modified feasible-based mechanism [5, 6], (ii) the penalty approach [7, 8], (iii) the augmented Lagrange multiplier approach [9, 10], (iv) the violated design points redirection [11, 12], (v) the fly-back mechanism method [13], (vi) the problem reformulation into an unconstrained multi-objective formulation [14, 15], and (vii) a series of sequential quadratic programming problems [16]. The distinctive features of conventional constraint handling schemes can be found in Aziz et al. [17], Kim et al. [18], and Jansen and Perez [10]. It is worth mentioning that many of these methods could not ensure a realization of strict constraint feasibility which would limit its possible use to practical applications [10]. Further, some of them require the adoption of problem-dependent or user-defined parameters such as penalty factors or Lagrange multipliers.
In this line of researches, the author recently proposed a considerably simple and efficient way to handle the constraint conditions in PSO when solving engineering optimization problems [18]. The key feature of the proposed constrained PSO method is that the given constrained optimization problem is transformed into an unconstrained problem without introducing problem-dependent or user-defined parameters such as penalty factors or Lagrange multipliers, although such parameters are usually required in general optimization algorithms. In this approach, particles of the swarm evolve as follows (see Kim et al. [18] for details). At the first stage of evolution, all particles try to move into a feasible region where constraint satisfaction is guaranteed. Thereafter, the particles try to explore the feasible region to find an optimal solution that satisfies the given constraint conditions. This constraint handling PSO scheme has been shown to outperform conventional methods for various continuous engineering optimization problems.
On the other hand, many types of engineering optimal design problems usually contain mixed integer-discrete-continuous- (MIDC-) type design variables together with the design constraints. They involve a high degree of nonlinearity due to the combinatorial aspect of the related integer-discrete variables. The development of global stochastic optimization techniques suitable for such optimization problems is a challenging and active research area in many engineering fields. However, a rigorous implementation of integer/discrete variables, even in the framework of unconstrained PSO schemes, still represents a challenge [29]. The authors discussed in Kim et al. [18] how the aforementioned constrained PSO scheme can be combined with conventional rounding-off techniques found in many studies (see, e.g., [2, 27, 38–40]). The numerical example in this study shows that the proposed method could solve constrained optimization problems with integer, discrete, and continuous design variables in a simple manner. However, the constrained-PSO-based approach with rounding-off technique tested in Kim et al. [18] may not provide sufficient optimization reliability, in the sense that the probability of obtaining an optimal design variable is not sufficiently high. In fact, such a shortcoming may chiefly be due to the poor particle diversification characteristics which frequently causes the potential problem of premature convergence of the swarm to local optima in a practical implementation. Further, this unfavorable phenomenon becomes more persistent in the case of constrained optimizations with MIDC variables. This means that the particles' diversity is undoubtedly an indispensable issue in improving the performance of population-based search algorithms. Furthermore, without adding an additional efficient diversity enhancing mechanism to the constrained PSO, it may be very difficult to reach the true global optimal solution of such mixed variable optimization problems.
To make up for the aforementioned deficiency, we propose a novel modified constrained PSO algorithm using a simple diversity enhancing mechanism with high applicability and reliability. Many diversity-guided PSO methods have recently been proposed (see, e.g., [41–46] and the references therein). However, their effectiveness was examined only via continuous optimization problems with no constraint conditions; that is, their methods cannot be directly applied to solve constrained MIDC optimization problems. The proposed approach is designed to have particles of the swarm evolve as follows. At the beginning stage of evolution, all particles try to move into a feasible region where constraint satisfaction is guaranteed. Here, the individual particle first freely evolves in a continuous search space regardless of the variable type based on the diversity-guided velocity update law with three phases of attraction, repulsion, and a combination of attraction and repulsion (the so-called in-between phase) [43, 47]. It is worth mentioning that the PSO scheme proposed by Pant et al. [43, 47] originally targets the optimization problems with only continuous design variables and no constraint conditions. Then, for integer or discrete design variables, the corresponding continuous values of particles are rounded off to the nearest integer or to discrete values. Thereafter, the particles continue to evolve in the aforementioned manner and try to explore such a feasible region to find an optimal solution that satisfies the given constraint conditions. Note that although it may not be considered the best possible approach, the rounded-off technique has been used with promising results as demonstrated in many studies [2, 18, 24, 27, 28, 38–40, 48, 49]. The introduced three-phase velocity control method has the following features (see Pant et al. [43, 47] for details):
if the population diversity is above an upper threshold, the particles attract each other,
if the diversity is below a lower threshold, the particles repel each other,
if the diversity is between upper and lower thresholds, then the individual particle is attracted by its own previous best position and is simultaneously repelled by the best known particle position.
This efficiently improves the population diversity in the framework of constrained PSO [18] and can reduce the probability of a premature convergence phenomenon (particles trapped in a local optimum due to the nature of mixed variables). Note that the procedure that transforms the given constrained optimization problem into an unconstrained problem introduces no problem-dependent extra parameters, such as penalty factors [23, 50] or Lagrange multipliers [51], which usually require time-consuming fine-tuning tasks. The proposed constrained PSO scheme, combined with the general integer and discrete variable handling technique and three phases of particle velocity control, can yield satisfactory optimal solution accuracy. We now demonstrate the scheme's distinctive features through several numerical examples.
2. Particle Swarm Optimizer for Constrained MIDC Optimization Problems
A novel PSO algorithm for constrained MIDC optimization problems with high practicality, simplicity, and implementability is proposed. Some of the advantages of the proposed method over conventional methods are discussed in detail.
2.1. Optimization Problem Description
In this study, we are interested in general constrained optimization problems that are mathematically formulated as follows:
subject to
where n
p
is the number of independent design variables, m is the number of constraint conditions,
where the nI-dimensional vector
where ℝ
n
I
, ℝ
n
D
, and ℝ
n
C
denote feasible subsets of integer, discrete, and continuous variables, respectively. The total number of design variables is n
p
∶ = nI + nD + nC. Note that in order to guarantee particle evolution within the given search space, it is sometimes required that the search space information be incorporated into a form of boundary constraints such that
to minimize f(
where 0 denotes a zero vector of an appropriate dimension. In Section 2.2, a novel PSO algorithm to find an optimal solution of the constrained MIDC problem defined in (5) is developed.
2.2. The Improved PSO for Constrained MIDC Problems
To solve the constrained MIDC optimization problem described by (5), we used the diversity enhanced particle evolution law that decreases the possibility of particles being trapped in a local optimum due to the mixed variable nature. Consider a swarm consisting of m
p
particles:
and its velocity
where i ∊ [1, 2, …, m
p
]. Then, to find an optimal solution for the constrained MIDC optimization problem in (5), each particle
where
where c1 is the cognitive scaling factor, c2 is the social scaling factor, and χ is the constriction factor defined as
where c1 and c2 are based on the authors' experience and were set at 2.02 for the example constrained MIDC optimization problems (c1 and c2 are usually set to 2.05 for optimization problems with only continuous design variables [18, 52]). The random numbers ζ
i
k
and ξ
i
k
are uniformly distributed in [0, 1] and represent the stochastic behavior of the PSO.
where
Note that the constriction factor χ in PSO inherently enables particles to search different regions efficiently by avoiding premature convergence to local optima [38, 54]. Therefore, the introduced three-phase evolution method in (10) could be a better way to solve MIDC optimization problems than the approach in [43]. On the other hand, although the evolution law of particles described in (9) and (10) has an effectiveness of improving the population diversity that prevents the premature convergence phenomenon in PSO, its main drawback is the lack of fine-searching ability in the current search area. Note that the convergence of the particle swarm is necessary to refine the optimal solution [55]. Therefore, to overcome such a problem, we applied the three-phase velocity update law described in (10) and the following law in consecutive order (see Figure 1):
that is, the three-phase velocity update procedure from (10) was applied to an early stage of evolution of particles, and after a predetermined number of iterations (kDIV), the conventional scheme described in (14) is introduced. We verified its effectiveness through several numerical examples, as discussed in Section 3.

Flowchart of the proposed PSO scheme for constrained MIDC optimization problems.
Next, we considered how to handle the mixed integer-discrete-continuous design variables
When particles in the swarm are initialized, each particle first freely evolves in a continuous search space regardless of the variable type based on the position and velocity updating law in (9) and (10). Therefore, all design variables take real values (i.e.,
Next, for an integer variable
Once all particles in the swarm have moved to their new positions according to the aforementioned procedure, then the given objective function f(
where h
max
(
Note that the procedure transforming the given constrained optimization problem into an unconstrained problem introduces no problem-dependent extra parameter such as penalty factors [23, 50] or Lagrange multipliers [51] that usually require time-consuming fine-tuning tasks. Based on the definition of a modified objective function ℒ(
From the preceding observation, the overall computational steps of the proposed PSO algorithm for constrained MIDC optimization problems are described as shown in Figure 1.
In the following sections, the effectiveness of the proposed PSO scheme for constrained MIDC optimization problems is demonstrated via several engineering optimal design example problems (it may be difficult to guarantee the performance of our method theoretically due to the probabilistic nature of PSO).
3. Numerical Experimentation
Several engineering optimal design problems were investigated to evaluate the effectiveness of the proposed PSO technique described in Section 2. The best results obtained by the proposed method are compared to problems reported in the literature that were solved by various evolutionary computation and other optimization algorithms. In the following examples, all computations were performed on the MATLAB platform.
3.1. Engineering Optimization Problems
Problem 1 (design of a pressure vessel described in [19–22]).
A cylindrical vessel is capped at both ends by hemispherical heads as shown in Figure 2. The objective is to minimize the total cost including the cost of the material, forming, and welding. There are four design variables: T s (the thickness of the shell), T h (the thickness of the head), R (the inner radius), and L (the length of the cylindrical section of the vessel not including the head). T s and T h are integer multiples of 0.0625 (the available thickness of rolled steel plates), and R and L are continuous. These design variables are specified as
The optimization problem is formulated as follows:
subject to
where the search space was set as 𝔻 ∶ = {

Configuration of a pressure vessel.
Problem 2 (design of a pressure vessel described in [23–27]).
For the cylindrical pressure vessel shown in Figure 2, the constrained optimization problem formulated as (19) subject to (20)–(23) was considered in [23–27]. Note that the constraint conditions described in (24) and (25) are not included in this case. Design variables x1 (∶ = T
s
) and x2 (∶ = T
h
) are integer multiples of 0.0625, and x3 (∶ = R) and x4 (∶ = L) are continuous. The search space was set as 𝔻 ∶ = {
Problem 3 (design of a speed reducer described in [24, 26]).
This example deals with a design problem of the speed reducer shown in Figure 3. The design parameters are as follows: b (the face width), m (the module of teeth), n (the number of teeth on the pinion), l1 (the length of shaft 1 between bearings), l2 (the length of shaft 2 between bearings), d1 (the diameter of shaft 1), and d2 (the diameter of shaft 2). The design variables are specified as
where x3 (∶ = n) is integer and the rest of
subject to
where 2.6 ≤ x1 ≤ 3.6, 0.7 ≤ x2 ≤ 0.8, 17 ≤ x3 ≤ 28, 7.3 ≤ x4 ≤ 8.3, 7.8 ≤ x5 ≤ 8.3, 2.9 ≤ x6 ≤ 3.9, 5.0 ≤ x7 ≤ 5.5.

Configuration of a speed reducer.
Problem 4 (design of a speed reducer described in [22, 32–34]).
It is assumed that for the same optimal design problem of the speed reducer presented in Problem 5, x3 is an integer variable; x1, x2, x4, and x5 are integer multiples of 0.1; and x6 and x7 are integer multiples of 0.01. In the cited studies, the minimization problem given in (27) was modified to be
Note also that 7.3 ≤ x5 ≤ 8.3 was considered in this case.
Problem 5 (design of a tension/compression coil spring described in [21, 22, 27, 35, 36, 57]). In this optimal design problem for tension/compression coil spring shown in Figure 4, x1 (∶ = N), x2 (∶ = D), and x3 (∶ = d) were considered as integer, continuous, and discrete variables, respectively, as follows:
The following optimization problem was investigated:
subject to
with
where the maximum work load P max = 1000. 0 lb, the maximum free length l max = 14.0 in, the minimum wire diameter d min = 0.2 in, the allowable maximum shear stress S = 189000. 0 psi, the maximum outside diameter of the spring D max = 3.0 in, the preload compression force P L = 300. 0 lb, the allowable maximum deflection under preload δ pm = 6.0 in, the deflection from preload position to maximum load position δ w = 1.25 in, and the shear modulus of the material G = 11. 5 × 106 psi.

Configuration of a tension/compression coil spring.
3.2. Numerical Experimental Results and Discussion
The performance of the proposed PSO algorithm was investigated using the five MIDC optimization problems. For brevity, the swarm size m
p
is 100, the maximum number of iterations is 3000, the number of experiments is 50, and c1 = c2 = 2.02 for all the examples. The maximum number of iterations for applying the three phases velocity update procedure described in (10) was set to kDIV = 2000. Fifty runs were performed for each of the five optimization problems, and the best solutions are summarized in Tables 1 to 7. The optimal design variables and the corresponding optimal objective function values are indicated in these tables. To further assess the effectiveness of the proposed technique, the results are compared with the best optima results available in the published literature. A comparison of the optimal objective function values in Table 1 (Problem 1) and Table 4 (Problem 3) demonstrates the superiority of the proposed PSO algorithm over conventional techniques. Also, Table 2 (Problem 2), Table 6 (Problem 4), and Table 7 (Problem 5) prove our method's ability to obtain results that coincide with results found in the literature. A statistical comparison of the solution quality analysis given in Table 3 (Problem 2), Table 5 (Problem 3), and Table 8 (Problem 5) demonstrates the remarkable reliability of the proposed PSO method. Note that the statistical analysis results for Problems 1, 4, and 5 show a nearly 100% confidence level (i.e., the standard deviations are all approximately zero); these are omitted here for the sake of brevity. In addition, the presented optimal design variables are all feasible solutions (i.e., they guarantee hℓ(
Optimal design of the pressure vessel—Problem 1.
Optimal design of the pressure vessel—Problem 2.
Statistical comparison for the pressure vessel design problem—Problem 2.
Optimal design of the speed reducer—Problem 3.
Statistical comparison for the speed reducer design problem—Problem 3.
Optimal design of the speed reducer—Problem 4.
Optimal design of the tension/compression coil spring—Problem 5.
Statistical comparison for the tension/compression coil spring—Problem 5.

Objective function values of Problem 2 in 50 experiments.
4. Conclusion
We presented a novel modified constrained PSO algorithm using a simple diversity enhancing mechanism with high applicability and reliability for finding optimal mixed integer-discrete-continuous design variables. The proposed approach causes the particles of the swarm to evolve as follows. At the beginning stage of evolution, all particles try to move into a feasible region where constraint satisfaction is guaranteed. Here, individual particles first freely evolve in a continuous search space regardless of the variable type based on the diversity-guided velocity update law that includes three phases: attraction, repulsion, and a combination of attraction and repulsion. The introduced three-phase velocity control method improves the population diversity in the framework of constrained PSO efficiently and thus can reduce the probability of a premature convergence phenomenon, particles being trapped in a local optimum due to the mixed variable nature. For integer or discrete design variables, the corresponding continuous values of particles are rounded off to the nearest integer or discrete values. Thereafter, the particles continue to evolve in the aforementioned manner and try to explore the feasible region to find an optimal solution that satisfies the given constraint conditions. Note that the procedure transforming the given constrained optimization problem into an unconstrained problem introduces no problem-dependent extra parameters such as penalty factors or Lagrange multipliers which usually require time-consuming fine-tuning tasks. Our numerical experiment verified that the proposed constrained PSO scheme, combined with a general integer and discrete variable handling technique and three phases of particle velocity control law, can yield satisfactory optimal solution accuracy.
Footnotes
Acknowledgments
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (no. 2012-012295) and the Chung-Ang University excellent freshman scholarship grants in 2013.
