Teaching-learning-based optimization with a fuzzy grouping learning strategy for global numerical optimization

Abstract

The Teaching-Learning-Based Optimization (TLBO) algorithm is a novel heuristic method that is inspired by the philosophy of teaching and learning in a class. In the “Teacher Phase” of the original TLBO algorithm, all learners are combined in one group and learn only from the teacher, which quickly leads to declining population diversity. Utilizing fuzzy K-means clustering to objectively divide all learners into smaller-sized groups more closely conforms to the modern idea of intra-class grouping for teaching and learning. Furthermore, fuzzy K-means clustering can objectively divide learners as nearly as possible according to their interests and abilities, which helps each learner to grow to his or her fullest extent. This paper presents a novel version of TLBO, TLBO with a Fuzzy Grouping Learning Strategy (FGTLBO), in which fuzzy K-means clustering is used to create K centers, each of which acts as the mean of its corresponding group. Performance and accuracy of the FGTLBO algorithm are examined on CEC2005 standard benchmark functions, and these results are compared with various other versions of TLBO. The experimental results verify that the FGTLBO algorithm is very competitive in terms of solution quality and convergence rate.

1 Introduction

Rao, et al. [21] initially proposed the Teaching-Learning-Based Optimization (TLBO) algorithm, which is based on swarm intelligence and applied to continuous, non-linear, multivariable, multimodal, large-scale optimization problems [22]. TLBO was inspired by the traditional teaching and learning phenomenon of a classroom. Compared with other evolutionary algorithms, such as Particle Swarm Optimization (PSO) [9], Differential Evolution (DE) [18], Artificial Bee Colony (ABC) [3], Group Search Optimizer (GSO) [24], Cuckoo Search (CS) [26], Water Cycle algorithm (WCA) [6], Differential Search algorithm (DSA) [15], Backtracking Search algorithm (BSA) [14], and Interior Search algorithm (ISA) [2], the TLBO algorithm has simple computational characteristics and few specific controlling parameters. These features have attracted great interest among researchers in recent years.

Since its introduction, the TLBO algorithm has been applied in various optimization problems such as parameters optimization of selected casting processes [20], multi-objective design optimization of a plate-fin heat sink [19], multi-objective optimization of heat exchangers [23], Dynamic Economic Emission Dispatch [25], Dynamic Voltage Restorer (DVR) compensator [13], and combinatorial problems on flow shop and job shop scheduling cases [1]. Niu, et al. put forward an improved TLBO algorithm to identify the parameters of PEM fuel cell and solar cell models [17]. Similarly, Ghasemi et al. proposed an improved TLBO algorithm using the L$\acute{\mathrm{e}}$vy mutation strategy for non-smooth optimal power-flow [12]. Li, et al. discussed a discrete TLBO algorithm for realistic flow-shop rescheduling problems [10]. Finally, Hosseinpour et al. used Teaching–Learning-Based optimization coupled with Simulated Annealing algorithm (SA-TLBO) method to determine the optimal placement of on-load tap changers in distribution networks [7].

Although TLBO has been successfully applied in many fields, Zou, et al. [5] report that the original TLBO algorithm features some undesirable dynamical properties that degrade its searching ability. One of the most important problems is that the TLBO algorithm tends to get trapped in local optima solutions because of diversity loss; in other words, over a certain period of time, each member in the population tends toward a fixed value.

Several variants have been introduced to improve the performance of the original TLBO. For example, a ring neighborhood topology was introduced into the original algorithm to maintain the exploration ability of the population and the diversity of individuals. Results show that the proposed method increased the solution quality [11]. The Gaussian sampling learning strategy was introduced into the original TLBO to improve its performance, and it balanced the exploration and exploitation in the “Teacher Phase” [4]. Subsequently, the quantum-behaved learning strategy was introduced into the original TLBO algorithm to enhance the exploitative nature of the population; results show that the proposed algorithm is a challenging method [5].

Although these variants on the TLBO algorithm show some improvements, they fail to focus objectively on more groups. To this aim, we present a novel version of the algorithm, TLBO with a fuzzy grouping learning strategy (FGTLBO). In FGTLBO, fuzzy K-means clustering is used to create K centers, which act as the means of their corresponding groups in the “Teacher Phase” in order to form groups objectively. This modification both improves the diversity of the population, which needs to be preserved in order to discourage premature convergence, and achieves balance between the explorative and exploitative tendencies inherent in pursuing better solutions.

The structure of this paper is organized as follows: Section 2 presents the original TLBO, which provides the necessary preparation for understanding the rest of the paper. Section 3 proposes FGTLBO. Section 4 analyzes the results of our comparative study and discusses the influence of the size of the group on the performance of the FGTLBO algorithm. Finally, Section 5 presents the conclusion and proposals for future work.

2 Overview of original TLBO algorithm

Inspired by the philosophy of teaching and learning in a traditional classroom, Rao, et al. developed the TLBO algorithm for the purpose of optimizing numerical problems. This algorithm considers the population as a group of learners, while different design variables are analogous to different subjects offered to learners. In the end, the learner’s grade is equivalent to the “fitness” value of the optimization problem. The best solution out of the entire population is considered to represent the teacher, who is generally the most knowledgeable person in a class and shares his knowledge with the learners.

This method’s operating procedure consists of two learning phases, namely the “Teacher Phase” and the “Learner Phase.” The “Teacher Phase” is analogous to learning from the teacher, whereas the “Learner Phase” means learning through interaction among learners. The phases are defined as follows: $X_{\mathit{new},\mathit{i}}=X_i+r(X_{\mathit{teacher}}-T_F{X}_{\mathit{mean}})$(1) where X_new, i is a newly generated individual according to X _i ; r is a random vector in which each element is a random number in the range [0,1]; X _teacher is the best individual among the current population; T _F represents a teaching factor that decides the value of the mean to be changed; and X _mean is the current mean value of all of the individuals. The value of T _F can be either 1 or 2, which is a heuristic step and decided randomly with equal probability.

2.1 Learner phase

In this phase, improvement in the knowledge of a learner X _i depends mainly on peer learning from an optimal learner X _j . Based on the knowledge levels of these two learners, two states may occur: if X _j is better than X _i , X _i will move towards X _j ; otherwise, it will be moved from X _i . The learner phase can be expressed as follows: $X_{\mathit{new},\mathit{j}}=X_i+r(X_j-X_i)$(2)$X_{\mathit{new},\mathit{j}}=X_i+r(X_i-X_j)$(3)

3 TLBO with a fuzzy grouping learning strategy (FGTLBO)

In the original TLBO, each learner is learning from a teacher in only one class, which easily leads to declining population diversity. On the other hand, utilizing fuzzy K-means clustering to group learners objectively conforms to the modern model of intra-class grouping for teaching and learning more closely. Motivated by these facts, we propose FGTLBO, a novel version of TLBO that incorporates a fuzzy grouping learning strategy.

3.1 Fuzzy K-means clustering

Fuzzy K-means clustering is a data clustering technique in which a dataset is grouped into K clusters such that every data point in the dataset belonging to a cluster will have a high degree of belonging or membership within that cluster, while another data point that lies far away from the center of the cluster will have a low degree of belonging or membership within that cluster. Traditional hard clustering demands that every point within a data set be assigned into a cluster precisely. Fuzzy K-means clustering is an extension of traditional hard clustering such that the corresponding clustering centers used in fuzzy K-means clustering are closer to the real centers of groups than those used in traditional hard clustering.

The clustering centers determined by these groups may be far away from each other if the centers are close to the actual centers of the groups under fuzzy k-means clustering. Conversely, the centers may be close to each other. According to formula in (4) the “Teacher Phase” of the original TLBO algorithm, each clustering center is considered to be the center of a certain group. If the centers of the these groups are close to one another, some individuals of the next generation will end up very similar; in other words, results easily fall into local optima, greatly reducing the effectiveness of the original TLBO algorithm. Therefore, the theory of fuzzy K-means clustering makes a suitable addition to the original TLBO algorithm.

Fuzzy K-means clustering was initially developed by Dunn and later generalized by Bezdek [8] by means of a family of objective functions. Fuzzy K-means clustering is an iterative method that minimizes the objective function defined as follows:

$J_m(U,C)=\sum _{i=1}^N\sum _{j=1}^K{u_{\mathit{ij}}^m\parallel x_i-c_j\parallel }^2,1<m<\infty$(5) where x _i represents feature data, m is a weighting exponent on each fuzzy member, u _ij is the membership degree of ith object with the jth cluster center, c _j is thr j-dimension center of the cluster, and ∥* ∥ is a matrix norm expressing the similarity between measured data and the center. Let U = (u _ij ) be a fuzzy matrix; then, the characteristics of u _ij are as follows: $u_{\mathit{ij}}\in [0,1],\forall i\in 1,2,\cdots ,N;j\in 1,2,\cdots ,K$(6)$\sum _{j=1}^K{u}_{\mathit{ij}}=1,\forall i\in 1,2,\cdots ,N$(7)$0<\sum _{i=1}^N{u}_{\mathit{ij}}<N,\forall j\in 1,2,\cdots ,K$(8)

The membership functions and cluster centers are updated by the following expressions: $u_{\mathit{ij}}=\frac{1}{{\sum }_{k=1}^K{\left[\frac{\parallel x_i-c_j\parallel }{\parallel x_i-c_k\parallel }\right]}^{\frac{2}{m-1}}}$(9) and $c_j=\frac{{\sum }_{i=1}^N{u}_{\mathit{ij}}^m{x}_i}{{\sum }_{i=1}^N{u}_{\mathit{ij}}^m}$(10)

This iteration will stop when $\underset{\mathit{ij}}{max}=\left|u_{\mathit{ij}}^{t+1}-u_{\mathit{ij}}^t\right|$≤e is satisfied, where e is a termination criterion between 0 and 1, and t is the iteration step.

3.2 Fuzzy grouping

Intra-class grouping for teaching and learning constitutes a method of classroom management that divides learners into two or more groups within the classroom to provide for individual differences. To more closely simulate the processes of teaching and learning in a modern class, we will divide all learners into small sized groups using fuzzy K-means clustering. Fuzzy K-means clustering can objectively divide learners as accurately as possible according to their interests and abilities, which will help grow their abilities and creativities to their fullest extents. The number K of cluster centers in fuzzy K-means clustering is identical to the number m of the dividing group strategy. For simplicity, we will divide all learners into groups based on Euclidean distance, and each group uses its own members to search for a better area in the search space. Once these groups are constructed, we can utilize them to update the corresponding group mean. Table 1 summarizes the pseudo code for fuzzy grouping as explained above.

3.3 The pseudo code for the FGTLBO algorithm

We developed FGTLBO by incorporating the fuzzy grouping learning strategy into the “Teacher Phase” of the original TLBO framework. Table 2 presents the pseudo code for the FGTLBO algorithm.

3.4 The global convergence analysis of the FGTLBO algorithm based on Markov chain

We consider only the minimum global optimization problem without loss of generality. Let x satisfy solution space Ω for the FGTLBO algorithm, namely x ∈ Ω. Let $X_{\mathit{teacher}}^{\left(t\right)}$ be the best individual in the tth evolutional generation. Note that $\{X_{\mathit{teacher}}^{\left(t\right)}.t=0,1,\cdots \}$ forms a discrete time stochastic process. Based on the parent individuals, the FGTLBO algorithm produces the next generation according to random variables. Let $X_{\mathit{teacher}}^{\left(t\right)}$ be the best individual of the parent groups, and let its corresponding state denote E _i . Let $X_{\mathit{teacher}}^{(t+1)}$ be the best individual of the progeny groups, and let its corresponding state denote E _j . This state transfer forms a discrete Markov chain, which shows that offspring only relate to their parent groups and have no relation to previous groups. Because the standard deviation adjusts dynastically with evolutional generation t, the Markov chain is not time homogeneous. This suggests that state transfer probability relates to evolutional generation t. Let $p_{\mathit{ij}}^{\left(t\right)}$ be the state transfer probability from generation t state transfer E _i to generation t + 1 state transfer E _j . The equation of $p_{\mathit{ij}}^{\left(t\right)}$ is as follows:

$p_{\mathit{ij}}^{\left(t\right)}=\{\begin{array}{c}\hfill p_{\mathit{ij}}^{\left(t\right)}=1-\sum _{E_i\in C}{p}_{\mathit{ij}}^{\left(t\right)}\mathit{if}f\left(E_j\right)\ge f\left(E_i\right)\hfill \\ \hfill p_{\mathit{ij}}^{\left(t\right)}=0,\mathit{if}f\left(E_j\right)<f\left(E_i\right)\hfill \\ \hfill \hfill \end{array}$(11)

Definition 1. For global solution x^*, if there exists a discrete time stochastic process $\{X_{\mathit{teacher}}^{\left(t\right)}.t=0,1,\cdots \}$ satisfying $\underset{t\to \infty }{lim}\left[p\left(f\right(X_{\mathit{teacher}}^{\left(t\right)})=f(x^{*}\left)\right)\right]=1$, then the algorithm converges to global solution x^* with probability 1.

Theorem1.The FGTLBO algorithm converges to global solutionx^*with probability 1.

Proof: Suppose that all learners are sorted in ascending order according to their results. There exists a transition probability matrix P^(t) of finite state Markov chain generation t steps satisfying the following: $P^{\left(t\right)}=\left(p_{\mathit{ij}}^{\left(t\right)}\right)\left|\Omega \right|\left|\Omega \right|=\left[\begin{array}{ccc}\hfill p_{11}^{\left(t\right)}\hfill & \hfill \hfill & \hfill \hfill \\ \hfill p_{21}^{\left(t\right)}\hfill & \hfill p_{22}^{\left(t\right)}\hfill & \hfill \hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \ddots \hfill \\ \hfill p_{\left[\Omega \right]1}^{\left(t\right)}\hfill & \hfill p_{\left[\Omega \right]2}^{\left(t\right)}\hfill & \hfill \dots p_{\left[\Omega \right]\left[\Omega \right]}^{\left(t\right)}\hfill \\ \hfill \hfill \end{array}\right]$(12) where P^(t) denotes a lower triangular matrix,$p_{\mathit{ij}}^{\left(t\right)}>0,i>j$, ${\sum }_{p_{\mathit{ij}}^{\left(t\right)}}=1$, and $p_{i11}^{\left(t\right)}=1$.

Based on the nature of the Markov chain P^(k) = P ^k and the convergence to a positive stable random matrix by the means of P^(k), $P^{\infty }=\underset{k\to \infty }{lim}{P}^k=(1,1,\mathit{cdots},1{)}^T(p_1,p_2,\cdots ,p_n)$(13) where vector (p₁, p₂, ⋯ , p _n ) only meets (p₁, p₂, ⋯ , p _n ) P = (p₁,  p₂, ⋯ , p _n ), ${\sum }_{i=1}^n{p}_i=1$, and $p_i=\underset{k\to \infty }{lim}{p}_{\mathit{ij}}^{\left(k\right)}>0$. The k denotes k ^th step Markov chain for transfer matrix P^(k). Moreover, the vector (p₁, p₂, ⋯ , p _n ) is a pre-multiplication feature vector of Eigenvalues for 1 and each component for a positive number. Consequently, $\underset{k\to \infty }{lim}{P}^k=(1,1,\cdots ,1{)}^T(1,0,\cdots ,0)=\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 10\hfill \\ \hfill ⋮⋮\ddots \hfill \\ \hfill 10\dots 0\hfill \\ \hfill \hfill \end{array}\right]$(14)

Furthermore, $\{\begin{array}{c}\hfill \underset{k\to \infty }{lim}{p}_{i1}^{\left(k\right)}=1,E_i\in \Omega \hfill \\ \hfill \underset{k\to \infty }{lim}{p}_{\mathit{ij}}^{\left(k\right)}=0,j\ne 1,E_i\in \Omega \hfill \\ \hfill \hfill \end{array}$(15)

Therefore, $\underset{t\to \infty }{lim}\left[f\left(X_{\mathit{teacher}}^{\left(t\right)}\right)=f\left(x^{*}\right)\right]=1$(16)

Thus, the FGTLBO algorithm can converge to a global solution with probability 1.

4 Experiments and comparisons

4.1 Benchmark functions used in experiments

A large set of CEC2005 tested benchmark functions [16] were used in experiments to assess the performance of the FGTLBO algorithm. Based on shape characteristics, the set of benchmark functions are grouped into unimodal functions (F₁ to F₅), basic multimodal functions (F₆ to F₁2), and expanded multimodal functions (F₁3 to F₁4).

4.2 Experimental environment, termination criteria, and control parameters

All experiments were conducted using Matlab7.9 on the same machine with a Celoron2.26 GHz CPU, 2GB of memory, and a Windows XP operating system. For the purpose of decreasing statistical errors, all experiments were independently run 25 times for 14 test functions of 30 variables, with 300,000 Function Evaluations (FES) as the stopping criterion. The FGTLBO algorithm uses as parameters N = 50 and K = m = 3. The parameters of the other algorithms agree well with the original papers.

4.3 Performance metric

The mean value F _mean and standard deviation SD of the error value function F (x) - F (x^*) were recorded to evaluate the performance of each algorithm, where F (x) and F (x^*) denote the test problem’s best fitness value and real global optimization value, respectively. Statistical analysis was used to compare the results obtained by the algorithms and to verify whether overall optimization performance differs significantly among various algorithms. To statistically compare the FGTLBO algorithm with four other TLBO algorithms, the Wilcoxons rank sum test [27] was applied at a 0.05 significance level to evaluate the median fitness values F _mean of two solutions from any two algorithms. This statistical tool is frequently employed in the literature to compare problem-solving success among the Computational-Intelligence algorithms.

4.4 Numerical experiments and results

This section compares the FGTLBO algorithm with four other TLBO variants. Corresponding tables present the experimental results, and the last three rows of each table summarize the comparison results. The best results are shown in bold.

4.4.1 Comparison of FGTLBO algorithm and four relevant TLBO algorithms

This section compares the FGTLBO algorithm with four relevant TLBO algorithms. Based on the statistical results in Table 3, it appears that none of the algorithms can perfectly solve the fourteen CEC2005 standard benchmark functions. Judging by the Wilcoxon’s rank sum test listed in the last three rows of Table 3, the FGTLBO algorithm outperforms the original TLBO algorithm on all test functions except F₄. For test function F₃, the mean value F _mean of the FGTLBO algorithm yields 4.07E-22, which is a significantly better result than those produced by the other four relevant TLBO algorithms.

The incorporation of a fuzzy grouping learning strategy allows the FGTLBO algorithm to achieve promising results on unimodal and multimodal functions. The FGTLBO algorithm outperforms TLBO, ETLBO, NSTLBO, and DGSTLBO on twelve, eight, thirteen, and thirteen of the fourteen test functions, respectively. The TLBO algorithm performs best on test function F₄, while the DGSTLBO algorithm performs worse than other four relevant algorithms on test functions F₁, F₃, F₅, F₆, and F₁₂. The ETLBO algorithm performs better than FGTLBO algorithm only on test functions F₄, F₆, F₁₂, and F₁₃. The NSTLBO algorithm performs worse than other four relevant algorithms on test functions F₂, F₄, F₉, F₁₀, F₁₁, F₁₃, and F₁₄. Notably, the DGSTLBO, NSTLBO, ETLBO, TLBO, and DATLBO algorithms perform similarly on test function F₄.

Figure 1 presents the convergence graphs of the five relevant TLBO algorithms on fourteen test functions for D = 30. Based on the convergence graphs, the FGTLBO algorithm shows better convergence, stability, and robustness in most cases than the other four algorithms for test functions F₁, F₂, F₃, F₅, F₆, F₈, F₉, F₁₀, F₁₁, F₁₂, F₁₃, and F₁₄. In the case of test function F₄, the convergence rate of the FGTLBO algorithm is similar to those of the other four relevant TLBO algorithms. For test function F₇, on the other hand, the convergence rate of the FGTLBO algorithm is worse than those of the other four algorithms.

The TLBO algorithm shows better convergence, stability, and robustness than the other four relevant TLBO algorithms only on test functions F₄ and F₁₂. The ETLBO algorithm shows better convergence than the TLBO algorithm only on test function F₁₂. The DGSTLBO algorithm shows worse convergence on test functions F₁, F₂, F₅, F₆, F₁₂, and F₁₃, and the NSTLBO algorithm shows worse convergence on test functions F₄, F₉, and F₁₀. Overall, the FGTLBO algorithm performs significantly better than the ETLBO, NSTLBO, DGSTLBO, and original TLBO algorithms. This analysis suggests that fuzzy grouping draws its strength from increasing the diversity of the population to discourage premature convergence. Therefore, the FGTLBO algorithm has a higher probability of efficiently exploring the search space and finding a globalsolution.

4.4.2 Influence of the number of groups on the performance of the FGTLBO algorithm

In order to consider the influence of different numbers of groups on the performance of the FGTLBO algorithm, the group quantity m was set from 2 to 11 while other parameters remained the same as described in Section 4.2. All of the experiments were conducted on test functions F₁ through F₁₄. Tables 4-5 show the statistical results for different numbers of groups. These results illustrate that FGTLBO (m = 3) demonstrated significantly better overall performance than other cases. Moreover, as m increased from 4 to 11, the performance of the FGTLBO algorithm did not improve, which shows that a situation with more groups is not optimal choice for the FGTLBO algorithm. Because smaller m values may result in premature convergence while larger m values greatly decrease the probability of finding the correct search direction. Therefore, we recommend that the number of groups m for the FGTLBO algorithm be set at 3.

5 Conclusions and future work

This paper presented FGTLBO, a new version of the TLBO algorithm in which a fuzzy K-means clustering strategy is used to create K centers, each of which acts as the mean of its corresponding group in the algorithm’s “Teacher Phase.” To more closely simulate the processes of teaching and learning in a modern classroom, we objectively divided all learners into small-sized groups. This conforms to the idea of modern intra-class groupings for teaching and learning. As fuzzy K-means clustering can objectively divide learners as nearly as possible on the basis of their interests and abilities, it can help each learner grow to his or her fullest extent. We examined the performance and accuracy of the FGTLBO algorithm on CEC2005 standard benchmark functions and compared the results with various classical versions of the TLBO algorithm. The experimental results verify that the FGTLBO algorithm is very competitive in terms of solution quality and convergence rate under most experimental conditions. Further research can assess how well the FGTLBO algorithm performs in parameter optimization of a multi-pass milling process.