Sage Journals: Discover world-class research

Abstract

In this paper, a novel modified optimization algorithm is presented, which combines Nelder-Mead (NM) method with a gradient-based approach. The well-known Nelder Mead optimization technique is widely used but it suffers from convergence issues in higher dimensional complex problems. Unlike the NM, in this proposed technique we have focused on two issues of the NM approach, one is shape of the simplex which is reshaped at each iteration according to the objective function, so we used a fixed shape of the simplex and we regenerate the simplex at each iteration and the second issue is related to reflection and expansion steps of the NM technique in each iteration, NM used fixed value of $α$ , that is, $α$ = 1 for reflection and $α$ = 2 for expansion and replace the worst point of the simplex with that new point in each iteration. In this way NM search the optimum point. In proposed algorithm the optimum value of the parameter $α$ is computed and then centroid of new simplex is originated at this optimum point and regenerate the simplex with this centroid in each iteration that optimum value of $α$ will ensure the fast convergence of the proposed technique. The proposed algorithm has been applied to the real time implementation of the transversal adaptive filter. The application used to demonstrate the performance of the proposed technique is a well-known convex optimization problem having quadratic cost function, and results show that the proposed technique shows fast convergence than the Nelder-Mead method for lower dimension problems and the proposed technique has also good convergence for higher dimensions, that is, for higher filter taps problem. The proposed technique has also been compared with stochastic techniques like LMS and NLMS (benchmark) techniques. The proposed technique shows good results against LMS. The comparison shows that the modified algorithm guarantees quite acceptable convergence with improved accuracy for higher dimensional identification problems.

Keywords

Nelder Mead optimization direct search adaptive filtering

Introduction

The Nelder-Mead (NM) is one of the best direct search based optimization technique for unconstrained multivariable complex problems.¹ This strategy was proposed by Nelder and Mead.² Due to its derivative free nature and ease of implementation it finds its applications in diverse disciplines, for example, engineering, computer science, chemistry, biosciences, etc.^3–12

NM is a very popular heuristic-based optimization strategy, however, a very small number of papers have addressed its convergence. In this regard, a detailed study was carried out in Torczon.¹³ An analytical expression for the convergence of pattern search methods was formulated. However, the NM algorithm was not considered in that study because of the varying structure of the simplex in each iteration which directly depends on the cost function. The convergence of NM strategy was further investigated by Lagarias et al.,¹⁴ where one and two dimensional strictly convex functions were studied. It was claimed that the algorithm converges for a convex function of one variable, whereas, for the two-variable functions the convergence can also be ensured. McKinnon et al.¹⁵ also highlighted the convergence issues of NM. It was concluded that if the objective function considered in the work of Lagarias et al.¹⁴ is thrice continuously differentiable, then the minimum point can be achieved, otherwise, the algorithm may converge to a non-stationary point. In Lagarias et al.¹⁶ a variant of the NM technique was proposed for the twice continuously differentiable cost function.

Besides the convergence issues, this optimization technique also suffers from dimensionality problems. The convergence performance is proportional to the dimension of the optimization problem. The optimization problem with lower dimension will converge faster as compared to higher dimension problem (see for instance Torczon,¹³ Lagarias et al.,¹⁶ and Han and Neumann¹⁷). This phenomenon is called the curse of dimensionality.¹⁸ Price et al.¹⁹ proposed a variant of NM which shows good results for higher dimensions with better convergence rate. Moreover, in Gao and Han²⁰ proposed a modified approach in which expansion, contraction, and shrinking become the function of the dimension of an optimization problem.

Many other modifications are also proposed in the literature. In Mehta and Dasgupta²¹ a modified NM algorithm was proposed for constrained optimization problems. Online implementation of NM method was proposed for computing systems and chemical processes in Poojary and colleagues^22–24 and Xiong and Jutan²⁵ respectively. In Marsililibelli and Castelli²⁶ an algorithm was proposed in which the parameters of simplex search are made adaptive according to the shape of the objective function. It also proposed a random search procedure for the initialization of simplex that assures the convergence of global minimum. In Musafer and Mahmood,²⁷ a free selective simplex for the downhill Nelder Mead simplex algorithm is proposed rather than the determinant simplex that forces its elements to perform a single operation,such as reflection.

Basically NM is a local search technique, however, some hybrid global optimization techniques were developed in Yıldız and colleagues,^28–39 which incorporate NM algorithm. The aim of hybridization is to accelerate global convergence.

Although many modifications are proposed in NM that address the issues of convergence and curse of dimensionality, but still there is scope to investigate new modifications that consider simplex structure like Spendley et al.⁴⁰ and optimal selection of parameters for NM operations that ensure convergence of the algorithm and give better performance for higher dimensions.

In this paper, a novel modified optimization algorithm is presented, which combines Nelder-Mead (NM) method with a gradient-based approach. In this technique, the simplex is initialized as a non-degenerated structure and its shape does not change with iterations. Therefore, it ensures the convergence of the algorithm even for higher dimensions. Moreover, due to the inculcation of the gradient based-approach, the modified NM algorithm solves the optimization problem in fewer iterations as compared to the NM. The Online implementation of this modified technique is also presented. Finally, this method is applied to a system identification problem where parameters are estimated in mean square error sense. In this case optimization problem is convex with a global minimum. This is a classical optimization problem in adaptive filter theory.⁴¹

The rest of the article is arranged as follows. The NM method is explained in Section 2, whereas, Section 3 presents the proposed algorithm. The online implementation of MNM is given in Section 4, the result discussion is presented in Section 5 and the article is concluded in Section 6.

Nelder-Mead

Nelder-Mead is a well known heuristic optimization method based on simplex structure. A simplex is hyper tetrahedron in n-dimensional space consisting of $n + 1$ points, where $n$ is the number of optimization variables. The simplex in one dimensional (1-D) space is a line segment, for 2-D space a triangle, and a tetrahedron for 3-D. NM algorithm starts by initializing the simplex using initial guess of the optimization variables. Simplex vertices are labeled as $x_{1}, x_{2} \dots x_{n + 1}$ , with their corresponding objective function values $f (x_{1}), f (x_{2}) \dots f (x_{n + 1})$ . In each iteration of NM the simplex vertices are sorted in ascending order according to the corresponding objective function values $f (x_{1}) < f (x_{2}) < \dots < f (x_{n + 1})$ . Let $x_{b} = x_{1}$ refers to the best point having smallest value of the cost function. Similarly $x_{w} = x_{n + 1}$ and $x_{s} = x_{n}$ are the worst and second most worst points respectively. In each iteration the algorithm tries to make a new simplex in which $x_{w}$ is replaced by $x_{n}$ such that $f (x_{n}) < f (x_{w})$ . The new point $x_{n}$ is obtained by equation (1), which represents the straight line starting from $x_{w}$ and passing through $x_{m}$ .

\begin{matrix} x_{n} = x_{m} + α (x_{m} - x_{w}) \\ where, \\ x_{m} = \frac{1}{n} \sum_{i = 1}^{n} x_{i} \end{matrix}

(1)

Where $x_{m}$ is the average of $n$ points excluding $x_{w}$ , and $α$ is the scalar parameter. The value of alpha is governed by four rules, which include Reflection, Expansion, Contraction(outside and inside), and Shrink. The algorithm 1 explains one iteration of the NM method.

Algorithm 1. Steps involved in NM algorithm.
1: Sorting of vertices: Vertices are sorted in ascending order according to their corresponding objective function values as explained earlier.
2: Midpoint: Midpoint is computed by equation (1)
3: Reflection: Solve equation (1) for $α = 1$ and define $x_{n}$ as $x_{r}$ , where $x_{r}$ is called the reflection point. Evaluate $f (x_{r})$ , if $(f (x_{b}) < f (x_{r}) < f (x_{s}))$ choose $x_{w} = x_{r}$
4: Expansion: If $(f (x_{r}) < f (x_{b}))$ then solve equation (1) for $α = 2$ , then the solution ( $x_{e} = x_{n}$ ) is called the expansion point. Evaluate $f (x_{e})$ , if $(f (x_{e}) < f (x_{b}))$ , then $x_{w} = x_{e}$
5: Contraction: If $(f (x_{s}) < f (x_{r}) < f (x_{w}))$ then set $α = 0.5$ in equation (1). This is called outside contraction and the solution is denoted by $x_{oc}$ . Evaluate $f (x_{oc})$ , and if $f (x_{oc}) < f (x_{r})$ , then $x_{w} = x_{oc}$ , then otherwise apply the shrink rule. If $f (x_{r}) > f (x_{w})$ , set $α = - 0.5$ in equation (1). This is called inside contraction and the solution is referred as $x_{ic}$ . Evaluate $f (x_{ic})$ , and if $(f (x_{ic}) < f (x_{w}))$ then replace $x_{w}$ with $x_{ic}$ , otherwise go to shrink rule.
6: Shrink: In this rule all the verices are shrinked by half except the best.

All the rules of NM algorithm are presented in the Figure 1.

Figure 1.

Operations performed on the simplex in Nelder-Mead’s algorithm for n = 2.

Modified Nelder Mead

The NM algorithm explained in the previous section requires direction of propagation of the simplex structure and the value of $α$ in each iteration. The algorithm heuristically computes these parameters. The shape of the simplex is updated in a direction which is calculated by a vector joining $x_{w}$ and the midpoint $x_{m}$ of all other vertices. The parameter $α$ has typically four choices: 1, 2, 0.5, and −0.5 which are based on the aforementioned rules.

A modified NM (MNM) algorithm is proposed in this section, which differs from the conventional NM method in two aspects: structure of the simplex and the selection of $α$ . In MNM the simplex is comprised of $2 n + 1$ vertices and its shape is also preserved throughout the optimization problem. However, in conventional NM the shape of the simplex can change in each iteration, and its structure can also degenerate especially in case of non smooth objective functions. Moreover, in MNM $α$ is analytically computed by solving an optimization problem, rather than heuristic selection of $α$ in the conventional NM.

Consider a generalized unconstrained optimization problem, given by equation (2)

min_{x} ρ = f (x_{1}, x_{2}, \dots, x_{n})

(2)

where $a \leq x \leq b \in ℜ^{n}$ is the vector of optimization variables and $ρ (x) : ℜ^{n} \to ℜ$ is the objective function.

In MNM method the simplex is a hyper tetrahedron structure in $n$ dimensional space. The simplex has $2 n + 1$ vertices: $v_{1}, v_{2} \dots v_{2 n + 1}$ , which are computed by using equation (3).

v_{i} = v_{c} + (- 1)^{i} d a_{j}, i = 1, 2, 3, \dots, 2 n

(3)

where $v_{c}$ is the centroid of the simplex, $d$ is the simplex size and the set ${a_{j}, j = 1, 2, \dots, n}$ represents orthonormal basis for the $n$ dimensional space containing the simplex.

The MNM method starts by initializing $v_{c}$ by using initial guess of the optimization variables, followed by sorting of the vertices in ascending order $v_{1}, v_{2} \dots v_{2 n + 1}$ according to their corresponding objective function values $ρ (v_{1}) < ρ (v_{2}) < \dots < ρ (v_{2 n + 1})$ , where $v_{1}$ is the best and $v_{2 n + 1}$ is the worst vertex. The mid point $v_{m}$ of all vertices is computed, excluding $v_{2 n + 1}$ . Then the new centroid is computed by equation (4).

\begin{matrix} v_{c} = v_{2 n + 1} + α^{*} Δ p \\ where \\ Δ p = v_{m} - v_{2 n + 1} \end{matrix}

(4)

where $α^{*}$ is defined in the subsequent paragraph.

The $n$ dimensional optimization problem in equation (2) can be transformed into one dimensional problem as defined in equation (5)

min_{α} ρ = f (v_{c} (α))

(5)

The solution of the above optimization problem is $α^{*}$ , which is obtained by solving equation (6)

\frac{d ρ (v_{c} (α *))}{d α^{*}} = 0

(6)

The Algorithm 2 presents all the steps of MNM.

Algorithm 2. Steps involved in MNM.
1: Initialization:
2: Choose the initial guess of simplex centroid $v_{c}$ and simplex size ( $d$ ).
3: Compute the orthonormal basis vectors for the space containing the simplex.
4: Iterative loop
5: Construction of Simplex: Simplex is generated by using equation (3).
6: Sorting of vertices:
7: Compute objective function at all $2 n + 1$ vertices.
8: Vertices are sorted in ascending order according to their corresponding objective function values as explained earlier.
9: Midpoint: Midpoint of all the vertices is computed excluding $v_{2 n + 1}$ .
10: Update Simplex Size: $while {ρ (v_{c}) < ρ (v_{2 n + 1})}$ set $d = \frac{d}{2}$ and repeat the steps 5 → 9, otherwise proceed to the next step.
11: Compute $α $ : Solve equation (6) to obtain $α $
12: New Centroid: Updated value of $v_{c}$ is found by solving equation (4)
13: Termination Criterion: If $ρ (v_{c}) \leq δ_{ρ}$ (tolerance for cost function) terminate the loop, otherwise go to step 5

Parameter estimation using MNM: An online implementation

In this section online implementation of the MNM algorithm for parameter identification is discussed.

Problem statement

The algorithm is used to estimate the parameters of an FIR system. The adaptive filter shown in Figure 2 estimates the parameters of the system in such a way that error $e (n)$ given by equation (7) is minimized.

e (n) = d (n) - y (n)

(7)

where $d (n)$ is the desired output of the system and $y (n)$ is response of the adaptive filter.

Figure 2.

N-taps transversal adaptive filter.⁴¹

The error $e (n)$ is minimized in mean square sense, it means that the model parameters are changed by continuously observing the input and the desired output. It is a sequential realization of wiener filters. The cost function $ζ$ for aforementioned problem is given by equation (8)

\begin{matrix} ζ = e^{2} (n) \\ = {d (n) - y (n)}^{2} \end{matrix}

(8)

where $y (n)$ is given by equation (9)

\begin{matrix} y (n) = \sum_{i = 0}^{N - 1} w_{i} (n) x (n - i) \\ = {\bar{w}}^{T} (n) \bar{x} (n) \end{matrix}

(9)

Equation (9) is the standard representation of digital filters in the literature. It describes the scalar product of two vectors $\bar{w}$ and $\bar{x}$ . Where $\bar{w} (n) = {[w_{0} (n), w_{1} (n), \dots, w_{N - 1} (n)]}^{T}$ are adaptive filter taps and $\bar{x} (n) = {[x (n - 1), x (n - 2), \dots, x (n - N + 1)]}^{T}$ are input samples and “ $n$ ” represents the sample or discrete time in $\bar{x}$ , $\bar{w}$ , and output y.

Problem formulation

The MNM is implemented for estimating parameters $(w^{T} (n))$ of the system. The optimum value of $w^{T} (n)$ is found by application of the procedure detailed in Algorithm 2. The centroid of the initial simplex is $w^{T} (0)$ which represents the initial value of the $N$ filter taps. Now by using the results in equations (4) and (9) the cost function in equation (8) takes the following form:

ζ = {[d (n) - {v_{2 n + 1} + α^{*} Δ p (n)}^{T} x (n)]}^{2}

(10)

Equation (10) highlights a very important property of the MNM algorithm. Now the optimization problem of $n$ dimensions has been reduced to finding an optimum value of the parameter $α$ , because it is the only unknown in equation (10). The value of $α^{*}$ is found by equation (6), which for the current problem is given by equation (11)

α^{*} = \frac{[d (n) - v_{c}^{T} (n) x (n)]}{{v_{m}^{T} (n) - v_{2 n + 1}^{T} (n)} x (n)}

(11)

Now the centroid for the new simplex is determined by equation (12)

v_{c}^{k + 1} (n) = v_{2 n + 1}^{k} (n) + α (v_{m}^{k} - v_{2 n + 1}^{k})

(12)

where $k$ refers to the kth iteration of the optimization routine.

Results and discussion

The performance of the proposed algorithm is evaluated by using an example of system identification problem given in Farhang-Boroujeny,⁴¹ which is described by block diagram in Figure 3, where $v (n)$ is zero mean, unit variance white gaussian noise, $x (n)$ is the output of coloring filter, and is applied as input to the system and the adaptive filter. The coloring filter, system, and the adaptive filter are given by equations (13), (14), and (15)

H (z) = \frac{\sqrt{1 - β^{2}}}{1 - β z^{- 1}}

(13)

P (z) = b_{0} + b_{1} z^{- 1} + b_{2} z^{- 2} + \dots + b_{N - 1} z^{- N + 1}

(14)

W (z) = w_{0} + w_{1} z^{- 1} + w_{2} z^{- 2} + \dots + w_{N - 1} z^{- N + 1}

(15)

where ${b_{i}, i = 0, 1, \dots, N - 1}$ are the system parameters, ${w_{i}, i = 0, 1, \dots, N - 1}$ are the taps of the adaptive filter and $0 \leq β \leq 1$ is a coloring parameter, which is used to control the eigen value spread of the correlation matrix of input $v (n)$ .

Figure 3.

System identification problem.

The aforementioned system identification problem is solved by using MNM, NM, LMS, and NLMS algorithms. The LMS and NLMS are used because they are widely used for such problems, moreover, NLMS is a bench mark for such applications. All the methods are online except NM, which is an offline technique requiring complete statistics of input for the system identification problem. The simulation results show comparison of above mentioned algorithms for identification of 2, 5, 10, and 25 taps FIR systems. The adptive filter taps are randomly initialized for all the techniques. The implementation of NM and MNM is presented in Algorithms 1 and 2 respectively, whereas, the pseudo-code for LMS and NLMS is given in Appendix. For simulations the simplex size for both MNM and NM algorithms is taken as $1$ , the maximum number of iterations allowed is $10, 000$ and the stopping criterion is $e_{r} < 0.005$ , where $e_{r}$ is defined by equation (16)

e_{r} = \frac{| w_{a} - w_{s} |}{w_{a}} \times 100

(16)

where $w_{a}$ represents optimum weights computed by Wiener Hopf’s equation⁴¹ given below as equation (17) and $w_{s}$ represents the simulated values of the filter taps.

W_{a} = R^{- 1} p

(17)

where, $R$ is the auto-correlation matrix of the input signal and “ $p$ ” is the cross-correlation vector of input “ $u$ ” and desired known signal “ $d$ .” The trajectories of filter taps for different algorithms are shown in Figures 4 to 7 for different values along with the contours of the performance surface.

Figure 4.

Effect of coloring on the trajectories of filter taps for different algorithms for $β = 0$ .

Figure 5.

Effect of coloring on the trajectories of filter taps for different algorithms for $β = 0.5$ .

Figure 6.

Effect of coloring on the trajectories of filter taps for different algorithms for $β = 0.75$ .

Figure 7.

Effect of coloring on the trajectories of filter taps for different algorithms for $β = 0.95$ .

The results in Figures 4 to 7 show the convergence of the algorithms in the presence of coloring effect in the input of the system and the adaptive filter. The coloring depends upon the value of $β$ in equation (13), when $β = 0$ , $x (n)$ is pure white noise, whereas, $x (n)$ is totally colored for $β = 1$ . The coloring is introduced to investigate the robustness of the aforementioned algorithms. It can be seen from the results that the number of iterations taken to reach the minimum value of the objective function $ζ_{\min}$ for all the algorithms vary by changing the amount of coloring in $x (n)$ . However, MNM and NM show more robustness to the coloring effect as compared to LMS and NLMS, especially LMS is very sensitive to the change in value of $β$ . The expression for $ζ_{\min}$ is given as

\begin{matrix} ζ_{\min} = E [d^{2} (n)] - p^{T} R^{- 1} p \\ where, \\ R = E [x (n) x^{T} (n)] \\ p = E [x (n) d (n)] \end{matrix}

where $R$ and $p$ are $N \times N$ autocorrelation matrix and $N \times 1$ cross correlation vector, respectively.

The quantitative analysis of the robustness of the algorithms is presented in Table 1 by computing standard deviation (std) of the number of iterations for all the techniques.

Table 1.

Effect of coloring on the number of iterations for all the algorithms.

$β$	Number of iterations
	MNM	NM	LMS	NLMS
	17	30	50	13
0.5	11	33	113	9
0.75	16	28	280	19
0.95	16	31	1910	12
Std	2.7	2.08	886.5	4.19

The results in Figures 8 and 9 show that the mean square error (MSE) for the algorithms decreases with the number of iterations. The error is computed by ensemble average of the sequence given by equation (8) over 100 simulation trials. It can be seen from Figures 8 and 9 that the performance of MNM is better than LMS and even comparable with NLMS. However, the NM algorithm shows poor performance and converges to a relatively higher value of MSE as compared to its counterparts. The robustness of MNM and NLMS due to the coloring effect can also be observed from the results.

Figure 8.

Learning curve plot of the algorithms for 2-taps system for $β = 0$ .

Figure 9.

Learning curve plot of the algorithms for 2-taps system for $β = 0.5$ .

Figures 10 to 13 show similar results for $5$ and $10$ taps adaptive filters. The simulation results show that the convergence rate of MNM algorithm is inversely proportional to the number of filter taps, however, the convergence of the algorithm is not effected by increasing the taps. Figure 14 demonstrate the estimation of 25-taps FIR filter using MNM and NM algorithms. These results show that the MNM algorithm gives sufficient performance for a large scale optimization problem, whereas, the conventional NM fails to solve the problems of higher dimensionality. The LMS and NLMS algorithms also estimate the filter taps successfully.

Figure 10.

Learning curve plot of the algorithms for 5-taps system for $β = 0$ .

Figure 11.

Learning curve plot of the algorithms for 5-taps system for $β = 0.5$ .

Figure 12.

Learning curve plot of the algorithms for 10-taps system for $β = 0$ .

Figure 13.

Learning curve plot of the algorithms for 10-taps system for $β = 0.5$ .

Figure 14.

Estimation of 25 taps LP FIR filter.

Conclusion

A novel modified optimization technique based on NM method has been proposed that transforms $n$ -dimensional optimization problem in to one-dimensional probelm for finding the optimum step size ( $α *$ ). Unlike NM the step size in MNM is computed analytically by gradient descent approach. An online implementation of the MNM is also proposed and is applied on the problem of online system identification. The performance of the proposed algorithm has been compared with LMS, NLMS, and NM. For the small scale optimization problems MNM outperforms NM and LMS, and its performance is quite similar to NLMS, which is considered as a bench mark for adaptive filter based online parameter estimation. The algorithm also shows robustness against the coloring in the input signal. However, for large scale optimization problem the convergence rate of MNM decreases.

The implementation of MNM on nonlinear optimization problems will further validate its effectiveness.

Footnotes

Appendix

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Sardar Muhammad Gulfam

Author biographies

Armaghan Mohsin is serving as Lecturer in Department of Physics at COMSATS University Islamabad (CUI), Islamabad, Pakistan. He is an experienced teacher and researcher and members of many societies and clubs including physics adventure club. His research interest include Signal processing and Optimization algorithms.

Yazan Alsmadi is a young distinguished scientist received the B.S. degree in Electrical Power Engineering from Yarmouk University, Jordan, and the M.S. and PhD degrees in Electrical and Computer Engineering from The Ohio State University (OSU), Columbus, OH, USA, in 2012 and 2015, respectively. He is currently an Assistant Professor in the Electrical Engineering Department at Jordan University of Science and Technology.

Ali Arshad Uppal received the MS degree in Computer Engineering from the University of Engineering and Technology Taxila, Pakistan, in 2012, and the Ph.D. degree in Electrical Engineering from the COMSATS Institute of Information Technology (CIIT), Islamabad, Pakistan, in 2016. He is also an Assistant Professor at COMSATS University Islamabad.

Sardar Muhammad Gulfam received his master’s degree from Tampere University of Technology Finland in 2010. He received his PhD degree in 2017 from COMSATS University Islamabad. Currently, he is working as an Assistant professor in the Department of Electrical and Computer Engineering, COMSATS University Islamabad, Islamabad, Pakistan.

References

Walters

Lloyd

Jr Morgan

, et al. Sequential simplex optimization: a technique for improving quality and productivity in research, development, and manufacturing (chemometrics series), 1991.

Nelder

Mead

. A simplex method for function minimization. Comput J 1965; 7(4): 308–313.

Caniyilmaz

Uçarkuş

Karaman

. Optimization of formulation ingredients and aging time for ice cream processing using combined design approach. J Food Process Preserv 2016; 40(6): 1325–1338.

Ijaz

Hamayun

Yan

, et al. Fractional order modeling and control of twin rotor aero dynamical system using nelder mead optimization. J Electr Eng Technol 2016; 11(6): 1863–1871.

Kear

Evans

Ellis

, et al. Computational aerodynamic optimisation of vertical axis wind turbine blades. Appl Math Model 2016; 40(2): 1038–1051.

Zhang

Shi

, et al. Microelectromechanical resonant accelerometer designed with a high sensitivity. Sensors 2015; 15(12): 30293–30310.

Kim

Bowen

Ozelkan

. Optimization of wastewater treatment plant operation for greenhouse gas mitigation. J Environ Manage 2015; 163: 39–48.

Cerda

Idris

. Optimization using the gradient and simplex methods. Talanta 2016; 148: 641–648.

Nobahari

Zandavi

Mohammadkarimi

. Simplex filter: a novel heuristic filter for nonlinear systems state estimation. Appl Soft Comput 2016; 49: 474–484.

10.

Bezerra

dos Santos

Santos

, et al. Simplex optimization: a tutorial approach and recent applications in analytical chemistry. Microchem J 2016; 124: 45–54.

11.

Mansoor

Nawaz

Gulfam

. Massive-mimo sparse uplink channel estimation using implicit training and compressed sensing. Appl Sci 2017; 7(1): 63.

12.

Gulfam

Nawaz

Ahmed

, et al. A novel 3d analytical scattering model for air-to-ground fading channels. Appl Sci 2016; 6(8): 207.

13.

Torczon

. On the convergence of the multidirectional search algorithm. SIAM J Optim 1991; 1(1): 123–145.

14.

Lagarias

Reeds

Wright

, et al. Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM J Optim 1998; 9(1): 112–147.

15.

McKinnon

. Convergence of the nelder–mead simplex method to a nonstationary point. SIAM J Optim 1998; 9(1): 148–158.

16.

Lagarias

Poonen

Wright

. Convergence of the restricted nelder–mead algorithm in two dimensions. SIAM J Optim 2012; 22(2): 501–532.

17.

Han

Neumann

. Effect of dimensionality on the Nelder-Mead simplex method. Optim Methods Softw 2006; 21(1): 1–16.

18.

Wright

. (1996). Direct search methods: Once scorned, now respectable. In: Numerical analysis: Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis (eds Griffiths

Watson

), pp. 191–208. USA: Addison-Wesley.

19.

Price

Coope

Byatt

. A convergent variant of the Nelder-Mead algorithm. J Optim Theory Appl 2002; 113(1): 5–19.

20.

Gao

Han

. Implementing the Nelder-Mead simplex algorithm with adaptive parameters. Comput Optim Appl 2012; 51(1): 259–277.

21.

Mehta

Dasgupta

. A constrained optimization algorithm based on the simplex search method. Eng Optim 2012; 44(5): 537–550.

22.

Sudheer

Raghavendra

Manjunath

. An online, derivative-free optimization approach to auto-tuning of computing systems. In: International Conference on Distributed Computing and Networking, 2010. Berlin, Heidelberg: Springer.

23.

Audet

Hare

. Derivative-free and blackbox optimization. Switzerland: Springer, 2017. doi:10.1007/978- 3-319-68913-5.

24.

Nawaz

Sharma

Wyne

, et al. Quantum machine learning for 6g communication networks: state-of-the-art and vision for the future. IEEE Access 2019; 7: 46317–46350.

25.

Xiong

Jutan

. Continuous optimization using a dynamic simplex method. Chem Eng Sci 2003; 58(16): 3817–3828.

26.

Marsililibelli

Castelli

. An adaptive search algorithm for numerical optimization. Appl Math Comput 1987; 23(4): 341–357.

27.

Musafer

Mahmood

. Dynamic hassan nelder mead with simplex free selectivity for unconstrained optimization. IEEE Access 2018; 6: 39015–39026.

28.

Yıldız

Sait

, et al. A new hybrid harris hawks-nelder-mead optimization algorithm for solving design and manufacturing problems. Mat Test 2019; 61(8): 735–743.

29.

Yildiz

. A novel hybrid whale-nelder-mead algorithm for optimization of design and manufacturing problems. Int J Adv Manuf Technol 2019; 105(12): 5091–5104.

30.

Meng

Wang

, et al. A comparative study of metaheuristic algorithms for reliability-based design optimization problems. Arch Comput Methods Eng 2021; 28: 1853–1869.

31.

Yıldız

Erdaş

. A new hybrid taguchi-salp swarm optimization algorithm for the robust design of real-world engineering problems. Mat Test 2021; 63(2): 157–162.

32.

Yıldız

Pholdee

Bureerat

, et al. Robust design of a robot gripper mechanism using new hybrid grasshopper optimization algorithm. Expert Syst 2021; 38(3): e12666.

33.

Gulfam

Nawaz

Ahmed

, et al. Angle and time of arrival characteristics of 3d air-to-ground radio propagation environments. Comput Commun 2017; 112: 22–37.

34.

Gao

Han

. Implementing the Nelder-Mead simplex algorithm with adaptive parameters. Comput Optim Appl 2012; 51(1): 259–277.

35.

Baghmisheh

MTV

Peimani

Sadeghi

, et al. A hybrid particle swarm-Nelder-Mead optimization method for crack detection in cantilever beams. Appl Soft Comput 2012; 12(8): 2217–2226.

36.

Karimi

Siarry

. Global simplex optimization – a simple and efficient metaheuristic for continuous optimization. Eng Appl Artif Intell 2012; 25(1): 48–55.

37.

Zahara

Kao

. Hybrid Nelder-Mead simplex search and particle swarm optimization for constrained engineering design problems. Expert Syst Appl 2009; 36(2): 3880–3886.

38.

Wang

. Parameter identification of chaotic systems by hybrid Nelder-Mead simplex search and differential evolution algorithm. Expert Syst Appl 2011; 38(4): 3238–3245.

39.

Chen

Lin

, et al. Parameters identification of photovoltaic models using hybrid adaptive Nelder-Mead simplex algorithm based on eagle strategy. Appl Energy 2016; 182: 47–57.

40.

Spendley

Hext

Himsworth

. Sequential application of simplex designs in optimisation and evolutionary operation. Technometrics 1962; 4(4): 441–461.

41.

Farhang-Boroujeny

. Adaptive filters: theory and applications. 1st ed. New York, NY: John Wiley & Sons, 1998.

A modified simplex based direct search optimization algorithm for adaptive transversal FIR filters

Abstract

Keywords

Introduction

Nelder-Mead

Modified Nelder Mead

Parameter estimation using MNM: An online implementation

Problem statement

Problem formulation

Results and discussion

Conclusion

Footnotes

Appendix

Declaration of conflicting interests

Funding

ORCID iD

Author biographies

References