Structural parameter optimization of radial basis function neural network based on improved genetic algorithm and cost function model

Abstract

This paper investigates the structural parameter optimization of RBF networks with the goal of economic control. The cost function and its implementation method are analyzed, and the cost function model of RBF neural network is established. The weights of RBF neural network are determined using recursive least squares method, then the crossover and mutation operators of genetic algorithm are improved and a new adaptive genetic algorithm is designed to implement the economic control of RBF neural network. The optimized network structure parameters are applied to the RBF neural network for simulation through function example, and the results obtained are compared with those of ordinary RBF network training. It shows that the method proposed in this paper is superior in both error and prediction values.

Keywords

Genetic algorithm RBF neural network cost function recursive least squares method structural parameter

Introduction

Selecting the scheme with the least equipment investment and the lowest system operation cost under the given control technical performance requirements and in various designs that are technically feasible, is the prototype definition of economic control. However, the research of control theory usually focuses on the technical performance of the system, such as controllability,^1,2 observability,^3–5 stability,^6,7 rapidity,⁸ accuracy,⁹ etc., while the economy of the system, that is, the economic cost paid to realize the control process, lacks due research and attention. “Economic control” is to select the scheme with the least equipment investment and the lowest system operating cost among various technically feasible designs under the given control technical performance requirements. In optimal control theory, the objective function is often the optimization of technical performance such as the highest control accuracy and the fastest control process, rather than the optimization of economic indicators with the least economic cost. The economical control is the optimal control with the technical performance index of the system as the constraint condition and the economic benefit as the objective function.

The existing economic cybernetics is based on the traditional control theory, with simple or local “hard” systems, linear relations, continuous events, and stability as the assumptions.^10–14 Usually, the research of control theory mainly focuses on the technical performance of the control system, such as controllability, observability, stability, robustness, rapidity, accuracy, reliability, etc., and rarely studies the economics of the control system, such as control equipment investment, system operating costs, and economic benefits after the system are put into operation.^15–17

Therefore, the main idea and method of solving the problem is to establish the mathematical model of the control object, and then use mathematical methods such as calculus or linear algebra to find the optimal solution. However, in reality, many practical systems are complex large-scale systems or giant systems, “soft” and “hard” flexible systems, and most of the system elements are highly nonlinear. These systems are time-varying and uncertain, some are discrete events, and some control is still non qualitative. Therefore, it is difficult to establish an accurate mathematical model to solve, or even impossible to solve by establishing a mathematical model. Existing research on intelligent control has not considered the economy and practicability of the control system. The performance indicators of the intelligent control system not only include the performance indicators of the traditional control system (such as stability, accuracy, rapidity, etc.) but also have some new performance indicators (such as intelligence). Expanding the research results of economic control to intelligent control, while applying intelligent control theory to economic control, and conducting research on the application of intelligent control, has opened up a new path for low-cost automation.

As can be seen from the above, economic control is to determine the optimal control structure parameter values when the economic cost is optimal. In view of this, this paper transforms the economic control implementation of radial basis function (RBF) neural networks into a structural optimization problem of neural networks. The cost function is used as the fitness function of the genetic algorithm and the determination of the weights of the neural network adopts the recursive least squares method. The improved adaptive genetic algorithm is used to optimize the structure of the RBF neural network, and the center value and width of the network are determined, thus achieving the goal of economic control of the neural network.

Cost function

RBF neural network (RBFNN) is a three-layer feedforward network:

(1) The first layer is the input layer, composed of signal source nodes, which connect the network with the external environment, and the number of nodes is determined by the dimensionality of the input signal;

(2) The second layer is the only hidden layer in the network, consisting of nodes with radial basis functions. Its function is to transform from the input space to the hidden layer space;

(3) The third layer is the output layer, which responds to input patterns, and its output nodes are calculated by linear combinations of basis functions given by hidden nodes.

The transformation from input space to hidden layer space is nonlinear, while the transformation from hidden layer space to output layer space is linear.

For a three-layer feedforward network^18–20 with an input layer, a hidden layer, and an output layer, it is assumed that the input, hidden, and output layer have p, q, and k neurons respectively, as shown in Figure 1.

Figure 1.

The topological structure of RBFNN.

In Figure 1, a = [a₁, a₂, …, a_p]^T $\in R^{p}$ is the input vector, $W \in R^{q \times k}$ is the output weight matrix and v = [v₁, v₂, …, v_k]^T is the output vector. Here, $ϕ_{p} (| | • | |)$ is the activation function of the p-th hidden layer node, c_p is the data center value of the p-th hidden layer node, and ||•|| is the distance between input vector and center vector which is Euclidean distance in this study. In RBFNN, the hidden layer nonlinearly maps the input space to a new space, and the output layer implements linear combinations in this new space. The adjustable parameters are the weights of these linear combinations, and the entire network performs a nonlinear transformation from R^p to R^k.

The number of samples to be trained is $M$ , and $Z$ and $V$ represent the actual and target matrices respectively. The input sample matrix is $a = (a_{1}, a_{2}, \dots, a_{M})$ , $K_{0}$ is the weight matrix between input layer and hidden layer containing the hidden layer threshold, and $K_{1}$ is the weight matrix between the hidden layer and the output layer. The net input of the hidden layer is (K₀)^TZ₀, and the net input of the output layer is (K₁)^TZ₁.

A = (a₁, a₂, …, a_s) is the given sample input, so the trained network should minimize the cost function:

cost = \min {({| | Z - V | |}_{F})}^{2}

(1)

where ||Z-V||_F is the Frobevenins norm of Z-V.

The dimensions of any given sample pair: A = (a₁, a₂, …, a_s) and V = (v₁, v₂, …,v_i) are p and k. (1) the number of neurons in the input and output layers is equal to the dimension of the input and output sample vectors respectively, (2) the transformation relationship of neurons is identity transformation, (3) the number of neurons in the hidden layer is equal to the total number of samples M, so the transformation relation f(a) is a zero-limited function, then there is a three-layer feedforward network that makes cost = min(||Z-V||_F)² = 0.

When the hidden layer output matrix Z is regarded as a constant, cost = (||V(Z⁺Z-I_M)||_F)² where I_M is the identity matrix, and Z⁺ is the Moor-Penrose generalized inverse of Z. This formula can be viewed as a function of variable Z.

If the transformation function $f (a)$ of the hidden layer of the three-layer feedforward neural network is a similar support set function and it is assumed that any two of the input samples are not equal to each other, there must be a free weight makes the hidden layer output matrix $Z$ strictly diagonally dominant. The Gauss function meets the requirements of similar support set function. When the number of nodes in the hidden layer of the network is equal to the number of samples, it is beneficial to realize accurate mapping without error in the network. Therefore, the Gaussian function is used as the radial basis function, and the number of hidden layer nodes is equal to the total number of samples, to achieve the lowest cost of the neural network.

The cost function of RBF Network is constructed as:

\cos t_{RBF} = \frac{1}{2 M} \sum^{M} \sum^{k} {(Outpu t_{ij} - AOutpu t_{ij})}^{2}

(2)

where Output_ij is the output of the RBF network for the $j$ th neuron on the i-th sample and AOutput_ij is the actual output of the RBF network for the j-th neuron on the $i$ th sample. Here, $1 \leq i \leq M$ and $1 \leq j \leq k$ . The quality of the network performance is determined by this cost function.^21–24

RLS learning algorithm for weight determination of RBFNN

For the learning of the weights W of RBFNN, this paper adopts the Recursive Least Square (RLS) algorithm.^25–29 The RLS learning algorithm corresponding to RBFNN is shown in Figure 2.

Figure 2.

RLS learning algorithm for weight determination of RBFNN.

Economic control implementation

The choice of the crossover rate and the mutation rate in the genetic algorithm is the key to the behavior and performance of the genetic algorithm, which directly affects the convergence of the algorithm. The crossover operator is mainly used to generate new individuals and realize the global search of the algorithm. It plays a key role in the genetic operation. The larger the crossover rate, the faster new individuals are generated. However, when the crossover rate is too large, it is more likely that the genetic model will be destroyed, so that the structure of individuals with high fitness will be destroyed soon; but if the crossover rate is too low, more individuals will be directly copied to the next generation, and the genetic search can get bogged down in sluggishness. Mutation operators are mainly used to maintain population diversity and prevent premature convergence. In the early stage of the algorithm, individuals in the population have relatively rich diversity, so the mutation probability should be set to a small value, thus improving the calculation speed of the algorithm. If the mutation rate is too small, it is difficult to generate a new individual structure, but if the mutation rate is too large, the algorithm becomes a pure random search and cannot reflect the characteristics of the genetic algorithm, and it is difficult to generate new genes, resulting in the premature algorithm. The selection of these two parameters by the standard genetic algorithm is generally determined by experience and trial and error, and the values are fixed. Usually, for the case where the mutation rate is constant, after many iterations, the quality of the population will tend to be consistent, thus forming inbreeding. According to the adaptive genetic algorithm, the crossover probability and mutation probability are directly adjusted linearly with the individual fitness in the population evaluation fitness and maximum fitness. When the fitness is closer to the maximum fitness, the crossover probability and mutation probability are smaller; when the fitness value is close to or equal to the individual with the maximum fitness value, the crossover probability, and mutation probability are close to or equal to zero. Because in the population in the early stage of evolution, the superior individuals are almost in a state of no change, and the superior individuals at this time are not necessarily the global optimal solution, which easily increases the possibility of evolution toward local convergence, the adaptive genetic algorithm is not ideal in the early stage of evolution.

This paper adopts the improved operator of genetic algorithm crossover and mutation. In this paper, the improved adaptive genetic algorithm is used to select and optimize the network center and width parameters, and the weight coefficient of the neural network is realized by the recursive least square method. The improvement of GA mainly includes the introduction of the adaptive genetic algorithm of floating-point coding, the adaptive adjustment of the exchange rate and the mutation rate, and the radial basis function neural network genetic optimization using the improved genetic algorithm as the learning algorithm of the RBF neural network is proposed. The algorithm improves the efficiency of global search optimization.

The genetic algorithm proposed in this paper first realizes the real numbering of the number string in GA, that is, the decimal number string replaces the binary number string to directly represent the parameters, to avoid the difficulty of coding due to the unclear value range of the network; Cancel The encoding and decoding process is improved, and the speed of the learning algorithm is improved; when the length of the digital string is unchanged (the calculation amount of the GA is unchanged), the calculation accuracy can be greatly improved. In this algorithm, the width of the RBF network is same as the center, which is obtained through the self-correction of the genetic algorithm. The center vector $c$ and the width $σ$ are each represented by a chromosome. The dimensionality of input vectors is p, the number of hidden nodes is q, and the population size is M, so the length of a center vector chromosome is q × p, and the center vector is a M × (q × p) matrix; the length of a width chromosome is q, and the width chromosome is a M × q matrix. Using the genetic algorithm of floating-point coding with the introduction of the adaptive mechanism as the learning algorithm of the RBF network can further improve the prediction accuracy of the network and improve the efficiency of global search optimization.

The scope of the random optimization search process of the genetic algorithm is basically within the initialized population range, and only mutation can make it jump out of the initial search space, so the initialization of the population is very important. Since the radial basis function center vector c and the width σ are separately encoded, they are also different in the initial population. A center parameter is initially a real number within the range of the input sample data, and the center width is initialized as a real number in the $[1, 4]$ interval.

The fitness function is a function to evaluate the quality of each chromosome in the population. The larger the value of the chromosome fitness function, the more likely it will be selected into the population, so the selection of the fitness function is very important. This paper adopts the method of using the RBF neural network cost function as the fitness function:

f = {\begin{matrix} \max (\cos t_{RBF}) - \cos t_{RBF} & \cos t_{RBF} < \max (\cos t_{RBF}) \\ 0 & \cos t_{RBF} \geq \max (\cos t_{RBF}) \end{matrix}

(3)

To maintain the stability and global convergence of the algorithm, we choose the best retention selection mechanism. First, the selection is performed according to the wheel selection mechanism, and then the individual with the highest fitness in the current solution is copied to the next generation population to ensure genetic inheritance. The final result obtained when the algorithm terminates must be the individual with the highest fitness that has appeared iteratively.

Crossover operation is the main search method of genetic algorithm. It combines with copy operation to make genetic algorithm have strong searchability. The mutation operation is an auxiliary search operation in the genetic algorithm, and its main purpose is to maintain the diversity of the population. In the operation of a genetic algorithm, if the crossover and mutation operators can be adjusted adaptively, it can not only enhance the search efficiency of GA but also expand the search space. Therefore, this paper adopts adaptive crossover and mutation operators. The implementation process of genetic algorithm to optimize RBFNN is shown in Figure 3.

Figure 3.

The implementation process of genetic algorithm to optimize RBFNN.

According to the theory of network cost function, the number of hidden layer nodes and radial basis function is selected. An adaptive genetic algorithm was used to optimize the neural network to obtain the best chromosome of the neural network, namely, the best center value $c$ and the best width value $σ$ of the RBF neural network. The weight of the neural network is learned by the recursive least square method so that the structure and parameters of the RBF network are completely determined, the optimized network is applied to the design, and the economical control can be realized.

Case study

Trigonometric function case

This paper mainly studies the problem of determining the optimal performance index of RBFNN for the economical control of logistics system. What to be optimized is the structural parameters of RBFNN, and the optimized evaluation index is the network cost of the control object. Therefore, the experimental result is that the network cost is the smallest. Corresponding network structure parameters. Since the direct cost of neural network control of sines and cosines is very small, the economic cost function of sines and cosines can be simplified to the benefit of the neural network control system, that is, the cost function model of the network can be regarded as the economic cost model of sines and cosines neural network control. This paper uses the function $y = \sin x + \cos x, x \in [0, 10]$ , and divides the sample into 50 sample intervals to generate the input and output data as the economical control research simulation object.

To test the superiority of the designed algorithm, the control simulation results of the economical control are compared with the results of ordinary RBF network control. The number of hidden layer nodes for both algorithms is chosen to be the same, both being 150. When using the economical control algorithm, the evolutionary generation is 1000 generations and the population size is 1000.

We used the economical control method described in the above to train the network, and recorded the optimal center value and width of the network to achieve the training target. The optimal parameters obtained are shown in Tables 1 and 2, where $c$ is the center of the network, and $σ$ is the network width.

Table 1.

The center of the network.

Parameter	Optimal value
$c_{1}$	0.8137
$c_{2}$	0.4825
$c_{3}$	0.1891
$c_{4}$	0.6549
$c_{5}$	0.8535
$c_{6}$	0.5475
$c_{7}$	0.8387
$c_{8}$	0.7861
$c_{9}$	0.9442
$c_{10}$	0.5314
$c_{11}$	0.7252
$c_{12}$	0.6320
$c_{13}$	0.4125
$c_{14}$	0.3878
$c_{15}$	0.2309

Table 2.

The network width.

Parameter	Optimal value
$σ_{1}$	2.0080
$σ_{2}$	1.0246
$σ_{3}$	4.3942
$σ_{4}$	3.6324
$σ_{5}$	1.4886
$σ_{6}$	3.2176
$σ_{7}$	1.3265
$σ_{8}$	1.2616
$σ_{9}$	1.0991
$σ_{10}$	2.5303
$σ_{11}$	3.1023
$σ_{12}$	4.2069
$σ_{13}$	1.2205
$σ_{14}$	2.3321
$σ_{15}$	1.8523

Since this paper takes the function as the research object, the input and output of the network are only 1, and the number of hidden nodes is selected as 15, so the center and width of the network are both $1 \times 15$ matrices. The optimized center value and width are used for simulation in the neural network, and the obtained results shows the order of magnitude of the economical control error is 1e-6. In detail, the error in the economical control learning is 9.2e-6, while the mean square error in the RBF network control learning is 0.005.

In terms of prediction, the value after $x = 10$ is regarded as the generalization ability test of the algorithm. By the predicted value comparison of the economical control and RBF network, it can be concluded that the algorithm designed in this paper can obtain a better fitting effect and has better generalization ability.

Discussions

With the development of neural network in theory and application, its application in all aspects has attracted more and more attention. Because of its strong approximation ability, neural network has been successfully applied to the establishment of nonlinear complex system models in various fields. However, in practical applications, the determination of the structure of neural network is still a thorny problem. This paper discusses the structure and parameter determination of RBF network in economic control. Ordinary genetic algorithm can no longer meet the demand, so the crossover and mutation operators that can be adjusted automatically are improved, and the improved genetic algorithm and the network cost function are used to realize the economical control of the RBF network. The performance of the economical control is tested by trigonometric function, and the best center value and width value of the network are obtained. From the economical control and RBF network control results, it can be seen that the economical control error is significantly smaller than that of RBF network, and the approximation effect and recursion are better. The feasibility of the designed algorithm can be verified through function examples.

Conclusions

In this paper, we use the improved adaptive genetic algorithm to optimize the structure of the RBF neural network, and determine the center value and width of the network. At last, the goal of economic control of the neural network is achieved. Economic control is currently an effective method for cost research, which can achieve optimal efficiency. However, due to the complexity of the actual system, the established model may have many unknown factors that have not been taken into account, resulting in certain errors. How to apply the network cost as well as possible to the neural network, fully utilize the advantages of the network cost function, and conveniently select the parameters of the neural network is a very worthwhile research problem. Moreover, when the research sample is large, the selection of the number of hidden layer nodes will be inconvenient. These will all become future research directions.

Footnotes

Handling Editor: Chenhui Liang

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is funding by the National Natural Science Foundation of China (Grant No: 52165061) and the Research Initiation Project of Wenzhou Polytechnic (Grant No: RC202307).

ORCID iD

Lianhui Li

References

Qiao

Efficiency measurement of urban and rural logistics supply chain system based on fuzzy algorithm. Comput Intell Neurosci 2022; 2022: 4753343.

Emadi

Sadeghian

Rabbani

, et al. A machine learning approach to the optimal control of the customer dynamics. J Appl Res Ind Eng 2024; 11: 212–228.

Radmanesh

Ebadi

MJ.

A local meshless radial basis functions based method for solving fractional integral equations. Comput Algorithms Numer Dimens 2023; 2: 35–46.

Hang

Gao

, et al. Using an integrated group decision method based on SVM, TFN-RS-AHP, and TOPSIS-CD for cloud service supplier selection. Math Probl Eng 2017; 2017: 3143502.

Mohamed

. Toward smart logistics: hybrization of intelligence techniques of machine learning and multi-criteria decision-making in Logistics 5.0. Multicriteria Algorithms Appl 2023; 1: 42–57.

Munyeshuri

Song

Akoko

, et al. Strategic application of cooperative game theory in mitigating labor shortages in post-pandemic logistics: a case study of Poland. J Oper Strateg Anal 2023; 1: 214–224.

Hang

Sun

, et al. A conjunctive multiple-criteria decision-making approach for cloud service supplier selection of manufacturing enterprise. Adv Mech Eng 2017; 9: 1687814016686264.

Ajalli

Saberifard

Zinati

Evaluation and ranking of performance indicators in humanitarian logistics using path analysis, fuzzy DEMATEL and SWARA. J Decis Oper Res 2022; 6: 1–20.

Nozari

Edalatpanah

SA.

Smart systems risk management in IoT-based supply chain. In: Garg

(ed.) Advances in reliability, failure and risk analysis. Industrial and Applied Mathematics. Singapore: Springer, 2023, pp.251–268.

10.

Zhang

Coastal ecological environment monitoring and protection system based on multisource information fusion decision. J Sens 2021; 2021: 5194700.

11.

Lei

Mao

Digital twin in smart manufacturing. J Ind Inf Integr 2022; 26: 100289.

12.

Liu

, et al. Sustainability assessment of intelligent manufacturing supported by digital twin. IEEE Access 2020; 8: 174988–175008.

13.

Fang

Feng

Qiu

, et al. Rail multi vehicle scheduling method for intermediate depot of steel plant based on improved genetic algorithm. Sci Program 2022; 2022: 4971638.

14.

Mao

Big data supported PSS evaluation decision in service-oriented manufacturing. IEEE Access 2020; 8: 154663–154670.

15.

Digital twin-driven performance optimization for hazardous waste landfill systems. Math Probl Eng 2022; 2022: 7778952.

16.

Mao

Sun

, et al. Digital twin driven green performance evaluation methodology of intelligent manufacturing: hybrid model based on fuzzy rough-sets AHP, multistage weight synthesis, and PROMETHEE II. Complexity 2020; 2020: 1–24.

17.

Zheng

Estimation of disease transmission in multimodal transportation networks. J Adv Transp 2020; 2020: 8898923.

18.

Wang

Liang

Cheng

, et al. Emission reduction benefits and economic benefits of China’s pilot policy on carbon emission trading system. Comput Intell Neurosci 2022; 2022: 5280900.

19.

Sohrabi

Pouri

Sabk Ara

, et al. Applying sustainable development to economic challenges of small and medium enterprises after implementation of targeted subsidies in Iran. Math Probl Eng 2021; 2021: 2270983.

20.

Liu

Zhu

CAD fractal art pattern design and evaluation based on RBF network algorithm. Adv Multimed 2022; 2022: 2736855.

21.

Wang

Multimedia vocal performance automation evaluation model based on RBF network. Math Probl Eng 2022; 2022: 3868389.

22.

Cheng

Zhao

The inverse solution algorithm and trajectory error analysis of robotic arm based on MQACA-RBF network. J Robot 2020; 2020: 7807952.

23.

Xiao

Luo

The application of RBF neural network model based on deep learning for flower pattern design in art teaching. Comput Intell Neurosci 2022; 2022: 4206857.

24.

Yang

Huang

An improved quality evaluation method for foreign trade English using GA-RBF neural network. Mob Inf Syst 2022; 2022: 3329908.

25.

Jiang

Prediction of short-term stock price trend based on multiview RBF neural network. Comput Intell Neurosci 2021; 2021: 8495288.

26.

Tan

Gao

Higher education teaching quality evaluation model based on improved RBF neural network. Wirel Commun Mob Comput 2022; 2022: 5495728.

27.

Yang

Risk prediction algorithm of social security fund operation based on RBF neural network. Int J Antennas Propag 2021; 2021: 6525955.

28.

Zhang

Research on the formation mechanism of MgO and Al2O3on composite calcium ferrite based on DA-RBF neural network. Comput Intell Neurosci 2022; 2022: 4327969.

29.

Liu

Yang

Zhang

, et al. Automatic generation of test cases based on genetic algorithm and RBF neural network. Mob Inf Syst 2022; 2022: 1489063.