Structural reliability analysis of multiple limit state functions using multi-input multi-output support vector machine

Abstract

Selecting and using an appropriate structural reliability method is critical for the success of structural reliability analysis and reliability-based design optimization. However, most of existing structural reliability methods are developed and designed for a single limit state function and few methods can be used to simultaneously handle multiple limit state functions in a structural system when the failure probability of each limit state function is of interest, for example, in a reliability-based design optimization loop. This article presents a new method for structural reliability analysis with multiple limit state functions using support vector machine technique. A sole support vector machine surrogate model for all limit state functions is constructed by a multi-input multi-output support vector machine algorithm. Furthermore, this multi-input multi-output support vector machine surrogate model for all limit state functions is only trained from one data set with one calculation process, instead of constructing a series of standard support vector machine models which has one output only. Combining the multi-input multi-output support vector machine surrogate model with direct Monte Carlo simulation, the failure probability of the structural system as well as the failure probability of each limit state function corresponding to a failure mode in the structural system can be estimated. Two examples are used to demonstrate the accuracy and efficiency of the presented method.

Introduction

During the past few decades, structural reliability methods have been gained increasing interest for rational treatment of the uncertainties in engineering structures. Despite the structural reliability community has achieved many advanced development on the theoretical research, serious computational obstacles arise when involving practical problems. For example, it usually involves the reliability assessment of multiple limit state functions (LSFs) in reliability-based design optimization (RBDO) of a structure, and many structures may have multiple failure modes which result in multiple LSFs. A structural reliability method capable of dealing multiple LSFs from the same system with a single run is more preferable for RBDO problems and multiple-LSF problems. When one examines the existing structural reliability methods, an importance fact is found that most of them are developed for a single LSF, for example, the most commonly used first-order reliability method (FORM) and second-order reliability method (SORM).^1–5 Although one may repeat to apply the existing structural reliability methods on each LSF, the computational and development effort cannot meet the demands in many practical cases. Therefore, the study of structural reliability methods which can deal with multiple LSFs simultaneously has progressively attracted attention recently, especially on computer-based simulation methods. Due to its excellent universality, direct Monte Carlo simulation (MCS) is suitable for a problem with multiple LSFs. However, the huge computational effort for small reliability level hinders its applications, just like the situation for a single LSF.^6–8 Based on subset simulation (SS), ⁶ Hsu and Ching ⁹ developed a simulation algorithm for the failure probabilities of multiple LSFs. In their algorithm, a principle variable is proposed to correlate with all LSFs of interest and drive the simulation to gradually approach the multiple failure regions. However, it is a non-trivial task for the determination of a proper principle variable. Li et al. ¹⁰ proposed a generalized SS to use unified intermediate events to resolve the sorting difficulty in the original one. In general, the generalized SS is much easy to carry out a reliability analysis of multiple LSFs simultaneously, compared with Hsu and Ching’s method. Like MCS, SS is also very time-consuming when structural analyses involve large numerical models, for example, finite element models. ¹¹

In practical problems, LSFs are established based on certain structural failure mechanism, for example, stress, displacement, and fatigue, and do not have analytical expressions. Furthermore, the failure regions usually possess complicated geometry and boundaries. In order to reduce the computational effort of structural reliability analysis, surrogate models, for example, response surface method (RSM),^12–20 artificial neural networks (ANNs),^21–24 and support vector machine (SVM),^25–29are generally suggested to approximate the actual LSF in a structural reliability problem. RSM may be the most popular methods among them. It aims to fit an LSF around the so-called design point as near as possible. However, its approximate precision is greatly influenced by an LSF’s complexity, polynomial form and order, and location of supporting points.^11,21 It has been proved that even a response surface with accuracy as one wish may produce an erroneous estimation for failure probability. ³⁰ Moreover, RSM is not applicable to the situation of multiple LSFs. ANN is one of the popular alternatives to RSM. Various kinds of ANNs may estimate the failure probabilities of multiple LSFs simultaneously with the aid of its high parallelism. However, in the case of a small number of training samples, the estimated results are greatly influenced by the initial parameter setting, and the training process is easy to fall into local optimum because ANN is mainly based on the principle of empirical risk minimization (ERM).^31–33 Recently, Chojaczyk et al. ²² provided a comprehensive review on the application of ANN in structural reliability analysis.

SVM was first proposed by Cortes and Vapnik ³¹ in 1995, which reveals many unique advantages in pattern recognition with small samples, nonlinear and high dimensions. It was further extended for nonlinear regression by Vapnik.^32,33 From the point of view of statistical learning theory, SVM has several advantages over RSM and ANN for the approximation of an LSF. ²⁵ SVM adopts the structural risk minimization (SRM) principle rather than the ERM principle so that SVM has better generalization ability. In addition, there is no local optimal issue when searching the algorithm parameters in an SVM. Due to its superior performance, SVMs have been developed to combine with conventional structural reliability methods to establish new structural reliability methods. Those advantages mentioned previously can inhere in the structural reliability methods based on various SVMs. Hurtado and Alvarez ²⁵ proposed an interesting method composed of SVM and stochastic finite element method to deal with structural reliability analysis by means of regarding the problem of structural reliability analysis as a pattern recognition problem. By combining SVM with FORM and MCS, the two approaches, SVM-based FORM and SVM-based MCS, were proposed by Li et al. ²⁶ for structural reliability analysis. Guo and Bai ²⁷ suggested a method that combines the least square SVM with MCS. It indicates the method based on the least square SVM is superior to the method based on traditional SVM. Dai et al. ²⁸ developed a SVM-based importance sampling to perform structural reliability analysis, in which SVM is used to construct the sampling density for the optimal important sampling density to reduce the number of training samples. Jiang et al. ²⁹ paid a special attention to generating uniform support vector for the SVMs’ application on structural reliability analysis.

Although SVMs have been widely used in reliability community, a series of traditional SVM models which only have one output are needed to be constructed when dealing with multiple LSFs simultaneously. ³⁴ This strategy may have very low computational efficiency, because both actual LSFs and their surrogate models are required to be called for a large number of times during surrogate model construction and reliability estimation.

As an alternative to the traditional SVMs, multiple-input multiple-output SVMs (MIMO-SVMs)^35–38 are good choice since they can model the multiple-input multiple-output relationship between the input random variables and the multiple failure modes in a structural system and have been applied in many engineering and science fields. The newly developed multiple-task least-squares SVMs (MTLS-SVMs) which is proposed by Xu et al.^35,36 aims for establishing a surrogate model between input parameters and multiple outputs for multiple classification and regression problems. This article presents a new method for structural reliability analysis of multiple LSFs, which utilizes the recently developed MTLS-SVM. To our best knowledge, this is the first attempt to employ MTLS-SVM for structural reliability analysis of multiple LSFs. A new random sampling method, combining the Latin hypercube sampling (LHS) and uniform sampling (US), is also developed to generate the training data (supporting points) set with a good coverage of the input random space. Then, the MTLS-SVM is trained to be a single surrogate model for all LSFs in the problem of interest. Finally, based on the trained MTLS-SVM model, MCS is employed to estimate the failure probabilities of all LSFs simultaneously. The computational efficiency and accuracy of the presented method are also studied in this article.

SVM technique

In this section, some basic concepts of SVM are briefly reviewed. More details of SVM can be referred to Vapnik. ³² It is well known that SVM is a statistical learning algorithm based on SRM, which can be used for classification and regression problems. Here, we focus on the regression problems, that is, function fitting problems.

Considering a training data set

$$S=\{({\mathbf{x}}_1,y_1),({\mathbf{x}}_2,y_2),\dots ,({\mathbf{x}}_l,y_l)\},{\mathbf{x}}_i\in R^n,y_i\in R$$

(1)

where l is sample size, ${\mathbf{x}}_i$ is the input parameters, and y is the scalar output, respectively.

The nonlinear relationship between the inputs and output can be described by a regression function obtained from SVM theory

$$f\left(\mathbf{x}\right)=\phi {\left(\mathbf{x}\right)}^T\mathbf{w}+b=\sum _{{\mathit{x}}_i\in \mathrm{SV}}({\alpha }_i-{\alpha }_i^{*})K({\mathbf{x}}_i,{\mathbf{x}}_j)+b$$

(2)

where $f(\mathit{x})$ is the regression function, $\phi (\mathbf{x})$ is a mapping function for a higher dimensional Hilbert space, $\mathbf{w}$ is the control parameter of the hyperplane in the Hilbert space, ${\alpha }_i$ and ${\alpha }_i^{*}$ are the Lagrange multipliers, and $K({\mathbf{x}}_i,{\mathbf{x}}_j)$ is the kernel function which meets the Mercer condition,^32,33 respectively. Furthermore, a small portion of samples in the whole training data set have non-zero ${\alpha }_i$ and ${\alpha }_i^{*}$ . These samples, which are used to construct a regression function rather than the whole data set, are called support vectors (SVs). The values of ${\alpha }_i$ , ${\alpha }_i^{*}$ , and b can be obtained by minimizing the following dual objective function with constraints

$$\begin{array}{c}\min \frac{1}{2}\sum _{i,j=1}^l({\alpha }_i-{\alpha }_i^{*})({\alpha }_j-{\alpha }_j^{*})K({\mathbf{x}}_i,{\mathbf{x}}_j)\hfill \\ \multicolumn{1}{c}{+\epsilon \sum _{i=1}^l({\alpha }_i+{\alpha }_i^{*})-\sum _{i=1}^l{y}_i({\alpha }_i^{*}-{\alpha }_i)}\\ s.t.\sum _{i=1}^l({\alpha }_i^{*}-{\alpha }_i)=0,0\le {\alpha }_i,{\alpha }_i^{*}\le c\hfill \\ (i,j=1,2,\dots ,l)\hfill \end{array}$$

(3)

where c is a regularization parameter used to measure the complexity and the loss of compromise, and ε is a linear insensitive loss function. According to the Karush–Kuhn–Tucker (KKT) complementary conditions, equation (3) has a global optimal solution because SVM converts a regression problem into a quadratic optimization problem.

The kernel function $K({\mathbf{x}}_i,{\mathbf{x}}_j)$ is one of key components of SVM. In SVM, a nonlinear problem in the input space is converted into a linear problem in a high-dimensional feature space. The inner product in the feature space will be replaced by the kernel function $K({\mathbf{x}}_i,{\mathbf{x}}_j)$ which meets the Mercer condition. Therefore, the expression of nonlinear transformation will not be needed any more. Several kinds of commonly used kernel functions are given below: (1) liner kernel function $K({\mathbf{x}}_i,{\mathbf{x}}_j)=〈{\mathbf{x}}_i,{\mathbf{x}}_j〉$ ; (2) polynomial kernel function $K({\mathbf{x}}_i,{\mathbf{x}}_j)=(〈{\mathbf{x}}_i,{\mathbf{x}}_j〉+1{)}^q$ ; (3) radial basis kernel function $K({\mathbf{x}}_i,{\mathbf{x}}_j)=\exp (-\Vert {\mathbf{x}}_i·{\mathbf{x}}_j{\Vert }^2/{\sigma }^2)$ ; and (4) sigmoid kernel function $K({\mathbf{x}}_i,{\mathbf{x}}_j)=\tanh (v〈{\mathbf{x}}_i,{\mathbf{x}}_j〉+a)$ .

The least square MIMO-SVM

The above-mentioned SVM technique is able to be applied to a system with a single output only. Thus, it cannot be used for dealing with a complex system with multiple outputs, for example, the demand of dealing with multiple LSFs in an RBDO problem as in this study. To overcome this issue, various MIMO-SVM techniques have been proposed to meet this demand.^35–38 In this article, a MTLS-SVM^35,36 is employed to build up a single surrogate model which can approximate multiple LSFs.

Considering a system has m outputs, the training sample size is l. The training data set in equation (1) becomes as

$$\begin{array}{c}S=\left\{{({\mathbf{x}}_i,{\mathbf{y}}_i)}_{i=1}^l\right\},{\mathbf{x}}_i\in R^n,{\mathbf{y}}_i\in R^m\hfill \\ i=1,2,\dots ,l\hfill \end{array}$$

(4)

where n is the dimension of input parameters. According to the Hierarchical Bayesian,^39,40 the control parameter ${\mathbf{w}}_i$ can be expressed as ${\mathbf{w}}_i={\mathbf{w}}_0+{\mathbf{v}}_i$ , which reflects the relationship and difference among the outputs. Here, the mean vector ${\mathbf{w}}_0$ is small when the output quantities are very different to each other, otherwise the vector is small. Therefore, ${\mathbf{w}}_0$ embodies the common information among the outputs, and ${\mathbf{v}}_i$ embodies the information of the specialty of the ith output.

To solve the regression problem of multiple outputs system, MIMO-SVM determines ${\mathbf{w}}_0$ , ${\mathbf{v}}_i$ , and $b_i$ simultaneously by minimizing the following objective function with constraints

$$\begin{array}{c}\min \frac{1}{2}{\mathit{w}}_0^T{\mathit{w}}_0+\frac{1}{2}\frac{\lambda }{m}\sum _{i=1}^m{\mathit{v}}_i^T{\mathit{v}}_i+c\sum _{i=1}^m{\mathit{\xi }}_i^T{\mathit{\xi }}_i\hfill \\ s.t.{\mathit{y}}_i={\mathit{Z}}_i^T({\mathit{w}}_0+{\mathit{v}}_i)+b_i{1}_l+{\mathbf{\xi }}_{\mathbf{i}}\hfill \\ (i=1,2,\dots ,m)\hfill \end{array}$$

(5)

where λ and c are two positive real regularized parameters, $1^l=(1,\dots ,1{)}^{\mathrm{T}}\in R^l$ , ${\mathbf{Z}}_i=(\varphi ({\mathbf{x}}_{i,1}),\varphi ({\mathbf{x}}_{i,2}),\dots ,\varphi ({\mathbf{x}}_{i,l}))$ , ${\mathit{\xi }}_i=({\xi }_{i,1},{\xi }_{i,2},\dots ,{\xi }_{i,l}{)}^{\mathrm{T}}$ . $\sum _{i=1}^m{\mathit{\xi }}_i^{\mathrm{T}}{\mathit{\xi }}_i$ is a quadratic loss function instead of the linear insensitive loss function ε in section “SVM technique.”

The Lagrange function is introduced to solve the optimization problem in equation (6)

$$L({\mathit{w}}_0,{\mathit{v}}_i,\mathit{b},{\mathit{\xi }}_i,{\mathit{\alpha }}_i)=\frac{1}{2}{\mathit{w}}_0^{\mathrm{T}}{\mathit{w}}_0+\frac{1}{2}\frac{\lambda }{m}\sum _{i=1}^m{\mathit{v}}_i^{\mathrm{T}}{\mathit{v}}_i+c\sum _{i=1}^m{\mathit{\xi }}_i^{\mathrm{T}}{\mathit{\xi }}_i-\sum _{i=1}^m{\mathit{\alpha }}_i^{\mathrm{T}}({\mathbf{Z}}_i^{\mathrm{T}}({\mathit{w}}_0+{\mathit{v}}_i)+b_i{1}_l+{\mathit{\xi }}_i-{\mathit{y}}_i)(i=1,2,\dots ,m)$$

(6)

where ${\mathit{\alpha }}_i=({\alpha }_{i,1},{\alpha }_{i,2},\dots ,{\alpha }_{i,l}{)}^{\mathrm{T}}$ are the Lagrange multipliers. The KKT conditions for equation (6) are given by

$$\frac{\partial L}{\partial {\mathit{w}}_0}=0,\frac{\partial L}{\partial {\mathit{v}}_i}=0,\frac{\partial L}{\partial b_i}=0,\frac{\partial L}{\partial {\mathit{\xi }}_i}=0,\frac{\partial L}{\partial {\mathit{\alpha }}_i}=0(i=1,2,\dots ,m)$$

(7)

The following linear equations are obtained

$$\{\begin{array}{c}{\mathit{w}}_0=\mathit{Z}\mathit{\alpha }\hfill \\ \multicolumn{1}{c}{{\mathit{v}}_i=\frac{m}{\lambda }{\mathit{Z}}_i{\mathit{\alpha }}_i}\\ \multicolumn{1}{c}{\sum _{i=1}^m{\mathit{\alpha }}_i=0}\\ \multicolumn{1}{c}{{\mathit{\alpha }}_i=2c{\mathit{\xi }}_i}\\ \multicolumn{1}{c}{{\mathit{y}}_i={\mathit{Z}}_i^T({\mathit{w}}_0+{\mathit{v}}_i)+b_i{1}_l+{\mathit{\xi }}_i}\end{array}i=1,2,\dots ,m$$

(8)

where $\mathbf{Z}=({\mathbf{Z}}_1,{\mathbf{Z}}_2,\dots ,{\mathbf{Z}}_m)$ , and $\mathit{\alpha }=({\mathit{\alpha }}_1^{\mathrm{T}},{\mathit{\alpha }}_2^{\mathrm{T}},\dots ,{\mathit{\alpha }}_m^{\mathrm{T}}{)}^{\mathrm{T}}$ . Then, one has a matrix equation

$$\left[\begin{array}{cc}{0}_{m\times m}\hfill & {\mathbf{O}}^{\mathrm{T}}\hfill \\ \multicolumn{1}{c}{\mathbf{O}}& \mathbf{H}\hfill \end{array}\right]·\left[\begin{array}{c}\mathbf{b}\hfill \\ \multicolumn{1}{c}{\mathit{\alpha }}\end{array}\right]=\left[\begin{array}{c}{0}_m\hfill \\ \multicolumn{1}{c}{\mathbf{y}}\end{array}\right]$$

(9)

where $\mathbf{O}=(1_{l_1},1_{l_2},\dots ,1_{l_m})$ is a block diagonal matrix, the positive definite matrix $\mathbf{H}={\mathbf{Z}}^T\mathbf{Z}+(1/2c){\mathbf{I}}_l+(m/\lambda )\mathbf{B}$ , ${\mathbf{I}}_l$ is unitary matrix, and $\mathbf{B}=({\mathbf{K}}_1,{\mathbf{K}}_2,\dots ,{\mathbf{K}}_m)$ is a block diagonal matrix with the element ${\mathbf{K}}_i={\mathbf{Z}}_i^{\mathrm{T}}{\mathbf{Z}}_i$ , respectively. Let the solution to equation (9) be ${\mathit{\alpha }}^{*}=({\mathit{\alpha }}_1^{*\mathrm{T}},{\mathit{\alpha }}_2^{*\mathrm{T}},\dots ,{\mathit{\alpha }}_m^{*\mathrm{T}}{)}^{\mathrm{T}}$ and $\mathbf{b}=(b_1^{*},b_2^{*},\dots ,b_m^{*}{)}^{\mathrm{T}}$ . Then, the regression estimate functions can be expressed as

$$\begin{array}{l}{f}_i\left(x\right)=\varphi {\left(x\right)}^{\mathrm{T}}\left(w_0^{\ast }+v_i^{\ast }\right)+b_i^{\ast }\\ =\varphi {\left(x\right)}^{\mathrm{T}}\left(Z{\alpha }^{\ast }+\frac{m}{\lambda }{Z}_i{\alpha }_i^{\ast }\right)+b_i^{\ast }\\ ={\displaystyle \sum _{i^{\prime }=1}^m{\displaystyle \sum _{j=1}^l{\alpha }_{i^{\prime },j}^{\ast }K\left(x_{i^{\prime },j},x\right)}}+\frac{m}{\lambda }{\displaystyle \sum _{j=1}^l{\alpha }_{i^{\prime },j}^{\ast }K\left(x_{i,j},x\right)}+b_i^{\ast }\\ i=1,2,\dots ,m\end{array}$$

(10)

More details about the MIMO-SVM algorithm can be seen in Xu et al.^35,36

Structural reliability analysis based on MIMO-SVM

It is well known that LSF is the foundation of the structural reliability analysis. In this article, MIMO-SVM is used to construct a single surrogate model of all multiple LSFs through a single run. On the basis of this surrogate model, MCS is then employed to calculate the failure probabilities of all LSFs simultaneously. This section starts with the development of a new random sampling method. Then, the implementation procedure of the presented method is provided.

The generation of training samples

For most of surrogate model methods, the training sample size and position are critical to construct an appreciate surrogate model which can meet the requirements of efficiency and accuracy. Although SVM techniques possess a good generalization performance compared with other machine learning algorithms, a small number of training samples is always preferable to construct a high accuracy surrogate model. From a general sense, MCS may be used to generate a training data set. However, random samples, generated according to the underlying probability distribution of random variables in MCS, usually cluster in a local domain and cannot completely represent the input space well. Moreover, MCS is not easy to obtain failure samples which represent structural characteristics in the failure domain, especially for small failure probability levels. The training data set without failure samples still may be used to train a surrogate model. However, its precision and generalization cannot be guaranteed. Thus, MCS is not suitable to generate the training data set for the purpose of training a proper surrogate model.

An LHS technique^41,42 is employed to generate the training samples in this article. In essence, LHS is a kind of multi-dimensional stratified sampling method in order to obtain a good coverage of input space and reduce the statistical uncertainty associated with MCS. Suppose that N training samples are required, the accumulative probability interval [0,1] of a random variable is divided into N non-overlapping subintervals

$$\{\begin{array}{c}\multicolumn{1}{c}{u_i=\frac{u}{N}+\frac{i-1}{N}}\\ \multicolumn{1}{c}{\frac{i-1}{N}<u_i<\frac{i}{N}}\end{array}i=1,2,\dots ,N$$

(11)

where u is a uniform random number in the interval [0,1] and $u_i$ is the random number in the ith subinterval $(i-1/N)<u_i<(i/N)$ , respectively. Then, the sampling in each subinterval is isoprobable and independent. This strategy avoids the large number of random sampling using MCS. The median LHS is adopted in this article, that is, $u=0.5$ . Let $\mathbf{u}=(u_1,u_2,\dots ,u_N{)}^{\mathrm{T}}$ be a random set of $u_i$ . The sampling point $x_i$ can be obtained according to the inverse cumulative distribution function of their marginal distributions

$$x_i=F_j^{-1}\left(u_i\right)i=1,2,\dots ,N$$

(12)

where $F^{-1}(u_i)$ is the inverse cumulative distribution function of the jth random variable. Without loss of generality, the input random variables are assumed to be independent. Even the input random variables are dependent, the above procedure still can be used to generate the training samples since the correlation among the input random variables is not needed to be considered at this stage. The above procedure can be easily extended to a problem of n-dimension. Suppose, the N sampling points are needed to be generated in an n-dimensional input space, first, the n-dimensional input space is split into N_n equal-sized hypercubes. Then in each hypercube, a random sample is generated. It should be pointed out that the N hypercubes need to be satisfied by the following requirement: only one hypercube can be chosen in any direction parallel with any of the axes.

If the associated failure probability of an LSF is too small, the training data set generated by LHS technique may not include any failure points. In order to address this issue, an improved sampling method, combining LHS technique and uniform sampling (US), is proposed to provide a good coverage of the whole input random space. This new method is denoted as LHS + US in this article. After generating $u_i$ by LHS technique, $x_i$ is obtained through a uniform mapping, that is, $u_i$ is mapped from the interval [0,1] to a compatible interval $[{\mu }_j-k{\sigma }_j,{\mu }_j+k{\sigma }_j]$ , instead of the complex inverse manipulation. Here, parameter k is a constant, which is used to describe the coverage of the training data set. Therefore, the selection of k should guarantee that there are enough failure points in the training data set. An empirical experience is that the value of k falls into an interval [3, 9] because the maximum allowable failure probability of a practical structure is less than 10⁻³. Here, a system with two LSFs is used to show the effect of parameter k. These two LSFs are given by

$$\begin{array}{c}{g}_1\left(\mathit{x}\right)=2-x_2+\exp {(-0.1x_1)}^2+{(0.2x_1)}^4\hfill \\ g_2\left(\mathit{x}\right)=4.5-x_1{x}_2\hfill \end{array}$$

(13)

where $x_1$ and $x_2$ are two standard normal variables.

Figure 1 shows 20 samples generated by LHS and LHS + US separately. Figure 1(a) presents the samples obtained by LHS technique. It can be seen that these 20 samples do not contain any failure samples and then cannot cover the input space well. Figure 1(b)–(d) shows the training samples obtained by LHS + US. It can be seen that the higher the value of k, the more the failure points will be generated and the better the coverage of random samples.

Figure 1.

Random sampling by LHS and LHS + US: (a) LHS, (b)LHS + US with k = 3, (c) LHS + US with k = 6, and (d) LHS + US with k = 9.

To select a proper value for k, a practical way is to start with k = 6 and check the percentage of failure samples. If this value is larger than 20%, the generation of training samples can be terminated. Otherwise, a larger value of k needs to be set.

Procedure of the presented method

Considering a structural system with m LSFs

$$\{\begin{array}{c}{g}_1\left(\mathit{X}\right)=g_1(x_1,x_2,\dots ,x_k)\hfill \\ \multicolumn{1}{c}{⋮}\\ \multicolumn{1}{c}{g_m\left(\mathit{X}\right)=g_1(x_1,x_2,\dots ,x_k)}\end{array}$$

(14)

The general procedure of the presented method is given below. First, the training samples required to construct a surrogate model are generated by LHS + US. Then, in order to select an appropriate kernel function, the corresponding parameters are determined by a grid-search method ⁴³ according to the selected kernel function and the training samples. After that, the sole surrogate model of all m LSFs is trained, based on the above information and the sampling algorithm presented in section “The generation of training samples.” Finally, MCS is employed to estimate the failure probabilities of all m LSFs based on the MIMO-SVM surrogate model. For the jth LSF, its failure probability is estimated as

$$P_{\mathrm{fj}}=P(g_j\left(\mathit{X}\right)<0)\approx \frac{n_j}{N},j=1,2,\dots ,m$$

(15)

where $g_j(\mathit{X})$ is the jth LSF from the trained surrogate model, N is the total number of samples or simulations in MCS, and $n_j$ is the total number of failure samples for LSF $g_j(\mathit{X})$ , respectively. The above procedure is summarized in Figure 2.

Figure 2.

Flowchart of the presented method.

Numerical examples

Two examples are used to illustrate the accuracy and efficiency of the presented method in this section, including one numerical example and one engineering problem. Example 1 is a two-dimensional structural system with four LSFs. It gives a visual representation of the effect of MIMO-SVM and the impact of training sample size and kernel function. Example 2 has 11 seven-dimensional LSFs, which is used to demonstrate the capacity of dealing with multiple LSFs by MIMO-SVM.

Example 1

Considering a reliability analysis problem of a structural system with 4 two-dimensional LSFs, it was taken and modified from Waarts. ¹¹ System failure is defined as $g_s(\mathbf{x})=\min \{g_1(\mathbf{x}),g_2(\mathbf{x}),g_3(\mathbf{x}),g_4(\mathbf{x})\}$ , and all four LSFs are given by

$$\begin{array}{c}{g}_1\left(\mathbf{x}\right)=0.1{(x_1-x_2)}^2-\frac{x_1+x_2}{\sqrt{2}}+3\hfill \\ g_2\left(\mathbf{x}\right)=x_1-x_2+3.5\sqrt{2}\hfill \\ g_3\left(\mathbf{x}\right)=0.1{(x_1-x_2)}^2+\frac{x_1+x_2}{\sqrt{2}}+3\hfill \\ g_4\left(\mathbf{x}\right)=-x_1+x_2+3.5\sqrt{2}\hfill \end{array}$$

(16)

where $x_1$ and $x_2$ are independent standard normal random variables. Both the failure probability of system and the failure probability of each LSF are required to be estimated simultaneously. In this problem, LHS + US is employed to generate random samples with k = 8, and sample size is chosen as 10 and 20 for illustrative purpose. The quadratic and cubic polynomial kernel functions and the radial basis kernel function are used for training MIMO-SVM surrogate model. Figure 3 shows the fitting results with different kernel functions and different sample size. MCS with 10⁶ samples is employed to estimate the failure probabilities of interest, based on the trained MIMO-SVM model. The failure probabilities of LSFs are summarized in Table 1.

Figure 3.

The LSFs fitted by MIMO-SVM with different kernel functions and different sample sizes: (a) Poly-2 with 10 samples, (b) Poly-3 with 10 samples, (c) Poly-2 with 20 samples, (d) Poly-2 with 40 samples, (e) RBF with 10 samples, (f) RBF with 20 samples, and (g) RBF with 40 samples.

Table 1.

The failure probabilities of all LSFs in Example 1 (× 10^–3).

LSFs	g 1(x)	g 2(x)	g 3(x)	g 4(x)	gs(x)
MCS	0.853	0.229	0.879	0.240	2.22
Single SVM	0.904	0.204	0.852	0.256	2.22
M-poly-2 (10)	0.904	0.204	0.852	0.256	2.22
M-poly-3 (10)	0.876	0.246	0.866	0.224	2.21
M-poly-2 (20)	0.904	0.204	0.852	0.256	2.22
M-RBF (10)	0.976	0.212	0.924	0.260	2.37
M-RBF (20)	0.904	0.204	0.848	0.256	2.21

LSF: limit state function; MCS: Monte Carlo simulation; SVM: support vector machine; RBF: radial basis function; MIMO-SVM: multi-input multi-output support vector machine.

M is MIMO-SVM.

The numbers in the parentheses are the sample size.

It can be seen from Figure 3 that the MIMO-SVM model possesses highly approximated precision when an appropriate number of failure samples is included in the training data set. However, the sample size seems to have a small influence on the approximated precision of the MIMO-SVM model when it is larger than 10. For this example, the MIMO-SVM models with different kernel functions may have similar precision. The simulation results from MCS (the second row in Table 1) are considered as “actual” ones, and they are used to compare to the computational results obtained by the proposed method. Table 1 indicates the presented method is as accurate as MCS. It is more efficient than the latter because it only needs a small number of training samples.

It is worth noting that the MIMO-SVM model is constructed only by one training run, while the number of training runs of a single-output SVM is identical with the number of LSFs in a system. Furthermore, a single-output SVM may require a different training data set for each LSF in the system. In this example, the MIMO-SVM model and the single-output SVM model have very similar precision, while the former needs as less as 10 training samples and the latter needs 40 training samples.

Example 2

As shown in Figure 4, a speed reducer ⁴⁴ has been designed to minimize its weight. There are 11 probabilistic constraints, and they represent bending, contact stress, longitudinal displacement, stress of the shaft, and geometry constraints. This design optimization problem has seven design variables: gear width (x₁), teeth module (x₂), number of teeth in the pinion (x₃), distance between bearings (x₄, x₅), and axis diameters (x₆ and x₇). Input random variables x₁–x₇ are mutually independent normal random variables with a standard deviation of 0.005. The objective function and the formulae of the probabilistic optimization are given by

$$\begin{array}{c}\mathrm{find}x=(x_1,\dots ,x_7)\hfill \\ \min f\left(x\right)=0.7854x_1{x}_2^2\hfill \\ \multicolumn{1}{c}{(3.3333x_3^2+14.0934x_3-43.0934)}\\ -1.508x_1(x_6^2+x_7^2)+7.477(x_6^3+x_7^3)+\hfill \\ 0.7854(x_4{x}_6^2+x_5{x}_7^2)\hfill \\ s.t.\mathrm{Pr}[g_i\left(x\right)>0]\le 1-\mathrm{\Phi }\left({\beta }_i\right),i=1,\dots ,11\hfill \end{array}$$

(17)

and

$$\begin{array}{c}{g}_1\left(\mathit{x}\right)=\frac{27}{x_1{x}_2^2{x}_3}-1,g_2\left(\mathit{x}\right)=\frac{397.5}{x_1{x}_2^2{x}_3^2}-1\hfill \\ g_3\left(\mathit{x}\right)=\frac{1.93x_4^3}{x_2{x}_3{x}_6^4}-1,g_4\left(\mathit{x}\right)=\frac{1.93x_5^3}{x_2{x}_3{x}_7^4}-1\hfill \\ g_5\left(\mathit{x}\right)=\frac{\sqrt{{(745x_4/\left(x_2{x}_3\right))}^2+16.9\times {10}^6}}{0.1x_6^3}-1100\hfill \\ g_6\left(\mathit{x}\right)=\frac{\sqrt{{(745x_5/\left(x_2{x}_3\right))}^2+157.5\times {10}^6}}{0.1x_7^3}-850\hfill \\ g_7\left(\mathit{x}\right)=x_2{x}_3-40,g_8\left(\mathit{x}\right)=5-\frac{x_1}{x_2}\hfill \\ g_9\left(\mathit{x}\right)=\frac{x_1}{x_2}-12,g_{10}\left(\mathit{x}\right)=\frac{1.5x_6+1.9}{x_4}-1,\hfill \\ \multicolumn{1}{c}{g_{11}\left(\mathit{x}\right)=\frac{1.1x_7+1.9}{x_5}-1}\\ {\beta }_1=\cdots {\beta }_{11}=3.0\hfill \\ 2.6\le x_1\le 3.6,0.7\le x_2\le 0.8,17\le x_3\le 28,\hfill \\ \multicolumn{1}{c}{7.3\le x_4\le 8.3,7.3\le x_5\le 8.3,2.9\le x_6\le 3.9,}\\ \multicolumn{1}{c}{5.0\le x_7\le 5.5}\end{array}$$

(18)

Figure 4.

Schematic speed reducer configuration.

It is obvious that all these 11 probabilistic constraints are required to be estimated in each iteration during the optimization process regardless of the usage of a double-loop or a single-loop optimization algorithm. Here, this study focuses on how to solve 11 probabilistic constraints simultaneously based on the presented method to reduce the total computational cost. Table 2 summarizes three groups of optimization results reported in the literature, ⁴⁴ and these optimization results are selected to perform our illustration.

Table 2.

The optimization results given by Lee and Lee. ⁴⁴

Optimization method	Objective function	Design variables
Deterministic	2992	(3.50, 0.70, 17.0, 7.30, 7.72, 3.35, 5.29)
PMA	3037	(3.60, 0.70, 17.8, 7.30, 7.79, 3.40, 5.34)
PMA+envelope function	3100	(3.60, 0.70, 17.2, 8.30, 8.30, 3.58, 5.45)

In this numerical example, 10 training samples are generated by LHS. Radial basis kernel function is chosen and the corresponding parameters are determined by grid-search method. The single MIMO-SVM surrogate model is trained according to the above information. Then, based on the trained MIMO-SVM model, MCS with 10⁵ samples is employed to estimate the failure probabilities of all 11 LSFs. In this article, the failure probabilities of all 11 probabilistic constraints for the three optimization cases are verified, that is, deterministic optimum, performance measure approach (PMA), and PMA + envelope function. ⁴⁴ The computational results are summarized in Table 3.

Table 3.

The failure probabilities of all LSFs in Example 2.

LSF	Deterministic		PMA		PMA + envelope function
	MCS	MIMO-SVM	MCS	MIMO-SVM	MCS	MIMO-SVM
g 1(x)	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
g 2(x)	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
g 3(x)	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
g 4(x)	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
g 5(x)	0.0923	0.0908	0.0000	0.0000	0.0000	0.0000
g 6(x)	0.1763	0.1768	0.0000	0.0000	0.0000	0.0000
g 7(x)	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
g 8(x)	0.4999	0.4978	4.5100 × 10−5	3.700 × 10−5	4.3000 × 10−5	4.6 × 10−5
g 9(x)	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
g 10(x)	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
g 11(x)	0.4667	0.4668	0.0157	0.0154	0.0000	0.0000

LSF: limit state function; MCS: Monte Carlo simulation; MIMO-SVM: multiple-input multiple-output SVM.

Similar to Example 1, a single MIMO-SVM surrogate model is constructed for all 11 LSFs, while 11 SVM surrogate models with one output are needed to be constructed in a traditional way. The training sample size is only 10 for the presented MIMO-SVM surrogate model, while at most 110 samples may be needed when using a single-output SVM model. It can be seen from Table 3 that the computational results obtained by the presented MIMO-SVM surrogate model with MCS are very close to those obtained directly by MCS. This indicates that the presented MIMO-SVM model has high accuracy under the condition of small number of samples and moderate dimensions.

Conclusion

It is well known that most of traditional structural reliability methods cannot be applied to deal with multiple LSFs simultaneously, when the failure probability of each LSF is of interest. A new structural reliability method using MIMO-SVM is presented to handle multiple LSFs for this issue which may arise in an RBDO problem and/or a problem with multiple failure modes. The main idea of the presented method is to construct a single surrogate model for all multiple LSFs using MIMO-SVM because all LSFs share the common input parameters and model (numerical or physical one). In order to obtain a good coverage of input parameter space, LHS and LHS + US are proposed to generate a training data set with a portion of failure samples. Finally, the failure probabilities of all LSFs are estimated by MCS based on the trained MIMO-SVM surrogate model. The most attractive advantages of this presented method are that all LSFs are approximated only using one training data set and the training operation is only run one time. These will be benefited for the estimation of probabilistic constraints in RBDO and structural reliability analysis with multiple failure modes. Numerical examples indicate that that the presented method needs a small amount of computational cost to achieve an accurate reliability analysis. Thus, it is suitable for RBDO with multiple probabilistic constraints and multiple modes reliability analysis.

A limitation of this study is the passive way of generation of training samples. Future work will involve combining the active learning techniques and MIMO-SVM to further reduce the computational cost and then apply it in practical RBDO.