An effective multivariate control chart for detecting small mean shifts using support vector data description

Abstract

Conventional multivariate cumulative sum control charts are more sensitive to small shifts than $T^2$ control charts, but they cannot get the knowledge of manufacturing process through the learning of in-control data due to the characteristics of their own structures. To address this issue, a modified multivariate cumulative sum control chart based on support vector data description for multivariate statistical process control is proposed in this article, which is named $D-\mathrm{MCUSUM}$ control chart. The proposed control chart will have both advantages of the multivariate cumulative sum control charts and the support vector data description algorithm, namely, high sensitivities to small shifts and learning abilities. The recommended values of some key parameters are also given for a better application. Based on these, a bivariate simulation experiment is conducted to evaluate the performance of the $D-\mathrm{MCUSUM}$ control chart. A real industrial case illustrates the application of the proposed control chart. The results also show that the $D-\mathrm{MCUSUM}$ control chart is more sensitive to small shifts than other traditional control charts (e.g. $T^2$ and multivariate cumulative sum) and a D control chart based on support vector data description.

Introduction

In modernized manufacturing process, inspired by applications of more advanced technologies and industrial expansion, a manufacturing system usually is composed of two or more correlated quality characteristics and hence it is essential to monitor and control all these quality variables simultaneously. Under this situation, an effective tool that can monitor several correlated variables simultaneously is indispensable. Therefore, Hotelling ¹ proposed a $T^2$ statistic for solving this problem. Based on it, he constructed a new control chart named $T^2$ control chart to monitor several correlated variables simultaneously. The most utilized method for monitoring large shifts is Hotelling’s $T^2$ control chart. ² However, it is relatively insensitive to small and moderate shifts in the process mean vector because it only takes the present information of the process into account. ³ To solve this problem, multivariate exponentially weighted moving average (MEWMA) control chart and multivariate cumulative sum (MCUSUM) control chart have been proposed. The MCUSUM control chart was first proposed by Woodall and Ncube. ⁴ They compared their MCUSUM control chart to Hotelling’s $T^2$ control chart based on average run length (ARL). The experimental results showed that their MCUSUM method was better than Hotelling’s $T^2$ control chart. Crosier ⁵ proposed two MCUSUM control charts. The first MCUSUM control chart reduced each multivariate observation to a scalar and then formed a MCUSUM of the scalars. The second MCUSUM control chart formed a CUSUM vector directly from the observations. Pignatiello and Runger ⁶ introduced two new formulations of the MCUSUM control charts, namely, $\mathrm{MC}1$ and $\mathrm{MC}2$ . Lowry et al. ⁷ presented the MEWMA control chart. They compared their MEWMA control chart to a MCUSUM control chart in ARL performance. The experimental results indicated that their method was similar to the MCUSUM control chart in detecting a shift in the mean vector of a multivariate normal distribution. For a review of multivariate control charts, Bersimis et al. ⁸ presented an excellent literature review.

The rapid development of information technology has promoted the progress of artificial intelligence under the background of big data era. Recently, a number of intelligent algorithms–based methods are applied in multivariate statistical process control (MSPC) and show a better performance than traditional methods. Among them, the unsupervised intelligent algorithm seems a promising method. Because the supervised intelligent algorithm requires that users should obtain, in advance, normal and abnormal pattern data sets. Prior knowledge of data sets on class label is essential for the supervised intelligent algorithm. In real industrial production, however, the great mass of collected data sets is normal data because abnormal data mean that there are some problems in production process. Therefore, it is costly and hard to collect abnormal pattern data sets.

There are two main categories of unsupervised methods applied in MSPC, namely, neural network and support vector–based methods. For neural network–based methods, self-organizing map (SOM) seems a promising method. Yu and Xi ⁹ proposed a novel SOM-based minimum quantization error (MQE) control chart. The experimental results showed that the MQE control chart might be a promising tool for multivariate quality control. For support vector–based methods, Sun and Tsung ¹⁰ first proposed a support vector data description (SVDD)-based control chart, ¹¹ which was named K chart. A case study demonstrated that the K chart could perform better than conventional charts when the underlying distribution of the quality characteristics was not multivariate normal. Gani et al. ¹² assessed the application of the K chart in a real industrial process. The industrial application showed that the K chart was sensitive to small shifts in mean vector and outperformed the $T^2$ control chart in terms of ARL. Ning and Tsung ¹³ provided an optimized design method of the SVDD-based K chart and the results confirmed that the proposed design scheme could improve the performance of K chart. He and Zhang ¹⁴ proposed a S-MCUSUM chart, which has an advantage of free distribution. However, their results showed that the cumulative sum (CUSUM) of T chart (i.e. MCUSUM chart) is somewhat better than the S-MCUSUM chart for multivariate normal distributed data. It should be pointed out that both the K chart and the S-MCUSUM chart focus on the situation when the data distribution departs from normality. When the distribution is multivariate normal, both charts would perform terribly, compared to conventional control charts.

All the cited literature above showed that the support vector–based method is a promising method for MSPC. Shi and Gindy ¹⁵ stated that the support vector method had been proved less vulnerable to overfitting problem and higher generalization ability since support vector method was designed to minimize structural risk, whereas previous neural network techniques were usually based on minimization of empirical risk.

Based on the above idea, this article proposes a modified MCUSUM control chart based on SVDD for MSPC, which is named $D-\mathrm{MCUSUM}$ control chart. Due to the characteristic of SVDD’s unsupervised learning, the construction of the $D-\mathrm{MCUSUM}$ control chart only requires normal process data easily collected in real-world manufacturing process and does not need the abnormal data hardly obtained in the real world. This characteristic makes the $D-\mathrm{MCUSUM}$ control chart more practical than other traditional control charts in realistic manufacturing process. At the same time, the $D-\mathrm{MCUSUM}$ control chart will become sensitive to small shifts due to the characteristic of the MCUSUM statistic.

It should be noted that the training process of SVDD for the proposed $D-\mathrm{MCUSUM}$ control chart is inputting $T^2$ value of the raw data to SVDD in the form of moving window vectors rather than directly inputting raw data to SVDD, which is adopted in other multivariate control charts based on SVDD (e.g. the K chart and the S-MCUSUM chart). Studies show that inputting data to SVDD in this way could get a better performance when the distribution of the quality characteristics is multivariate normal. To evaluate the performance of the $D-\mathrm{MCUSUM}$ control chart, a simulation study is performed. Comparison results show that the $D-\mathrm{MCUSUM}$ control chart has a better sensitivity in small shifts than other traditional control charts (e.g. $T^2$ and MCUSUM) and a D control chart based on SVDD. In this study, the choices of some key parameters (e.g. window size) are also provided for a better application.

The remainder of this article is organized as follows: The fundamental principles of SVDD and MCUSUM are reviewed briefly in section “Review of SVDD and MCUSUM.” Then, the approach of using the proposed chart for monitoring the manufacturing process is presented in section “Approach for monitoring multivariate manufacturing process.” The simulation studies of the $D-\mathrm{MCUSUM}$ control chart are shown in section “Simulation studies” and an illustrative example is given in section “Illustrative example.” The conclusions are given in section “Conclusion.”

Review of SVDD and MCUSUM

SVDD

Inspired by the development of the support vector classifier, Tax and Duin ¹¹ proposed a new support vector–based method, which was named SVDD. Different from two-class classifiers, such as support vector machines, ¹⁶ SVDD is proposed to solve one-class problems. Its target is to find a minimum hypersphere that can enclose all or most of the given data, and then it can make use of the hypersphere to monitor whether the current data are in control. Based on this, extension of the SVDD’s original idea to MSPC is rational.

When given a training set $T=\{x_i\in R^d,i=1,\dots ,N\}$ , N is the number of the training set and $x_i$ is a d dimension column vector. The target of SVDD is to give a description of the training set. Practically, the description is to find a hypersphere that can contain all or most of the training sets. The hypersphere is characterized by center a and radius $R>0$ and it must be as small as possible, namely, a small volume of the sphere. We can get the optimized hypersphere through minimizing $R^2$ . The minimized process can be equivalent to

$$F(R,a)=R^2+C\sum _i{\xi }_i$$

(1)

where ${\xi }_i$ is a slack variable. The target of its introduction is to avoid overfitting which often appears in the neural network method. The parameter C is the penalty parameter and its function is to control the trade-off between the volumes and the errors.

In the process of the minimization, it has to be under constraints

$$\Vert x_i-a{\Vert }^2\le R^2+{\xi }_i,{\xi }_i\ge 0\forall i$$

(2)

Through Lagrange multipliers, equation (2) can be incorporated into equation (1) and this will generate a new equation

$$\begin{array}{l}L\left(R,a,{\alpha }_i,{\gamma }_i,{\xi }_i\right)=R^2+C{\displaystyle \sum _i{\xi }_i}-{\displaystyle \sum _i{\alpha }_i}\\ \left\{R^2+{\xi }_i-\left({\Vert x_i\Vert }^2-2a·x_i+{\Vert a\Vert }^2\right)\right\}-{\displaystyle \sum _i{\gamma }_i{\xi }_i}\end{array}$$

(3)

with the Lagrange multipliers ${\alpha }_i,{\gamma }_i\ge 0$ . L should be minimized with respect to $R,a,{\xi }_i$ and maximized with respect to ${\alpha }_i$ and ${\gamma }_i$ for meeting the Karush–Kuhn–Tucker condition. ¹⁷ Three new constraints can be obtained by setting partial derivatives to zero

$$\frac{\partial L}{\partial R}=0:\sum _i{\alpha }_i=1$$

(4)

$$\frac{\partial L}{\partial a}=0:a=\frac{\sum _i{\alpha }_i{x}_i}{\sum _i{\alpha }_i}=\sum _i{\alpha }_i{x}_i$$

(5)

$$\frac{\partial L}{\partial {\xi }_i}=0:C-{\alpha }_i-{\gamma }_i=0$$

(6)

Because ${\alpha }_i,{\gamma }_i\ge 0$ , the Lagrange multipliers ${\gamma }_i$ can be removed from equation (6) and a new constraint can be obtained

$$0\le {\alpha }_i\le C$$

(7)

By substituting equations (4)–(6) into equation (3), equation (3) can be rewritten as follows

$$L=\sum _i{\alpha }_i(x_i·x_i)-\sum _{i,j}{\alpha }_i{\alpha }_j(x_i·x_j)$$

(8)

For determining whether a new point z is enclosed by the hypersphere, the distance between the center of the sphere and the point z must be calculated. The point z will not be rejected when the distance is smaller or equal to the radius

$$D_z^2=(z·z)-2\sum _i{\alpha }_i(z·x_i)+\sum _{i,j}{\alpha }_i{\alpha }_j(x_i·x_j)\le R^2$$

(9)

where $D_z$ is the distance between the center of the sphere and the point z.

For giving a more flexible data description, the inner products $(x_i·x_j)$ can be replaced by a kernel function $K(x_i·x_j)$ . After the process applied in equations (8) and (9), two new equations can be obtained

$$L=\sum _i{\alpha }_{i}K(x_i·x_i)-\sum _{i,j}{\alpha }_i{\alpha }_{j}K(x_i·x_j)$$

(10)

$$D_z^2=K(z·z)-2\sum _i{\alpha }_{i}K(z·x_i)+\sum _{i,j}{\alpha }_i{\alpha }_{j}K(x_i·x_j)\le R^2$$

(11)

Any kernel functions have to satisfy Mercer’s theorem. ¹⁸ There are two kinds of kernel functions commonly used in practice: a polynomial kernel and a Gaussian kernel.

The polynomial kernel is as follows

$$K(x_i,x_j)={(x_i·x_j)}^d$$

(12)

where the free parameter $d\in N^{+}$ gives the degree of the polynomial kernel.

The Gaussian kernel is as follows

$$K(x_i,x_j)=\exp (-\frac{{\Vert x_i-x_j\Vert }^2}{s^2})$$

(13)

where s is the width parameter.

According to Lazzaretti et al., ¹⁹ the SVDD with the Gaussian kernel can generate an irregular-shaped hypersphere and have a better performance in data description. Therefore, the Gaussian kernel will be used in this article.

The parameters C and s are set to 1 and 5 in this article, and more detailed analysis is presented in section “Choice of SVDD parameters.” It should be noted that getting the optimal performance of proposed method is not the target of this study. The target of this study is obtaining a general performance to show the generality of our methods. Interested readers can refer to Wang et al. ²⁰ They used genetic algorithm (GA) to search the optimal parameter combination of support vector regression (SVR) ²¹ and the results showed that their GA-SVR method was better than the mathematical regression model and artificial neural network (ANN).^22–24

MCUSUM

According to Pignatiello and Runger, ⁶ CUSUM control charts ²⁵ are often used instead of Shewhart control charts ²⁶ when the detection of small shifts is of interest in univariate statistical process control. In other words, the CUSUM statistic has a better sensitivity for small shifts. ²⁷ Based on an analogy of the univariate statistical process control, two multivariate extensions of the CUSUM control charts and the Shewhart control charts are generated naturally. They are Hotelling’s $T^2$ control charts ¹ and Runger’s MCUSUM control charts. ⁶ Just like the characteristics of the CUSUM control charts in univariate process, the MCUSUM control charts accumulate information from the past and present observations to give quick signals for small shifts. The $T^2$ control charts only observe the current data and ignore the sequential information of the whole observations; therefore, it is insensitive to small shifts but sensitive to large ones compared with the MCUSUM control charts.

Pignatiello and Runger proposed two statistics to construct different MCUSUM control charts, that is, $\mathrm{MC}1$ and $\mathrm{MC}2$ statistics. It should be pointed out that $\mathrm{MC}2$ statistic is adopted to construct the new control chart proposed in this article because preliminary investigations conducted showed that $\mathrm{MC}2$ is a better choice.

Here, $\mathrm{MC}2$ statistic will be introduced briefly. The $\mathrm{MC}2$ statistic proposed by Pignatiello and Runger is constructed based on Hotelling’s $T^2$ statistic. The $T^2$ statistic and the $\mathrm{MC}2$ statistic are shown in equations (14) and (15), respectively

$$T^2={(x-\mu )}^T{\mathrm{\Sigma }}^{-1}(x-\mu )$$

(14)

where x denotes the quality characteristics and $\mu$ is their mean. $\mathrm{\Sigma }$ is the covariance matrix. We assume that x is an independent and identically distributed variable and subject to multivariate normal distribution

$$\mathrm{MC}{2}_t=max\{0,\mathrm{MC}{2}_{t-1}+T_t^2-k\},k\ge 0t\in N^{+}$$

(15)

where $\mathrm{MC}{2}_t$ is the value of $\mathrm{MC}2$ at time t. It should be noted that $\mathrm{MC}{2}_0=0$ . $T_i^2$ is the value of $T^2$ at time t. k is a key parameter for $\mathrm{MC}2$ and it will have a great impact on the performance of MCUSUM control charts. Therefore, it is crucial to choose an appropriate value of k.

Approach for monitoring multivariate manufacturing process

D-MCUSUM control chart based on SVDD

In this study, a modified multivariate control chart based on SVDD is proposed to monitor the shift of manufacturing process, which is called $D-\mathrm{MCUSUM}$ control chart. The $D-\mathrm{MCUSUM}$ control chart can be constructed by a $D-\mathrm{MCUSUM}$ statistic. The $D-\mathrm{MCUSUM}$ statistic is constructed by a $D-\mathrm{value}$ statistic, which is also the basis of D control chart. Therefore, the $D-\mathrm{value}$ statistic will be presented first.

D-value and D chart

When the SVDD is trained by the collected normal data, the trained SVDD will generate a hypersphere characterized by “center” and “radius” that can represent the learned normal data space. After getting the trained SVDD, a $D-\mathrm{value}$ can be defined to calculate the “distance” of input vector from the center of learned normal data space. The $D-\mathrm{value}$ can represent how far away the current observed data deviate from the in-control state. The larger the “distance” is, the larger the possibility that the current observed data have gone in out-of-control state will be. The $D-\mathrm{value}$ can be computed as follows

$$\begin{array}{l}D-value={\Vert t-a\Vert }^2=K(t·t)-2{\displaystyle \sum _i{\alpha }_{i}K}(t·x_i)\\ +i,j{\displaystyle \sum {\alpha }_i{\alpha }_{j}K(x_i·x_j)}\end{array}$$

(16)

where t represents a n-dimensional moving window vector and the moving window vector consists of n $T^2$ values of the raw data. a represents the center of normal data space that SVDD learned.

The $D-\mathrm{value}$ is the monitoring statistic of the D chart. Extremely high $D-\mathrm{value}$ , that is, exceeding the control limit of the D chart, means that the input vector is an outlier and the process is out of control.

D-MCUSUM statistic

Although $D-\mathrm{value}$ , in a sense, can be used to judge the current state of input vector, it shares a similar drawback with $T^2$ statistic. Both of them only observe the current data and ignore the sequential information of the whole observations, and therefore it is insensitive to small shifts. To solve this problem, a modified MCUSUM statistic based on $D-\mathrm{value}$ is proposed in this article. The new statistic is named $D-\mathrm{MCUSUM}$ . The $D-\mathrm{MCUSUM}$ statistic can be calculated using equation (17)

$$\begin{array}{l}D-MCUSU{M}_t\\ =\max \left\{0,D-MCUSU{M}_{t-1}+D-valu{e}_t-k\right\},k\ge 0t\in N^{+}\end{array}$$

(17)

where $D-\mathrm{MCUSU}{M}_t$ represents the value of $D-\mathrm{MCUSUM}$ at time t and $D-\mathrm{MCUSU}{M}_0$ is zero. $D-\mathrm{valu}{e}_t$ is the value of $D-\mathrm{value}$ at time t. k is a key nonnegative parameter and it should be chosen carefully.

The $D-\mathrm{MCUSUM}$ chart can monitor the multivariate manufacturing process through the $D-\mathrm{MCUSUM}$ statistic. When the $D-\mathrm{MCUSUM}$ value exceeds its control limit, the $D-\mathrm{MCUSUM}$ chart will give an alarm.

A multivariate monitoring model based on the D-MCUSUM control chart

From what we have introduced above, an effective $D-\mathrm{MCUSUM}$ control chart–based multivariate monitoring model is proposed. Figure 1 shows the whole application of the monitoring model. This model provides an intelligent system of collecting data automatically, monitoring, and alarm. The monitoring model is composed of two modules, that is, offline learning module and online monitoring module.

Figure 1.

$D-\mathrm{MCUSUM}$ control chart–based multivariate monitoring model.

In the offline learning module, a large number of in-control data will be collected automatically from various sensors embedded in manufacturing system because a learning model trained by data with a lot of noises must lead to an unsatisfactory result. Therefore, before inputting the collected in-control data to SVDD to train SVDD, some data preprocessing methods will be adopted to deal with the raw in-control data such as denoising and standardization. After preprocessing procedure, the distribution parameters of the in-control data will be estimated to calculate the $T^2$ value of in-control data, and then the $T^2$ value of in-control data will be fed to SVDD to train SVDD in the form of moving window vectors. After getting the trained SVDD model, the control limit of the $D-\mathrm{MCUSUM}$ control chart will be determined to make in-control ARL approximately equal to setting value.

In the online monitoring module, some data preprocessing methods are also needed to deal with the raw data. After this, the $T^2$ value of monitoring data will be fed to trained SVDD model in the form of moving window vectors to calculate the value of the $D-\mathrm{MCUSUM}$ statistic, which can be used to judge the current state of the manufacturing process. If the value of the $D-\mathrm{MCUSUM}$ statistic exceeds the control limit, an alarm signal should be given. When an alarm signal occurs, this means that the process is in out-of-control state and related measures should be taken to conduct attribution analysis to find out assignable causes and solve it. After this process, the manufacturing system will get back to normal.

Generation of simulation data

To evaluate the performance of the $D-\mathrm{MCUSUM}$ control chart based on ARL, a Monte Carlo simulation ²⁸ is adopted in this article to generate the required multivariate data.

In the multivariate manufacturing process, there will usually be two or more correlated quality characteristics. In this study, a bivariate manufacturing process model is adopted to generate simulation data and it can be expressed in equation (18)

$$Y_t=\mu +F_t$$

(18)

where $Y_t$ is the quality characteristic at time t when the process is in control. $\mu$ is the process mean vector. $F_t$ is subject to a bivariate normal distribution with unit variances and a correlation coefficient of $\rho$ (i.e. ${\mu }_0=\left[\begin{array}{c}0\hfill \\ \multicolumn{1}{c}{0}\end{array}\right]$ , ${\mathrm{\Sigma }}_0=\left[\begin{array}{cc}1\hfill & \rho \hfill \\ \multicolumn{1}{c}{\rho }& 1\hfill \end{array}\right]$ ). In this study, we assume that $\mu =\left[\begin{array}{c}0\hfill \\ \multicolumn{1}{c}{0}\end{array}\right]$ .

In this study, the mean shifts pattern is adopted in the out-of-control process. If an out-of-control pattern occurs in the process at time t, this can be expressed in equation (19)

$$X_t=Y_t+\delta ,t\ge n$$

(19)

where $X_t$ is the quality characteristic at time t when the process is out of control. n is the window size and it will be introduced in section “Moving window analysis method.” $\delta$ is the shifts vector $(\delta =\left[\begin{array}{c}{\delta }_1\hfill \\ \multicolumn{1}{c}{{\delta }_2}\end{array}\right])$ . Eight different kinds of shifts are selected to cover the whole range of shifts patterns (i.e. ${\delta }_1,{\delta }_2=0.25\sigma ,0.5\sigma ,0.75\sigma ,1.0\sigma ,1.5\sigma ,2.0\sigma ,2.5\sigma ,3.0\sigma ,\sigma =1$ in this study). Different shifts vectors represent different magnitudes of shifts and the shifts vector $\delta$ consists of any combinations of these shifts. In total, 44 different shifts vectors are selected in this article (for details of the 44 shifts vectors, readers can refer Appendix 1).

Performance evaluation method

Moving window analysis method

For getting more real simulation results, a moving window analysis method^29,30 is adopted in this article to input data to SVDD. There will be n observation points in the moving window each time. The dimension of the space that SVDD learned is determined by the moving window size. Every observation point is the $T^2$ value of the raw data and the $T^2$ value can be calculated using equation (14). Preliminary investigations proved that inputting data to SVDD in this way would get a better performance compared to directly inputting raw data to SVDD. Figure 2 briefly shows the application of the moving window analysis method.

Figure 2.

The application of the moving window analysis method.

The application of the moving window analysis method mainly consists of the following steps:

• Step 1. Setting t (observation point) to n (window size).

• Step 2. Taking the most recent n observation points, $Z_i=T_i^2(i=t-n+1,\dots ,i=t-1,t)$ to generate a n-dimensional moving window vector.

• Step 3. Making $t=t+1$ to generate next new window vector after last window vector is used.

In this article, we assume that the process is initially in control. This assumption can be considered more practical because in the real-world monitoring process an out-of-control signal often occurs after a period in which the process is in control, and the starting point of the out-of-control signal is usually unknown. ³¹

It should be pointed out that the moving window size will last a great effect on the proposed control chart. Therefore, it is essential to set an appropriate value of window size. For more details of this, it will be discussed in section “Choice of window size.”

ARL

ARL is a widely used performance criteria in the field of control charts. The ARL, in a sense, can indicate the sensitivity of control charts. ³² The ARL is defined as the average number of observations between the occurrence of out-of-control state and the alarm of out-of-control signal. In general, there are two kinds of ARLs, that is, in-control ARL and out-of-control ARL. The two kinds of ARLs are also related to the probabilities of two types of errors, that is, type I error and type II error. For in-control ARL, the larger the probability of type I error is, the shorter the in-control ARL will be. For out-of-control ARL, the larger the probability of type II error is, the longer the out-of-control ARL will be. There is usually a compromise between the two errors. Because when the type I error decreases, the type II error will increase. When the type II error decreases, however, the type I error will increase. An excellent control chart should minimize both the errors simultaneously. In other words, an excellent control chart should have the characteristics of longer in-control ARL and shorter out-of-control ARL.

It should be noted that an in-control ARL of 350 is adopted in this article to evaluate the performance of different control charts and the in-control ARL of 350 will make control charts have a relative small type I error. Specific steps of calculating the in-control ARL are summarized as follows:

1. Setting window number m to 1, t (observation point) to n (window size).

2. Feeding a window vector with n observation points (in-control data) to SVDD and calculating the $D-\mathrm{MCUSUM}$ value.

3. If the $D-\mathrm{MCUSUM}$ value exceeds the control limit, proceeding to Step 4; otherwise, incrementing m and t by 1 and returning to Step 2.

4. Recording the value of the window number m as the run length of this trial.

For the calculation of out-of-control ARL, it is almost the same as in-control ARL. The only difference between them is the window vectors. For in-control ARL, every observation point of the window vectors is in-control data. For out-of-control ARL, however, initial several window vectors will include out-of-control data partially and every observation point of later window vectors is out-of-control data. The reason for this is that we assume that the process was initially in control. We can take an example to introduce this process. For instance, when window number $m=1$ , the window vector consists of $n-1$ in-control observation points and 1 out-of-control observation point. When window number $m=2$ , the new window vector will include more out-of-control observation points due to the movement of the window and it will be made up of $n-2$ in-control observation points and 2 out-of-control observation points. On the analogy of this, when window number $m\ge n$ , the window vector completely consists of out-of-control observation points.

It should be noted that all the ARLs are computed from 10,000 independent simulation runs to get a stable result.

Control limit

In this study, a fixed control limit h is used to determine the state of the current process. When a $D-\mathrm{MCUSUM}$ value exceeds the control limit h, an alarm signal should be given, that is to say, the process is in out-of-control state. ³³

All the control limits are obtained based on in-control ARL of 350. The control limits will be tuned to make in-control ARL approximately equal to 350. All the control limits are computed from 10,000 independent simulations.

Simulation studies

Choice of window size

Before conducting simulation experiments, several key parameters should be determined. An appropriate window size will construct a better monitoring model. Therefore, a study will be conducted to select an appropriate widow size based on the ARL performances of different window sizes. Six kinds of different window sizes $(n=1,2,4,12,16,25)$ are selected to compute the ARL performance. In this study, the correlation coefficient $\rho$ is set to 0.8 and other parameters are set to what we above have introduced. By several simple trail-and-error experiments, the parameter k of different window sizes is selected to obtain a relative optimal performance. An appropriate control limit h is adjusted to make in-control ARL approximately equal to 350. To show the result better, a non-centrality parameter $\lambda$ is introduced in this study, which is also called Mahalanobis distance. ³⁴ The parameter $\lambda$ is expressed in equation (20) and can represent the magnitude of shifts vector $\delta$

$$\lambda =\sqrt{{\delta }^T{\mathrm{\Sigma }}^{-1}\delta }$$

(20)

where $\delta$ is the shifts vector.

In general, a large shift will have a better out-of-control ARL, that is, a smaller out-of-control ARL. In the study, we first take a shifts vector $(\delta =\left[\begin{array}{c}1.5\hfill \\ \multicolumn{1}{c}{0.5}\end{array}\right])$ as a case to study the ARL performance of different window sizes and the result is shown in Figure 3. The control limit h and the parameter k of different window sizes are shown in Figure 4. From Figure 3, we can find out that a small window size will have a better ARL performance. For confirming this, another study with different shifts vectors is conducted and the result is shown in Figure 4. The parameter $\lambda$ is used to show the performance of different window sizes in Figure 4.

Figure 3.

Performance of different window sizes for a specific shifts pattern.

Figure 4.

Performance of different window sizes.

As is shown in Figure 4, window size of 1 gets the best performance overall and this window size should be the best choice of the $D-\mathrm{MCUSUM}$ control chart. The result also reveals a phenomenon that the larger the window size becomes, the worse performance the $D-\mathrm{MCUSUM}$ control chart will get. This phenomenon can be explained in this way. A large window usually can contain more observation points and this is equivalent to consider sequential information. The proposed $D-\mathrm{MCUSUM}$ control chart is a kind of modified method based on sequential information. Therefore, the superposition of two different methods based on sequential information will make the $D-\mathrm{MCUSUM}$ control chart perform terribly and a small window size should be a better choice for the $D-\mathrm{MCUSUM}$ control chart. At the same time, a small window size can get a more simplified structure of SVDD and less data cost compared to a large window size. Therefore, window size n is set to 1 for the $D-\mathrm{MCUSUM}$ control chart in this study.

Choice of SVDD parameters

The performance of SVDD is very dependent on the parameters C and s. Both will greatly affect the tightness of the data description boundary. To show this phenomenon, 200 two-dimensional banana data are generated. Figure 5 shows the different boundaries of the SVDD with different C and s. As shown in Figure 5, a small C or s will lead to a tighter boundary, which is equivalent to an overfitting phenomenon. However, a large C or s will generate a looser boundary. Therefore, the two parameters that become too small or too large can lead to an unsatisfactory data description boundary of SVDD due to overfitting or underfitting problem. Therefore, for avoiding the two problems, the parameters C and s are set to 1 and 5, respectively, to get a relative optimal detection performance in this article. After all, getting the optimal performance of the proposed method is not the target of this study. The target of this study is obtaining a general performance to show the generality of our methods.

Figure 5.

Examples of SVDD with different values of C and s: (a) small values of C and s and (b) large values of C and s.

Simulation results and discussions

There may be various kinds of correlations between variables in the multivariate manufacturing process. Therefore, four different experiments are designed to testify the universality of the $D-\mathrm{MCUSUM}$ control chart with four different correlation coefficients $(\rho =0.2,0.4,0.6,0.8)$ , which evenly covers the whole range of correlations. The four experiments will compare the performance of the $D-\mathrm{MCUSUM}$ control chart with $T^2$ control chart, MCUSUM control chart, and a control chart based on the $D-\mathrm{value}$ statistic, which is called D control chart. All these experiments will adjust the control limit of every control chart to make in-control ARL approximately equal to 350 and compare the performances of different control charts based on out-of-control ARL. All the performances of different control charts are computed from 10,000 simulations. The parameter k of MCUSUM and $D-\mathrm{MCUSUM}$ control chart is set to 3.5 and 0.25, respectively. Because the shift magnitude $\lambda$ of the same shifts vector with different correlation coefficients will differ, a new statistic $\phi$ will be proposed to represent shift magnitude for the unities of different correlation coefficients and it is expressed in equation (21). Based on the statistic $\phi$ , shifts can be divided into two categories: small shifts $(0<\phi \le 0.55)$ and large shifts $(0.55<\phi \le 1)$

$${\phi }_i=\sqrt{\frac{{\lambda }_i}{{\lambda }_{max}}},i\in N^{+}$$

(21)

where ${\lambda }_{max}=max\{{\lambda }_i\},i\in N^{+}$ .

Figure 6 shows the out-of-control ARL performance of the $D-\mathrm{MCUSUM}$ control chart and other control charts. The detailed simulation results are shown in Table 2. It should be noted that the new statistic $\phi$ is used in Figure 6 and Table 2 to represent the shift magnitudes. Table 1 shows the control limits and in-control ARLs of four control charts.

Figure 6.

Comparisons of out-of-control ARL performance between the $D-\mathrm{MCUSUM}$ control chart and other control charts with different correlation coefficients: (a) $\rho =0.2$ , (b) $\rho =0.4$ , (c) $\rho =0.6$ , and (d) $\rho =0.8$ .

Table 1.

The control limits and in-control ARLs of four control charts.

Control charts	Control limit h				In-control ARL
	$\rho =0.2$	$\rho =0.4$	$\rho =0.6$	$\rho =0.8$	$\rho =0.2$	$\rho =0.4$	$\rho =0.6$	$\rho =0.8$
$D-\mathrm{MCUSUM}$	2.5065	2.5255	2.4655	2.5104	350.5035	350.3923	350.4878	350.3231
D	1.6873	1.7155	1.6885	1.6865	349.7261	350.1049	348.3808	348.2077
MCUSUM	10.1683	10.1685	10.1455	10.1685	351.1598	351.5810	350.2910	351.7247
$T^2$	11.7425	11.7555	11.7355	11.7459	351.1126	351.4924	350.7202	349.6433

As is shown in Figure 6 and Table 2, the $D-\mathrm{MCUSUM}$ control chart shows better performance in small shifts and similar performance in large shifts in comparison with other control charts. This can be explained that the CUSUM method will make the control chart become sensitive to small shifts but insensitive to large shifts, and then the $D-\mathrm{MCUSUM}$ control chart is modified based on CUSUM method. Simulation experiments of four different correlations also testify the universality of the $D-\mathrm{MCUSUM}$ control chart’s application in the multivariate manufacturing process. Just like the relationship between $T^2$ control chart and MCUSUM control chart, the $D-\mathrm{MCUSUM}$ control chart performs better than the D control chart in small shifts. The results show that the $D-\mathrm{MCUSUM}$ control chart is better than the D control chart and the $T^2$ control chart as a whole.

Table 2.

Out-of-control ARL performance of the $D-\mathrm{MCUSUM}$ control chart and other control charts in different correlation coefficients.

$\rho$	$\varphi$	$D-\mathrm{MCUSUM}$	D	MCUSUM	$T^2$	$\rho$	$\varphi$	D-MCUSUM	D	MCUSUM	$T^2$
0.2	0.2567	253.9886	285.8531	278.8852	288.2971	0.4	0.2758	245.4567	279.3781	270.2994	275.4603
	0.2887	221.8214	261.4734	249.9286	259.6716		0.2887	231.3961	269.2053	253.3999	273.0281
	0.3630	128.2183	186.5659	156.4963	189.4702		0.3745	133.2048	193.4750	160.0261	192.9588
	0.3674	123.1266	183.9557	152.4247	183.8305		0.3901	116.2130	172.6217	145.0038	175.2115
	0.4083	84.3458	144.7005	106.9400	143.6124		0.4083	98.7643	157.1369	124.8453	157.6975
	0.4421	59.5509	111.1378	77.0972	113.6983		0.4580	60.5040	111.3153	78.6217	113.9788
	0.4446	58.3990	110.3721	75.7169	112.3166		0.4667	55.9986	104.3085	73.3149	107.8091
	0.4631	48.0845	94.7760	62.6927	94.9795		0.4777	48.9994	96.6664	64.8027	97.2066
	0.5000	32.7846	69.3213	40.7426	70.7594		0.5000	39.9770	81.9663	50.8158	82.6201
	0.5085	30.2544	64.3928	37.2718	66.1736		0.5297	30.1378	64.6902	36.9232	63.9378
	0.5134	28.6773	60.4290	35.3570	62.0157		0.5316	29.3738	63.1033	36.7117	62.5404
	0.5196	27.4576	58.5584	32.9684	59.3035		0.5464	25.5616	54.5120	31.2014	55.8837
	0.5442	21.4121	47.8168	25.1603	46.7015		0.5516	24.4865	52.6978	29.1609	53.4201
	0.5774	15.9981	34.6881	17.5721	34.1137		0.5774	19.5373	41.7590	22.3441	43.2046
	0.6225	11.2921	22.5940	11.3854	22.5122		0.6476	11.2315	22.2606	11.1845	22.9729
	0.6252	11.1192	21.8792	11.0010	22.6156		0.6487	11.2802	22.5373	11.2564	22.3257
	0.6287	10.7652	21.1789	10.6276	21.3330		0.6570	10.6904	20.3525	10.4473	20.9628
	0.6364	10.1839	19.8648	9.7374	19.6783		0.6601	10.3485	20.2655	10.0209	20.3152
	0.6550	9.0391	16.9288	8.5197	16.5212		0.6756	9.4684	16.8601	8.7326	17.2722
	0.7071	6.4554	10.0914	5.6206	10.4599		0.7071	7.6418	13.1362	6.7948	13.0885
	0.7191	5.9980	9.1464	5.1205	9.2000		0.7470	6.0175	9.2168	5.1787	9.3161
	0.7197	5.9863	9.1534	5.1749	9.2691		0.7491	5.9731	9.0789	5.0712	9.0148
	0.7243	5.8175	8.7508	4.9646	8.8174		0.7518	5.8658	8.7338	4.9633	8.8643
	0.7260	5.8326	8.5652	4.9248	8.6250		0.7631	5.5199	7.9508	4.6938	8.0985
	0.7349	5.5339	7.8655	4.6434	8.0241		0.7727	5.3118	7.4074	4.3984	7.2793
	0.7695	4.5925	5.8760	3.7476	5.8454		0.7801	5.0614	6.8285	4.2195	6.9025
	0.8034	3.9203	4.3795	3.1360	4.4692		0.8165	4.2731	5.1169	3.3935	5.1631
	0.8055	3.8932	4.3724	3.0680	4.4363		0.8350	3.9378	4.4502	3.1120	4.4449
	0.8055	3.9186	4.3450	3.0618	4.3613		0.8375	3.9338	4.3672	3.0830	4.3956
	0.8117	3.8320	4.1993	2.9863	4.1917		0.8448	3.7902	4.0434	2.9520	4.1478
	0.8117	3.8108	4.0795	3.0180	4.1440		0.8448	3.7629	4.1040	2.9648	4.0348
	0.8165	3.7234	4.0028	2.9188	4.0116		0.8565	3.5852	3.7754	2.8254	3.7552
	0.8350	3.4836	3.4973	2.6658	3.4827		0.8722	3.3991	3.3349	2.6064	3.3338
	0.8700	3.0429	2.7151	2.2783	2.7361		0.8722	3.4047	3.4277	2.6235	3.3004
	0.8804	2.9498	2.5040	2.1912	2.5310		0.9129	2.9572	2.5433	2.2036	2.5933
	0.8807	2.9522	2.5145	2.1833	2.5501		0.9159	2.9068	2.5213	2.1639	2.5061
	0.8832	2.9189	2.4470	2.1467	2.5259		0.9174	2.9290	2.5250	2.1586	2.5257
	0.8842	2.8951	2.4778	2.1486	2.4705		0.9208	2.8860	2.4184	2.1349	2.4586
	0.8891	2.8820	2.3815	2.1036	2.3860		0.9292	2.8035	2.3242	2.0701	2.3283
	0.9001	2.7816	2.2573	2.0139	2.2516		0.9335	2.7721	2.2530	2.0423	2.2921
	0.9129	2.6870	2.0826	1.9242	2.0684		0.9409	2.6967	2.1711	1.9805	2.1627
	0.9263	2.5860	1.9258	1.8167	1.9408		0.9554	2.6256	2.0232	1.8580	2.0009
	0.9604	2.3979	1.6261	1.6199	1.6234		0.9620	2.6008	1.9257	1.8147	1.9328
	1.0000	2.2338	1.3768	1.4378	1.3903		1.0000	2.3756	1.5946	1.6061	1.5985
0.6	0.2730	246.7352	275.3110	261.5399	275.7333	0.8	0.2296	255.9418	279.2270	272.1011	279.4227
	0.2887	225.6029	264.7358	247.1154	267.1485		0.2887	171.2341	224.5105	196.7966	227.5523
	0.3666	134.0535	190.4406	158.5348	195.3805		0.3247	122.7317	184.0456	151.1345	181.8675
	0.3861	112.7346	169.9206	134.5662	168.6867		0.3344	110.3849	169.4939	137.7567	170.1923
	0.4083	91.3271	148.9824	112.8563	147.5331		0.3920	57.9819	108.3482	75.4650	108.7417
	0.4480	60.4765	110.9889	78.7550	112.3050		0.3976	53.8151	102.1068	68.5254	102.5694
	0.4592	54.0858	99.7778	69.0208	103.4629		0.4083	47.5655	91.9886	61.2505	93.2415
	0.4729	47.0537	92.3266	60.5623	92.3104		0.4359	34.6737	73.6881	43.9189	73.1105
	0.5000	35.7459	73.8544	45.4530	74.3843		0.4480	29.9774	63.4866	38.0006	63.9506
	0.5184	30.1035	63.8003	36.3126	64.3154		0.4592	26.7207	57.7851	31.9910	58.7144
	0.5209	29.0018	62.7123	36.2085	62.6955		0.4729	23.0196	50.6097	27.5709	50.6340
	0.5395	24.6568	52.2460	29.3958	52.4017		0.5000	17.4493	37.6307	19.7812	38.5945
	0.5460	23.2908	51.0766	27.3169	50.2564		0.5209	14.3796	30.6451	15.6011	29.9307
	0.5774	17.8195	37.2802	19.7237	37.3225		0.5544	10.7731	20.9612	10.4712	21.2292
	0.6336	11.3434	22.5476	11.1329	22.2113		0.5623	10.2070	19.4880	9.6911	19.6145
	0.6349	11.2266	22.2278	10.9174	22.6919		0.5774	8.8787	16.3395	8.3423	16.6846
	0.6493	10.1159	19.3491	9.7366	19.1643		0.5792	8.8602	16.0162	8.1383	15.7541
	0.6687	8.7403	16.0308	8.1250	16.1269		0.6165	6.7601	10.8921	5.9371	10.8909
	0.6745	8.5055	15.1641	7.7782	15.4865		0.6336	5.9932	8.9203	5.1089	9.1043
	0.7071	6.9575	11.3051	6.0516	11.1632		0.6493	5.4269	7.8291	4.5702	7.8675
	0.7331	5.9424	8.9254	5.0587	8.8660		0.6605	5.0465	6.8390	4.2112	6.9181
	0.7366	5.8248	8.7066	4.9789	8.6506		0.6687	4.8311	6.3444	3.9811	6.3927
	0.7443	5.6003	8.0931	4.6627	8.0964		0.7001	4.0958	4.7863	3.2379	4.7281
	0.7630	5.0481	6.9185	4.1652	6.9038		0.7071	3.9755	4.3894	3.1304	4.4098
	0.7722	4.8191	6.4342	3.9057	6.4013		0.7071	3.9530	4.4148	3.1208	4.3763
	0.7876	4.4808	5.5294	3.6440	5.6390		0.7260	3.6044	3.7897	2.7879	3.7108
	0.8165	3.9135	4.3616	3.1228	4.4218		0.7260	3.6120	3.7307	2.8043	3.7306
	0.8165	3.9198	4.3940	3.1133	4.4057		0.7366	3.4438	3.4215	2.6366	3.4605
	0.8290	3.7117	3.9313	2.9105	3.9958		0.7752	2.9514	2.5088	2.1730	2.5229
	0.8290	3.7059	4.0324	2.9129	4.0396		0.7752	2.9515	2.5137	2.1639	2.5295
	0.8438	3.5391	3.5471	2.6975	3.6066		0.7759	2.9364	2.4811	2.1647	2.4933
	0.8633	3.2432	3.1288	2.4936	3.0930		0.7840	2.8494	2.4016	2.0636	2.3454
	0.8633	3.2617	3.0989	2.4916	3.0777		0.7953	2.7669	2.1991	1.9818	2.1864
	0.8867	3.0107	2.6709	2.2596	2.6813		0.8068	2.6528	2.0356	1.8909	2.0260
	0.8960	2.8906	2.5201	2.1610	2.4837		0.8165	2.5881	1.9045	1.8019	1.9075
	0.8979	2.8814	2.4734	2.1411	2.4601		0.8190	2.5749	1.8651	1.7823	1.8810
	0.9129	2.7684	2.2452	2.0178	2.2788		0.8409	2.4222	1.6475	1.6498	1.6691
	0.9129	2.7790	2.2748	2.0080	2.2717		0.8719	2.2697	1.4309	1.4698	1.4301
	0.9183	2.7462	2.2029	1.9781	2.1879		0.8765	2.2439	1.4078	1.4613	1.4168
	0.9345	2.6089	2.0021	1.8566	2.0005		0.9022	2.1644	1.2791	1.3430	1.2784
	0.9457	2.5413	1.8959	1.7807	1.8928		0.9129	2.1345	1.2319	1.2969	1.2434
	0.9539	2.4874	1.7793	1.7412	1.8113		0.9340	2.0890	1.1637	1.2238	1.1641
	0.9759	2.3792	1.6267	1.6121	1.6436		0.9668	2.0395	1.0954	1.1410	1.0925
	1.0000	2.2745	1.4745	1.4934	1.4697		1.0000	2.0170	1.0463	1.0744	1.0471

The results also show that the $D-\mathrm{MCUSUM}$ control chart can detect small shifts faster than other traditional control charts (e.g. $T^2$ and MCUSUM). Furthermore, the $D-\mathrm{MCUSUM}$ control chart can make full use of the in-control data and continually improve its structure through learning. Therefore, it can adapt itself to different manufacturing processes. This is essential for the real-world application.

Illustrative example

In this section, a real industrial case derived from Alt ³⁵ is provided to this study to demonstrate the applicability of the proposed control chart. Two quality characteristics of stiffness and bending strength from a lumber manufacturing plant are presented to represent a bivariate process. The in-control vector mean and covariance matrix are calculated based on the in-control data and they are given as follows

$${\mu }_0=\left[\begin{array}{c}265\hfill \\ \multicolumn{1}{c}{470}\end{array}\right],{\mathrm{\Sigma }}_0=\left[\begin{array}{cc}100\hfill & 66\hfill \\ \multicolumn{1}{c}{66}& 121\hfill \end{array}\right]$$

The in-control ARL of 350 is acceptable in this study.

We assume that a small shift magnitude of $\phi =0.5$ is important enough to be detected as soon as possible in this case, and then we begin to monitor sample of 120 observations by the proposed control chart and other control charts. The parameter k of MCUSUM and $D-\mathrm{MCUSUM}$ control chart is set to 3.5 and 0.25, respectively. The monitoring results are shown in Figure 7. Appendix 1 provides the monitoring data and other related statistics for different charts.

Figure 7.

The $D-\mathrm{MCUSUM}$ control chart and other control charts for monitoring the illustrative example: (a) the $T^2$ control chart, (b) the MCUSUM control chart, (c) the D control chart, and (d) the $D-\mathrm{MCUSUM}$ control chart.

As is shown in Figure 7, the proposed control chart gives a faster out-of-control signal than other control charts at the 35th sample point. Moreover, the proposed control chart gives a continuous abnormal alarm after the 35th sample point. This shows the stability of the proposed control chart for detecting small shifts. This is a very important characteristic for practical applications and this characteristic is unavailable for other control charts. Therefore, this case gives a practical demonstration that the $D-\mathrm{MCUSUM}$ control chart is quite effective in detecting small shifts.

Conclusion

In this article, we propose a modified MCUSUM control chart based on SVDD for detecting small mean shifts, which is named $D-\mathrm{MCUSUM}$ control chart. Some key parameters (e.g. window size) are discussed and give some recommended values for a better application. In addition, the proposed chart adopts the $T^2$ value of the raw data as the input features of SVDD instead of the raw data, and studies reveal that inputting data to SVDD in this way could effectively improve the performance of the proposed chart when the distribution of the quality characteristics is multivariate normal.

To evaluate the performance of the $D-\mathrm{MCUSUM}$ control chart, a simulation study with four different correlations (evenly cover the whole range of correlations) is conducted to show the robustness of the proposed chart. According to simulation results, compared to the D control chart, the $D-\mathrm{MCUSUM}$ control chart shows a better performance in the detection of small shifts due to considering sequential information. Compared to other traditional control charts (e.g. $T^2$ and MCUSUM), the simulation results and a real industrial case also show the sensitivity of the $D-\mathrm{MCUSUM}$ control chart for small shifts.

Besides these, the $D-\mathrm{MCUSUM}$ control chart has other advantages. The $D-\mathrm{MCUSUM}$ control chart can adapt itself to different manufacturing processes through continual learning. This characteristic is not available to other traditional control charts. Furthermore, the learning of the $D-\mathrm{MCUSUM}$ control chart only requires normal process data (easily collected in real-world manufacturing process) and do not need the abnormal data (hardly obtained in the real world). This can save a great deal of cost in data acquisition.

From the above empirical analysis, all the results show that the $D-\mathrm{MCUSUM}$ control chart is an effective, promising, and robust tool to monitor multivariate manufacturing process, especially of small shifts.

Some issues could be the topics of future research. As pointed out in section “SVDD,” the parameter optimization of SVDD still requires further investigation. Moreover, integrating the $D-\mathrm{MCUSUM}$ control chart with other control charts to improve the performance is also a future research topic.