Quality-related fault diagnosis based on k -nearest neighbor rule for non-linear industrial processes

Abstract

The fault diagnosis approaches based on k-nearest neighbor rule have been widely researched for industrial processes and achieve excellent performance. However, for quality-related fault diagnosis, the approaches using k-nearest neighbor rule have been still not sufficiently studied. To tackle this problem, in this article, we propose a novel quality-related fault diagnosis framework, which is made up of two parts: fault detection and fault isolation. In the fault detection stage, we innovatively propose a novel non-linear quality-related fault detection method called kernel partial least squares-k-nearest neighbor rule, which organically incorporates k-nearest neighbor rule with kernel partial least squares. Specifically, we first employ kernel partial least squares to establish a non-linear regression model between quality variables and process variables. After that, the statistics and thresholds corresponding to process space and predicted quality space are appropriately designed by adopting k-nearest neighbor rule. In the fault isolation stage, in order to match our proposed non-linear quality-related fault detection method kernel partial least squares-k-nearest neighbor seamlessly, we propose a modified variable contributions by k-nearest neighbor (VCkNN) fault isolation method called modified variable contributions by k-nearest neighbor (MVCkNN), which elaborately introduces the idea of the accumulative relative contribution rate into VC k-nearest neighbor, such that the smearing effect caused by the normal distribution hypothesis of VC k-nearest neighbor can be mitigated effectively. Finally, a widely used numerical example and the Tennessee Eastman process are employed to verify the effectiveness of our proposed approach.

Introduction

With the rapid development of industry, modern industrial systems are expanding toward the direction of large scale and complexity. To ensure safety and reliability in industrial production, multivariate statistical process monitoring (MSPM) as a kind of data-driven approach has been extensively studied and successfully applied to actual industrial processes.^1–3 In MSPM, process-related fault detection is a popular research task, of which the mainstream method is principal component analysis (PCA). ⁴ Many researchers have improved PCA for better application in fault detection. For example, Xiu et al. ⁵ proposed a novel Laplacian regularized robust PCA method that can effectively capture the intrinsic non-linear geometric information. Other process-related fault detection methods have canonical correlation analysis (CCA), ⁶ non-negative matrix factorization (NMF),^7,8 and so on. Another particularly important research direction for MSPM is quality-related fault diagnosis,^9–11 in which quality-related fault detection and fault isolation are two key tasks. Quality-related fault detection belongs to a supervised learning task in machine learning area, ¹² and it aims to detect whether a fault that affects product quality occurs in the industrial system. When a fault detection algorithm indicates some faults exist in the system, fault isolation attempts to locate the faulty sensors. By the diagnosis of quality-related faults, unnecessary downtime and cost brought by the quality-unrelated faults can be greatly reduced, and the risky faulty sensors can also be located as quickly as possible. Therefore, quality-related fault diagnosis has been a research hotspot recently.^13–15

Compared with process samples in real industry, quality samples usually have a large time lag to collect and a relatively rare quantity. Hence, the direct use of quality samples cannot meet the requirement of real-time online monitoring. To tackle this situation, the commonly adopted idea is to first establish a regression model between process variables and quality variables, and then extract the quality-related features from process variables, which will replace quality variables to realize quality-related online fault detection. Currently, two mainstream frameworks based on this idea are least squares (LS)–based and partial least squares (PLS)–based approaches. Zhou et al. ¹⁶ comprehensively analyzed the defect of PLS ¹⁷ for quality-related fault detection and proposed a total projection to latent structures (TPLS) model. ¹⁸ Yin et al.^19,20 performed singular value decomposition (SVD) method on coefficient matrix of LS and PLS separately, and presented modified partial least squares (MPLS) and improved partial least squares (IPLS). Since all above are linear methods which are unsuitable for non-linear processes, the kernel trick has been widely adopted for non-linear fault detection. Its main idea is to map the original process variables into Reproducing Kernel Hilbert Space (RKHS) through some kernel function, thus making the process variables linearly separable in such kernel space. ²¹ By introducing the kernel trick, many linear methods can be transformed into non-linear versions.^14,22,23 However, all of the above methods design statistics without considering the local characteristics among samples.

k-Nearest neighbor (k-NN) rule is a classical machine learning method, which is usually adopted as a classifier.^24,25 Because it is capable of mining local characteristics between near neighbors, ²⁶ k-NN rule has been modified to propose a fault detection method based on the k-NN rule, namely, FD-k-NN. ²⁷ It provides a promising direction for the solution of the above problems to some extent. The statistics of FD-k-NN are designed by fully considering the Euclidean distance measurement among local neighbor samples. Due to its excellent performance, k-NN-based fault detection methods have been studied extensively.^28–30 Although these methods have been employed into various tasks, such as multirate sampling process ³¹ and multimode process, ³² quality-related fault detection methods based on k-NN rule have still not been well-established so far.

Fault isolation is a successor task of fault detection, which is utilized to locate fault sensor variables. Many classical fault isolation approaches have been proposed. Contribution plot and reconstruction-based contribution (RBC) are two most commonly used isolation methods, but they have relatively obvious smearing effect. To deal with this problem, many other fault isolation techniques have been addressed in previous works.^33–36 Unfortunately, these methods fail to be used in k-NN-based fault detection. To handle the problem, Zhou et al. ³⁷ proposed a novel fault isolation method based on k-NN rule, VCk-NN, which makes it possible to determine the failed sensors after detecting faults using FD-k-NN. Compared with previous methods, VCk-NN suffers from less effect of fault smearing. Nevertheless, VCk-NN assumes that the process samples follow a multivariate normal distribution, which greatly restricts its effect of fault isolation.

In this article, we propose a novel quality-related fault diagnosis framework, which is made up of two parts: non-linear quality-related fault detection and fault isolation. For quality-related fault detection task, the motivation of proposing kernel partial least squares (KPLS) by combining KPLS with k-NN rule is that (1) KPLS can effectively utilize process variables to obtain predictive quality variables, which will replace the actual quality variables that cannot get in real time, and (2) k-NN rule can mine information between a test sample and the nearest template samples for a better detection effect compared with only using a single test sample. For fault isolation task, the motivation to improve VCk-NN to MVCk-NN is that the hypothesis is sometimes not satisfied, and in this case, VCk-NN still has a smearing effect, so MVCk-NN is presented to deal with this problem by mitigating the influence of faulty variables on faultless variables. The main contributions of this article are summarized as follows:

• We propose a new quality-related non-linear fault diagnosis framework based on k-NN rule, including a new quality-related non-linear fault detection method KPLS-k-NN and a new fault isolation method modified VCk-NN.

• KPLS-k-NN is proposed by combining KPLS with k-NN rule. Quality-related statistics of KPLS-k-NN take into full consideration the local neighbor information among predicted quality samples, which greatly improve the detection rate (DR) for quality-related faulty samples.

• Modified VCk-NN is established by introducing the idea of the relative variable contributions of accumulative relative contribution rate (ARCR) into VCk-NN, which does not need the assumption that the process samples obey a multivariate normal distribution and has more precise isolation results in identifying latent fault root cause than VCk-NN.

The rest of this article is arranged as follows: first, give some relevant preliminaries. Afterward, the k-NN scheme for quality-related non-linear fault diagnosis is proposed in a detailed presentation. Then, the simulation results are provided and discussed. Finally, we conclude the article and present our future work.

Preliminaries

Let the non-linear process contain an input data matrix $\mathbf{X}$ which records $N$ samples of $m$ process variables and an output data matrix $\mathbf{Y}$ including $N$ samples of $l$ quality variables, that is

$$\mathbf{X}={\left[{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_N\right]}^{\mathrm{T}}\in {ℝ}^{N\times m}$$

(1)

$$\mathbf{Y}={[{\mathbf{y}}_1,{\mathbf{y}}_2,...,{\mathbf{y}}_N]}^{\mathrm{T}}\in {\mathbb{R}}^{N\times l}$$

(2)

where ${\mathbf{x}}_i\in {\mathbb{R}}^m$ and ${\mathbf{y}}_i\in {\mathbb{R}}^l(i=1,...,N)$ represent the $i\mathrm{th}$ sample of $\mathbf{X}$ and $\mathbf{Y}$ , respectively. All samples are supposed to obey normal distribution.

The implementation of KPLS method is divided into two steps as follows. First, kernel trick ²¹ is introduced into KPLS model to effectively deal with the non-linear relationship among variables. Given a kernel function $\varphi$ , it maps the original samples $x_i(i=1,2,\dots ,N)$ into a high-dimension kernel space $\mathcal{F}$ , which is defined as ${\mathbf{x}}_i\in {\mathbb{R}}^m\to \varphi ({\mathbf{x}}_i)\in {\mathbb{R}}^f$ , where $f$ is the dimension of $\mathcal{F}$ . Thus, the process matrix $\mathbf{X}$ is transformed into feature matrix $\mathbf{\Phi }$ as

$$\mathbf{\Phi }={[\varphi \left({\mathbf{x}}_1\right),\varphi \left({\mathbf{x}}_2\right),\dots ,\varphi \left({\mathbf{x}}_N\right)]}^{\mathrm{T}}\in {\mathbb{R}}^{N\times f}$$

(3)

As a necessary step, $\varphi ({\mathbf{x}}_i)$ needs to be processed to zero mean vector $\overline{\varphi }$ , that

$$\overline{\varphi }\left({\mathbf{x}}_i\right)=\varphi \left({\mathbf{x}}_i\right)-\overline{\varphi }$$

(4)

$$\overline{\varphi }=\frac{1}{N}\sum _{i=1}^N\varphi \left({\mathbf{x}}_i\right)=\frac{1}{N}{\mathbf{\Phi }}^{\mathrm{T}}{1}_N$$

(5)

where $1_N=[1,1,\dots ,1{]}^{\mathrm{T}}\in {\mathbb{R}}^N$ . Hence, the zero mean matrix of $\mathbf{\Phi }$ can be obtained by

$$\begin{array}{c}\multicolumn{1}{c}{\overline{\mathbf{\Phi }}={[\overline{\varphi }\left({\mathbf{x}}_1\right),\overline{\varphi }\left({\mathbf{x}}_2\right),\dots ,\overline{\varphi }\left({\mathbf{x}}_N\right)]}^{\mathrm{T}}}\\ \multicolumn{1}{c}{=({\mathbf{I}}_N-\frac{1}{N}{1}_N{1}_N^{\mathrm{T}})\mathbf{\Phi }}\end{array}$$

(6)

where ${\mathbf{I}}_N$ is the identity matrix. The concrete form of $\mathbf{\Phi }$ is unknowable, but its kernel matrix $\mathbf{K}=\mathbf{\Phi }{\mathbf{\Phi }}^{\mathrm{T}}\in {\mathbb{R}}^{N\times N}$ can be artificial setting. In this article, the radial basis function (RBF) kernel is used to calculate the elements in $\mathbf{K}$ , which is

$${\mathbf{K}}_{i,j}=\exp (-\frac{{\Vert {\mathbf{x}}_i-{\mathbf{x}}_j\Vert }^2}{2{\sigma }^2})(i,j=1,2,\dots ,N)$$

(7)

where $\sigma$ is the kernel parameter that needs to be set according to experience. The zero mean of $\mathbf{K}$ is calculated as follows

$$\overline{\mathbf{K}}=\overline{\mathbf{\Phi }}{\overline{\mathbf{\Phi }}}^{\mathrm{T}}=({\mathbf{I}}_N-\frac{1}{N}{1}_N{1}_N^{\mathrm{T}})\mathbf{K}({\mathbf{I}}_N-\frac{1}{N}{1}_N{1}_N^{\mathrm{T}})$$

(8)

Second, PLS model is established between $\overline{\mathbf{\Phi }}$ and $\mathbf{Y}$ in space $\mathcal{F}$ . Thus, the KPLS model is constructed as

$$\{\begin{array}{c}\overline{\mathbf{\Phi }}=\mathbf{T}{\mathbf{P}}^{\mathrm{T}}+{\overline{\mathbf{\Phi }}}_r\hfill \\ \multicolumn{1}{c}{\mathbf{Y}=\mathbf{U}{\mathbf{Q}}^{\mathrm{T}}+{\mathbf{Y}}_r}\end{array}$$

(9)

where $\mathbf{T}\in {\mathbb{R}}^{N\times a}$ and $\mathbf{P}\in {\mathbb{R}}^{f\times a}$ are the score matrix and the loading matrix of $\overline{\mathbf{\Phi }}$ , respectively, $\mathbf{U}\in {\mathbb{R}}^{N\times a}$ and $\mathbf{Q}\in {\mathbb{R}}^{l\times a}$ are the score matrix and the loading matrix of $\mathbf{Y}$ , respectively, ${\overline{\mathbf{\Phi }}}_r$ and ${\mathbf{Y}}_r$ are the residual matrices, and $a$ represents the number of latent variables.

The iterative calculation algorithm of KPLS has been elaborated in Jiao et al., ¹⁴ in which we can get the score matrix $\mathbf{U}$ and $\mathbf{R}$ that is $\mathbf{R}={\overline{\mathbf{\Phi }}}^{\mathrm{T}}\mathbf{U}({\mathbf{T}}^{\mathrm{T}}\overline{\mathbf{K}}\mathbf{U}{)}^{-1}$ . For the score matrix $\mathbf{T}$ and the loading matrix $\mathbf{Q}$ , the following equations hold

$$\mathbf{T}\mathbf{=}\overline{\mathbf{\Phi }}\mathbf{R}$$

(10)

$$\mathbf{Q}\mathbf{=}{\mathbf{Y}}^{\mathrm{T}}\mathbf{T}$$

(11)

Obviously, the score vector ${\mathbf{t}}_{\mathrm{new}}$ of $\overline{\varphi }({\mathbf{x}}_{\mathrm{new}})$ is

$${\mathbf{t}}_{\mathrm{new}}={\mathbf{R}}^{\mathrm{T}}\overline{\varphi }\left({\mathbf{x}}_{\mathrm{new}}\right)$$

(12)

Methodology

FD-k-NN as a popular fault detection approach has ability in determining whether a fault has occurred in the process, but it cannot estimate whether the fault occurred will have an impact on the production quality.

Therefore, in this section, we propose k-NN-based fault diagnosis scheme. Its fault detection scheme KPLS-k-NN is designed as follows: first, FD-k-NN is employed to monitor the process space. Then, KPLS is adopted to obtain the predicted quality samples ${\mathbf{Y}}_p$ of training process samples $\mathbf{X}$ in predicted quality space, where quality-related fault detection will be carried out. When KPLS-k-NN indicates that the system has failed, a new fault isolation method MVCk-NN is given for better locating the faulty sensor variables.

The proposed KPLS-k-NN for fault detection

Fault detection in process space

FD-k-NN is designed through following the principle: any normal samples should be close to other normal samples to some extent, while for a faulty sample, it should deviate from normal samples. Usually, the degree of deviation is measured by k-NN distance, which is defined as the average square distance between the test sample and its k-NNs from the training normal samples. When the k-NN distance of a sample exceeds the threshold, it is considered as a faulty sample, otherwise, it is judged as a normal sample. The details of the algorithm are as follows.

Model building

Given the training samples $\mathbf{X}$ , the model is built according to the following procedures:

• 1. Find the k-NNs for each sample ${\mathbf{x}}_i$ in $\mathbf{X}$ and compute all the Euclidean distance, that

$$d_{\mathrm{ij}}=\Vert {\mathbf{x}}_i-{\mathbf{x}}_j{\Vert }_2,j\in k-\mathrm{NNs}\left({\mathbf{x}}_i\right)$$

(13)

where $k-\mathrm{NNs}({\mathbf{x}}_i)$ represents the set of k-NNs of ${\mathbf{x}}_i$ .

• 2. Calculate the k-NN distance of ${\mathbf{x}}_i$ . k-NN distance is adopted as the statistics $D_{\mathbf{x}}^2({\mathbf{x}}_i)$ as

$$D_{\mathbf{x}}^2\left({\mathbf{x}}_i\right)=\frac{1}{k}\sum _{j=1}^k{d}_{\mathrm{ij}}^2$$

(14)

• 3. Determine the threshold $D_{\mathbf{x},\alpha }^2$ :

$D_{\mathbf{x}}^2$ is rearranged in descending order as $D_{\mathbf{x}\_\mathrm{arrange}}^2$ , of which $(1-\alpha )-\mathrm{empirical}\mathrm{quartile}$ is chosen as $D_{\mathbf{x},\alpha }^2$ in Zhou et al., ³⁷ that is, $D_{\mathbf{x}\_\mathrm{arrange}}^2(\lfloor N(1-\alpha )\rfloor )$ .

Fault detection

For a new incoming test sample ${\mathbf{x}}_{\mathrm{new}}$ , the sample categories are inferred as the following steps:

1. Find ${\mathbf{x}}_{\mathrm{new}}\mathrm{\text{'}}\mathrm{s}$ k-NNs from $\mathbf{X}$ .

2. Compute ${\mathbf{x}}_{\mathrm{new}}\mathrm{\text{'}}\mathrm{s}$ k-NN distance $D_{\mathbf{x}}^2({\mathbf{x}}_{\mathrm{new}})$ .

3. Compare $D_{\mathbf{x}}^2({\mathbf{x}}_{\mathrm{new}})$ with the threshold $D_{\mathbf{x},\alpha }^2$ :

If $D_{\mathbf{x}}^2({\mathbf{x}}_{\mathrm{new}})>D_{\mathbf{x},\alpha }^2$ , then ${\mathbf{x}}_{\mathrm{new}}$ is considered as a faulty sample. Otherwise, ${\mathbf{x}}_{\mathrm{new}}$ is a normal sample.

Fault detection in predicted quality space

Given the training process samples $\mathbf{X}$ and training quality samples $\mathbf{Y}$ . To obtain the predicted quality samples ${\mathbf{Y}}_p$ of training process samples $\mathbf{X}$ , KPLS ⁴¹ is adopted to obtain ${\mathbf{Y}}_p$ (the prediction value of $\mathbf{Y}$ ), since it has the ability to utilize $\mathbf{X}$ to effectively mine the essential information of $\mathbf{Y}$ . We set the coefficient matrix of $\mathbf{X}$ and ${\mathbf{Y}}_p$ as $\mathbf{M}$ , which is

$$\mathbf{M}=\mathbf{U}{({\mathbf{T}}^{\mathrm{T}}\overline{\mathbf{K}}\mathbf{U})}^{-1}{\mathbf{T}}^{\mathrm{T}}\mathbf{Y}$$

(15)

According to the calculation of equations (10) and (11), the predictive output ${\mathbf{Y}}_p$ is calculated by

$${\mathbf{Y}}_p=\mathbf{T}{\mathbf{Q}}^{\mathrm{T}}=\overline{\mathbf{\Phi }}{\overline{\mathbf{\Phi }}}^{\mathrm{T}}\mathbf{U}{({\mathbf{T}}^{\mathrm{T}}\overline{\mathbf{K}}\mathbf{U})}^{-\mathbf{1}}{\mathbf{T}}^{\mathrm{T}}\mathbf{Y}=\overline{\mathbf{K}}\mathbf{M}$$

(16)

Similar to process space, we call the space where ${\mathbf{y}}_p(i=1,\dots ,N)\in {\mathbf{Y}}_p$ is located predicted quality space.

At this point, FD-k-NN is employed to perform the detection of quality-related faults in predicted quality space. We need to find the k-NNs for each sample ${\mathbf{y}}_p$ in ${\mathbf{Y}}_p$ , and obtain the ${\mathbf{y}}_p\mathrm{\text{'}}\mathrm{s}$ k-NN distance $D_{\mathbf{y}}^2({\mathbf{y}}_p)$ according to equation (14). Notice that since $\mathbf{X}$ and ${\mathbf{Y}}_p$ generally do not obey the same distribution, for $\mathbf{x}$ and ${\mathbf{y}}_p$ in the same point, their corresponding k-NNs might not be the samples in the same points. Hence, the k-NNs for $\mathbf{x}$ and ${\mathbf{y}}_p$ should be calculated separately. In addition, our KPLS-k-NN method does not need to carry out any variable transformation in the original space, and it directly depends on the Euclidian distance of the variables in the original space as an index to quantify the discrepancy between test samples and normal samples.

Different from the threshold in Zhou et al., ³⁷ in this article, kernel density estimation (KDE) ⁴² , as a non-parameter probability density estimation method of random variable, is utilized to determine the threshold for two monitoring spaces, which can be referred in detail in Parzen. ³⁸ Thus, corresponding to $D_{\mathbf{x}}^2$ and $D_{\mathbf{y}}^2$ , we will get their thresholds $D_{\mathbf{x},\alpha }^2$ and $D_{\mathbf{y},\alpha }^2$ .

For a new incoming test sample ${\mathbf{x}}_{\mathrm{new}}$ , its predicted quality ${\mathbf{y}}_{p,\mathrm{new}}$ is calculated by equations (12) and (15) as follows

$${\mathbf{y}}_{p,\mathrm{new}}=\mathbf{Q}{\mathbf{t}}_{\mathrm{new}}={\mathbf{M}}^{\mathrm{T}}\overline{\mathbf{\Phi }}\overline{\varphi }\left({\mathbf{x}}_{\mathrm{new}}\right)={\mathbf{M}}^{\mathrm{T}}{\overline{\mathbf{k}}}_{\mathrm{new}}$$

(17)

To determine whether ${\mathbf{y}}_{p,\mathrm{new}}$ is a faulty sample, we find ${\mathbf{y}}_{p,\mathrm{new}}\mathrm{\text{'}}\mathrm{s}$ k-NNs from ${\mathbf{Y}}_p$ , and compute ${\mathbf{y}}_{p,\mathrm{new}}\mathrm{\text{'}}\mathrm{s}$ k-NN distance $D_{\mathbf{y}}^2({\mathbf{y}}_{p,\mathrm{new}})$ .

Finally, detection logic is performed by comparing $D_{\mathbf{y}}^2({\mathbf{y}}_{p,\mathrm{new}})$ with the threshold $D_{\mathbf{y},\alpha }^2$ : if $D_{\mathbf{y}}^2({\mathbf{y}}_{p,\mathrm{new}})>D_{\mathbf{y},\alpha }^2$ , then ${\mathbf{x}}_{\mathrm{new}}$ is considered as a quality-related faulty sample. Otherwise, ${\mathbf{x}}_{\mathrm{new}}$ is a normal sample or a quality-unrelated faulty sample.

The whole scheme of KPLS-k-NN

By combining fault detection in process space as well as in predicted quality space, our proposed the whole KPLS-k-NN non-linear quality-related fault detection scheme is summarized as follows:

Offline modeling

1. Normalize the training process sample $\mathbf{X}$ and training quality samples $\mathbf{Y}$ to the zero mean and unit variance.

2. Obtain the coefficient matrix $\mathbf{M}$ of KPLS by equation (15) and get ${\mathbf{Y}}_p$ using equation (16).

3. Calculate $\mathbf{x}\mathrm{\text{'}}\mathrm{s}$ k-NN distance $D_{\mathbf{x}}^2(\mathbf{x})$ and ${\mathbf{y}}_p\mathrm{\text{'}}\mathrm{s}$ k-NN distance $D_{\mathbf{y}}^2({\mathbf{y}}_p)$ to get the thresholds $D_{\mathbf{x},\alpha }^2$ and $D_{\mathbf{y},\alpha }^2$ .

Online detection

For a new incoming test sample ${\mathbf{x}}_{\mathrm{new}}$ :

1. Obtain ${\mathbf{y}}_{p,\mathrm{new}}$ using equation (17).

2. Compute the statistics $D_{\mathbf{x}}^2({\mathbf{x}}_{\mathrm{new}})$ and $D_{\mathbf{y}}^2({{\mathbf{y}}_{p,}}_{\mathrm{new}})$ .

Detection logic

1. If $D_{{\mathbf{x}}_{\mathrm{new}}}^2\le D_{\mathbf{x},\alpha }^2$ and $D_{{\mathbf{y}}_{\mathrm{new}}}^2\le D_{\mathbf{y},\alpha }^2$ , the system is fault-free.

2. If $D_{{\mathbf{y}}_{\mathrm{new}}}^2>D_{\mathbf{y},\alpha }^2$ , the system has quality-related faults.

3. If $D_{{\mathbf{y}}_{\mathrm{new}}}^2\le D_{\mathbf{y},\alpha }^2$ and $D_{{\mathbf{x}}_{\mathrm{new}}}^2>D_{\mathbf{x},\alpha }^2$ , the system has some quality-unrelated faults.

Notice that our KPLS-k-NN is essentially a supervised quality-related fault detection method, which is designed by combining KPLS with FD-k-NN. The above seems to be a simple combination, but FD-k-NN as an unsupervised method is successfully applied to complete a supervised fault detection task.

The proposed MVCk-NN for fault isolation

The KPLS-k-NN-based fault detection approach has been presented in the above section, which can effectively judge whether there are some faults in the process and whether the faults are related to product quality. Next, when KPLS-k-NN indicates the system exists some faults, a fault isolation method matched with KPLS-k-NN will be needed to locate faulty variables. Here, we propose a new fault isolation method MVCk-NN in detail.

In general, the sensor fault as a kind of system faults is classified as the additive fault. Hence, a process sample $\mathbf{x}\in {\mathbb{R}}^m$ can be expressed as

$$\mathbf{x}={\mathbf{x}}^{*}+\mathbf{f}={\mathbf{x}}^{*}+\sum _{i=1}^m{\xi }_i{f}_i$$

(18)

where ${\mathbf{x}}^{*}\in {\mathbb{R}}^m$ denotes the fault-free component of $\mathbf{x}$ and $\mathbf{f}\in {\mathbb{R}}^m$ denotes the fault component. If $\mathbf{f}\mathbf{=}0$ , $\mathbf{x}$ is a normal sample, otherwise a faulty sample. ${\xi }_i\in {\mathbb{R}}^m(i=1,\dots m)$ is the fault-direction indicator vector of the ith process variable among $m$ variables, of which the ith element is $1$ and other elements are $0$ .

To locate which sensors cause the statistics $D_{\mathbf{x}}^2$ to alarm faults, the contributions from all variables of $\mathbf{x}$ to $D_{\mathbf{x}}^2$ need to be calculated. We decompose $D_{\mathbf{x}}^2$ as follows

$$D_{\mathbf{x}}^2=\frac{1}{k}\sum _{j=1}^k\Vert \mathbf{x}-{\mathbf{x}}_j{\Vert }_2^2=\sum _{i=1}^m\left\{\frac{1}{k}\sum _{j=1}^k\left[{\xi }_i^T(\mathbf{x}-{\mathbf{x}}_j)\right]\right\}$$

(19)

Then, define the contribution from ith variable of $\mathbf{x}$ to $D_{\mathbf{x}}^2$ as

$$C(\mathbf{x},i)=\frac{1}{k}\sum _{j=1}^k\left[{\xi }_i^T(\mathbf{x}-{\mathbf{x}}_j)\right]$$

(20)

Obviously, by equations (19) and (20), $D_{\mathbf{x}}^2$ is the sum of the contributions of all variables, namely, $D_{\mathbf{x}}^2=\sum _{i=1}^{m}C(\mathbf{x},i)$ .

Next, we discuss the influence of the fault magnitude on variable contributions. Set $\mathbf{x}$ include at least one fault. Without loss of generality, we assume the ith variable be faulty, and others be normal or faulty, that

$$\mathbf{x}={\mathbf{x}}^{*}+{\xi }_i{f}_i+\sum _{j=1,j\ne i}^m{\xi }_j{f}_j(f_i\ne 0)$$

(21)

By equations (20) and (21), we have

$$C(\mathbf{x},i)=\frac{1}{k}\sum _{j=1}^k\left[{\xi }_i^T({\mathbf{x}}^{*}-{\mathbf{x}}_j+{\xi }_i{f}_i+\sum _{j=1,j\ne i}^m{\xi }_j{f}_j)\right]$$

(22)

According to neighborhood relationship, a reasonable hypothesis is given ³⁷ as follows.

Hypothesis 1

Any variable’s fault magnitude is much larger than the Euclidean distance between the variable and its neighbors on the same dimension, that

$$\Vert f_i{\Vert }_2\ge \Vert {\left[{\mathbf{x}}^{*}\right]}_i-{\left[{\mathbf{x}}_j\right]}_i{\Vert }_2$$

(23)

where $[·{]}_i$ represents the ith component of vector.

Due to ${\xi }_i^T{\xi }_i=1$ , ${\xi }_i^T{\xi }_j=0,\forall j\ne i$ , and Hypothesis 1, it can be obtained that

$$C(\mathbf{x},i)\approx \frac{1}{k}\sum _{j=1}^k{f}_i^2=f_i^2$$

(24)

It indicates that the contribution of each fault variable to the statistics is approximately equal to the square of the fault magnitude, and this method hardly suffers from smearing effect when the Hypothesis 1 is satisfied.

The above are VCk-NN-based variable contributions, but there exists a drawback that Hypothesis 1 sometimes cannot be enough satisfied, which causes that the contribution of fault variables may not be significantly different from that of normal variables, so as to increase the smearing effect.

Therefore, our proposed MVCk-NN introduces the idea of ARCR ³⁵ into VCk-NN to obtain the relative variable contributions instead of absolute ones, so that the smearing effect is further eliminated. The detailed steps are illustrated as follows:

• 1. For a new sample ${\mathbf{x}}_{\mathrm{new}}\in {\mathbb{R}}^m$ , when $m=1$ , it means process variable is a single source variable. In this case, once a fault occurs, the faulty variable must be this process variable. However, when $m\ge 2$ , we need to execute the following Steps 2–5 to isolate faulty process variables.

• 2. Normalize each $C({\mathbf{x}}_{\mathrm{new}},i)$ obtained by equation (20) to guarantee that all variables contribute more or less the same to ${\mathbf{x}}_{\mathrm{new}}$

$$\hat{C}({\mathbf{x}}_{\mathrm{new}},i)=\frac{C({\mathbf{x}}_{\mathrm{new}},i)}{\frac{1}{N}\sum _{j=1}^{N}C({\mathbf{x}}_j,i)}$$

(25)

• 3. To eliminate the smearing effect, $\hat{C}({\mathbf{x}}_{\mathrm{new}},i)$ is divided by the sum of all the $m$ normalized contributions to obtain relative variable contribution $C_r({\mathbf{x}}_{\mathrm{new}},i)$

$$C_r({\mathbf{x}}_{\mathrm{new}},i)=\frac{\hat{C}({\mathbf{x}}_{\mathrm{new}},i)}{\sum _{i=1}^m\hat{C}({\mathbf{x}}_{\mathrm{new}},i)}$$

(26)

• 4. The recommended experience threshold is given in Peng et al. ³⁵

$$\theta =\frac{{\Vert C_r\left({\mathbf{x}}_{\mathrm{new}}\right)\Vert }_2+\frac{1}{m}}{2}$$

(27)

where $C_r({\mathbf{x}}_{\mathrm{new}})=[C_r({\mathbf{x}}_{\mathrm{new}},1),\dots ,C_r({\mathbf{x}}_{\mathrm{new}},m){]}^T$ .

• 5. If $C_r({\mathbf{x}}_{\mathrm{new}},i)$ is more than $\theta$ , the ith variable of ${\mathbf{x}}_{\mathrm{new}}$ can be considered as a root cause of faults, otherwise not.

Through the above derivation, the flow chart of the proposed k-NN-based quality-related non-linear fault diagnosis scheme is summarized in Figure 1.

Figure 1.

The quality-related non-linear fault diagnosis framework of k-NN.

Case study

This section applies a widely typical numerical and a real industrial Tennessee Eastman (TE) process benchmark to validate the effectiveness of our proposed method. Two fault evaluation indexes are adopted for performance evaluation. In the fault detection stage, our method KPLS-k-NN will be compared with the state-of-the-art approach total kernel projection to latent structures (TKPLS) ²² and the most recent SVD-based non-linear method modified kernel least squares (MKLS) ²³ to show its superiorities. In the fault isolation stage, our MVCk-NN will be compared with VCk-NN. Besides, in the experiment, the confidence level $\alpha$ is set to 0.99.

Evaluation index

Two evaluation indexes—that is, the fault DR and the false alarm rate (FAR)—are used in our experiments, which are defined as follows

$$\mathrm{D}\mathrm{R}=\frac{\mathrm{Number}\mathrm{of}\mathrm{effective}\mathrm{alarms}}{\mathrm{Total}\mathrm{faulty}\mathrm{samples}}\times 100\%$$

(28)

$$\mathrm{F}\mathrm{AR}=\frac{\mathrm{Number}\mathrm{of}\mathrm{false}\mathrm{alarms}}{\mathrm{Total}\mathrm{faulty}\mathrm{samples}}\times 100\%$$

(29)

where an effective alarm represents a quality-related faulty sample is detected, while a false alarm represents a quality-unrelated faulty sample is detected. For performance evaluation, when a quality-related fault occurs, DR is adopted as the key indicator, while when a quality-unrelated fault happens, then FAR is used as the key indicator. More details on DR and FAR can be referred to Wang and Jiao. ²³

Typical numerical example

The following numerical example introduced in Peng et al. ²² is applied

$$\{\begin{array}{c}{x}_1~N\left({1,0.01}^2\right),x_2~N\left({1,0.01}^2\right)\hfill \\ \multicolumn{1}{c}{x_3=\sin \left(x_1\right)+e_1}\\ \multicolumn{1}{c}{x_4=x_1^2-3x_1+4+e_2}\\ \multicolumn{1}{c}{x_5=x_2^2+\cos \left(x_2^2\right)+1+e_3}\\ \multicolumn{1}{c}{y=x_3^2+x_3{x}_4+x_1+v}\end{array}$$

(30)

where $e_i~N(0,{0.001}^2)(i=1,2,3)$ , $v~N(0,0.{005}^2)$ , and $e_i$ and $v$ denote the noises. $\mathbf{x}=[x_1,x_2,x_3,x_4,x_5{]}^{\mathrm{T}}$ is the process variable, and $y$ is the quality variable.

From the above equation, we can see that $y$ can only be affected by $x_1$ , $x_3$ , and $x_4$ , but not by $x_2$ and $x_5$ . Hence, when a fault occurs in $x_1$ , it will have influence on quality variable $y$ . While a fault happens in $x_2$ , it will not do anything to $y$ . We generate 400 normal samples as training samples for offline modeling, 400 normal samples as validation samples for selecting the hyper-parameters, and 400 test samples which include 200 normal samples and 200 faulty samples for online detection. The fault scenarios are as follows:

• Fault 1: step bias occurs in $x_1$ : $x_1=x_1^{*}+f$ .

• Fault 2: ramp change occurs in $x_1$ : $x_1=x_1^{*}+(t-200)f$ .

• Fault 3: step bias occurs in $x_2$ : $x_2=x_2^{*}+f$ .

• Fault 4: ramp change occurs in $x_2$ : $x_2=x_2^{*}+(t-200)f$ , where $x_1^{*}$ and $x_2^{*}$ are the normal values of $x_1$ and $x_2$ , respectively, $t$ $(201\le t\le 400)$ is the sequence number of the $t\mathrm{th}$ sample, and $f$ is the fault magnitude. It can be seen that Fault 1 and Fault 2 are quality-related faults, while Fault 3 and Fault 4 are quality-unrelated faults.

The model parameters are $N=400$ , $m=5$ , and $l=1$ . For the choice of hyper-parameters, that is, $A$ , $c$ , and $k$ , we utilize cross-validation method to determine them. In detail, in the normal state, several sets of parameters with the minimum of sum of FARs of process variables and quality variables will be selected as the alternative hyper-parameters. In addition, we also need to consider the predictive performance of KPLS meanwhile. Therefore, we select one from the alternative hyper-parameters that has the minimum prediction error as final hyper-parameters. In this experiment, the hyper-parameters are set as $A=2$ , $c=2\times {10}^4$ , and $k=5$ . The significance level $\alpha$ is 0.01. To make the experiment more comprehensive, the magnitude of Fault 1 is changed in turn that $f=0.2,0.4,0.6,0.8$ , while for Fault 2, $f=0.002,0.003,0.004,0.005$ .

The results of quality-related faults

Fault detection

The detection results for Fault 1 and Fault 2 of KPLS-k-NN, MKLS, and TKPLS are displayed in Table 1. As shown in Table 1, the DRs of three methods to detect Fault 1 are all $100\%$ , and the DRs of three methods for Fault 2 are almost above $90\%$ . So, three methods all give right detection results.

Table 1.

Detection results of KPLS-k-NN, MKLS, and TKPLS for quality-related Fault 1 and Fault 2 (%).

Fault 1								Fault 2
f	KPLS-k-NN		MKLS		TKPLS		f	KPLS-k-NN		MKLS		TKPLS
	$D_{\mathbf{y}}^2$	$D_{\mathbf{x}}^2$	$T_{\mathrm{mkls}}^2$	$Q_{\mathrm{mkls}}$	$T_{\mathrm{ky}}^2\&Q_{\mathrm{kr}}$	$T_{\mathrm{ko}}^2\&T_{\mathrm{kr}}^2$		$D_{\mathbf{y}}^2$	$D_{\mathbf{x}}^2$	$T_{\mathrm{mkls}}^2$	$Q_{\mathrm{mkls}}$	$T_{\mathrm{ky}}^2\&Q_{\mathrm{kr}}$	$T_{\mathrm{ko}}^2\&T_{\mathrm{kr}}^2$
0.2	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	0.002	${\left(92\right)}^a$	${(93.5)}^a$	${\left(95\right)}^a$	${\left(93\right)}^a$	${(94.5)}^a$	${(86.5)}^a$
0.4	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	0.003	${\left(96\right)}^a$	${(96.5)}^a$	${\left(97\right)}^a$	${(96.5)}^a$	${(96.5)}^a$	${\left(93\right)}^a$
0.6	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	0.004	${(97.5)}^a$	${(97.5)}^a$	${(98.5)}^a$	${\left(98\right)}^a$	${\left(98\right)}^a$	${(92.5)}^a$
0.8	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	0.005	${\left(97\right)}^a$	${(97.5)}^a$	${\left(99\right)}^a$	${(97.5)}^a$	${(98.5)}^a$	${(95.5)}^a$

KPLS-k-NN: kernel partial least squares-k-nearest neighbor.

where “a” refers to DRs.

Furthermore, without loss of generally, we set $f=0.4$ in Fault 1 and observe the statistics values of three methods. From Figure 2, it is obvious that for these three methods, almost of all statistics of normal samples are below the threshold, while those faulty samples are significantly above the threshold. Therefore, three methods all have a good detection ability for quality-related faults.

Figure 2.

Detection results of quality-related Fault 1 with $f=0.4$ by (a) KPLS-k-NN, (b) MKLS, and (c) TKPLS.

Fault isolation

When $f=0.4$ , the Fault 1 isolation results of MVCk-NN and VCk-NN are shown in Figure 3. According to the setting of Fault 1, variable $x_1$ is the root cause of Fault 1, while $x_3$ and $x_4$ are also affected by $x_1$ , but they are not root causes. In Figure 3(a), VCk-NN regards $x_1$ , $x_3$ , and $x_4$ as the root cause of Fault 1, while from Figure 3(b), MVCk-NN can only identify $x_1$ as the root cause of Fault 1. By contrast with VCk-NN, our proposed MVCk-NN has higher accuracy in the fault isolation task.

Figure 3.

Isolation results of quality-related Fault 1 with $f=0.4$ by (a) MVCk-NN and (b) VCk-NN.

The results of quality-unrelated faults

Fault detection

The results to detect Fault 3 and Fault 4 using three methods are shown in Table 2. Because the quality-unrelated statistics of these three approaches are all over $90\%$ , they can all warn that some process faults have happened in the system. The FARs of the statistics using TKPLS are over $90\%$ , so TKPLS fails to identify the true natures of these faults. For MKLS, although its FARs of the statistics $T_{\mathrm{mkls}}^2$ are all less than $1\%$ , they are still not equal to $0$ . However, using KPLS-k-NN, the FARs of the statistics $D_{\mathbf{y}}^2$ are all $0$ , so it achieves the best performance in comparison methods.

Table 2.

Detection results of KPLS-k-NN, MKLS, and TKPLS for quality-unrelated Fault 3 and Fault 4 (%).

Fault 1								Fault 2
f	KPLS-k-NN		MKLS		TKPLS		f	KPLS-k-NN		MKLS		TKPLS
	$D_{\mathbf{y}}^2$	$D_{\mathbf{x}}^2$	$T_{\mathrm{mkls}}^2$	$Q_{\mathrm{mkls}}$	$T_{\mathrm{ky}}^2\&Q_{\mathrm{kr}}$	$T_{\mathrm{ko}}^2\&T_{\mathrm{kr}}^2$		$D_{\mathbf{y}}^2$	$D_{\mathbf{x}}^2$	$T_{\mathrm{mkls}}^2$	$Q_{\mathrm{mkls}}$	$T_{\mathrm{ky}}^2\&Q_{\mathrm{kr}}$	$T_{\mathrm{ko}}^2\&T_{\mathrm{kr}}^2$
0.2	${\left(0\right)}^b$	${\left(100\right)}^a$	${\left(0\right)}^b$	${\left(100\right)}^a$	${\left(100\right)}^b$	${\left(100\right)}^a$	0.002	${\left(0\right)}^b$	${\left(92\right)}^a$	${\left(1\right)}^b$	${\left(92\right)}^a$	${(91.5)}^b$	${\left(92\right)}^a$
0.4	${\left(0\right)}^b$	${\left(100\right)}^a$	${(0.5)}^b$	${\left(100\right)}^a$	${\left(100\right)}^b$	${\left(100\right)}^a$	0.003	${\left(0\right)}^b$	${\left(96\right)}^a$	${(0.5)}^b$	${(96.5)}^a$	${\left(96\right)}^b$	${\left(96\right)}^a$
0.6	${\left(0\right)}^b$	${\left(100\right)}^a$	${(0.5)}^b$	${\left(100\right)}^a$	${\left(100\right)}^b$	${\left(100\right)}^a$	0.004	${\left(0\right)}^b$	${(95.5)}^a$	${(0.5)}^b$	${(95.5)}^a$	${(95.5)}^b$	${\left(96\right)}^a$
0.8	${\left(0\right)}^b$	${\left(100\right)}^a$	${\left(0\right)}^b$	${\left(100\right)}^a$	${\left(100\right)}^b$	${\left(100\right)}^a$	0.005	${\left(0\right)}^b$	${\left(97\right)}^a$	${(0.5)}^b$	${\left(97\right)}^a$	${(96.5)}^b$	${(97.5)}^a$

KPLS-k-NN: kernel partial least squares-k-nearest neighbor.

where “a” refers to DRs and “b” refers to FARs.

We set $f=0.003$ in Fault 2 to valid the fault detection and isolation performance for quality-unrelated faults. The values of three methods’ statistics are shown in Figure 4. From Figure 4, when Fault 2 occurs, quality-unrelated statistics of three methods are all over their own thresholds, so they all indicate some faults have occurred. Because $Q_{\mathrm{kr}}^2$ of TKPLS is almost higher than its threshold in all faulty samples, it misunderstands that the faults are quality-related, so it fails. Since quality-related statistics of both the KPLS-k-NN method and MKLS are lower than the thresholds, they both can give the correct results: the system exists some quality-unrelated faults. Compared with $T_{\mathrm{mkls}}^2$ of MKLS, it is very obvious that $D_{\mathbf{y}}^2$ of KPLS-k-NN can achieve a better margin, because it is much lower than the threshold. Therefore, for quality-unrelated faults, KPLS-k-NN can make a clearer division between quality-related and quality-unrelated samples.

Figure 4.

Detection results of quality-unrelated Fault 2 with $f=0.003$ by (a) KPLS-k-NN, (b) MKLS, and (c) TKPLS.

Fault isolation

When $f=0.003$ , the Fault 2 isolation results of two methods are displayed in Figure 5. Variable $x_2$ is the root cause of Fault 2. Although $x_5$ is not root cause, it will be affected by $x_2$ . In Figure 3, VCk-NN considers $x_2$ and $x_5$ both as root cause variables, while our MVCk-NN only determines that $x_2$ is the root cause of Fault 2, which is consistent with the fact.

Figure 5.

Isolation results of quality-related Fault 2 with $f=0.003$ by (a) MVCk-NN and (b) VCk-NN.

Through the above analysis of the experimental results, our method KPLS-k-NN does a much excellent job for quality-related fault detection task than other comparison methods. Besides, when KPLS-k-NN indicates some faults happen, our proposed MVCk-NN has high accuracy in fault root cause diagnosis and has prominent advantages over VCk-NN.

TE benchmark

TE process is a real simulation benchmark of industrial process, which has been widely utilized for the simulation and verification of various control and MSPM approaches, ³⁹ and its structure flowchart is shown in Figure 6. The variables in this process contain two blocks of variables: the XMV block of 11 manipulated variables and the XMEAS block of 41 measured variables which include 22 process and 19 analysis variables. In this simulation, 22 process variables (XMEAS (1–22)) and 11 manipulated variables (XMV (1–11)) are chosen to be process input $\mathbf{X}$ , and select purge gas analysis component G (XMEAS (35)) as the quality output $\mathbf{Y}$ .

Figure 6.

The structure flowchart of the TE process.

The training data set and validation data set contain 500 and 960 fault-free data samples, respectively. For the test data set, it is composed of 21 different fault sets, each of which includes 960 data samples and they are displayed in Table 3. The fault categories can be roughly divided into the following ones: step faults, random variation faults, slow drift faults, sticking faults, constant position faults, and some unknown faults. The detailed fault information is described in Downs and Vogel ⁴⁰ and the website (http://depts.washington.edu/control/LARRY/TE/download.html).

Table 3.

Fault types in the TE process.

Fault	Description	Type
IDV(1)	A/C Feed ratio, B composition constant (Stream 4)	Step
IDV(2)	B composition, A/C ratio constant (Stream 4)	Step
IDV(3)	D feed temperature (Stream 2)	Step
IDV(4)	Reactor cooling water inlet temperature	Step
IDV(5)	Condenser cooling water inlet temperature	Step
IDV(6)	A feed loss (Stream 1)	Step
IDV(7)	C header pressure loss (Stream 4)	Step
IDV(8)	A, B, C feed composition (Stream 4)	Random variation
IDV(9)	D feed temperature (Stream 2)	Random variation
IDV(10)	C feed temperature (Stream 4)	Random variation
IDV(11)	Reactor cooling water inlet temperature	Random variation
IDV(12)	Condenser cooling water inlet temperature	Random variation
IDV(13)	Reactor kinetics	Slow drift
IDV(14)	Reactor cooling water valve	Sticking
IDV(15)	Condenser cooling water valve	Sticking
IDV(16)	Unknown	Unknown
IDV(17)	Unknown	Unknown
IDV(18)	Unknown	Unknown
IDV(19)	Unknown	Unknown
IDV(20)	Unknown	Unknown
IDV(21)	Valve (Stream 4)	Constant position

TE: Tennessee Eastman.

To classify the process faults into the category of affecting $\mathbf{Y}$ and that of not affecting, the criterion of Zhou et al. ¹⁶ is adopted, that is, if $n_y/n_t>10\%$ , then the faults are considered to be quality-related, otherwise quality-unrelated, where $n_y/n_t$ denotes the affected rate of $\mathbf{Y}$ . According to the validation data set, the model parameters are $N=500$ , $m=33$ , and $l=1$ and the hyper-parameters are set as $A=10$ , $c=2\times {10}^4$ , and $k=10$ .

The results of fault detection

For the quality-related faults, detection results of KPLS-k-NN, MKLS and TKPLS are presented in Table 4. We can see that TKPLS performs better than KPLS-k-NN and MKLS since all its FDRs are higher than the corresponding ones of the other methods. However, KPLS-k-NN and MKLS still provide satisfactory results, with most of their statistics above being $10\%$ . The DRs of KPLS-k-NN are significantly superior to MKLS in IDV(6) and IDV(18).

Table 4.

Detection results of KPLS-k-NN, MKLS, and TKPLS for quality-related faults in the TE process (%).

Fault number	$n_y/n_t$	KPLS-k-NN		MKLS		TKPLS
		$D_{\mathbf{y}}^2$	$D_{\mathbf{x}}^2$	$T_{\mathrm{mkls}}^2$	$Q_{\mathrm{mkls}}$	$T_{\mathrm{ky}}^2\&Q_{\mathrm{kr}}$	$T_{\mathrm{ko}}^2\&T_{\mathrm{kr}}^2$
IDV(1)	22.72	${(82.25)}^a$	${(99.75)}^a$	${(34.88)}^a$	${(99.88)}^a$	${(99.75)}^a$	${(99.75)}^a$
IDV(2)	72.16	${(85.75)}^a$	${(98.63)}^a$	${(86.88)}^a$	${(98.50)}^a$	${(98.50)}^a$	${(98.75)}^a$
IDV(5)	14.98	${(13.88)}^a$	${(36.63)}^a$	${(23.50)}^a$	${(33.38)}^a$	${(33.38)}^a$	${(33.25)}^a$
IDV(6)	95.63	${(94.13)}^a$	${\left(100\right)}^a$	${(17.75)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${(99.25)}^a$
IDV(7)	18.23	${(29.88)}^a$	${\left(100\right)}^a$	${(32.63)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$	${\left(100\right)}^a$
IDV(8)	58.93	${(62.25)}^a$	${(98.63)}^a$	${(79.00)}^a$	${(97.75)}^a$	${(98.50)}^a$	${(97.75)}^a$
IDV(10)	11.99	${(60.12)}^a$	${(63.88)}^a$	${(26.75)}^a$	${(59.38)}^a$	${(73.25)}^a$	${(87.25)}^a$
IDV(12)	61.67	${(67.38)}^a$	${(99.63)}^a$	${(76.63)}^a$	${(99.25)}^a$	${(99.50)}^a$	${(99.13)}^a$
IDV(13)	66.67	${(83.00)}^a$	${(95.38)}^a$	${(79.13)}^a$	${(95.25)}^a$	${(95.50)}^a$	${(94.75)}^a$
IDV(18)	85.39	${(86.00)}^a$	${(90.88)}^a$	${(24.38)}^a$	${(90.50)}^a$	${(91.13)}^a$	${(89.88)}^a$
IDV(20)	12.48	${(10.00)}^a$	${(65.25)}^a$	${(28.38)}^a$	${(63.50)}^a$	${(59.75)}^a$	${(50.25)}^a$
IDV(21)	24.97	${(46.88)}^a$	${(50.75)}^a$	${(30.50)}^a$	${(48.63)}^a$	${(64.00)}^a$	${(42.88)}^a$

KPLS-k-NN: kernel partial least squares-k-nearest neighbor.

where “a” refers to DRs.

For the quality-unrelated faults, Table 5 gives the detection results. We can see the corresponding statistics $D_{\mathbf{x}}^2$ , $Q_{\mathrm{mkls}}$ , and $T_{\mathrm{ko}}^2\&T_{\mathrm{kr}}^2$ of all comparative methods have very close values, indicating that the three statistics are equally effective to detect system faults. TKPLS fails to judge the correlation between faults and quality, because its statistics for these faults are all above $10\%$ . For difficult-to-detect faults IDV(9), IDV(15)–IDV(17), both KPLS-k-NN and MKLS fail, but they enable to give correct results for the other faults. Compared with MKLS, KPLS-k-NN performs much better, because the FARs of KPLS-k-NN is much less than MKLS. Because KPLS-k-NN has good detection ability for different 21 kinds of faults, it indicates that the model trained by KPLS-k-NN has perfect generalization ability.

Table 5.

Detection results of KPLS-k-NN, MKLS, and TKPLS for quality-unrelated faults in the TE process (%).

Fault number	$n_y/n_t$	KPLS-k-NN		MKLS		TKPLS
		$D_{\mathbf{y}}^2$	$D_{\mathbf{x}}^2$	$T_{\mathrm{mkls}}^2$	$Q_{\mathrm{mkls}}$	$T_{\mathrm{ky}}^2\&Q_{\mathrm{kr}}$	$T_{\mathrm{ko}}^2\&T_{\mathrm{kr}}^2$
IDV(3)	7.74	${(1.13)}^b$	${(14.37)}^a$	${(7.25)}^b$	${(14.00)}^a$	${(15.38)}^b$	${(6.88)}^a$
IDV(4)	6.74	${(1.88)}^b$	${(99.75)}^a$	${(5.63)}^b$	${(98.50)}^a$	${(87.50)}^b$	${(99.50)}^a$
IDV(9)	4.99	${(0.75)}^b$	${(11.13)}^a$	${(5.63)}^b$	${(10.88)}^a$	${(12.13)}^b$	${(7.12)}^a$
IDV(11)	9.99	${(3.13)}^b$	${(78.00)}^a$	${(9.50)}^b$	${(74.00)}^a$	${(70.13)}^b$	${(75.63)}^a$
IDV(14)	6.24	${(7.63)}^b$	${\left(100\right)}^a$	${(3.38)}^b$	${\left(100\right)}^a$	${\left(100\right)}^b$	${\left(100\right)}^a$
IDV(15)	6.24	${(3.13)}^b$	${(19.25)}^a$	${(10.50)}^b$	${(14.63)}^a$	${(18.38)}^b$	${(12.63)}^a$
IDV(16)	3.75	${(39.25)}^b$	${(59.88)}^a$	${(20.00)}^b$	${(52.50)}^a$	${(63.13)}^b$	${(74.38)}^a$
IDV(17)	9.49	${(32.75)}^b$	${(95.50)}^a$	${(13.25)}^b$	${(93.50)}^a$	${(95.25)}^b$	${(90.38)}^a$
IDV(19)	8.36	${(0.50)}^b$	${(26.13)}^a$	${(1.00)}^b$	${(19.88)}^a$	${(43.13)}^b$	${(13.63)}^a$

KPLS-k-NN: kernel partial least squares-k-nearest neighbor.

where “a” refers to DRs and “b” refers to FARs.

The results of fault isolation

We select IDV(1) to verify the fault isolation effect of MVCk-NN by compared with VCk-NN. IDV(1) is a step fault which represents the change of A/C feed ration. The root cause of IDV(1) is the variable of $x_{25}$ . When IDV(1) happens, variables $x_1$ , $x_{26}$ , $x_4$ , and $x_{18}$ will deviate from the normal operation state in turn because of the influence of $x_{25}$ . Figure 7 displays the relative variable contributions of all process variables from 161th sample point to 960th sample point. From Figure 7(a), we can see that for the most part, VCk-NN identifies $x_{25}$ , $x_1$ , $x_4$ , and $x_{18}$ as the root cause of IDV(1). Whereas in Figure 7(b), MVCk-NN only needs to consider variables $x_{25}$ and $x_1$ as the root cause. Therefore, our proposed MVCk-NN has a relatively better ability to locate the variables causing faults than VCk-NN.

Figure 7.

Fault isolation results for IDV(1) by (a) MVCk-NN and (b) VCk-NN.

According to the above results, we can conclude that the proposed method is effective in quality-related non-linear fault diagnosis and has prominent advantages over traditional methods. Compared with the SVD-based methods, KPLS-k-NN directly utilizes the relationship among the predicted quality samples to design the statistics, which considers the information of the nearest neighbor structure. Besides, KPLS-k-NN directly detects the process space, avoiding the problem that the residual statistics may have a large component of variance, such as MKLS. Moreover, compared with VCk-NN, the proposed MVCk-NN has advantages in both variable contribution and threshold setting.

Conclusion

In this article, we present a novel quality-related non-linear fault diagnosis method based on k-NN, which is especially suitable for non-linear industrial processes with relatively small training samples. It consists of a new quality-related fault detection method KPLS-k-NN and a new fault isolation method MVCk-NN. First, KPLS is applied to establish a regression model between process variables and quality variables in order to obtain predictive quality samples. Then, FD-k-NN method is used to design two corresponding statistics, that is, $D_{\mathbf{x}}^2$ and $D_{\mathbf{y}}^2$ for the process space and the predicted quality space, respectively. They will make a judgment on whether there is a fault happening in the system, and whether the fault is related to quality when a fault occurs. When KPLS-k-NN detects faults, in order to locate root cause variables of faults, MVCk-NN is proposed by introducing the idea of ARCR into VCk-NN. Finally, the proposed method is applied to detect the faults in case study and the effectiveness is validated. Due to the superior fitting ability of the deep neural networks in comparison to KPLS, further work will focus on how to combine deep neural networks with k-NN rule to detect quality-related faults.