Abstract
Quantifying the current and expected future performance of a machining process no doubt is essential for continuous improvement of product quality and productivity. However, process capability evaluation for small batch production runs is a challenging work, because the assumptions required by traditional evaluation approaches based on statistical process control techniques are commonly not satisfied in real world. In this article, a sensitivity analysis–based process capability evaluation method for small batch production runs is proposed, and a new capability index is also presented correspondingly. In this method, an error propagation model between machining errors and input errors is first established using weighted least squares support vector machine. Then, the sensitivity distribution of machining errors versus input errors is characterized by a set of eigenvalues and eigenvectors in the variation space of input errors. Third, the safe variation space of input errors is solved according to the specification limits of quality characteristics. Finally, the process capability is evaluated by comparing the fitness between the safe variation space and the tolerance space of input errors. A practical case is addressed to validate the feasibility and effectiveness of the proposed method, and the results demonstrate that the method can measure a real small batch machining process effectively and get rid of the common assumption of independent and identical distribution, which is needed by traditional methods.
Keywords
Introduction
In today’s global competitive marketplace, adequate and reliable product quality is one of the critical factors of gaining the market competition advantages for manufacturing companies.1,2 Quantifying the current and expected future performance of a machining process no doubt is essential for continuous improvement of product quality and productivity. Process capability indices (PCIs) that have been widely used in manufacturing industries and service industries provide numerical measures on whether or not a manufacturing process is capable to meet the predetermined specification level of a product.
3
And the common used PCIs, such as
Focusing on the problem of process performance analysis for small batch production runs, a novel sensitivity analysis–based method for dynamic process capability evaluation (DPCE) is proposed in this article, and a new PCI is also presented. It is expected to establish a more practical measuring method for small batch machining processes which are usually in dynamically stable state. Because there is no unambiguous definition of small batch production, it is necessary to clarify the connotation of small batch production investigated in this study. In the implementation of control charts, 25 or more subgroups which consist of at least 100 observations need to be used to establish reliable control limits. Quessenberry 9 suggested that 300 observation samples are used to establish the control limits of X chart. From the perspective of statistical quality control, when the available quality samples are not enough to implement Shewhart control charts, the production run can be called as small batch production. The purpose of this work is to analyze the process capability for this case.
The rest of this article is organized as follows. After a brief review about process capability evaluation for small batch machining processes in section “Literature review,” the logic framework of the proposed DPCE method is put forward in section “Logic framework of DPCE based on sensitivity analysis.” Then, the methodologies for construction of machining error propagation model based on weighted least squares support vector machine (WLS-SVM), sensitivity analysis of machining errors and evaluation of process capability are stated in sections “Construction of machining error propagation model based on WLS-SVM,”“Sensitivity analysis of machining errors” and “Evaluation of process capability based on sensitivity distribution” respectively. In section “Case study and discussions,” a practical case is studied to demonstrate the feasibility and effectiveness of the proposed method, and some discussions are also given. Finally, conclusions and future work are drawn in section “Conclusion.”
Literature review
As for the issue of process capability evaluation for small batch production runs, some important work has been done from the late 1980s. In the classic approaches based on statistical process control (SPC) techniques, it is a critical problem to accurately estimate the process parameters using available few samples. Aiming at the problem of small sample size, Cheng and Spiring,
10
Saxena and Singh
11
and Miao et al.
12
adopted Bayesian approach to estimate the process parameters. Balamurali
13
developed new appropriate capability indices on the basis of classic PCIs and further used bootstrap technique to estimate their confidence intervals. Ke et al.
14
employed six bootstrap methods to construct upper confidence bounds of incapability index
For achieving a consistent performance of a process, Taguchi introduced the concept of process robustness in manufacturing, which facilitate the application of robust design concept in engineering processes and products. 23 The existing approaches to measure the robustness and performance of a process or product can be classified into three categories, namely index-based approaches, experimental-based approaches and sensitivity-based approaches. 24 Among these approaches, most of index-based approaches are based on the concept of variance. In the most popular experimental-based approach, namely response surface methodology (RSM), it is assumed that the errors follow normal distribution and the error variance is constant and independent of mean. Hence, the two kinds of approaches are not suitable for small batch production. 25 To study robustness using sensitivity-based approaches, it is essential to construct an appropriate response model, which can avoid the assumption of IIND. Consequently, this kind of methodologies may be promising approaches to solve the issue of DPCE for small batch production. In the last decade, the trend of the study is to apply sensitivity-based approaches in measuring the robustness of a process or a product. 26 And sensitivity analysis–based methodologies have been used to solve the problems of performance or robustness evaluation in design and manufacturing fields. Zhu and Ting, 27 and Caro et al. 28 proposed a novel robust design technique based on the performance sensitivity distribution of a mechanism. Gunawan and Azarm 29 present a robust design optimization method based on the notion of a sensitivity region, which is a measure of how far a feasible design is from the boundary of a feasible domain in the parameter variation space. Dai and Scott 30 used sensitivity analysis and cluster analysis to improve both efficiency and effectiveness of a scale-based product family. In manufacturing fields, Zhang et al. 31 used sensitivity analysis approach to evaluate the impact of each process-level error on the product quality, so as to make the optimization of fixture design as well as workpiece in-process tolerance allocation. Abellan-Nebot et al. 32 employed the sensitivity analysis of candidate process plans to identify critical fixtures and manufacturing stations or operations. In the works mentioned above, the sensitivity analysis–based methods are used to define evaluation criteria, in order to optimize a design, select an optimal design from several different schemes, perform workpiece in-process tolerance allocation or identify critical fixtures and manufacturing stations. However, the nature of process capability evaluation is to compare the fitness between process performance and the predetermined specification limits. Therefore, these methodologies for robust design and process optimization cannot be applied in capability evaluation for machining processes. To measure the robustness of multi-stage manufacturing processes, Mondal et al. 33 adopted sensitivity matrices (Jacobian matrices) to construct the variation models in single-stage process. Nevertheless, their method is only appropriate for large batch production, because plenty of samples are required to estimate the standard deviation of controllable input variables. Even so, these applications demonstrate that sensitivity analysis–based method has a promising potential in measuring the performance or robustness of a system.
Based on the work of Zhu and Ting, 27 a novel sensitivity analysis–based DPCE method is proposed in this study. Their objectives are to present a robust parameter design technique based on the theory of performance sensitivity distribution. And the performance index of a system is defined as the ratio of the volume of safe space and real space, which ignores the position relationship between these two spaces. In this work, the focused issue is to evaluate the process capability for small batch production runs. And the capability index of a machining process is measured by considering not only the size of the safe space, but also the position relationship between safe variation space and real variation space. Additionally, a WLS-SVM-based approach to modeling error propagation for machining process is proposed in this study.
Logic framework of DPCE based on sensitivity analysis
Understanding machining error variation from the perspective of sensitivity
It is well known that machining errors are affected by the couple effect of the geometrical errors of machining system and the errors caused by cutting force, heat and vibration. If the machining system is seen as a black box, these errors can be categorized into two classes. One is referred to as system errors, which include geometrical errors of machine tool components and the errors caused by cutting force, heat and vibration. The other can be referred to as input errors, which include the geometrical errors of workpiece (machining error in previous stage), datum errors, fixture errors and installation errors. Usually, input errors have tolerance limits and can be easily measured and controlled in machining processes. System errors are relatively stable during a period of time and cannot be easily inspected and controlled. When there are no assignable causes, machining process is in statistical control state and machining errors are not sensitive to the input errors relatively. When some assignable causes occur, machining process is out of control and machining errors are sensitive to input errors. Consequently, the sensitivity of machining errors against input errors reveals whether the machining process is in control or not to some extent.
Logic framework of sensitivity analysis–based DPCE
Based on the cognition mentioned above, a novel DPCE method based on sensitivity analysis is proposed. As shown in Figure 1, its logic framework mainly contains three steps that are addressed in detail as follows:
Step 1: The error propagation relationships between machining errors and input errors are modeled based on WLS-SVM using the latest history data. The key problems include determining the structure of WLS-SVM model, selection of kernel function, weight assignment and training WLS-SVM.
Step 2: The sensitivity equations of machining errors against input errors are solved according to the regression model constructed in step 1. Then, the error sensitivity distribution against input errors is analyzed and characterized by a set of eigenvalues and eigenvectors in the variation space of input errors.
Step 3: The safe variation space of input errors is depicted as an ellipsoid according to the tolerance limits of quality characteristics. Then, dynamic process capability is evaluated by comparing the fitness between the safe variation space and tolerance space of input errors, and one dynamic PCI is defined. After that, the procedures of calculating the proposed dynamic PCI are stated.

Logic framework of DPCE based on sensitivity analysis.
In the following three sections, the three key enable technologies, namely construction of WLS-SVM-based error propagation model, sensitivity analysis of machining errors and evaluation of process capability based on sensitivity are addressed, respectively.
Construction of machining error propagation model based on WLS-SVM
Modeling machining error propagation relationships is the foundation of error sensitivity analysis. The performance of error propagation model directly affects the accuracy of sensitivity analysis of machining error as well as process capability evaluation. Least squares support vector machine (LS-SVM) has excellent performance of modeling nonlinear relationship even in the case of small sample size.34,35 We can obtain the optimal estimate of LS-SVM regression model under the assumption that residual errors of estimate function follow normal distribution. 36 However, the assumption is often not realistic in small batch production runs. Therefore, WLS-SVM is employed to construct a more robust regression model describing the complicated error propagation relationships between machining errors and input errors. The principle and mathematical formulation of WLS-SVM can be referred to the work of Suykens et al. 37
Determining the structure of WLS-SVM model
The purpose of constructing error propagation model is to analyze the sensitivity of machining errors against input errors, in which the machining system is treated as a black box. The inputs of WLS-SVM model include machining errors in previous stage, datum error, fixture error and install errors. And its output is the machining error of a quality characteristic, such as diameter error, roundness error and cylindricity error. Because a SVM model has only one output, it is necessary to construct a WLS-SVM-based error propagation model for each quality characteristic. The WLS-SVM-based error propagation model of a quality characteristic is shown in Figure 2. The input vector can be formulated as

WLS-SVM-based error propagation model of a quality characteristic.
Selection of kernel function
The performance of SVMs relies on the kernel function and its parameters. Therefore, an appropriate kernel function should be selected while constructing SVM models. There are two most popular kernel functions. One is polynomial function, and the other is Gaussian radial basis function (RBF). Among the two common kernel functions, RBF has more excellent generalization performance in handling nonlinear problems and is the most used kernel function for classification and function estimation. 38 Hence, it is selected as the kernel function of WLS-SVM-based error propagation model. According to the mathematical forms of WLS-SVM and RBF, the error propagation model between the kth quality characteristic and its input errors can be formulated as follows
where N denotes the number of support vectors,
Weight assignment
The weights of training samples can also be calculated using weighting functions, such as Huber function 39 and Hampel function 40 , according to the model error. Hereinto, Hampel weighting function shows exciting performance in robust estimation, especially in the case that there are extreme outliers in training samples. Consequently, Hampel function is chosen to assign the weights for training data. Thus, the weight of a training sample can be determined according to the residual errors of LS-SVM model which are trained by the same training data
where
where
Training WLS-SVM model
In the practical applications, the historical data of latest finished N workpieces are taken as the training samples after a workpiece is finished. The N samples form an observation window to measure the performance of the current machining process. When the next workpiece is finished, the input error and machining error of another latest finished N workpieces are taken as the training samples by moving the observation window one step toward and so on. A precise error propagation model is the foundation of sensitivity analysis and capability evaluation. And the number of training samples is one of the critical factors influencing the precision of WLS-SVM-based models. However, the error propagation model trained using too many historical data may not be able to reflect the current state of a machining process. To take a compromise between model errors and timeliness, an appropriate number of training samples should be obtained by trial and error. The training procedures are listed as follows: 37
Step 1: Train a LS-SVM model through cross-validation using the acquired latest history data
Step 2: Calculate
Step 3: Assign the weight
Step 4: Solve the quadratic programming equations of WLS-SVM using the same history data
Sensitivity analysis of machining errors
Once the error propagation models are constructed, sensitivity analysis of machining errors can be carried out to evaluate the current stability of the machining process. The procedures of sensitivity analysis on the basis of the models constructed above are detailed in this section.
Sensitivity matrix of machining errors
As mentioned in section “Determining the structure of WLS-SVM model,” suppose the input vector of error propagation model is a n-dimensional vector, namely
where
where
Sensitivity distribution of machining errors
The sensitivity of machining errors is analyzed using the theory of performance sensitivity distribution proposed by Zhu and Ting. 27 According to the attribute of Jacobian matrix, the machining errors of all quality characteristics can be approximated by the linear model of equation (1)
where
Obviously, the n input errors are independent. The n-dimensional fluctuation space of these input errors is called as variation space
The relationships between machining errors and input errors can be further depicted by taking a norm of the vector
where the operator ‖‖ indicates two-norm. Let
From the definition of two-norm, it is known that
and diagonal matrix
where
Equation (12) describes the sensitivity distribution of machining error in n-dimensional variation space of input errors. According to the feature of matrix
1. If
Equation (13) defines a family of n-dimensional hyper-ellipsoids. The length of a semi-principal axis
For a certain value of
2. If

Sensitivity distribution of machining errors (n = 3): (a) r = 3 and (b) r = 2.
According to the geometric meaning of equation (13), equation (14) can be interpreted as an n-dimensional hyper-cylindroid with
Evaluation of process capability based on sensitivity distribution
Determination of safe variation space
To ensure machining errors be in their corresponding tolerance limits, the machining errors of all quality characteristics need to meet the following constraints
where
For a machining system, suppose
Then, the problem of process capability evaluation can be transformed to comparing the fitness between safe variation and actual variation space, which is defined by the tolerance limits of all input errors. However, only the linearity relationship between machining error vector
In general, the linear ranges of different machining systems are not the same. They usually rely on the characteristics of a machining system. Some investigations demonstrated that the linear relationships could be held when the component variations are less than 3%–5% of the nominal dimensions. 41 It should be pointed out that a cylindroid safe variation space will be converted to an ellipsoid safe space after their infinite principal axes are reduced to finite lengths. The principal axis directions of the modified safe space are the same as those of the original safe space, because the error sensitivity distribution reflected by these axes remains the same.
To limit the input error vector
where
to amend the corresponding semi-principal axis length in the modified safe space
where
The following discussions mainly aim at the corrected safe variation space rather than the original one. For avoiding confusion,
Definition of dynamic PCI
According to the definition of process capability, we can measure the performance of a process by comparing the fitness between the safe variation space and the tolerance space of input errors. However, directly using the ratio of the volume of safe variation space to that of real variation space as a measure of process capability may result in error prone conclusions, because the direction of each principal axis also affects the amount of the safe variation space intersecting with the real space. Consequently, the capability of a machining process should be measured by considering not only the size of the safe space but also the directions of the principal axes, namely taking the position relationship between safe variation space and real variation space into considerations to reflect the actual process capability.
The safe variation space is a hyper-ellipsoid and the directions of their principal axis may not coincide with those of the coordinate axis, while the real variation space is a block. Therefore, the position relationship between them may include the following four situations, as shown in Figure 4 (

Position relationship between safe space and real variation space (n = 2): (a) Sr ⊆ Ss, (b) Ss ⊆ Sr and (c) Sr ⊆ Ss, Ss ⊆ Sr and Sr ∩ Ss ≠ ∅.
From the above, a new PCI
where

Equidistant offset block space
If the outputs of a machining process are not relatively sensitive to input error variation, the process capability is adequate. The more adequate is the process capability, the larger is the size of
The traditional PCIs are defined under the assumption that the machining process is in statistical control. The proposed PCI is defined for the processes which are in dynamically stable state and reflects the capability of the processes ensuring machining errors in related tolerance limits. Therefore, it is called as dynamic PCI.
Calculation of the proposed PCI
To compute the value of
Step 1: Along the direction of each coordinate axis, discretize the real variation space into finite n-dimensional block units with a certain step h and compute the center coordinate of each unit.
Step 2: Judge whether each block unit falls inside the safe variation space one by one according to equation (21) and cumulate the volumes of all space units that fall inside the safe variation space to gain the volume of the intersection
where
Step 3: Compare the values between
Step 4: Initialize the value of
Step 5: Construct a equidistant offset block space
Step 6: Judge whether each vertex of
Step 7: Compute the value of the index
The general idea behind a PCI is to compare what a process “should do” with what it “is actually doing.”
42
In the classic definitions of PCIs, the tolerance band of a quality characteristic is what a process “should do” and the process variability and centering are what it “is actually doing.” Whereas in the proposed PCI, the real variation space indicates what a process “should do” and the safe space indicates what it “is actually doing.” It should be pointed out that the physical significance of the PCI is similar with that of the classic PCIs
Case study and discussions
In this section, a practical case is addressed to demonstrate the feasibility and effectiveness of the proposed approach for process capability evaluation. The corresponding computing program is coded in MATLAB and runs on a DELL OPTIPLEX 360 with a 2.93 GHz processor and 2 GB random access memory (RAM). Finally, some discussions are also stated.
A practical case
The deep-hole boring process for the outer barrel part of an airplane landing gear is taken as an example. And the drawing of the outer barrel part is shown in Figure 6. The key quality characteristics include diameter, roundness and coaxiality, which are dimensioned with red color, and their specification limits are [0, 0.04], [0, 0.02] and [0, 0.03], respectively. The positioning datum are outer cylinder A, outer cylinder B and left end face. Consequently, the corresponding input errors of machining process include the alignment error of datum A and datum B, the diameter error and roundness error in the previous stage. Their allowed tolerance limits are [0, 0.05], [0, 0.05], [−0.1, 0.1] and [0, 0.04], respectively. Because of the customized trait of airplane products, the lot size of outer barrel parts in a batch production run is commonly small and ranges usually from ten to several tens. Therefore, their machining processes are typical small batch production runs. Meanwhile, the cost of each blank is very high, which ranges from 20,000 to tens of thousands dollars. A quality accident may result in huge financial losses. Consequently, it is necessary to real-time implement process capability evaluation while machining this type of parts. In a small batch production, the input errors and corresponding machining errors of sequential 25 workpieces are acquired, as shown in Table 1. To observe the fluctuation of the process capability, a window size 20 is adopted to construct the error propagation models for three key quality characteristics and evaluate the current process performance. The analysis procedures for the process capability are stated as follows:
Step 1: Establish the error propagation model based on WLS-SVM. To find a compromise size to balance model errors and timeliness, one experiment has been conducted to compare different sizes of observation window. Different window sizes with 12, 16, 20 and 24 samples are considered. The mean absolute relative error (MARE) is taken as evaluation criterion. The MATLAB LS-SVM Toolbox by De Brabanter et al. 43 was used to construct WLS-SVM-based error propagation model and evaluate the performance of different window sizes. The comparison results are presented in Table 2. According to the results, the window size with 20 and 24 samples produced almost the same MARE (4.78% and 4.75%), and there is no significant improvement on MARE from the window size with 20–24. Therefore, the window size 20 is selected in this application case.

Drawing of the outer barrel part.
Acquired input errors and machining errors of sequential 25 workpieces.
Performance comparison for different window sizes.
Afterward, the observations of the first 20 workpieces are taken as the training samples of WLS-SVM-based error propagation models for three key quality characteristics. Then, another three error models are constructed by moving the observation window one step toward and so on. Moving the observation window from part no. 20 to 24 successively, 15 regression models for three key quality characteristics are constructed, respectively. The hyper-parameters
Step 2: Solve the Jacobian matrix. According to equations (5) and (6), the Jacobian matrix
Step 3: Construct the error propagation characteristic matrix and calculate the eigenvalues and eigenvectors. The error propagation characteristic matrix can be attained according to the equation
Step 4: Modify the safe variation space and calculate the eigenvalues of the safe space. According to equation (15), the value of
Obtained hyper-parameters of 15 WLS-SVM-based error propagation models.
Sensitivity of quality characteristics to input errors.
Eigenvalues of error propagation characteristic matrix
From Figure 6, it is known that both the nominal values of the first two input errors are equal to 175. And it is known that both the nominal values of the last two input errors are equal to 144.8 according to its technique documents. Let the modification coefficient
Step 5: Calculate the volumes of
Modified eigenvalues of characteristic matrix
Lengths of the semi-principal axes.
According to the calculation procedures of the proposed PCI, we can calculate the volumes of the intersection between the safe space and real variation space, and the volume of the equidistant offset block from the real space with a step 0.001. Then, the capability index
Values of
The run chart of the proposed PCI is plotted in Figure 7, from which two phenomena can be seen: (1) all of the values of

Run chart of the proposed PCI.
Discussion
The five steps mentioned above show the feasibility of using the proposed method to evaluate the capability of a real small batch production run in a job-shop. It also indicates that the sensitivity-based process capability evaluation method is systematic and effective, because the five steps transform one by one smoothly and logically. Among these steps, the error propagation model is constructed using statistical learning method (WLS-SVM) without normality detection and distribution parameters estimation, which need to be done in traditional process capability evaluation methodologies; the latter four steps are carried out on the foundation of the first step and also need not the assumption conditions. It clearly reveals how to evaluate the performance of a small batch production run and avoid the required assumptions for SPC techniques. Unlike the traditional PCIs, the proposed PCI measures the process capability from a new perspective of the sensitivity of machining errors. Although the physical meanings of the proposed process performance indices are not similar with those of traditional PCIs, they also reveal the process performance quantitatively by relating its capability to the predetermined specification limits. Consequently, the proposed method can be used to perform process capability analysis and get rid of the influence caused by the assumptions which are the foundations of traditional capability evaluation methods but are commonly violated in real machining processes.
It should be pointed out that constructing error propagation model plays a critical role in the presented method and the model precision directly affects the evaluating result of process capability. However, the number of training data influences the performance of a WLS-SVM regression model significantly. Insufficient training data may generate imprecise error propagation model and lead to error conclusions of process performance. When the number of workpieces in a production batch is very small or the machining process is at starting stage, the available observation data are rare limited and usually less than 20. In this case, we can adopt the first-order terms of Taylor series expansion 28 to establish the linear approximation model of error propagation relationships, so as to evaluate the process capability based on sensitivity analysis.
Although the above practical case focuses mainly on a small batch production run, the sensitivity analysis–based process capability evaluation method can also be applied in mass production without any change, in which we can acquire plenty of observation data to train WLS-SVM-based error propagation. Moreover, the proposed method for evaluating process capability in a single machining process is the base of further measuring the performance of multi-stage machining processes.
In all, this work attempts to evaluate the process performance for small batch production runs from an entirely different viewpoint to terminate the trouble resulted by the assumption of independent and identical distribution. And the results of application case show that the method is promising.
Conclusion
A sensitivity analysis–based process capability evaluation method for small batch production runs is proposed in this article, and a corresponding capability index is also presented. In this method, an error propagation model between machining errors and input errors is first established using WLS-SVM. Then, the sensitivity distribution of machining errors versus input errors is characterized by a set of eigenvalues and eigenvectors in the variation space of input errors. Finally, the safe variation space of input errors is depicted as an ellipsoid and the process capability is evaluated by comparing the fitness between the safe variation space and the tolerance space of input errors. The results of a practical case demonstrate that the proposed method is feasible and effective in real world and does not need the assumption of independent and identical distribution, which are essential for traditional process capability evaluation methods. In a word, this work is an attempt to measure the performance of small batch production runs from a novel perspective of the sensitivity of machining errors, and the results of an application case demonstrate that it is promising.
In the future, one important work is to investigate the relationship between the proposed PCI and process yield, which will further promote its practicability. Another ongoing work is to further investigate how to evaluate the performance of multi-stage machining processes based on the proposed method.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research work is supported by the National Basic Research Program of China (973 program) with grant No. 2011CB706805. The authors hereby thank the MOST of China for the financial aids.
