Uncertainty propagation on a nonlinear measurement model based on Taylor expansion

Abstract

In this paper, the propagation of uncertainty on a nonlinear measurement model is presented using a higher-order Taylor series. As the derived formula is based on a Taylor series, it is necessary to compute the partial derivatives of the nonlinear measurement model and the correlation among the various products of the input variables. To simplify the approximation of this formula, most previous studies assumed that the input variables follow independent Gaussian distributions. However, in this study, we generate multivariate random variables based on copulas and obtain the covariances among the products of various input variables. By applying the derived formula to various cases regardless of the error distribution, we obtained the results that coincide with those of a Monte-Carlo simulation. To apply high-order Taylor expansion, the nonlinear measurement model should be continuous within the range of the input variables to allow for differentiation, and be an analytic function in order to be represented by a power series. This approach may replace some time-consuming Monte-Carlo simulations by choosing the appropriate order of the Taylor series, and can be used to check the linearity of the uncertainty.

Introduction

In describing the propagation of uncertainty through a system, the first-order Taylor series is highly recommended by the Guide to the Expression of Uncertainty in Measurement (GUM). ¹ However, it is difficult to accurately predict the uncertainty of a nonlinear measurement model using a first-order Taylor expansion. Thus, the GUM suggests adding higher-order terms of the Taylor expansion when the model cannot be represented linearly. The high-order terms are obtained from the second order Taylor expansion with the assumption that the input variables have normal distribution.

Monte-Carlo (MC) simulation is widely used for nonlinear measurement model since it can predict distributions as well as standard deviations of measurement model outputs. It is strongly recommended that sufficient number of simulations are required to achieve high accuracy of the estimation. For example, the GUM recommends a minimum of 200,000 simulations to obtain a 95% confidence. ² Some simulations such as finite element method (FEM) numerical analysis, statistical parameter estimations, and complex signal processing often take hours to days.^3–5 Therefore, in these cases, MC simulation is not suitable for the uncertainty propagation on the nonlinear measurement model. Various studies on uncertainty propagation have been conducted, and they are well summarized in Liu et al.,^6,7 Meng et al., ⁸ and Ouyang et al. ⁹ Recent studies showed that the probability density function (pdf) of the output of the measurement model could be predicted even through the input has arbitrary distribution. Most studies, however, assume independent input variables or approximate by truncating high orders terms.

Recently, Wang and Iyer ¹⁰ added third-order terms to reduce the error produced by the truncation of the Taylor expansion, and showed that the uncertainty of the nonlinear measurement model could be predicted more accurately than using the technique recommended in GUM. Zhang extended the Taylor series to an infinite number of terms and proposed some methods to calculate the variance of the system. ¹¹ Mekid and Vaja ¹² calculated the uncertainty of the measurement model up to the third order using the skewness and kurtosis of the input variables. Xue et al. ¹³ derived the variances and covariances of the model based on the infinite expansion of the Taylor series. These approaches assume that the input variables are independent (or dependent) normal distributions, thus removing terms that are difficult to calculate analytically. Thus, they cannot be applied when the input variables do not obey a normal distribution and/or are correlated with each other.

In this paper, we re-derive the Taylor series for the covariance of a nonlinear measurement model regardless of distribution and correlation of the variables. We also show the method how to apply the derived formula to simple numerical simulation. In the numerical approach, we calculated the non-analytic terms in the formula (such as the covariance between products of input variables) from the multivariate random variables generated based on copulas. To obtain the partial derivatives of the system, an n-point stencil is applied across the multiple dimensions. As a result, the formula based on the higher-order Taylor expansion can be fully applied to covariance calculations on the nonlinear measurement model, irrespective of the distribution of the input variables.

This paper is organized as follows. In Section 2, the covariance of the nonlinear measurement model is re-derived using a Taylor series expansion. In Section 3, we show examples of applying the derived formula by simple numerical approach. Section 4 discusses the limitations of Taylor approximation, and Section 5 presents the conclusions to this study.

Theory

In probability theory and statistics, covariance is a measure of the degree of correlation between two random variables, x₁ and x₂. By applying the linearity of expectations, this can be represented as the “expected value of their product” minus the “product of their expected values.”

$$\begin{array}{c}\mathrm{cov}(x_1,x_2)=〈(x_1-〈x_1〉)(x_2-〈x_2〉)〉\hfill \\ =〈x_1{x}_2-x_1〈x_2〉-〈x_1〉x_2+〈x_1〉〈x_2〉〉\hfill \\ =〈x_1{x}_2〉-〈x_1〉〈x_2〉-〈x_1〉〈x_2〉+〈x_1〉〈x_2〉\hfill \\ =〈x_1{x}_2〉-〈x_1〉〈x_2〉\hfill \end{array}$$

(1)

where 〈·〉 denotes the expectation of the argument. The approximate output of a system using the first-order Taylor expansion is

$$\begin{array}{c}\multicolumn{1}{c}{y_i=f_i(x_1+z_1,\cdots ,x_m+z_m)\approx f_i(x_1,\cdots ,x_m)}\\ \multicolumn{1}{c}{+\sum _{k=1}^m\frac{\partial f_i(x_1,\cdots ,x_m)}{\partial x_k}{z}_k}\end{array}$$

(2)

where x_k is the input variable, z_k is the error in x_k, and y is the output of the function. f is a real-valued function that is differentiable on x₁, …, x_m; for simplicity, f(x₁, …, x_m) will be represented as f. Thus, the covariance using the first-order Taylor expansion is given by:

$$\begin{array}{c}\multicolumn{1}{c}{\mathrm{cov}(y_i,y_j)\approx \mathrm{cov}(f_i+\sum _{k=1}^m\frac{\partial f_i}{\partial x_k}{z}_k,f_j+\sum _{p=1}^m\frac{\partial f_j}{\partial x_p}{z}_p)}\\ \multicolumn{1}{c}{=\mathrm{cov}(\sum _{k=1}^m\frac{\partial f_i}{\partial x_k}{z}_k,\sum _{p=1}^m\frac{\partial f_j}{\partial x_p}{z}_p)}\\ \multicolumn{1}{c}{=〈\sum _{k=1}^m\frac{\partial f_i}{\partial x_k}{z}_k\sum _{p=1}^m\frac{\partial f_j}{\partial x_p}{z}_p〉-〈\sum _{k=1}^m\frac{\partial f_i}{\partial x_k}{z}_k〉}\\ \multicolumn{1}{c}{〈\sum _{p=1}^m\frac{\partial f_j}{\partial x_p}{z}_p〉}\\ \multicolumn{1}{c}{=\sum _{k=1}^m\frac{\partial f_i}{\partial x_k}\sum _{p=1}^m\frac{\partial f_j}{\partial x_p}(〈z_k{z}_p〉-〈z_k〉〈z_p〉)}\\ \multicolumn{1}{c}{=\sum _{k=1}^m\sum _{p=1}^m\frac{\partial f_i}{\partial x_k}\frac{\partial f_j}{\partial x_p}\mathrm{cov}(z_k,z_p)}\end{array}$$

(3)

If there are m random variables, an m×m covariance matrix $\mathbf{\Sigma }$ can be obtained.

$${\Sigma }_{i,j}=\mathrm{cov}(y_i,y_j)(i,j=1,\cdots ,m)$$

(4)

The covariance matrix ${\mathbf{\Sigma }}^y$ on the outputs y = (y₁, …, y_u) can be expressed as a product of matrices using the above approximate equation.

$${\mathbf{\Sigma }}^y=\mathbf{J}{\mathbf{\Sigma }}^z{\mathbf{J}}^{\mathrm{T}}$$

(5)

$$\mathbf{J}=\left[\begin{array}{ccc}\frac{\partial f_1}{\partial x_1}\hfill & \cdots \hfill & \frac{\partial f_1}{\partial x_m}\hfill \\ \multicolumn{1}{c}{⋮}& \ddots \hfill & ⋮\hfill \\ \multicolumn{1}{c}{\frac{\partial f_u}{\partial x_1}}& \cdots \hfill & \frac{\partial f_u}{\partial x_m}\hfill \end{array}\right]$$

(6)

Here, J is the Jacobian matrix of f = (f₁, …, f_u), and J^T is the transpose of J. This formula is known to describe the propagation of uncertainty in linear measurement model. ¹

Equation (5), however, is not appropriate when the measurement model f is not linear. Thus, we require an accurate model for the nonlinear f and a method of propagating the uncertainty through it. In this paper, we use a higher-order Taylor expansion to obtain the covariance matrix of the nonlinear measurement model. It is not guaranteed that the covariance matrix will be expressed as a simple product of matrices, similar to equation (5). The approximation of the outputs using the second-order Taylor expansion is given by

$$\begin{array}{c}\multicolumn{1}{c}{y_i=f_i(x_1+z_1,\cdots ,x_m+z_m)\approx f_i(x_1,\cdots ,x_m)}\\ \multicolumn{1}{c}{+\sum _{k=1}^m\frac{\partial f_i}{\partial x_k}{z}_k+\frac{1}{2}\sum _{k,h=1}^m\frac{{\partial }^2{f}_i}{\partial x_k\partial x_h}{z}_k{z}_h}\end{array}$$

(7)

The additive law of covariance is useful to simplify the Taylor expansion in equation (1).

$$\begin{array}{c}\multicolumn{1}{c}{\mathrm{cov}(a{x}_1+b{x}_2,c{x}_3+d{x}_4)=\mathrm{ac}\mathrm{cov}(x_1,x_3)}\\ \multicolumn{1}{c}{+\mathrm{ad}\mathrm{cov}(x_1,x_4)+\mathrm{bc}\mathrm{cov}(x_2,x_3)+\mathrm{bd}\mathrm{cov}(x_2,x_4)}\end{array}$$

(8)

where a, b, c, and d are constants. Using equation (8), we can obtain approximations of the covariance of two outputs of the system:

$$\begin{array}{c}\multicolumn{1}{c}{\mathrm{cov}(y_i,y_j)\approx \mathrm{cov}(f_i+\sum _{k=1}^m\frac{\partial f_i}{\partial x_k}{z}_k+\frac{1}{2}\sum _{k,h=1}^m\frac{{\partial }^2{f}_i}{\partial x_k\partial x_h}{z}_k{z}_h,f_j+\sum _{p=1}^m\frac{\partial f_j}{\partial x_p}{z}_l+\frac{1}{2}\sum _{p,q=1}^m\frac{{\partial }^2{f}_j}{\partial x_p\partial x_q}{z}_p{z}_q)}\\ \multicolumn{1}{c}{=\mathrm{cov}(\sum _{k=1}^m\frac{\partial f_i}{\partial x_k}{z}_k+\frac{1}{2}\sum _{k,h=1}^m\frac{{\partial }^2{f}_i}{\partial x_k\partial x_h}{z}_k{z}_h,\sum _{p=1}^m\frac{\partial f_j}{\partial x_p}{z}_p+\frac{1}{2}\sum _{p,q=1}^m\frac{{\partial }^2{f}_j}{\partial x_p\partial x_q}{z}_p{z}_q)}\\ \multicolumn{1}{c}{=\sum _{k,p=1}^m\frac{\partial f_i}{\partial x_k}\frac{\partial f_j}{\partial x_p}\mathrm{cov}(z_k,z_p)+\frac{1}{2}\sum _{k,p,q=1}^m\frac{\partial f_i}{\partial x_k}\frac{{\partial }^2{f}_j}{\partial x_p\partial x_q}\mathrm{cov}(z_k,z_p{z}_q)}\\ \multicolumn{1}{c}{+\frac{1}{2}\sum _{k,h,p=1}^m\frac{{\partial }^2{f}_i}{\partial x_k\partial x_h}\frac{\partial f_j}{\partial x_p}\mathrm{cov}(z_k{z}_h,z_p)+\frac{1}{4}\sum _{k,h,p,q=1}^m\frac{{\partial }^2{f}_i}{\partial x_k\partial x_h}\frac{{\partial }^2{f}_j}{\partial x_p\partial x_q}\mathrm{cov}(z_k{z}_h,z_p{z}_q)}\end{array}$$

(9)

Similarly, the covariance of two outputs for the r-th order can be obtained based on the r-th order Taylor expansion.

$$\begin{array}{c}\multicolumn{1}{c}{\mathrm{cov}(y_i,y_j)\approx \sum _{p,q=1}^r\frac{1}{p!}\frac{1}{q!}\sum _{i_1,\cdots i_p,j_1,\cdots j_q=1}^m}\\ \multicolumn{1}{c}{\frac{{\partial }^p{f}_i(x_1,\cdots ,x_n)}{\partial x_{i_1}\cdots \partial x_{i_p}}\frac{{\partial }^q{f}_j(x_1,\cdots ,x_n)}{\partial x_{j_1}\cdots \partial x_{j_q}}\mathrm{cov}(z_{i_1}\cdots z_{i_p},z_{j_1}\cdots z_{j_q})}\end{array}$$

(10)

As a result, the covariance of the nonlinear measurement model can be represented as the covariance of the various products of input variables with their partial derivatives. Note that the analytic solution for the covariance of the product of variables has not known when they are not normal distribution. This makes equation (10) difficult to apply to the real problem. Equation (10), however, can be applied using a numeric calculation if we can compute the numerical partial derivatives and the numerical covariance for the product of input variables. We will show this in following section.

Numerical approach

As mentioned in the previous section, in order to apply equation (10), the partial difference of the measurement model and the covariance between the products of the input variables are required. There are various ways to find precisely the partial difference. In this study, we simply applied n-point stencil to focus on the applicability of the derived equation. It is beyond the scope of this paper to analytically calculate the covariance between the product of the input variables. Instead, we calculated it numerically and applied it to equation (10).

Numerical partial differentiation

The n-point stencil is widely used to obtain numerical derivatives. For example, the first partial derivative using a 5-point stencil is as follows:

$$f^{\prime }(x)\approx \frac{1}{12h}(f(x-2h)-8f(x-h)+8f(x+h)-f(x+2h))$$

(11)

The n-point stencil can also be used to compute derivatives on the multivariable. However, we use a multi-dimensional m-point stencil to simplify the numerical implementation, as shown in Figure 1. For O variables, we require n^O points. For the first-order partial derivatives, equation (11) is applied along each axis. As the order of the differentiation increases, such as for ∂²f/∂²x, equation (11) is modified appropriately; the formula can be found in Fornberg. ¹⁴ Note that this approach does not require additional induction to derive the appropriate formulas.

Figure 1.

Multi-dimensional n-point stencil used in this paper.

Covariance of various products of input variances

Equation (10) requires the covariances of various products of input variables, such as $\mathrm{cov}(z_{i_1}\cdots z_{i_p},z_{j_1}\cdots z_{j_q})$ . Although there have been some studies on the covariance between products of random variables,^15,16 we found that these approximations are not sufficiently accurate for this study. Instead, we generated multivariate random variables from the covariances of input variables using the relevant copula and found the covariances of the products of various combinations.

In applying the copula, multivariate normal random numbers are generated using Cholesky decomposition and converted to uniform random vectors. ¹⁷ These vectors are then transformed back to random numbers with the target marginal distributions using the inverse cumulative distribution function in MATLAB. ¹⁸ Depending on the marginal distribution, there may be a difference in the correlations between the target and the generated variables because of the nonlinearity of the transform. To overcome this issue, we apply Spearman’s ranked correlation. In a normal distribution, the Pearson correlation coefficient is related to Spearman’s rho as: ¹⁹

$$\rho =2\sin \left(\frac{\pi }{6}{\rho }_s\right)$$

(12)

where ρ_s and ρ are Spearman’s and Pearson’s correlation coefficients, respectively. In this paper, to focus on the numerical approach, equation (12) is applied repeatedly until the generated random variables meet the target covariances. The detailed procedure is descried in the appendix.

Results

We tested the covariances of the nonlinear measurement model derived from the previous section using four nonlinear functions and five random variable distributions. The nonlinear functions are y₁ = x₁³+x₂²+2, y₂ = x₁²+x₂², y₃ = x₁²x₂, and y₄ = x₁x₂. The distributions considered here are the normal, exponential, uniform, gamma, and Rayleigh distributions; details are summarized in Table 1. We calculated the covariance of y₁ and y₂ and the covariance of y₃ and y₄ for the various distributions of input variables using a MC simulation and first-, second-, and third-order Taylor approximations. The MC simulations consisted of 1,000,000 runs; all results are summarized in Table 2.

Table 1.

Distribution of test variable.

Distribution	Denote	Mean	Variance
Normal	N(μ, σ2)	μ	σ 2
Exponential	Exp(λ)	λ −1	λ −2
Uniform	U(a, b)	$\frac{1}{2}(a+b)$	$\frac{1}{12}(b-a{)}^2$
Gamma	Γ(k, θ)	kθ	kθ 2
Rayleigh	R(σ)	$\sigma \sqrt{\frac{\pi }{2}}$	$\frac{4-\pi }{2}{\sigma }^2$

Table 2.

Comparison of covariance using the MC simulation and the Taylor expansion.

Type	Distribution ofx1 and x2	Covarianceof input	Monte-Carlo	First order	Second order	Third order
${\mathbf{\Sigma }}^{(y_1,y_2)}$	N(0, 2), N(0, 3)	$\left[\begin{array}{cc}2.0031\hfill & 1.2050\hfill \\ \multicolumn{1}{c}{1.2050}& 3.0045\hfill \end{array}\right]$	$\left[\begin{array}{cc}137.82\hfill & 20.84\hfill \\ \multicolumn{1}{c}{20.84}& 31.90\hfill \end{array}\right]$	$\left[\begin{array}{cc}4.45\hfill & 4.33\hfill \\ \multicolumn{1}{c}{4.33}& 4.22\hfill \end{array}\right]\times {10}^{-5}$	$\left[\begin{array}{cc}18.09\hfill & 20.98\hfill \\ \multicolumn{1}{c}{20.98}& 31.89\hfill \end{array}\right]$	$\left[\begin{array}{cc}137.82\hfill & 20.84\hfill \\ \multicolumn{1}{c}{20.84}& 31.90\hfill \end{array}\right]$
	Exp(1/ $\sqrt{2}$ ), Exp(1/ $\sqrt{3}$ )	$\left[\begin{array}{cc}2.0010\hfill & 1.2166\hfill \\ \multicolumn{1}{c}{1.2166}& 2.9960\hfill \end{array}\right]$	$\left[\begin{array}{cc}6308.6\hfill & 1146.5\hfill \\ \multicolumn{1}{c}{1146.5}& 350.7\hfill \end{array}\right]$	$\left[\begin{array}{cc}158.9\hfill & 107.3\hfill \\ \multicolumn{1}{c}{107.3}& 75.9\hfill \end{array}\right]$	$\left[\begin{array}{cc}1416.1\hfill & 645.7\hfill \\ \multicolumn{1}{c}{645.7}& 350.7\hfill \end{array}\right]$	$\left[\begin{array}{cc}6308.6\hfill & 1146.5\hfill \\ \multicolumn{1}{c}{1146.5}& 350.7\hfill \end{array}\right]$
	U(– $\sqrt{6}$ , $\sqrt{6}$ ), U(–3, 3)	$\left[\begin{array}{cc}2.0013\hfill & 1.2032\hfill \\ \multicolumn{1}{c}{1.2032}& 2.9982\hfill \end{array}\right]$	$\left[\begin{array}{cc}38.17\hfill & 8.24\hfill \\ \multicolumn{1}{c}{8.24}& 12.43\hfill \end{array}\right]$	$\left[\begin{array}{cc}2.56\hfill & 3.02\hfill \\ \multicolumn{1}{c}{3.02}& 3.83\hfill \end{array}\right]\times {10}^{-5}$	$\left[\begin{array}{cc}7.21\hfill & 8.22\hfill \\ \multicolumn{1}{c}{8.22}& 12.43\hfill \end{array}\right]$	$\left[\begin{array}{cc}38.17\hfill & 8.24\hfill \\ \multicolumn{1}{c}{8.24}& 12.43\hfill \end{array}\right]$
	Γ(5, $\sqrt{2/3}$ ), Γ(3, 1)	$\left[\begin{array}{cc}2.0009\hfill & 1.2086\hfill \\ \multicolumn{1}{c}{1.2086}& 3.0074\hfill \end{array}\right]$	$\left[\begin{array}{cc}8031.0\hfill & 1675.0\hfill \\ \multicolumn{1}{c}{1675.0}& 486.9\hfill \end{array}\right]$	$\left[\begin{array}{cc}2353.1\hfill & 753.6\hfill \\ \multicolumn{1}{c}{753.6}& 280.7\hfill \end{array}\right]$	$\left[\begin{array}{cc}5430.2\hfill & 1427.2\hfill \\ \multicolumn{1}{c}{1427.2}& 486.9\hfill \end{array}\right]$	$\left[\begin{array}{cc}8031.0\hfill & 1675.0\hfill \\ \multicolumn{1}{c}{1675.0}& 486.9\hfill \end{array}\right]$
	R $\left(\sqrt{\frac{4}{(4-\pi )}}\right)$ , R $\left(\sqrt{\frac{6}{(4-\pi )}}\right)$	$\left[\begin{array}{cc}2.0016\hfill & 1.2260\hfill \\ \multicolumn{1}{c}{1.2260}& 3.0021\hfill \end{array}\right]$	$\left[\begin{array}{cc}4339.9\hfill & 1135.1\hfill \\ \multicolumn{1}{c}{1135.1}& 402.8\hfill \end{array}\right]$	$\left[\begin{array}{cc}1455.1\hfill & 592.4\hfill \\ \multicolumn{1}{c}{592.4}& 278.5\hfill \end{array}\right]$	$\left[\begin{array}{cc}2964.6\hfill & 968.9\hfill \\ \multicolumn{1}{c}{968.9}& 402.8\hfill \end{array}\right]$	$\left[\begin{array}{cc}4339.9\hfill & 1135.1\hfill \\ \multicolumn{1}{c}{1135.1}& 402.8\hfill \end{array}\right]$
${\mathbf{\Sigma }}^{(y_3,y_4)}$	N(0, 2), Exp(1/ $\sqrt{3}$ )	$\left[\begin{array}{cc}1.9979\hfill & 1.2009\hfill \\ \multicolumn{1}{c}{1.2009}& 3.0004\hfill \end{array}\right]$	$\left[\begin{array}{cc}140.44\hfill & 42.00\hfill \\ \multicolumn{1}{c}{42.00}& 17.40\hfill \end{array}\right]$	$\left[\begin{array}{cc}0.0\hfill & -0.01\hfill \\ \multicolumn{1}{c}{-0.01}& 5.99\hfill \end{array}\right]$	$\left[\begin{array}{cc}23.89\hfill & 8.50\hfill \\ \multicolumn{1}{c}{8.50}& 17.40\hfill \end{array}\right]$	$\left[\begin{array}{cc}140.44\hfill & 42.00\hfill \\ \multicolumn{1}{c}{42.00}& 17.40\hfill \end{array}\right]$
	Exp(1/ $\sqrt{2}$ ), U(–3, 3)	$\left[\begin{array}{cc}2.0037\hfill & 1.1713\hfill \\ \multicolumn{1}{c}{1.1713}& 3.0000\hfill \end{array}\right]$	$\left[\begin{array}{cc}477.39\hfill & 71.90\hfill \\ \multicolumn{1}{c}{71.90}& 14.35\hfill \end{array}\right]$	$\left[\begin{array}{cc}12.01\hfill & 8.50\hfill \\ \multicolumn{1}{c}{8.50}& 6.00\hfill \end{array}\right]$	$\left[\begin{array}{cc}74.07\hfill & 31.26\hfill \\ \multicolumn{1}{c}{31.26}& 14.35\hfill \end{array}\right]$	$\left[\begin{array}{cc}477.39\hfill & 71.90\hfill \\ \multicolumn{1}{c}{71.90}& 14.35\hfill \end{array}\right]$
	U(– $\sqrt{6}$ , $\sqrt{6}$ ), Γ(3, 1)	$\left[\begin{array}{cc}2.0003\hfill & 1.2096\hfill \\ \multicolumn{1}{c}{1.2096}& 3.0030\hfill \end{array}\right]$	$\left[\begin{array}{cc}66.46\hfill & 27.19\hfill \\ \multicolumn{1}{c}{27.19}& 25.72\hfill \end{array}\right]$	$\left[\begin{array}{cc}0.00\hfill & -0.04\hfill \\ \multicolumn{1}{c}{-0.04}& 18.01\hfill \end{array}\right]$	$\left[\begin{array}{cc}28.78\hfill & 6.86\hfill \\ \multicolumn{1}{c}{6.86}& 25.72\hfill \end{array}\right]$	$\left[\begin{array}{cc}66.46\hfill & 27.19\hfill \\ \multicolumn{1}{c}{27.19}& 25.72\hfill \end{array}\right]$
	Γ(5, $\sqrt{2/3}$ ), R $\left(\sqrt{\frac{6}{(4-\pi )}}\right)$	$\left[\begin{array}{cc}2.0021\hfill & 1.2202\hfill \\ \multicolumn{1}{c}{1.2202}& 2.9946\hfill \end{array}\right]$	$\left[\begin{array}{cc}4640.6\hfill & 650.2\hfill \\ \multicolumn{1}{c}{650.2}& 102.1\hfill \end{array}\right]$	$\left[\begin{array}{cc}1686.6\hfill & 354.4\hfill \\ \multicolumn{1}{c}{354.4}& 77.4\hfill \end{array}\right]$	$\left[\begin{array}{cc}3381.7\hfill & 561.8\hfill \\ \multicolumn{1}{c}{561.8}& 102.1\hfill \end{array}\right]$	$\left[\begin{array}{cc}4640.6\hfill & 650.2\hfill \\ \multicolumn{1}{c}{650.2}& 102.1\hfill \end{array}\right]$
	R( $\sqrt{\frac{4}{(4-\pi )}}$ ), N(0, 3)	$\left[\begin{array}{cc}2.0019\hfill & 1.2265\hfill \\ \multicolumn{1}{c}{1.2265}& 3.0025\hfill \end{array}\right]$	$\left[\begin{array}{cc}730.04\hfill & 140.57\hfill \\ \multicolumn{1}{c}{140.57}& 30.79\hfill \end{array}\right]$	$\left[\begin{array}{cc}160.36\hfill & 59.31\hfill \\ \multicolumn{1}{c}{59.31}& 21.94\hfill \end{array}\right]$	$\left[\begin{array}{cc}389.57\hfill & 105.27\hfill \\ \multicolumn{1}{c}{105.27}& 30.79\hfill \end{array}\right]$	$\left[\begin{array}{cc}730.04\hfill & 140.57\hfill \\ \multicolumn{1}{c}{140.57}& 30.79\hfill \end{array}\right]$

All random variables were generated to have the covariance matrix $\left[\begin{array}{cc}2\hfill & 1.2\hfill \\ \multicolumn{1}{c}{1.2}& 3\hfill \end{array}\right]$ . The covariances of the generated random variables (denoted as “Covariance of input” in Table 2) exhibit only small differences from the target covariances according to Spearman’s ranked correlation. As expected, the first- and second-order Taylor approximations are quite different from the MC simulation results, but the covariances of the MC simulations and the third-order Taylor approximation are exactly the same, regardless of the distribution of random variables. Note that the MC simulation results for the variance of y₂ and y₄ are the same as those given by the second-order Taylor approximation, and they do not change as the order of the Taylor approximation increases. This means that we can determine the order of Taylor expansion, although the exact mathematical definition of the function is unknown. This will be described in more detail in the following section.

Limitation of Taylor approximation

In this section, we study the limitation of the covariance based on Taylor approximation. The function f(x)  = 1/(1 − x) is used as the nonlinear function. For the accuracy of calculation and convenience of analysis, the derivative of this function has been derived analytically and only one random variable is examined. The results of the Taylor series expansion of f are shown in Figure 2, where x = 2 and z = 0.9 and 1.01 (see equation (7)). For z = 0.9, the Taylor approximation converges to a value of 1/(1 − 2.9) when the differential order exceeds 40. For z = 1.01, however, the Taylor approximation diverges gradually. This is because the function f becomes discontinuous when the denominator is zero. Therefore, for the above function to be expressed as a Taylor series, the value of z should be between −1 and 1. ²⁰ This corresponds well to the fact that the differentiable functions only can be represented as Taylor series. Figure 3 shows the variance of f as a function of the Taylor expansion’s order. When the variance of the input variable is 0.03, the variance of f using the proposed method approaches the result of the MC simulations (dashed line). However, it diverges from the MC simulation results for the case of 0.05, because the distribution of the random variable f(x) deviates from the continuous range, as shown in Figure 4(b). Therefore, the nonlinear measurement model approximating with the Taylor series should be a local continuous function that can be differential within the input variable range. It should also be an analytic function that can be expressed as the sum of the power series. Figure 3 additionally reveals that in the case of the measurement models that are difficult to describe mathematically, such as EVM and eye pattern measurement of complex signals,^5,21 increasing the order of Taylor series and examining its convergence will be helpful in determining the differential order. Note that equations (9) and (10) show rapidly increasing number of calculation as the order of the Taylor series increase. For example, as the order increases to r, the amount of partial differential computation increases to $\sum _{k=1}^{r}2(k+1)$ . Therefore, it is important to choose the appropriate order when applying the Taylor approximation.

Figure 2.

Taylor approximation for the output f.

Figure 3.

Taylor approximation for the variance of output.

Figure 4.

Input and output values in MC simulation: (a) for input variance of 0.03 and (b) for input variance of 0.05.

Conclusion

In this paper, we have re-derived a formula for estimating the covariances of nonlinear measurement model using high-order Taylor expansions. The presence of these high-order terms means that the expression is analytically intractable. Many previous studies have simplified these terms by assuming that the input variables obey normal distributions, whereas we generated multivariate random variables with arbitrary distributions and calculated these terms numerically. The estimation results were shown to be exactly the same as those given by MC simulations. We also showed that the nonlinear measurement model should be an analytic function that can be differentiated within the range of the input variables to apply high-order Taylor approximation.

The method used in Section 3 (Numerical Approach) may be used instead of time-consuming MC simulations such as the uncertainty evaluation on the finite element method (FEM) numerical analysis, statistical parameter estimations, and complex signal processing. It can also be used to evaluate the linearity of the uncertainty. This paper has focused on showing that the uncertainty propagation of nonlinear measurement model can be represented as high-order Taylor expansion and it can be applied in real problem using numerical approaches. Thus, it is left to the future studies that how to precisely generate multivariable random variables.