Sage Journals: Discover world-class research

Abstract

A method is presented for improving the localization accuracy of a mobile manipulator. The system consists of an industrial manipulator mounted on a mobile platform to perform fuselage drilling/riveting tasks in indoor environment. The localization is based on dual laser range finders mounted on the system and a number of artificial landmarks preset in the environment. Localization based on trilateration method is presented in this study. The adaptive unscented Kalman filter method is used to improve the accuracy of localization as well. The proposed method has been verified through experiments and the localization accuracy of mobile manipulator can be improved to 8 mm in most cases.

Keywords

Mobile manipulator AUKF Localization accuracy

Introduction

Mobile manipulator is a mobile platform stocked with robotic manipulator which can be applied to commercial or industrial field. The mobile manipulator has already been employed in many fields, such as material handling,¹ assembling task,² transporting,³ and so on. The localization issue is the most fundamental and critical one for mobile manipulator to accomplish the tasks. One of the challenges for localization is to suppress or remove the influence of the variable noise in the environment.^4,5

The localization accuracy is an important indicator for mobile manipulator. A number of approaches were taken to improve it. Röwekämper et al.⁶ used the particle filters to improve localization accuracy of the robot by an integrated pose estimation system with precise path execution. Moreno et al.⁷ provided Markov chain Monte Carlo algorithms combined with the differential evolution methods to efficiently solve optimization problems for global localization. Jiang and Xi⁸ proposed a multi-sensor fusion method based on camera and laser range finder to enhance the accuracy of mobile manipulator. This method can obtain more environmental information so that the precise position can be acquired. To improve localization accuracy of the system, Mu et al.⁹ presented an ultrasonic sensor scanning method based on a novel ultrasonic sensor ranging algorithm which was developed for the recognition of the inclined plate and obtained the location of the ultrasonic sensor relative to the inclined plate reference frame. Liu et al.¹⁰ utilized the multiple sonar sensors by accumulating the sonar data to overcome the sparseness and uncertainty of sensor performance. Lin et al.¹¹ presented a method for robot to calculate its accurate location using graph-based optimization in indoor environment. Kim and Chung¹² designed a novel scan matching algorithm based on the laser range finder to improve the localization performance of a mobile robot in glass-walled environments. Yang et al.¹³ proposed a method using the geomagnetic sensors to estimate the positions of mobile robots in indoor environment. Cho et al.¹⁴ developed a measurement error observer to remove large errors caused by wall penetrating line-of-sight (LOS) signals and non-LOS signals in the process of localization, and this method used received signal strength indicator data of wireless local area network. Mi and Takahashi¹⁵ used high-frequency-band radio frequency identification (RFID) module in order to determine the position in indoor environment. Mankotia et al.¹⁶ proposed a localization algorithm based on Bluetooth which used iterative trilateration and Cuckoo search algorithm and implemented them in Monte Carlo simulation.

Kalman filter (KF) is a method to enhance the localization accuracy which was first presented by Schmidt. This method was used to predict the orbit in the Apollo Program.¹⁷ Then KF was applied in the fields of control, guidance, navigation, and communication. Song et al.¹⁸ used it for enhancing the navigation accuracy by fusing the gyroscope and encoder data. Chuang et al.¹⁹ presented KF and fuzzy compensation which fused the data of magnetometer and gyroscope so that the localization accuracy can be improved. Liu et al.²⁰ proposed an algorithm that was the KF localization algorithm based on symmetric double-sided two-way ranging method for solving the problem of node localization. Wang et al.²¹ presented the hybrid ultra-wideband Doppler system employing the KF to enhance measurement accuracy and improve system robustness. Ma and Wang²² utilized KF by combining the rough location estimation and the instant velocity for RFID tracking to enhance the position estimation.

The drawback of KF is that Gaussian noise statistics is applicable to linear systems, while the practical systems are generally nonlinear systems. The unscented KF (UKF) method which was first proposed by Julier and Uhlmann²³ can be applied to nonlinear systems and is not limited to assuming that the distributions of noise sources are Gaussian. Liu et al.²⁴ proposed a modified UKF method for solving the less precision or divergence issue of time difference of arrival localization. This method judged whether the system states suddenly changed, and then introduced the suboptimal fading factor to the predicted covariance matrix for decreasing the error of state mutation. Kim et al.²⁵ used UKF method to improve the localization accuracy of system which included laser range finders, gyro, and encoders. The laser range finders calculated the absolute position of mobile manipulator, while the gyro and encoders provided the local information. Gao et al.²⁶ presented an optimal data fusion method based on the adaptive fading UKF method for the stochastic nonlinear multi-sensor systems. This method had a two-level fusion structure. The bottom-level structure used an adaptive fading UKF method based on the Mahalanobis distance to improve the adaptability and robustness of local state estimations against process-modeling error. The top-level structure utilized the principle of linear minimum variance to calculate globally optimal state estimation. Wang et al.²⁷ proposed an adaptively robust UKF method to reduce the effect of the dynamics model error and the measurement model error simultaneously. Bahraini et al.²⁸ proposed a new adaptive UKF (AUKF) algorithm with RGB-D sensors providing both RGB as well as depth information in order to improve the accuracy of SLAM (simultaneous localization and mapping), and an adaptive systematic search method was performed to find the suitable value for the scaling parameter and to improve the accuracy of estimation. Hong et al.²⁹ proposed AUKF algorithm combining data of odometer, ultrasonic position module, and electronic compass to decrease the effect of accumulated error caused by noise.

For the above research achievements, the methods are mainly based on natural landmarks or relative localization. In this article, the AUKF localization method based on artificial landmarks is presented for improving localization accuracy of mobile manipulator in indoor environment.

Problem formulation

The mobile manipulator with Mecanum wheels was developed in Shanghai University, as shown in Figure 1. This system includes a manipulator mounted onto a wheeled mobile manipulator. Two laser range finders that are placed on the right front and left rear of the system, respectively, are used to localize with preset artificial landmarks. In this article, the mobile manipulator can implement fuselage drilling or riveting tasks in the aerospace manufacturing field.^30

–36

Figure 1.

Wheeled mobile manipulator developed in Shanghai University.

The coordinate systems are established on the mobile manipulator. As shown in Figure 1, the coordinate systems include the world coordinate system ${X_{G} Y_{G} Z_{G}}$ of which the origin O_G is built on the ground in the working environment according to the actual situation. The coordinate system of the base of manipulator ${X_{B} Y_{B} Z_{B}}$ is built on the base, and the origin O_B of ${X_{B} Y_{B} Z_{B}}$ is set in the center of base. The coordinate system of the tool central point (TCP) ${X_{M} Y_{M} Z_{M}}$ is built on the TCP, and the origin O_M of ${X_{M} Y_{M} Z_{M}}$ is set in the center of TCP. The sensor coordinate systems ${X_{L 1} Y_{L 1} Z_{L 1}}$ and ${X_{L 2} Y_{L 2} Z_{L 2}}$ of the origins $O_{L 1}$ and $O_{L 2}$ are built in the center of the laser range finder, respectively. As shown in Figure 2, the headstock of mobile manipulator is defined as the positive $Y_{L 2}$ direction and the positional relationship between the dual laser range finders and the base of the manipulator is shown, and only movement along X_G and Y_G directions with rotation along Z_G direction is considered in this article.

Figure 2.

Coordinate systems of the mobile manipulator.

The position and pose of the mobile manipulator in the world coordinate system can be defined as equation (1)

{}^{G}R = [{}^{G}x, {}^{G}y, {}^{G}θ]

where ${}^{G}x$ and ${}^{G}y$ are the positions of the mobile manipulator in the world coordinate system, and ${}^{G}θ$ is the azimuth angle between the positive X_G direction and the positive X_B direction.

In Figure 2, B ₁, B ₂, L ₁, and L ₂ represent the distance from the dual laser range finders to the center of base of manipulator, and α represents the angle between the positive X_B of ${X_{B} Y_{B} Z_{B}}$ and the positive $Y_{L 2}$ of ${X_{L 2} Y_{L 2} Z_{L 2}}$ . B_i is a reflector in the environment.

When the laser range finders are applied, the localization accuracy of the system cannot be high because of environmental noise, mechanical vibration, or system error.^37,38 Particularly, the localization accuracy is greatly affected by environmental noise.

Modeling of localization based on reflectors

Localization can be classified into absolute or relative localization. The latter one has an accumulated error, while the former one does not. For absolute localization, it is a common way that many reflectors preset in the environment as landmarks which can be identified and measured accurately. In this research, the feature of each reflector will be extracted from the collected data. The feature extraction algorithm composes of three steps: (i) preprocessing raw data, (ii) clustering and identifying, and (iii) feature extraction.

Preprocessing raw data

In this step, a set of ordered data sequences ${{(ρ_{i}, {}^{L}ϕ _{i}), λ_{i}} | i = 1, \dots, n}$ is obtained from the laser range finders, where $(ρ_{i}, {}^{L}ϕ _{i})$ is the polar form and $λ_{i}$ is the intensity value of the ith data point. $ρ_{i}$ is the distance of ith data point and ${}^{L}ϕ _{i}$ is the ith azimuth of the data point in sensor coordinate system. In this step, the scanned points are converted into Cartesian form as below equation

{\begin{matrix} {}^{L}x ​_{i} = ρ_{i} cos {}^{L}ϕ ​_{i} \\ {}^{L}y ​_{i} = ρ_{i} sin {}^{L}ϕ ​_{i} \end{matrix}

Clustering and identifying

Clustering which classifies different objects is the fundamental step before identifying. If the distance between adjacent scanned points is shorter than the threshold δ, these points will be assigned into a cluster. Considering that measurement errors of laser range finders are relevant to the measured distance, an adaptive clustering method³⁹ is employed. The threshold δ is defined as below equation

δ = | | R_{h} - P_{i - 1} | | + 3 σ_{r}

| | R_{h} - P_{i - 1} | | = ρ_{i - 1} \cdot sin (Δ ϕ) / sin (β - Δ ϕ)

where δ composes of $| | R_{h} - P_{i - 1} | |$ and $σ_{r}$ . $| | R_{h} - P_{i - 1} | |$ is the distance between point $R_{h}$ and point $P_{i - 1}$ and can be computed by sine theorem as equation (4). As shown in Figure 3, $R_{h}$ is intersection point between the extension line of the line $\vec{O_{L} P_{i}}$ and a circle of which the center point is $P_{i - 1}$ , and $P_{i - 1}$ represents the (i-1)th data point. In this article, $σ_{r}$ is the measurement error and is set as 3 mm. $Δ ϕ$ is the angle resolution of the sensor. β is the acute angle between line $\vec{O_{L} P_{i - 1}}$ and line $\vec{P_{i - 1} R_{h}}$ _.

Figure 3.

Geometric diagram of the threshold.

Identifying reflectors can be implemented by the intensity threshold of object and diametral threshold of reflectors in this study.

After clustering, the data sets include two kinds: reflector sets and non-reflector sets. In equation (5), to identify cylindrical reflector, the diameter of measured reflectors D is constrained in the interval $[D_{min}, D_{max}]$ . The other constraint is the given intensity threshold $λ_{δ}$ .

{\begin{matrix} λ_{max} > λ_{δ} \\ D_{min} \leq D \leq D_{max} \end{matrix}

$D_{min}$ , $D_{max}$ , and $λ_{max}$ should be set according to actual situation. $λ_{max}$ is the maximum intensity threshold of a cluster. The artificial landmarks are wrapped around by high reflective material which can be distinguished easily from the environment.

Feature extraction

After identifying, a set of reflectors of which center point is denoted by $C_{i}$ can be obtained. $ρ_{C_{i}} and {}^{L}{_{C_{i}}}$ represent distance and azimuth of $C_{i}$ , respectively, in sensor coordinate system. As shown in Figure 4, $(ρ_{C_{i}}, {}^{L}{_{C_{i}}})$ can be obtained with reflector data set ${{(ρ_{i}, {}^{L}ϕ _{i}), λ_{i}} | i = m, \dots, n}$ as

ρ_{C_{i}} = {\begin{matrix} (ρ_{(\frac{n + m + 1}{2})} + ρ_{(\frac{n + m - 1}{2})}) / 2 + D / 2, if (n - m) is odd \\ ρ_{(\frac{n + m}{2})} + D / 2, if (n - m) is even \end{matrix}

{}^{L}{​_{C_{i}}} = {\begin{matrix} ({}^{L}ϕ ​_{(\frac{n + m + 1}{2})} + {}^{L}ϕ ​_{(\frac{n + m - 1}{2})}) / 2, if (n - m) is odd \\ {}^{L}ϕ ​_{(\frac{n + m}{2})}, if (n - m) is even \end{matrix}

Figure 4.

Extraction of the center of the cylindrical reflector.

In Figure 4, both $P_{n}$ and $P_{m}$ denote reflector data points.

Localization based on trilateration method

A series of $C_{i}$ can be obtained after feature extraction step. For localization of mobile manipulator, the number of scanned reflectors should be more than three at any time.

The trilateration method is used for localization. As shown in Figure 5(a), theoretically, three circles can intersect at one point. However, for the error factor of the measurement, these circles do not usually intersect at one point, as shown in Figure 5(b). In Figure 5(a), $C_{1} ({}^{G}x _{1}, {}^{G}y _{1})$ , $C_{2} ({}^{G}x _{2}, {}^{G}y _{2})$ , and $C_{3} ({}^{G}x _{3}, {}^{G}y _{3})$ represent reflectors in the environment and the distance from reflectors to mobile manipulator is $ρ_{C_{1}}$ , $ρ_{C_{2}}$ , and $ρ_{C_{3}}$ , respectively.

Figure 5.

(a) The theoretical situation and (b) The actual situation.

The least squares estimation principle is applied to determine the intersection point. $O_{B} ({}^{G}x, {}^{G}y)$ that denotes the position of mobile manipulator in global coordinate frame can be calculated as

{({}^{G}x, {}^{G}y)}^{T} = {(A^{T} A)}^{- 1} A^{T} b

where:

A = [\begin{matrix} 2 ({}^{G}x ​_{1} - {}^{G}x ​_{n}) & 2 ({}^{G}y ​_{1} - {}^{G}y ​_{n}) \\ ⋮ & ⋮ \\ 2 ({}^{G}x ​_{n - 1} - {}^{G}x ​_{n}) & 2 ({}^{G}y ​_{n - 1} - {}^{G}y ​_{n}) \end{matrix}]

b = [\begin{matrix} {}^{G}{x_{1}^{2}} - {}^{G}{x_{n}^{2}} + {}^{G}{y_{1}^{2}} - {}^{G}{y_{n}^{2}} + {}^{L}{ρ_{B_{n}}^{2}} - {}^{L}{ρ_{B_{1}}^{2}} \\ ⋮ \\ {}^{G}{x_{n - 1}^{2}} - {}^{G}{x_{n}^{2}} + {}^{G}{y_{n - 1}^{2}} - {}^{G}{y_{n}^{2}} + {}^{L}{ρ_{B_{n}}^{2}} - {}^{L}{ρ_{B_{n - 1}}^{2}} \end{matrix}]

The position of system ${({}^{G}x, {}^{G}y)}^{T}$ can be obtained from equation (8).

The azimuth angle ${}^{G}θ _{i}$ can be attained from every reflector as

{}^{G}θ ​_{i} = a tan 2 ({}^{G}y ​_{i} - {}^{G}y, {}^{G}x ​_{i} - {}^{G}x) - {}^{L}ϕ ​_{B_{i}}

${}^{G}θ$ is mean value which can be obtained from the following equation

{}^{G}θ = \frac{1}{n} \sum_{i = 1}^{n} {}^{G}θ ​_{i}

Modeling of localization with AUKF method

Localization with UKF method

Owing to environmental noise, mechanical vibration, system error, and locational error of the reflectors, the localization accuracy of mobile manipulator is poor in the abovementioned method. The AUKF method is utilized to improve localization accuracy.

For nonlinear state-space models, UKF method can be used to accurately acquire the posterior mean and covariance to the second order or third order of Taylor series expansion.

The state-estimation involves equation of state and equation of measurement for KF. The equations are defined as

X_{k} = f_{k - 1} X_{k - 1} + W_{k - 1}

Z_{k} = h_{k} X_{k} + V_{k}

where $X_{k}$ represents the position state of reflectors and $Z_{k}$ is the observed signal of reflectors. $W_{k}$ is the process noise and $V_{k}$ is the observation noise. The dynamic model $f_{k}$ and measurement model $h_{k}$ are unit matrix,⁴⁰ while the mobile manipulator moves at a low speed.

The position state of the reflectors $X_{k}$ and the observed signal of the reflectors $Z_{k}$ are defined as

X_{k} = [\begin{matrix} {}^{L}x ​_{k} \\ {}^{L}y ​_{k} \\ {}^{L}ϕ ​_{k} \end{matrix}] = [\begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} {}^{L}x ​_{k - 1} \\ {}^{L}y ​_{k - 1} \\ {}^{L}ϕ ​_{k - 1} \end{matrix}] + [\begin{matrix} w_{x} \\ w_{y} \\ w_{ϕ} \end{matrix}]

Z_{k} = [\begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} {}^{L}x ​_{k} \\ {}^{L}y ​_{k} \\ {}^{L}ϕ ​_{k} \end{matrix}] + [\begin{matrix} v_{x} \\ v_{y} \\ v_{ϕ} \end{matrix}]

When the system noise is nonzero mean, equations (13) and (14) can be expressed in the following form equivalently

{\begin{matrix} X_{k} = f_{k - 1} X_{k - 1} + q + μ_{k - 1} \\ Z_{k} = h_{k} X_{k} + r + η_{k} \\ E [W_{k}] = q \\ E [V_{k}] = r \end{matrix}

where $μ_{k - 1}$ and $η_{k}$ are independent and Gaussian white noise of zero mean, and their variances are Q and R, respectively.

The following step introduces unscented transformation (UT) into UKF method. The UT selects seven sample points (sigma points) to predict the position state of the reflectors at the next moment.

To calculate the estimate value of the position state at k time, the sigma matrix $η_{k}$ is calculated by ${\hat{x}}_{k - 1}$ which is the estimate value of the position state of the reflectors and $P_{k - 1}$ which is the posteriori covariance matrix of the estimated error at (k-1) time.

{\hat{x}}_{k - 1} = {[\begin{matrix} {}^{L}{\hat{x}} ​_{k - 1} & {}^{L}{\hat{y}} ​_{k - 1} & {}^{L}{\hat{ϕ}} ​_{k - 1} \end{matrix}]}^{T}

η_{k} = [\begin{matrix} {\hat{x}}_{k - 1} & {\hat{x}}_{k - 1} \pm {(\sqrt{(ξ + 3) P_{k - 1}})}_{i} \end{matrix}]; i = 0, 1, 2

ξ = ψ^{2} (3 + κ) - 3

where ξ is a scaling parameter. κ is also a scaling parameter and usually set to 0.⁴¹ ψ is usually set to a small positive value and is to control the distribution of the sigma points.⁴²

The corresponding weight W which is associated with sigma points is defined as follows

W = {[\begin{matrix} W_{m} & W_{c} \end{matrix}]}^{T} = [\begin{matrix} \frac{ξ}{ξ + 3} & {(\frac{1}{2 (ξ + 3)})}_{i} \\ \frac{ξ}{ξ + 3} + 1 - ψ^{2} + ν & {(\frac{1}{2 (ξ + 3)})}_{i} \end{matrix}] i = 1, \dots, 6

where ν is applied to incorporate prior knowledge of the distribution of the position of the reflectors. $W_{m}$ and $W_{c}$ have seven elements, respectively.

For the convenience of expression, the process of UT can be replaced with equation (22)

[η_{k}, W] = UT ({\hat{x}}_{k - 1}, P_{k - 1}, ψ, ν, κ, D)

where D represents the dimension of the position state of the reflectors.

As shown in Figure 6, the process of UKF method consists of prediction phase and update phase. The prediction phase can be divided into four steps.

Figure 6.

The flow diagram of UKF. UKF: unscented Kalman filter.

The first step of prediction phase calculates the sigma matrix $η_{k}$ and the weight matrix $W$ . The values ψ, ν, and κ are selected by UT at (k-1) time.

[η_{k}, W] = UT ({\hat{x}}_{k - 1}, P_{k - 1}, 1, 1.2, 0.98, 3)

The second step of prediction phase calculates the priori estimate ${\hat{x}}_{k | k - 1}$ and the priori covariance matrix of estimated error $P_{k | k - 1}$ using system dynamic function with equation (24)

({\hat{x}}_{k | k - 1}, P_{k | k - 1}) \leftarrow (η_{k}, W, q, Q)

The third step of prediction phase is the correction step. In correction step, ${\hat{x}}_{k | k - 1}$ and $P_{k | k - 1}$ can be corrected by UT

[ω_{k}, W] = U T ({\hat{x}}_{k | k - 1}, P_{k | k - 1}, 1, 1.2, 0.98, 3)

The last step of prediction phase calculates the observed predictive value ${\hat{y}}_{k | k - 1}$ , the reciprocal covariance matrix $P_{x_{k} y_{k}}$ , and covariance matrix of observed variables $P_{y_{k} y_{k}}$ using measurement function with equation (26)

({\hat{y}}_{k | k - 1}, P_{x_{k} y_{k}}, P_{y_{k} y_{k}}) \leftarrow (ω_{k}, W, r, R)

In Figure 7, the process of specific calculation is shown for equations (24) and (26).

Figure 7.

The process of calculation with equations (24) and (26).

According to the prediction phase, the process steps of update phase can be obtained as follows

K_{k} = P_{x_{k} y_{k}} P_{y_{k} y_{k}} ​^{- 1}

{\hat{x}}_{k} = {\hat{x}}_{k | k - 1} + K_{k} (Z_{k} - {\hat{y}}_{k | k - 1})

P_{k} = P_{k | k - 1} - K_{k} P_{y_{k} y_{k}} K_{k} ​^{T}

where $K_{k}$ is the gain matrix. ${\hat{x}}_{k}$ is the estimate value of the position state of the reflectors in the sensor coordinate system at k time. $P_{k}$ is the a posteriori covariance matrix of estimated error at k time. $Z_{k}$ is the observed value of reflectors at k time.

The localization accuracy with reflectors can be improved by UKF method from the above steps.

Localization with AUKF method

The traditional UKF method is used to solve the issue when the prior statistical characteristic of the noise is known. However, it is unknown in most cases. At this time, the estimation accuracy of the mobile manipulator will be reduced, and the filtering results may diverge. The AUKF method based on maximum a posterior estimation and exponential weighting fading memory is introduced.

The exponential weighting value d_k satisfies the following condition at k time

d_{k} = \frac{1 - b}{1 - b^{k}}

where b is forgetting factor, and 0 < b < 1. b is set as 0.98 in this article.

The estimated means of noise ${\hat{q}}_{k}$ and estimated variances of noise ${\hat{Q}}_{k}$ can be calculated as

{\begin{matrix} ε_{k} = Z_{k} - {\hat{y}}_{k | k - 1} \\ τ_{k} = f (η_{k, i}) - {\hat{x}}_{k | k - 1} \\ {\hat{q}}_{k} = (1 - d_{k}) {\hat{q}}_{k - 1} + d_{k} [{\hat{x}}_{k} - \sum_{i = 0}^{6} W_{m, i} f (η_{k, i})] \\ {\hat{Q}}_{k} = (1 - d_{k}) {\hat{Q}}_{k - 1} + d_{k} [K_{k} ε_{k} ε_{k} ​^{T} K_{k} ​^{T} + P_{k} - \sum_{i = 0}^{6} W_{c, i} τ_{k} τ_{k} ​^{T}] \end{matrix}

where ${\hat{q}}_{k}$ and ${\hat{Q}}_{k}$ replace the real mean of noise q and real variances of noise Q, respectively.

The estimated means of noise ${\hat{r}}_{k}$ and the estimated variances of noise ${\hat{R}}_{k}$ can be calculated as

{\begin{matrix} ε_{k} = Z_{k} - {\hat{y}}_{k | k - 1} \\ μ_{k} = h (ω_{k, i}) - {\hat{y}}_{k | k - 1} \\ {\hat{r}}_{k} = (1 - d_{k}) {\hat{r}}_{k - 1} + d_{k} [Z_{k} - \sum_{i = 0}^{6} W_{m, i} h (ω_{k, i})] \\ {\hat{R}}_{k} = (1 - d_{k}) {\hat{R}}_{k - 1} + d_{k} [ε_{k} ε_{k} ​^{T} - \sum_{i = 0}^{6} W_{c, i} μ_{k} μ_{k} ​^{T}] \end{matrix}

where ${\hat{r}}_{k}$ and ${\hat{R}}_{k}$ replace the real mean of noise r and R, respectively.

Experiment

In this research, the scanning range of laser range finder is from 0° to 270° and the angular resolution is 0.25°.

In this article, the localization of mobile manipulator is divided into two steps. The first step is coarse localization with dual laser range finders, and the second step is precision localization⁴³ with the industrial camera which is mounted on the front of system with one calibration panel preset on the fuselage. The precision localization step requires the mobile manipulator to move to a position with small tolerance so that the calibration panel can be caught by the camera. Based on the industrial camera parameters, this tolerance is small, so the improvement of coarse localization accuracy is important in the whole process.

In the experiment, three cases with different number of reflectors are considered. The number is three, four, and five, respectively. Taking five reflectors as example, as shown in Figures 8 and 9, the nominal positions of five reflectors are (2235,282), (1945,0), (0,0), (0,1055), and (245,1720) in the world coordinate system, and the unit is millimeter. All nominal positions are obtained by OptiTrack^® in the experiments. OptiTrack is a highly accurate motion capture system while no less than three markers mounted on reflectors or lasers are captured by at least two cameras simultaneously, and the measurement accuracy of OptiTrack can reach submillimeter level. In addition, OptiTrack is applied to validate experimental data in this article.

Figure 8.

Experimental mobile manipulator.

Figure 9.

Experiment setup.

The positional relations between the dual laser and the base of the manipulator can also be measured by OptiTrack and defined as follows

{\begin{matrix} {}_{B}^{L}T = [{}_{L}^{C}T]]^{- 1} {}_{B}^{C}T \\ {}_{B}^{C}T = {}_{M}^{C}T {}_{TCP}^{M}T {}_{B}^{TCP}T \end{matrix}

where ${}_{B}^{L}T$ is the 4 × 4 transformation matrix and represents the position and pose of the manipulator base coordinate system {B} relative to the sensor coordinate systems {L} which are mounted with markers. ${}_{L}^{C}T$ represents the position and pose of the sensor coordinate systems {L} relative to the camera coordinate system {C}. ${}_{B}^{C}T$ represents the position and pose of the manipulator base coordinate system {B} relative to the camera coordinate system {C}. ${}_{M}^{C}T$ represents the position and pose of the coordinate system of markers {M} mounted on TCP relative to the camera coordinate system {C}. ${}_{TCP}^{M}T$ represents the position and pose of the TCP coordinate system {TCP} relative to the coordinate system of markers {M}. ${}_{B}^{TCP}T$ represents the position and pose of the manipulator base coordinate system {B} relative to the TCP coordinate system {TCP}. The positional relations between the dual laser range finders and the base of the manipulator are shown in Figure 2, and the height difference between the laser and the manipulator base is 911.3 mm through the experiments. L ₁ is equal to 1943.8 mm and L ₂ is equal to 405.3 mm. B ₁ is equal to 705.5 mm and B ₂ is equal to 732.7 mm. α is equal to 2.8°

The localization data were obtained at 20 different positions. The nominal coordinates of these positions are listed in Table 1.

Table 1.

The nominal coordinates of mobile manipulator in the world coordinate system.

Position 1 (mm)	Position 2 (mm)	Position 3 (mm)	Position 4 (mm)	Position 5 (mm)
(743,720)	(691,1203)	(1097,1327)	(1527,1427)	(1740,1703)
Position 6 (mm)	Position 7 (mm)	Position 8 (mm)	Position 9 (mm)	Position 10 (mm)
(1905,1620)	(2370,1635)	(2475,1647)	(2605,1650)	(2847,1703)
Position 11 (mm)	Position 12 (mm)	Position 13 (mm)	Position 14 (mm)	Position 15 (mm)
(2557,1477)	(2727,1303)	(3207,1467)	(3407,1137)	(3574,1317)
Position 16 (mm)	Position 17 (mm)	Position 18 (mm)	Position 19 (mm)	Position 20 (mm)
(3787,1137)	(3940,1588)	(4182,1387)	(4175,1687)	(4232,1899)

Three experiments for each case were carried out. One is not to use any filter method, and the other is to use KF method or AUKF method. The error of localization can be obtained from the nominal position and the actual position. In Figure 10, the nominal positions are shown as the center points of all circles, while the radius represents the error of localization. It can be seen that the AUKF method localizes more accurately than the other two cases.

Figure 10.

The localization error of mobile manipulator. (a) The localization through three reflectors. (b) The localization through four reflectors. (c) The localization through five reflectors.

The localization error of different cases is listed in Table 2. The purple circle represents the case without any filter. The green circle represents the case with KF, and the red circle represents the case with AUKF method. According to the results, it can be seen almost every position, and AUKF method can significantly improve the localization accuracy, that is, the radius of red circle is smaller than green one or purple one. From another perspective, AUKF method can reduce the impact of the number of reflectors onto the localization accuracy. Under the circumstances, there was no correlation between the accuracy and the number of reflectors.

Table 2.

The values of localization error in Figure 10.

Number of reflectors	Position 1	Position 2	Position 3	Position 4	Position 5
3	9.9 mm () 18.6 mm () 28.2 mm ()	19.1 mm () 28.3 mm () 50.3 mm ()	2.7 mm () 10.2 mm () 15.9 mm ()	6.0 mm () 24.9 mm () 46.9 mm ()	0.8 mm () 9.7 mm () 13.0 mm ()
4	1.4 mm () 8.4 mm () 11.6 mm ()	18.3 mm () 26.2 mm () 38.6 mm ()	9.5 mm () 12.9 mm () 23.3 mm ()	0.4 mm () 8.5 mm () 13.1 mm ()	6.0 mm () 7.2 mm () 18.7 mm ()
5	7.4 mm () 9.0 mm () 21.9 mm ()	18.4 mm () 23.5 mm () 31.9 mm ()	3.7 mm () 11.5 mm () 27.2 mm ()	2.0 mm () 7.6 mm () 10.1 mm ()	3.4 mm () 6.4 mm () 9.9 mm ()
Number of reflectors	Position 6	Position 7	Position 8	Position 9	Position 10
3	3.3 mm () 8.9 mm () 18.1 mm ()	3.6 mm () 22.0 mm () 52.9 mm ()	6.4 mm () 7.7 mm () 13.8 mm ()	5.6 mm () 15.6 mm () 33.5 mm ()	0.8 mm () 22.8 mm () 43.0 mm ()
4	2.7 mm () 6.8 mm () 12.9 mm ()	1.2 mm () 18.5 mm () 44.3 mm ()	0.7 mm () 11.6 mm () 20.5 mm ()	3.0 mm () 8.2 mm () 24.4 mm ()	5.5 mm () 15.0 mm () 21.9 mm ()
5	1.2 mm () 2.4 mm () 2.5 mm ()	6.7 mm () 10.9 mm () 16.7 mm ()	2.0 mm () 9.9 mm () 21.6 mm ()	2.7 mm () 13.2 mm () 20.2 mm ()	2.5 mm () 7.6 mm () 24.3 mm ()
Number of reflectors	Position 11	Position 12	Position 13	Position 14	Position 15
3	0.4 mm () 11.8 mm () 14.9 mm ()	0.9 mm () 17.0 mm () 20.9 mm ()	6.9 mm () 31.8 mm () 57.1 mm ()	1.6 mm () 18.5 mm () 44.7 mm ()	12.9 mm () 31.3 mm () 58.5 mm ()
4	0.9 mm () 6.7 mm () 7.1 mm ()	4.8 mm () 14.8 mm () 17.3 mm ()	2.5 mm () 17.2 mm () 19.7 mm ()	6.2 mm () 26.9 mm () 36.7 mm ()	5.4 mm () 18.5 mm () 29.8 mm ()
5	2.3 mm () 2.9 mm () 4.6 mm ()	3.8 mm () 9.1 mm () 13.3 mm ()	4.3 mm () 5.1 mm () 10.4 mm ()	7.2 mm () 11.7 mm () 19.4 mm ()	3.8 mm () 14.5 mm () 20.8 mm ()
Number of reflectors	Position 16	Position 17	Position 18	Position 19	Position 20
3	12.3 mm () 25.1 mm () 45.4 mm ()	1.4 mm () 30.8 mm () 54.4 mm ()	1.9 mm () 28.1 mm () 52.1 mm ()	2.0 mm () 27.8 mm () 57.8 mm ()	0.5 mm () 24.8 mm () 56.6 mm ()
4	0.4 mm () 26.8 mm () 35.5 mm ()	5.0 mm () 22.4 mm () 39.2 mm ()	5.1 mm () 17.9 mm () 33.9 mm ()	4.8 mm () 14.2 mm () 19.6 mm ()	2.5 mm () 24.0 mm () 35.1 mm ()
5	3.3 mm () 12.9 mm () 22.3 mm ()	5.8 mm () 13.8 mm () 16.5 mm ()	1.7 mm () 11.3 mm () 14.8 mm ()	5.1 mm () 6.7 mm () 16.9 mm ()	5.2 mm () 12.2 mm () 16.3 mm ()

It can be seen from Table 2, the accuracy can be improved to 8 mm except position 2 with AUKF method in most cases. It is mainly because the mobile manipulator is too close to the reflector (0, 1055) mm at position 2. It can be seen in Table 3, the distance between the mobile manipulator and this reflector is only 706 mm at position 2.

Table 3.

The distance between the mobile manipulator and the reflector (0, 1055; mm).

Distance 1	Distance 2	Distance 3	Distance 4	Distance 5
815	706	1130	1572	1856
Distance 6	Distance 7	Distance 8	Distance 9	Distance 10
1987	2243	2545	2672	2920
Distance 11	Distance 12	Distance 13	Distance 14	Distance 15
2592	2738	3233	3408	3584
Distance 16	Distance 17	Distance 18	Distance 19	Distance 20
3788	3976	4195	4223	4315

The maximum error of AUKF localization method is 19.1 mm, while the error of KF method is 31.8 mm, and the method without any filter is 58.5 mm. AUKF method can improve the localization accuracy of mobile manipulator with reflectors.

Conclusion

In this article, a method for improving localization accuracy of mobile manipulator used for fuselage drilling/riveting is presented. This article proposes an algorithm based on AUKF method to improve the localization accuracy with artificial landmarks. The model of AUKF method has been established, and the experiments are carried out in the real scenes. Through the experiment results, it can be found that, in most cases, the AUKF method can improve the localization accuracy of mobile manipulator compared with KF method or no filter method. In the future, the robustness of feature matching will be increased and this AUKF method will also be used in the localization with nature landmarks.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China grant no. 61803251 and the Shanghai Municipal Science and Technology Commission grant no. 17511107801.

ORCID iD

Tao Song

References

Nuelle

Schulz

M J

Aden

. Force sensing, low-cost manipulator in mobile robotics. In: International conference on control, IEEE, Nagoya, Japan, 24–26 April 2017, pp. 196–201. DOI: 10.1109/ICCAR.2017.7942686.

Hamner

Koterba

Shi

. An autonomous mobile manipulator for assembly tasks. Auton Robot 2010; 28(1): 131–149.

Huang

Liu

. Design and experiment of handling manipulator for hatching egg vehicle. Chinese J Eng Design 2017; 24: 601–608.

Chopra

Balachandran

Karki

. Noise effects on localization of mobile sensor platforms. In: Proceedings of SPIE-The International Society for Optical Engineering, San Diego, California, United States, 8–11 March 2010, pp. 1–8. DOI:10.1117/12.847605.

Yin

Zoubir

. RSS-based sensor network localization in contaminated Gaussian measurement noise. In: IEEE international workshop on computational advances in multi-sensor adaptive processing, France, 15–18 December 2013, pp. 121–124. DOI: 10.1109/CAMSAP.2013.6714022.

Röwekämper

Sprunk

Tipaldi

. On the position accuracy of mobile robot localization based on particle filters combined with scan matching. IEEE/RSJ Int Conf Int Robot 2012; 57(1): 3158–3164.

Moreno

Martín

Munoz

. differential evolution Markov chain filter for global localization. J Intell Robot Syst 2016; 82(3-4): 513–536.

Jiang

. Target object identification and localization in mobile manipulations. In: Proceedings of the 2011 IEEE international conference on robotics and biomimetics, Phuket, Thailand, 7–11 December 2011, pp. 144–149. DOI: 10.1109/ROBIO.2011.6181276.

Zhang

Huang

. Omni-directional scanning localization method of a mobile robot based on ultrasonic sensors. Sensors 2016; 16(12): 1–14.

10.

Liu

Sun

Fang

. Robotic room-level localization using multiple sets of sonar measurements. IEEE Trans Instrum Meas 2016; 66: 2–13.

11.

Lin

. Graph-based SLAM in indoor environment using corner feature from laser sensor. In: Automat IEEE, Hefei, China, 19–21 March 2017, pp. 1211–1216. DOI: 10.1109/YAC.2017.7967597.

12.

Kim

Chung

. Localization of a mobile robot using a laser range finder in a glass-walled environment. IEEE Trans Ind Elect 2016; 63(6): 3616–3627.

13.

Yang

Kim

Kuc

. Geomagnetic localization of mobile robot. In: IEEE international conference on mechatronics, Federat Univ Gippsland, Gippsland, Australia, 13–15 February 2017, pp. 123–127. DOI: 10.1109/ICMECH.2017.7921091.

14.

Cho

. Measurement error observer-based IMM filtering for mobile node localization using WLAN RSSI measurement. IEEE Sens J 2016; 16(8): 2489–2499.

15.

Takahashi

. Design of an HF-band RFID system with multiple readers and passive tags for indoor mobile robot self-localization. Sensors 2016; 16(8): 1200.

16.

Mankotia

Agrawal

Singh

. Error minimization in Bluetooth based indoor localization of a mobile robot using cuckoo search algorithm. In: International conference on medical imaging IEEE, Greater Noida, India, 7–8 November 2014, pp. 283–288. DOI: 10.1109/MedCom.2014.7006019.

17.

Kalman

. A new approach to linear filtering and prediction problems. J Basic Eng Trans 1960; 82: 35–45.

18.

Song

. A new method of AGV navigation based on Kalman filter and a magnetic nail localization. In: IEEE international conference on robotics and biomimetics IEEE, Qingdao, China, 3–7 December 2016, pp. 952–957. DOI: 10.1109/ROBIO.2016.7866447.

19.

Chuang

Hou

Chen

. Indoor intelligent mobile robot localization using fuzzy compensation and Kalman filter to fuse the data of gyroscope and magnetometer. IEEE Trans Ind Elect 2015; 62(10): 1–1.

20.

Liu

Zou

Wang

. Kalman filter localization algorithm based on SDS-TWR ranging. Telkomnika Indonesian J Elect Eng 2013; 11(3): 1436–1448.

21.

Wang

Tang

Ferreras

. An improved indoor localization solution using a hybrid UWB-doppler system with Kalman filter. In: Radio Wireless Symposium IEEE, Anaheim, CA, 15–18 January 2018, pp. 181–183. DOI: 10.1109/RWS.2018.8304980.

22.

Wang

. Fusion of RSS and phase shift using the Kalman filter for RFID tracking. IEEE Sens J 2017; 17(11): 3551–3558.

23.

Julier

Uhlmann

. A new extension of the Kalman filter to nonlinear systems. In: Proceedings of SPIE -The International Society for Optical Engineering, 1997, 3068, pp. 182–193.

24.

Liu

Xiang

Mao

. Time difference localization algorithm based on modified unscented Kalman filter. In: Chinese Automation Congress IEEE, Wuhan, China, 27–29 November 2015, pp. 1879–1884. DOI: 10.1109/CAC.2015.7382810.

25.

Kim

Jung

Kim

. Positioning accuracy improvement of laser navigation using unscented Kalman filter. Robot Auton Syst 2013; 62(9): 1241–1247.

26.

Gao

. Multi-sensor optimal data fusion based on the adaptive fading unscented Kalman filter. Sensors 2018; 18(20): 488.

27.

Wang

Sun

. Adaptively robust unscented Kalman filter for tracking a maneuvering vehicle. J Guid Control Dyn 2014; 37(5): 1696–1701.

28.

Bahraini

Bozorg

Rad

. A new adaptive UKF algorithm to improve the accuracy of SLAM. Int J Robot 2018; 5(1): 1–12.

29.

Hong

Guo

. Robot indoor localization algorithm based on adaptive unscented Kalman filter. J Comput Meas Control 2018; 26(1): 238–247. DOI: 10.16526/j.cnki.11-4762/tp.2018.01.059.

30.

Song

Guo

. Optimization of a mobile manipulator for a wheeled manipulator. J Mech Rob 2016; 8(6): 061007-1–061007-14.

31.

Guo

Fang

Song

. Tracking and localization for omni-directional mobile industrial robot using reflectors. Adv Manuf 2018; 6(1): 118–125.

32.

Guo

Song

. Tip-over stability analysis for a wheeled mobile manipulator. J Dyn Sys Meas Control 2017; 139(5): 054501-1–054501-7.

33.

Song

Guo

. Slip analysis for a wheeled mobile manipulator. J Dyn Sys Meas Control 2017; 140(2): 021005-1–021005-12.

34.

Guo

Jin

Bao

. Accuracy analysis of omnidirectional mobile manipulator with Mecanum wheels. Adv Manuf 2016; 4(4): 363–370.

35.

Song

Guo

. Design, analysis, and applications of mobile manipulators. In: Novel Design and Applications of Robotics Technologies, IGI Global, 2018, pp. 26–65. DOI: 10.4018/978-1-5225-5276-5.ch002.

36.

Song

Guo

. Path tracking of a wheeled mobile manipulator through improved localization and calibration. Mobile Robot 2018; 1: 151–175.

37.

Shahgholian

Tahmasebi

Khaloozadeh

. A fuzzy extended Kalman filter for mobile robot localization in high amount of measurement noise. In: International conference on control, Kumaracoil, India, 16–17 December 2016, pp. 333–338. DOI: 10.1109/ICCICCT.2016.7987969.

38.

Wang

Jing

Jia

. An indoor mobile localization strategy for robot in NLOS environment. Int J Distrib Sens N 2013; 2013(1): 893–900.

39.

Feng

Fang

. Natural landmark extraction method for mobile robot. Robots 2010; 32(4): 540–546.

40.

Zhao

Zhou

Chen

. Application of Kalman filter in indoor positioning system for real-time tracking. Wuhan Univ J Nat Sci 2009; 55(6): 696–700.

41.

Chen

. Research of localization for indoor mobile robot based on unscented Kalman filter. China Meas Test 2011; 37(5): 1–5.

42.

Liu

. An adaptive UKF filtering algorithm for GPS position estimation. In: International conference on wireless communications IEEE, Beijing, China, 24–26 September 2009, pp. 1–8. DOI: 10.1109/WICOM.2009.5305046.

43.

Guo

Diao

. Research on end positioning for wheeled mobile robot in aviation drilling. Indus Control Comput 2017; 30(6): 83–87. DOI: 10.3969/j.issn.1001-182X.2017.06.036.

Improvement of localization with artificial landmark for mobile manipulator

Abstract

Keywords

Introduction

Problem formulation

Modeling of localization based on reflectors

Preprocessing raw data

Clustering and identifying

Feature extraction

Localization based on trilateration method

Modeling of localization with AUKF method

Localization with UKF method

Localization with AUKF method

Experiment

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References