Pose detection of parallel robot based on improved RANSAC algorithm

Abstract

For the factors of complex image background, unobvious end-effector characteristics and uneven illumination in the pose detection of parallel robot based on binocular vision, the detection speed, and accuracy cannot meet the requirement of the closed-loop control. So a pose detection method based on improved RANSAC algorithm is presented. First, considering that the image of parallel robot is rigid and has multiple corner points, the Harris–Scale Invariant Feature Transform algorithm is adopted to realize image prematching. The feature points are extracted by Harris and matched by Scale Invariant Feature Transform to realize good accuracy and real-time performance. Second, for the mismatching from prematching, an improved RANSAC algorithm is proposed to refine the prematching results. This improved algorithm can overcome the disadvantages of mismatching and time-consuming of the conventional RANSAC algorithm by selecting feature points in separated grids of the images and predetecting to validate provisional model. The improved RANSAC algorithm was applied to a self-developed novel 3-degrees of freedom parallel robot to verify the validity. The experiment results show that, compared with the conventional algorithm, the average matching time decreases by 63.45%, the average matching accuracy increases by 15.66%, the average deviations of pose detection in Y direction, Z direction, and roll angle $\beta$ decrease by 0.871 mm, 0.82 mm, and 0.704°, respectively, using improved algorithm to refine the prematching results. The real-time performance and accuracy of pose detection of parallel robot can be improved.

Introduction

Parallel robot has received wide attention in recent years for the advantages of strong rigidity, powerful carrying capacity, stable structure, high precision and low movement inertia, and so on. ¹ The pose of end-effector is an important parameter reflecting the kinematic state in control research of parallel robot. Accurate pose detection can effectively reduce the error from kinematic model resolving and realize high-performance closed-loop control of parallel robot. However, the pose detection of end-effector for parallel robot is still difficult for the factors of the expensive detection equipment, the complex detection method, and the limited detection range.^2,3 Currently, the detection methods mainly include appending measuring robot, locating by dual-frequency interferometer, multisensors, and machine vision.^4–6 Compared with other methods, machine vision has the advantages of noncontact, powerful adaptability, high cost performance, and so on. ⁷ It is suitable to solve the problems of pose detection of parallel robot, which features the multi-DOF (degrees of freedom) movement, the complex movement trajectory and the difficulty of direct detection.

According to the number of cameras, the visual pose detection method can be classified into monocular vision, binocular vision, and multivision. Compared with monocular vision, binocular vision has no strict requirements for the geometric constraints among the markers, which significantly reduces the designing requirements of markers. Besides, binocular vision can be used for data fusion, which can improve the detection accuracy and robustness. ⁸ Though multivision has higher detection accuracy, it needs multiple images to match, which leads to the increasing matching difficulty and time. So binocular vision has been widely used. However, for the binocular vision applied to the pose detection for realizing closed-loop control of parallel robot, it is difficult to place landmark on the end-effector. In addition, for the factors of complex image background, unobvious end-effector characteristics, and uneven illumination, it is difficult to achieve the space matching accurately and rapidly. And the detection speed and accuracy cannot meet the requirement of the closed-loop control.^9,10 Therefore, this paper improves the matching algorithm in pose detection without landmark on end-effector and focuses on increasing the speed and accuracy of pose detection of parallel robot by improving and optimizing the binocular vision algorithm. Furthermore, the improved method is applied to a self-developed novel 3-DOF parallel robot to verify the validity. It lays the foundation for the real-time high-performance closed-loop control of parallel robot.

The pose detection process based on binocular vision includes the images acquisition, the feature point extraction, the stereo matching, the camera calibration, and the pose estimation. Among all the processes, stereo matching is the most important. The speed and accuracy of stereo matching determine the speed and accuracy of pose detection directly. ¹¹ It is difficult to realize matching accurately and rapidly in the pose detection of parallel robot due to the external influences, such as the indoor lighting change, the uneven illumination, and the noise. Thus, it is necessary to study the matching methods. The matching methods mainly include the gray-based matching ¹² and the feature-based matching. ¹³ The former calculates the minimization of error sum of squares of all the image pixels to obtain transformation parameters among images with iterative computation. It could increase the amount of computation, slow down the computing speed and lead to poor robustness. While the feature-based matching algorithm can extract image feature points stably and realize image matching using gray information to derive symbol feature, which can improve calculating speed and robustness. Compared with the marker-free method, the marker method is simple and efficient in the feature-based matching. However, the markers are easily confused and obscured.^14,15 In addition, because of the multi-DOF movement, the complex movement trajectory, and the small end-effector, it is difficult to find a suitable position to attach marker in parallel robot. So in this paper, the pose detection of parallel robot is realized using marker-free method. ¹⁶

As a milestone of feature matching, the matching method of Scale Invariant Feature Transform (SIFT) is proposed by Lowe ¹⁷ based on 2D image scale–space theory. In order to detect the stable feature points efficiently, Lowe ¹⁸ optimized LoG spot detection method by adopting the differential Gauss extreme DoG operator as the detection function. The SIFT algorithm based on DoG operator can not only deal with the matching problem in the conditions of translation, rotation, and scale transformation among images, but also remain comparatively stable ability of feature matching.^19,20 Based on the above advantages, the SIFT algorithm has been widely used on the occasions of the static image registration, the panoramic image mosaic, the face recognition, and others, which do not require good real-time performance.^21,22 However, for matching with high-speed requirement in pose detection of parallel robot, the conventional SIFT algorithm is complex and needs more time to match. ²³ The Harris corner detection algorithm is easy and stable. It is not susceptible to light and rigid geometric distortion, but it is sensitive to noise, and it has got the scale invariance. ²⁴ Therefore, some scholars have put forward the matching algorithm by combining Harris with SIFT. For example, a Harris–SIFT algorithm is proposed and validated in matching of binocular vision in Zhao et al. ²⁵ by combining Harris significance with SIFT descriptor. This method effectively improves the real-time performance of matching algorithm, but the information of corner points detected by Harris is less than the information of feature points detected by SIFT. The Harris–SIFT algorithm also has the problem of mismatching. ²⁶

Compared with other refining methods of image matching, the RANSAC algorithm can estimate high-precision parameters from the data set containing a large number of outliers, which are seriously deviated from normal range and unable to accord with the mathematical model. It has been widely applied in vision detection due to the stronger robustness.^27,28 However, the accuracy of the RANSAC algorithm depends on the iterative number. More iterations may need more time. Generally, the loss function is modified to improve the RANSAC algorithm. ²⁹ For example, the joint probability distribution of the inlier, which may be described by model and the outlier error are used to verify the model. It can improve the accuracy of model estimation. ³⁰ For the factors of complex image background, unobvious end-effector characteristics and uneven illumination in the pose detection of parallel robot based on binocular vision, the distribution of points in matching data set is uneven, and the data set contains many outliers. So it is difficult to realize high-accuracy pose detection of parallel robot using the conventional RANSAC algorithm and improved RANSAC algorithm based on modified loss function.

For the above problems, the binocular vision is used in pose detection of parallel robot in this paper. The matching problem of pose detection is studied in priority. In order to solve the problem of poor real-time performance of matching, the Harris–SIFT algorithm is adopted to realize image prematching considering that the images of parallel robot without landmark on end-effector are rigid images and have multiple corner points in this paper. For the problems of mismatching in Harris–SIFT algorithm, an improved RANSAC algorithm is proposed to refine the prematching results. The improved algorithm can remove the mismatching points in Harris–SIFT algorithm by selecting feature points in separated grids of the images and predetecting to validate provisional model. The improved algorithm can improve real-time performance and accuracy of image matching simultaneously. As a result, the method improves the accuracy of pose detection of parallel robot and provides a feasible solution for the real-time high-performance closed-loop control of parallel robot.

RANSAC algorithm and improvement

RANSAC algorithm

The data set after prematching contains the inliers and the outliers. The outlier is produced by the error detection, calculation, or assumption. The RANSAC algorithm is a robust model parameter estimation algorithm. It calculates the mathematical model parameters and tests the model using the remaining points to get effective data samples. As a result, the efficiency of image processing is improved using fewer points to estimate model parameters. It makes up for the deficiency of general parameter estimation method using all points.

According to the basic idea of RANSAC algorithm, the target transformation model H between two images needs to be calculated by Sampson distance. Then, the matching points are tested by the target model. The mismatching points are removed at last. The selection of target model and the representation of Sampson distance are as following:

• 1. Calculating the target model H between two images: The eight matching point pairs are randomly selected to calculate the provisional candidate model F usually in RANSAC algorithm. The Sampson distance d of the matching point pairs corresponding to the model F is calculated. The threshold value k is set. The points, which meet the condition of d < k are selected as inliers. Furthermore, the model F, which includes most inliers, is optimized to obtain the target model H. Finally, the matching points are re-estimated by target candidate model H. The mismatching point pairs are removed. Projection transformation model is expressed as follows

$$\left[\begin{array}{c}x\hfill \\ \multicolumn{1}{c}{y}\\ \multicolumn{1}{c}{1}\end{array}\right]=H\left[\begin{array}{c}{x}^{\prime }\hfill \\ \multicolumn{1}{c}{y^{\prime }}\\ \multicolumn{1}{c}{1}\end{array}\right]$$

(1)

where (x, y, 1) and (x’, y’, 1) represent the homogeneous coordinates of matching point pairs m(x, y) and m (x’, y’), respectively.

• 2. The representation of Sampson distance d: for any matching point pair m(x, y) and m(x’, y’), the representation of Sampson distance d in candidate model F can be calculated using Equation (2)

$$d=\frac{m^T\mathrm{Fm}}{{\left(\mathrm{Fm}\right)}_1^2+{\left(\mathrm{Fm}\right)}_2^2+{\left(F^T{m}^{\prime }\right)}_1^2+{\left(F^T{m}^{\prime }\right)}_2^2}$$

(2)

The improvement of RANSAC algorithm

The conventional RANSAC algorithm is capable of interpreting and smoothing data containing gross errors. However, for the factors of complex image background, unobvious end-effector characteristics and uneven illumination in the pose detection of parallel robot based on binocular vision, it is difficult to detect the pose based on the conventional RANSAC algorithm accurately and rapidly. It has the following problems:

• The two candidate points could be considered to be one point if the distance between the two points is too close when selecting a sample randomly. It will lead to low-accuracy of the fundamental matrix.

• Because the prematching data set of the parallel robot images includes outliers, it is necessary to look for the support set of candidate models every time when selecting sample set randomly. Too much time is needed to look for the support set when there are many errors in observation data set.

So, an improved RANSAC algorithm is proposed by selecting feature points in separated grids of the images and predetecting to validate provisional model. First, for the problem (a) mentioned above, instead of random sampling, the sampling method selecting feature points in separated grids is proposed. The image is divided into blocks and the blocks without matching points are removed. For increasing the sparsity of the sample distribution and preparing for validating provisional model by predetecting, the samples are selected randomly from different blocks to constitute a random sample set with sparse distribution. Second, for the problem (b) mentioned above, the provisional model is validated by predetecting. The validating point is selected randomly from sample set to validate the provisional model. It can remove the invalid model in advance to avoid useless calculation in a large number of support set.

These improvements not only make the model parameters more accurate, but also overcome the deficiency of conventional RANSAC algorithm, which has to detect all the observed data points after determining the provisional model parameters. The improved algorithm reduces the amount of computation and improves the detection efficiency. Thus, for the original matching point pairs, the improved RANSAC algorithm is as follows:

1. Selecting feature points in separated grids.

2. ii. The eight matching point pairs are randomly selected from sample set S to obtain provisional candidate model F.

Validating provisional model by predetecting.

The ninth point is validated whether it is the support set of candidate model F: if it is yes, this provisional candidate model F is a candidate model F. Otherwise, the nine matching point pairs are reselected and process 1 is repeated.

1. Detecting all matching point pairs by candidate model F and the threshold k to get m, which is the number of the support set for candidate model.

2. If $m\ge$ threshold t, the target model F is obtained. Otherwise, the nine matching point pairs are reselected and processes 1, 2, and 3 are repeated.

3. Optimizing target model F and determining the target model H ultimately.

4. The counter plus 1 when random sample set is selected every time. If the model parameters are not found after repeating k times, the program is terminated.

Pose detection of parallel robot

The improved RANSAC algorithm is applied to pose detection of a self-developed novel 3-DOF parallel robot to verify the validity. The prototype of parallel robot is shown in Figure 1. The robot is mainly composed of the platform, connecting rods, columns, and sliders. The moving platform is connected to the fixed platform by three limbs. Each body of the column guide is equipped with slider. Pose of end-effector is changed by the motor driving slider. The parallel robot can realize 2D movement and 1D rotation. The capacity of rotation can reach 115°. It breaks through the limit of the rotation capacity in conventional parallel robot.

Figure 1.

Prototype of the novel 3-DOF parallel robot.

For the pose detection of the novel 3-DOF parallel robot, images are acquired by left and right cameras, and feature points are extracted based on Harris operator first. Second, feature points are prematched by Harris–SIFT algorithm to improve the matching speed. Third, an improved RANSAC algorithm is proposed to solve the mismatching problem. Finally, the matching points after refining are taken into the binocular vision coordinate model to get the pose of parallel robot.

Image acquisition and feature point extraction

Image acquisition

This paper depends on the system of MICROVISION binocular stereovision as a development platform. Camera model is MV-1300FM. The cameras are installed in front of parallel robot. The height from ground and the angle between cameras can be adjusted flexibly. The original images of parallel robot are acquired by left and right cameras at the same time, which are shown in Figure 2. The original image size is 1280 × 1024 and the format is bmp.

Figure 2.

Original images of the parallel robot.

Feature point extraction based on Harris

Because parallel robot has obvious rigid-body characteristics, we extract the feature points in images using the Harris. It has high re-examination rate in terms of rigid geometric distortion and brightness changes, and it is simple and stable. ³⁰ In feature point extraction based on Harris, the second-order matrix of pixels is expressed as follows

$$\mathit{M}=\left[\begin{array}{cc}{I}_x^2\hfill & I_x{I}_y\hfill \\ \multicolumn{1}{c}{I_x{I}_y}& I_y^2\hfill \end{array}\right]$$

(3)

In the formula, $I_x$ and $I_y$ , respectively, represent the gradient of pixel gray value in $x$ and $y$ directions. $I_{\mathrm{xy}}$ represents the second mixed partial derivative. Then the Harris response function of pixels is as follows

$$R=\det \mathit{M}-k{\left(\mathrm{trace}\mathit{M}\right)}^2$$

(4)

The det M is the determinant of the matrix M . The $\mathrm{trace}\mathit{M}$ is the authentic of the matrix M . The k is the empirical constant, which is usually in the range of 0.04 to 0.06. The threshold value is set as t. The response R is computed based on the Harris operator. If R > t, it is a feature point. Otherwise, it is not. In this work, the threshold value t is set to be 0.8. As shown in Figure 3, according to the Harris operator, the corner points in images can be accurately extracted as the feature points. The left and right images are acquired by left and right cameras, respectively.

Figure 3.

Extracted feature points using Harris.

Stereo matching and three-dimensional coordinate calculation of mark points

Stereo prematching based on Harris–SIFT algorithm

Feature detection based on Harris is deemed to be stable, but sensitive to noise. It even cannot achieve scale invariance. In order to solve these problems, we combine Harris with SIFT to achieve the prematching of robot images. The feature points are extracted by Harris and matched by SIFT, thereby the feature points have common feature of the two algorithms that can improve real-time performance and accuracy. ²⁷

Application of improved RANSAC algorithm

The improved RANSAC algorithm is applied to the stereo matching to refine prematching result. So the real-time performance and accuracy of matching can be improved. The matching point pairs refined by conventional RANSAC algorithm and improved RANSAC algorithm from prematching results of Harris–SIFT algorithm are shown in Figure 4. Compared with the conventional RANSAC algorithm (Figure 4(a)), the matching result of improved RANSAC algorithm (Figure 4(b)) contains fewer mismatching point pairs, and the matching accuracy can be improved.

Figure 4.

Matching point pairs refined by conventional RANSAC algorithm and improved RANSAC algorithm from prematching results of Harris–SIFT algorithm. (a) Matching point pairs refined by conventional RANSAC algorithm. (b) Matching point pairs refined by improved RANSAC algorithm.

Camera model

In order to calculate the actual three-dimensional coordinates of spatial points according to the image coordinates of matching point pairs, the pinhole model is used to analyze the camera imaging model, as shown in Figure 5. According to the camera calibration method of Zhang, ³¹ four coordinate systems are defined: (1) The world coordinate system $O_w-X_w{Y}_w{Z}_w$ , which is customized depending on the circumstances to describe the coordinates of objects in three-dimensional space. (2) The camera coordinate system, $O_c-X_c{Y}_c{Z}_c$ , in which origin O_c is set in the optic center of camera, and Z_c axis coincides with the optical axis and is perpendicular to the image plane. (3) The image coordinate system, $O_i-X_i{Y}_i$ , in which origin O_i is set at the center of image. (4) The pixel coordinate system o-uv, in which origin o is set at the upper left of image and unit is pixel.

Figure 5.

Pinhole imaging model.

According to pinhole imaging model, the any point M in space $(X_w,Y_w,Z_w)$ is projected to the point m(u, v) in the image. The relationship between them is as follows

$$Z_{C}m={\mathit{M}}_{\mathit{1}}{\mathit{M}}_{\mathit{2}}M$$

(5)

where Z_C is the distance from focus to the image plane, ${\mathit{M}}_1$ is the internal parameter matrix, ${\mathit{M}}_2$ is the external parameter matrix. The specific conversion relationship among the coordinate systems is as follows:

• 1. For spatial point M and projective point m, the transformation between $M_C$ (coordinate of M in the camera coordinate system) and $M_I$ (coordinate of m in the image coordinate system) can be expressed as following equation

$$M_I=\frac{1}{Z_C}\left[\begin{array}{cccc}f\hfill & 0\hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& f\hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 1\hfill & 0\hfill \end{array}\right]M_C$$

(6)

where Z_C is the distance from focus to image plane, $f$ is the focal length. Besides, the transformation between $M_I$ (coordinate of m in the image coordinate system) and $m$ (coordinate of m in the pixel coordinate system) can be expressed as follows

$$m=\left[\begin{array}{ccc}\frac{1}{d_x}\hfill & \frac{-\cos \theta }{d_x}\hfill & u_0\hfill \\ \multicolumn{1}{c}{0}& \frac{1}{d_y*\sin \theta }\hfill & v_0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 1\hfill \end{array}\right]M_I$$

(7)

where $(u_0,v_0)$ is the origin $O_{\mathrm{i}}$ coordinate of image coordinate system in pixel coordinate system, $d_x$ and $d_y$ are, respectively, distances between adjacent pixels in x and y directions, and $\theta$ is the angle between horizontal axis and vertical axis of pixel coordinate system.

According to Equations (6) and (7), the transformation between $M_C$ (coordinate of M in the camera coordinate system) and $m$ (coordinate of m in the pixel coordinate system) can be expressed as

$$\begin{array}{c}{Z}_C{M}_I=\left[\begin{array}{ccc}\frac{1}{d_x}\hfill & \frac{-\cot \theta }{d_x}\hfill & u_0\hfill \\ \multicolumn{1}{c}{0}& \frac{1}{d_y*\sin \theta }\hfill & v_0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 1\hfill \end{array}\right]\left[\begin{array}{cccc}f\hfill & 0\hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& f\hfill & 0\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& 0\hfill & 1\hfill & 0\hfill \end{array}\right]\hfill \\ M_C={\mathit{M}}_1{M}_C\hfill \end{array}$$

(8)

where ${\mathit{M}}_1$ is the internal parameter matrix, controlled by internal camera configuration.

• 2. The transformation between $M_W$ (coordinate of M in the world coordinate system) and $M_C$ (coordinate of M in the camera coordinate system) can be expressed as

$$M_C=\left[\begin{array}{cc}\mathit{R}\hfill & \mathit{T}\hfill \\ \multicolumn{1}{c}{0}& 1\hfill \end{array}\right]M_W={\mathit{M}}_2{M}_W$$

(9)

where R is a rotation matrix, T is a translation vector, and M₂ is the extrinsic parameter matrix.

For binocular vision model, the internal and external parameters of left and right cameras can be obtained by camera calibration. Assuming that ${\mathit{R}}_{\mathit{l}}$ , ${\mathit{T}}_{\mathit{l}}$ and ${\mathit{R}}_{\mathit{r}}$ , ${\mathit{T}}_{\mathit{r}}$ , respectively, represent the translation vectors and the rotation matrices of left and right cameras, the structural parameters of binocular vision system can be expressed as

$$\{\begin{array}{c}\mathit{R}={\mathit{R}}_{\mathit{r}}{\mathit{R}}_{\mathit{l}}^{-1}\hfill \\ \multicolumn{1}{c}{\mathit{T}={\mathit{T}}_r-{\mathit{R}}_{\mathit{r}}{\mathit{R}}_{\mathit{l}}^{-1}{\mathit{T}}_{\mathit{l}}}\end{array}$$

(10)

Coordinate calculation of mark point

The coordinate of projective point imaged by the left and right cameras from the same spatial mark point of the end-effector of parallel robot can be acquired by matching. The spatial mark point $P_W$ is imaged by the left and right cameras, respectively, as projective points $P_l$ and $P_r$ . The projection matrices, respectively, are ${\mathit{M}}_{\mathit{l}}$ and ${\mathit{M}}_{\mathrm{r}}$ . According to the projection theorem

$$\{\begin{array}{c}{s}_l{P}_l={\mathit{M}}_{\mathit{l}}{P}_W\hfill \\ \multicolumn{1}{c}{s_r{P}_r={\mathit{M}}_{\mathit{r}}{P}_W}\end{array}$$

(11)

where $s_l$ and $s_r$ indicate different scale factors, respectively. Then the equations are

$$s_l\left[\begin{array}{c}{u}_1\hfill \\ \multicolumn{1}{c}{v_1}\\ \multicolumn{1}{c}{1}\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}^1\hfill & m_{12}^1\hfill & m_{13}^1\hfill & m_{14}^1\hfill \\ \multicolumn{1}{c}{m_{21}^1}& m_{22}^1\hfill & m_{23}^1\hfill & m_{24}^1\hfill \\ \multicolumn{1}{c}{m_{31}^1}& m_{32}^1\hfill & m_{33}^1\hfill & m_{34}^1\hfill \end{array}\right]\left[\begin{array}{c}X\hfill \\ \multicolumn{1}{c}{Y}\\ \multicolumn{1}{c}{Z}\\ \multicolumn{1}{c}{1}\end{array}\right]$$

(12)

$$s_r\left[\begin{array}{c}{u}_2\hfill \\ \multicolumn{1}{c}{v_2}\\ \multicolumn{1}{c}{1}\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}^2\hfill & m_{12}^2\hfill & m_{13}^2\hfill & m_{14}^2\hfill \\ \multicolumn{1}{c}{m_{21}^2}& m_{22}^2\hfill & m_{23}^2\hfill & m_{24}^2\hfill \\ \multicolumn{1}{c}{m_{31}^2}& m_{32}^2\hfill & m_{33}^2\hfill & m_{34}^2\hfill \end{array}\right]\left[\begin{array}{c}X\hfill \\ \multicolumn{1}{c}{Y}\\ \multicolumn{1}{c}{Z}\\ \multicolumn{1}{c}{1}\end{array}\right]$$

(13)

where $(u_1,v_1,1)$ and $(u_2,v_2,1)$ , respectively, represent homogeneous coordinates of projective points $P_l$ and $P_r$ . The $(X,Y,Z,1)$ is the homogeneous coordinates of the spatial mark point $P_W$ in world coordinate system. The $m_{\mathrm{ij}}^k$ (k = 1, 2; i = 1, 2, 3; j = 1, 2, 3, 4) represents the ith row and jth column of the left and right projection matrices. Utilizing the least square method, Equations (12) and (13) can be solved and (X, Y, Z) can be obtained.

Parameter solution of parallel robot

Parallel robot coordinate system

According to the characteristics of robot, the parallel robot coordinate system is defined as Figure 6. The $\left\{B\right\}=\{O-\mathrm{XYZ}\}$ and $\left\{T\right\}=\{O^{\prime }-X^{\prime }{Y}^{\prime }{Z}^{\prime }\}$ , respectively, represent fixed and moving coordinate system. The origin $O$ of the fixed coordinate system $O-\mathrm{XYZ}$ is located in the midpoint of $Q_1$ and $Q_2$ , and $Z$ axis is perpendicular to the plane of fixed platform. The origin of moving coordinate system $O^{\prime }-X^{\prime }{Y}^{\prime }{Z}^{\prime }$ is located in the midpoint of $P_1$ and $P_2$ , and $Z^{\prime }$ axis is perpendicular to the plane of moving platform. The fixed coordinate system is set as the world coordinate system. $P_1$ and $P_2$ are set as mark points. Then the change of parallel robot end-effector pose can be represented by the change of center point pose of moving platform. So, the pose of parallel robot includes the position of point $O^{\prime }$ and three attitude angles of parallel robot, which are pitch angle $\alpha$ , roll angle $\beta$ , and yaw angle $\gamma$ .

Figure 6.

Coordinate systems of the novel 3-DOF parallel robot.

Position parameter calculation

In Figure 6, points $P_1$ and $P_2$ are set as mark points, and the space coordinates of $P_1(X_1,Y_1,Z_1)$ and $P_2(X_2,Y_2,Z_2)$ are known, the midpoint of parallel robot moving platform is also the midpoint between point $P_1$ and $P_2$ . So its position coordinate is expressed as follows

$$X_0=\frac{X_1+X_2}{2},Y_0=\frac{Y_1+Y_2}{2},Z_0=\frac{Z_1+Z_2}{2}$$

(14)

Pose parameter calculation

For the moving platform coordinate $\left\{T\right\}=\{O^{\prime }-X^{\prime }{Y}^{\prime }{Z}^{\prime }\}$ , the origin $O^{\prime }$ can be located in the origin $O$ of world coordinate system by translation. The attitude angles $(\alpha ,\beta ,\gamma )$ of parallel robot represent the angles that the moving platform coordinate $\left\{T\right\}=\{O^{\prime }-X^{\prime }{Y}^{\prime }{Z}^{\prime }\}$ of parallel robot need to rotate for coinciding with the world coordinate $\left\{B\right\}=\{O-\mathrm{XYZ}\}$ . It is specifically expressed as pitch angle $\alpha$ , which rotates around $X$ axis, roll angle $\beta$ , which rotates around $Y$ axis, yaw angle $\gamma$ , which rotates around $Z$ axis. Rotation matrix is expressed as

$$\begin{array}{cc}\multicolumn{1}{c}{R}& =\left[\begin{array}{ccc}\cos \beta \cos \gamma \hfill & \cos \beta \sin \gamma \hfill & -\sin \beta \hfill \\ \multicolumn{1}{c}{\sin \alpha \sin \beta \cos \gamma }& \sin \alpha \sin \beta \sin \gamma +\cos \alpha \cos \gamma \hfill & \sin \alpha \cos \beta \hfill \\ \multicolumn{1}{c}{\cos \alpha \sin \beta \cos \gamma }& \cos \alpha \sin \beta \sin \gamma -\sin \alpha \cos \gamma \hfill & \cos \alpha \cos \beta \hfill \end{array}\right]\hfill \\ \multicolumn{1}{c}{}& =\left[\begin{array}{ccc}{R}_{11}\hfill & R_{12}\hfill & R_{13}\hfill \\ \multicolumn{1}{c}{R_{21}}& R_{22}\hfill & R_{23}\hfill \\ \multicolumn{1}{c}{R_{31}}& R_{32}\hfill & R_{33}\hfill \end{array}\right]\hfill \end{array}$$

(15)

Then, the pitch angle $\alpha$ , the roll angle $\beta$ , and the yaw angle $\gamma$ are, respectively, expressed as

$$\{\begin{array}{c}\alpha =\arctan \left(\frac{R_{23}}{R_{33}}\right)\hfill \\ \multicolumn{1}{c}{\beta =-\arcsin R_{13}}\\ \multicolumn{1}{c}{\gamma =\arctan \left(\frac{R_{12}}{R_{11}}\right)}\end{array}$$

(16)

The parallel robot can do movement with three DOF. So the translation in $Y$ axis and $Z$ axis, and roll angle $\beta$ can be acquired. So the pose parameters of parallel robot are expressed as $(Y,Z,\beta )$ . Then, all the pose parameters of the parallel robot are obtained.

Experiment and analysis

Experimental platforms are as follows: (1) Hardware platform: The operating system is Microsoft Windows 7. The processor is Intel(R) Core(TM)2 Duo. The frequency is 2.66 GHz. The memory is 2 GB. The high-speed digital image acquisition cards are MV-1394A and MV1394B. The camera model is MV-1300FM. The lens model is AFT-0814MP. The calibration target model is AFT-CT430. (2) Software platform: The CCAS MICROVISION binocular stereovision development platform is used. It uses visual C++6.0 integrated development tools to prepare the human–computer interaction interface. The API function interface of each module is implemented by combining C++programming language with third-party software MATLAB.

Test method

In order to verify the advantages of improved RANSAC compared with conventional RANSAC algorithm in accuracy and speed, three-group experiments of pose detection are carried out. First, three-group images of parallel robot are acquired from different directions in three-group experiments, respectively. The improved RANSAC and conventional RANSAC algorithm are used for each group of images to refine the prematching results of Harris–SIFT algorithm. Then, these two methods are compared in the accuracy and time for each group. Finally, the extracted mark points are taken into the camera model to calculate the pose parameters. Then the acquired pose parameters are compared with actual pose parameters, which are obtained by laser distance measuring instrument of INR-Laser Scanner H and electronic Honeywell-HMR3100 to verify the validity of the pose detection method.

Result and analysis

The matching results of three-group images are shown in Figures 7–9 and Tables 1–3. In Figures 7–9, the left half of each image represents the images acquired by left camera, and the right half of each image represents the images acquired by right camera. Figures 7(a)–9(a) and Figures 7(b)–9(b) are the results of refining by conventional RANSAC and improved RANSAC algorithm, respectively. Figures 7(c)–9(c) is the mismatching point pairs in the prematching results of Harris–SIFT algorithm. It shows that the prematching results of Harris–SIFT algorithm in the three-group experiments contain many mismatching point pairs (Figures 7(c)–9(c)). Compared with the conventional RANSAC algorithm (Figures 7(a)–9(a)), the matching results of improved RANSAC algorithm (Figures 7(b)–9(b)) contain fewer mismatching point pairs, and the matching accuracy can be improved. The detailed data of the three-group experiments are shown in Tables 1–3. The MA can be calculated by Equation (17)

$$\mathrm{MA}=\frac{\mathrm{MP}-M{P}_m}{\mathrm{MP}}$$

(17)

Figure 7.

Matching results of the images acquired by the left-oriented cameras. (a) Matching results refined by conventional RANSAC algorithm from prematching results of Harris–SIFT algorithm. (b) Matching results refined by improved RANSAC algorithm from prematching results of Harris–SIFT algorithm. (c) Mismatching point pairs in the prematching results of Harris–SIFT algorithm.

Figure 8.

Matching results of the images acquired by the right-oriented cameras. (a) Matching results refined by conventional RANSAC algorithm from prematching results of Harris–SIFT algorithm. (b) Matching results refined by improved RANSAC algorithm from prematching results of Harris–SIFT algorithm. (c) Mismatching point pairs in the prematching results of Harris–SIFT algorithm.

Figure 9.

Matching results of the images acquired by the left-oriented camera and the right-oriented camera. (a) Matching results refined by conventional RANSAC algorithm from prematching results of Harris–SIFT algorithm. (b) Matching results refined by improved RANSAC algorithm from prematching results of Harris–SIFT algorithm. (c) Mismatching point pairs in the prematching results of Harris–SIFT algorithm.

Table 1.

Matching results of the images acquired by the left-oriented cameras.

Algorithm	Conventional RANSAC	Improved RANSAC
T	2.73 s	1.10 s
MP	181	44
MPm	35	3
MA	80.66%	93.18%

T: total time of calculation; MP: total number of the matching point pairs after refining the prematching results; MP_m: number of the mismatching point pairs; MA: matching accuracy.

Table 2.

Matching results of the images acquired by the right-oriented cameras.

Algorithm	Conventional RANSAC	Improved RANSAC
T	2.45 s	0.89 s
MP	134	31
MPm	29	2
MA	78.36%	93.55%

T: total time of calculation; MP: total number of the matching point pairs after refining the prematching results; MP_m: number of the mismatching point pairs; MA: matching accuracy.

Table 3.

Matching results of the images acquired by the left-oriented camera and the right-oriented camera.

Algorithm	Conventional RANSAC	Improved RANSAC
T	1.97 s	0.61 s
MP	18	5
MPm	2	0
MA	88.89%	100.0%

T: total time of calculation; MP: total number of the matching point pairs after refining the prematching results; MP_m: number of the mismatching point pairs; MA: matching accuracy.

From the matching results in Figures 7–9 and Tables 1–3, we can see that for the problems of low-accuracy and time-consuming in matching due to lots of outliers in prematching data set of parallel robot images, the improved RANSAC algorithm can effectively improve the accuracy and speed of pose detection of parallel robot. The average time of the method based on the conventional RANSAC algorithm is 2.38 s, and the average accuracy is 82.64%. The average time of the proposed method is 0.87 s, and the average accuracy is 95.58%. Compared with the conventional RANSAC algorithm, the average time of the proposed method decreases by 63.45%, which improves the real-time performance of system. The average matching accuracy increases by 15.66% using improved RANSAC in experiments. From Tables 1–3, we can see that the time of pose detection can be decreased to 0.61 s, and the accuracy can be improved simultaneously. The proposed method reduces the time of end-effector pose detection for parallel robot based on binocular vision from second level to millisecond level, which can meet the time requirement of the servo cycle of parallel robot. It provides a feasible solution for real-time detection of end-effector pose of parallel robot.

We take out the mark points of end-effector from feature points, and take it into camera model to get the pose parameters. Specific operation is as follows: Experiments are accomplished by grouping according to the different deviation angle between the left and right cameras. The robot moves by fixed track and the images are acquired regularly. The pose parameters at each measuring point are got from the images based on the conventional RANSAC algorithm and the improved RANSAC algorithm, respectively. The result is compared with actual pose. The tracking errors can be calculated by Equation (18)

$$\begin{array}{c}\mathrm{\Delta }Y=Y-Y_{\mathrm{real}}\hfill \\ \mathrm{\Delta }Z=Z-Z_{\mathrm{real}}\hfill \\ \mathrm{\Delta }\beta =\beta -{\beta }_{\mathrm{real}}\hfill \end{array}$$

(18)

The $\mathrm{\Delta }Y$ , $\mathrm{\Delta }Z$ , $\mathrm{\Delta }\beta$ , respectively, represent tracking error in Y direction, Z direction, and roll angle $\beta$ . The $Y$ , $Z$ , $\beta$ represent the detection pose parameters at each measuring point. The $Y_{\mathrm{real}}$ , $Z_{\mathrm{real}}$ , ${\beta }_{\mathrm{real}}$ represent the actual pose parameters at each measuring point. The tracking errors from experiments are shown in Figure 10. The data come from 12 measuring points in the three-group experiments, respectively. The $\mathrm{\Delta }Y$ , $\mathrm{\Delta }Z$ , $\mathrm{\Delta }\beta$ are analyzed by comparing the experiment results of conventional RANSAC algorithm with the experiment results of improved RANSAC algorithm in every group, respectively, as shown in Figure 10(a)–(c).

Figure 10.

Pose tracking errors of parallel robot. (a) Tracking errors based on the images acquired by left-oriented cameras. (b) Tracking errors based on the images acquired by right-oriented cameras. (c) Tracking errors based on the images acquired by the left-oriented camera and the right-oriented camera.

Figure 10 shows that the tracking errors of P₁ and P₇ in the Y direction and the tracking errors of P₄ and P₁₀ in the Z direction are larger than others. In the above-mentioned measuring points, because of the significant change of direction, the changes of speed and acceleration are obvious. In addition, the inertia also can affect the pose accuracy. Figure 10 indicates that the movement trend of detection three-dimensional pose based on binocular vision in the experiments corresponds to actual pose.

Based on the conventional RANSAC algorithm, the gross errors occurred in P₁₂ (Figure 10(a)) and P₆ (Figure 10(b)), which come from mismatching. In addition, the average deviation of the measuring points can be got according to the tracking error. In the experiments based on the conventional RANSAC algorithm, the average deviation in Y direction is 1.278 mm, the average deviation in Z direction is 1.207 mm, and the average deviation of roll angle $\beta$ is 1.025°. In the experiments based on the improved RANSAC algorithm, the average deviation in Y direction is 0.407 mm, which decreases by 0.871 mm, the average deviation in Z direction is 0.387 mm, which decreases by 0.82 mm, and the average deviation of roll angle $\beta$ is 0.321°, which decreases by 0.704°.

From Figure 10, we can see that the tracking error and the average deviation are decreased using the proposed method. In addition, it can reduce gross error, which is easy to generate in pose detection based on the conventional RANSAC algorithm. Compared with the detection method, which is used in Sui et al., ³² the average deviation in Z direction is reduced from 4.09 mm to 0.387 mm. It shows that the improved algorithm not only improves the real-time performance, but also improves the accuracy of the pose detection.

In conclusion, the RANSAC algorithm proposed in this paper is adopted to refine the prematching results of Harris–SIFT operator. Then, the refined matching results are taken into the camera model, and the pose parameters of end-effector of parallel robot are obtained. As a result, the accuracy and speed of end-effector pose detection of parallel robot are efficiently improved.

Conclusion

In the end-effector pose detection of parallel robot based on binocular vision, the factors of complex image background, unobvious end-effector characteristics and uneven illumination make the image matching difficult to achieve accurately and rapidly, and affect the accuracy and real-time performance of pose detection. Therefore, in order to improve the accuracy and speed of end-effector pose detection of parallel robot based on binocular vision and further realize the closed-loop control of parallel robot, this paper improves the conventional RANSAC algorithm and proposes a pose detection method of parallel robot based on the improved RANSAC algorithm. The following conclusions can be drawn:

1. A method of selecting feature points in separated grids of images is proposed to improve the conventional RANSAC algorithm. It solves the problem of inaccurate fundamental matrix caused by random sample selection. It increases the sparsity of sample distribution and improves the accuracy of image matching.

2. A method of predetecting to validate provisional model is proposed to further improve the conventional RANSAC algorithm. It solves the problem that too much time is needed to look for the support set when there are many errors in observation data set. It reduces invalid calculation of the support set and improves the speed of image matching.

3. An image matching method based on Harris–SIFT algorithm and improved RANSAC algorithm is proposed. It solves the problem of mismatching in the prematching results of Harris–SIFT algorithm to improve the accuracy and real-time performance of matching.

4. Based on proposed image matching method, the end-effector pose detection of parallel robot is realized, which improves the accuracy and speed of pose detection. The proposed pose detection method of parallel robot was applied to a self-developed novel 3-DOF parallel robot to verify the validity. The experimental results show that, compared with the pose detection method based on the conventional RANSAC algorithm, the average deviations of end-effector poses in Y direction, Z direction and roll angle β decrease by 0.871 mm, 0.82 mm and 0.704° using the proposed pose detection method, respectively. The proposed pose detection method based on the improved RANSAC algorithm can improve the accuracy and speed of end-effector pose detection of parallel robot, which provides a feasible solution for real-time detection of end-effector pose for parallel robot. It also lays the foundation for further realizing the high-performance closed-loop control of parallel robot.