Sage Journals: Discover world-class research

Abstract

Parallel robots have a growing range of applications due to their appealing characteristics (high speed and acceleration, increased rigidity, etc.). However, several open problems make it difficult to model and control them. Low computational-cost algorithms are needed for high speed tasks where high accelerations are required. This article develops the nonlinear camera-space manipulation method and makes use of an extended Kalman filter (EKF) for the estimation of the camera-space manipulation parameters. This is presented as an alternative to the traditional method which can be time consuming while reaching convergence. The proposed camera-space manipulation parameter identification was performed in positioning tasks for a parallel manipulator and the experimental results are reported. Results show that it is possible to estimate the set of camera-space manipulation parameters by means of an extended Kalman filter. Using the proposed Kalman filter method we observed a significant reduction of the computational effort when estimating the camera-space manipulation parameters. However, there was no significant reduction of the robot’s positioning error. The proposed extended Kalman filter implementation requires only 2 ms to update the camera-space manipulation parameters compared to the 85 ms required by the traditional camera-space manipulation algorithm. Such time reduction is beneficial for the implementation of the method for a wide range of high speed and industrial applications. This article presents a novel use of an extended Kalman filter for the real-time estimation of the camera-space manipulation parameters and shows that it can be used to increase the positioning accuracy of a parallel robot.

Keywords

Vision-based control camera-space manipulation extended Kalman filter parallel robot

Introduction

Industrial robot control has been an active area of research for several decades. Many efforts have been devoted to develop simple, yet robust and reliable algorithms to control complex robotic tasks. Furthermore, interest in the modeling and control of parallel robots has increased in recent years as a result of the versatility of these architectures. Some of their applications include pick and place operations, machine-tooling, and the performance of medical procedures.

Parallel manipulators consist of several closed kinematic chains that connect the end-effector to the base. This configuration makes them stiffer, relative to their total mass, than serial robots and yields an increased accuracy and velocity of the end-effector with a higher payload capacity. On the other hand, a disadvantage of parallel robots is their typically low cost-effectiveness due to their complex kinematics and rather expensive control units, as well as their poor workspace to robot-dimension ratio.¹

Moreover, their use is still sporadic due to several open problems inherent to the algorithms used to control them.² Indeed, the forward kinematic problem (FKP) of a parallel robots has multiple solutions and there is no standard technique to calculate them.³ Even though particular FKP solutions may exist, their calculation is rather time consuming⁴ and finding a complete solution becomes more complex with the number of degrees of freedom (DOF) of the end-effector. Also, numerical problems may be encountered even for robots with very simple architectures.⁵ Furthermore, the singularities of a parallel robot exist in greater number than in a serial robot. For example, an anthropometric arm with 6 DOF has 16 singularities, while a Stewart’s platform with the same number of end-point DOFs has 43.⁶

Another issue is the calibration of the kinematic model: the uncertainties in the manufacture and assembly of the robot makes it difficult to obtain an accurate Jacobian which is needed in most control approaches.^7
–9 Parallel robots require the calibration of a larger number of structural parameters when compared to serial robots. For instance, for a 3-DOF serial robot, having one link per DOF, nine parameters are needed to properly define its kinematic model (link length, link twist and link offset). One of the simplest parallel robots, 3-DOF delta robot, has 7 links and therefore requires the specification of 21 parameters.¹⁰

There is a significant amount of work devoted to developing control approaches for parallel robots.¹¹ In particular, vision-based control techniques are a popular option for industrial robots’ pick-and-place tasks. Indeed, vision-based control is robust to different kinds of errors such as disturbances coming from the dynamic environment and uncertainties of the robot’s kinematic model.¹² It also directly provides information about the relative position of the end-effector with respect to its environment regardless of the uncertainties of the dynamic environment and of the robot’s kinematic model. On the other hand, new uncertainties associated to the vision sensors must be addressed. In particular, we need to account for extrinsic uncertainties coming from the sensor’s model when the model is used to reconstruct signals and intrinsic uncertainties coming from the sensor itself (i.e. illumination conditions, discretization noise, etc.). These new uncertainties are independent of payload and robot architecture and thus easier to deal with.

Several vision-based control methods can be found in the literature. For example, visual servoing, camera-space manipulation (CSM), stereo vision, and so on.^{13

–27} These methods are used mainly to control serial manipulators and more recently, mobile robots.

The CSM can be considered as a calibration-free, vision-based control technique for robots and manipulators. To be successfully implemented, CSM requires neither an a priori knowledge of the robot’s Jacobian nor the calibration of the robot’s geometric or cameras’ parameters. The CSM is based on optimization techniques that minimize the end-effector positioning error within the field of view of the sensors used to control the maneuvers. It relies on the estimation of the relationship between the position of visual markers located on the robot and their corresponding position in images taken by at least two cameras. This relationship is defined implicitly in the so-called observation equations.²⁴ Compared to other vision-based control techniques, CSM is impervious to noise as sensitivity to it is eliminated by using historical data to average out the zero-mean uncertainty corrupting the image features. The CSM has been successfully applied to the control of a variety of 3-D positioning tasks with a wide range of applications including space exploration, warehouse maneuvering, industrial tasks, and mobile-robot control.^{21,24,28

–33} Using the CSM, these tasks have been executed reliably, robustly, and with high precision with both serial and mobile robots. Meanwhile, validation of the CSM method for use with parallel manipulators is an open area of research.

There has been little research to apply vision-based control methods to parallel robots; most of the reported works rely on visual servoing techniques. A 2-DOF redundant planar parallel manipulator is controlled using visual servoing to perform drawing paths with positioning errors of 0.1–1.0 mm, an improvement from 1 to 3.6 mm when visual servoing is not used.³⁴ A similar control strategy was used to control a parallel delta type robot (Robotenis) with three translational DOFs.¹⁶ The robot was able to interact with objects moving at speeds as high as 1 m/s, with a tracking error smaller than 20 mm. Further use of this technique has included force–position controls. For example, the Isoglide4-T3R1, an isotropic parallel structure with 4 DOF (3 in translation and 1 in rotation) was controlled in this way.¹⁷ Simulation results achieved positioning errors of less than 1 mm. Finally, position–velocity controls have also been developed: the Adept Quattro, a delta type parallel robot, is controlled via a visual servoing approach where positions and velocities are estimated simultaneously using vision.³⁵ Precisions of 4 mm at a velocity of 0.025 m/s are reported.

More recently, the CSM has proven its value for the control of parallel robots.³⁶ This work utilized an academic prototype to provide a proof of concept and reported an average positioning error of 3.17 ± 1.14 mm. Still work has to be done in order to make the most of the intrinsic advantages of an industrial parallel robot. In particular, low computational-cost algorithms are needed to control parallel robots in high speed tasks where high accelerations are required.

When dealing with robot-manipulator control, a nonlinear estimation procedure for the estimation of the CSM parameters has traditionally been preferred. The traditional CSM parameter estimation is an iterative method which recalculates the vision parameters using a global optimization method requiring all previous measurements. This can become time consuming as the number of measurements increases. On the other hand, the extended Kalman filter (EKF) corrects the previous parameter estimate and without the global optimization may decrease computation time becoming a viable alternative for CSM-based vision control. Additionally, it can avoid potential numerical instabilities which could occur with iterative methods.³⁷ Such numerical instabilities linked to the pin-hole camera model have been previously reported, however, instabilities of the orthographic camera model, have not yet been proven. The orthographic camera model and not the pin-hole model is used in this work.

In this article, we compare two approaches for producing the so-called camera-space kinematics, associated to CSM control approach. Specifically, we looked at the performance of an EKF against the original nonlinear CSM formulation. The use of Kalman filters for vision-based robot control is well established³⁸; however, this work is the first to apply it to the CSM method in parallel robots. This article shows that using the EKF over the traditional CSM method decreases computation time while estimating vision parameters. Parameters obtained using the EKF maintain the same positional accuracy compared to the traditional method while better suited for real-time vision-based control of a parallel robot manipulator.

Methods

The pin-hole camera model approximated by means of an orthographic camera model

The mapping between a 3-D physical space and a camera bi-dimensional image-plane can be approximately derived from a perspective projection described using the “pin-hole camera model” (Figure 1).^39,40

Figure 1.

Camera-fixed reference system.

This mapping is expressed as a function of the so-called view parameters, represented by vector $C = [C_{1}, \dots, C_{6}]^{T}$ , which define the relationship between the physical location of the visual features ( $x_{i} = [x_{i}, y_{i}, z_{i}]^{T}$ ) obtained through the forward kinematics of the robot and their corresponding image-plane location ( $x_{c_{i}}$ , $y_{c_{i}}$ ). The resulting approximation of the pin-hole camera model, detailed in the Appendix 1 “CSM model details,” is

[\begin{matrix} x_{c_{i}} \\ y_{c_{i}} \end{matrix}] = [\begin{matrix} g_{x} (x_{i}; C) \\ g_{y} (x_{i}; C) \end{matrix}]

The camera dependent $C$ parameters are computed after acquiring a number of m samples. These samples consist of the location of the visual markers attached to the robot’s end-effector expressed in the operational space and their corresponding camera-space projection. The parameters are obtained through an iterative procedure that involves the minimization of the following scalar function

J (C) = \sum_{i = 1}^{m} W_{i} ({[x_{c_{i}} - g_{x} (x_{i}; C)]}^{2} + {[y_{c_{i}} - g_{y} (x_{i}; C)]}^{2})

where subindex i denotes an observation and W_i is the sample’s associated weighting factor.

Target estimation

The estimation of the 3-D coordinates of the target point ( $x_{r} = {[X_{r}, Y_{r}, Z_{r}]}^{T}$ ) is obtained from its measured position in the camera frame considering the current vision parameters. This is accomplished by minimizing the following function

F (x_{r}) = \sum_{j = 1}^{n c} {[x_{c_{r}}^{j} - g_{x} (x_{r}; C^{j})]}^{2} + {[y_{c_{r}}^{j} - g_{y} (x_{r}; C^{j})]}^{2}

where $n c$ is the number of participating cameras, and $x_{c_{r}}^{j}$ and $y_{c_{r}}^{j}$ give the image coordinates in pixels of the target point in the j-th camera. When the positioning task is finished, a designated point on the robot’s end-effector ( $x_{d}$ ) should coincide with the target ( $x_{r}$ ). The relationship between the position of $x_{d}$ and that of the visual marker ( $x_{i}$ ) is always known through the end-effectors’ geometry. The minimization of equation (3) is shown explicitly in the Appendix 1 “Target tracking.”

Flattening

The applicability of the orthographic model can be increased after reducing its associated errors by means of flattening. The flattening procedure consists of modifying the raw camera samples in such a way that they become more consistent with an orthographic projection.^37,41 The modified samples $(x_{f}_{i}, y_{f}_{i})$ are obtained from the original detected image-plane location $(x_{c}_{i}, y_{c}_{i})$ as follows

x_{f}_{i} = x_{c}_{i} \frac{Z_{i}}{Z_{r}}; y_{f}_{i} = y_{c}_{i} \frac{Z_{i}}{Z_{r}}

Considering the coordinate system attached to the camera as shown in Figure 1, the origin of this coordinate system is located at the focal point of the camera while the X and Y axes are parallel to the image plane’s horizontal and vertical axes, respectively. The Z axis is oriented along the optical axis of the camera. Therefore, Z_i in equation (4) represents the Z-coordinate of the i-th detected sample, whereas Z_r is the roughly anticipated value of the Z-distance of the target at the maneuver’s termination along each camera axis. In the experiments reported herein, Z_r is continuously updated following the evolution of the estimated view parameters when new samples are obtained, as described in the section “Target estimation” and Appendix 1 “CSM model details.”

Extended Kalman filter

Consider the nonlinear system described by

C_{k} = F_{k - 1} (C_{k - 1}) + w_{k - 1}

for the k th measurement. The state vector $C_{k}$ is determined by the nonlinear state transition function $F_{k - 1}$ and a zero mean process noise $w_{k}$ with covariance $Q_{k}$ . An estimate of $C_{k}$ can be found from a number of $y_{k}$ measurements. Each measurement is a function of the states and a zero mean measurement noise $v_{k}$ with covariance $R_{k}$ and can be expressed as

y_{k} = H_{k} (C_{k}) + v_{k}

When estimating a set of constant states, the EKF may be written as follows⁴²

\begin{matrix} P_{k}^{-} = P_{k - 1}^{+} + Q_{k - 1} \\ S_{k} = H_{k} ({\hat{C}}_{k - 1}) P_{k}^{-}^{T} H_{k} ({\hat{C}}_{k - 1}) + R_{k} \\ K_{k} = P_{k}^{-} H_{k} ({\hat{C}}_{k - 1}) S_{k}^{- 1} \\ {\hat{C}}_{k} = {\hat{C}}_{k - 1} + K_{k} (y_{k} - H_{k} ({\hat{C}}_{k - 1})) \\ P_{k}^{+} = (I - K_{k} H_{k} ({\hat{C}}_{k - 1})) P_{k}^{-} \end{matrix}

where $P_{k}^{-}$ and $P_{k}^{+}$ are the covariance matrices before and after the state update ( ${\hat{C}}_{k - 1}$ ), I is an identity matrix of suitable size, and $K_{k}$ is the filter gain. In general $R = E (v v^{T})$ .

Approximation of view parameters by means of an EKF

There is evidence that an EKF can be used to obtain a good, even if not optimal in the least squares sense, approximation of the view parameters ( $C$ ).⁴² These parameters are used to define a mapping between the manipulator’s joint coordinates and the camera-space appearance of a number of visual features. Considering that the view parameters of a stationary camera are constant, the evolution of these parameters at a time instant k is defined by

{\dot{C}}_{k} = 0

This expression can be used to define the state transition matrix as an identity matrix which operates on the value of the view parameters at any instant where the k th measurements is available⁴²

F_{k} = I

For every visual feature that is detected, its camera-space location will be represented using its flattened coordinate $x_{f_{k}} = [x_{f_{k}}, y_{f_{k}}]^{T}$ while the physical location of the feature obtained using the nominal kinematic model of the manipulator will be represented as $x_{k} = [X_{k}, Y_{k}, Z_{k}]^{T}$ . Then the observation vector $y_{k}$ and the matrix $H_{k} (C_{k - 1})$ are

y_{k} = {[x_{f_{k}}, y_{f_{k}}]}^{T}

H_{k} (C_{k - 1}) = {[\begin{matrix} x_{k} & 0 \\ 1 & 0 \\ 0 & x_{k} \\ 0 & 1 \end{matrix}]}^{T} [\begin{matrix} \frac{\partial b_{1}}{\partial C_{1}} & \dots & \frac{\partial b_{1}}{\partial C_{6}} \\ ⋮ & ⋮ \\ \frac{\partial b_{8}}{\partial C_{1}} & \dots & \frac{\partial b_{8}}{\partial C_{6}} \end{matrix}]

In equation (10), the parameters $b_{1}, \dots, b_{8}$ are evaluated using the current estimate of the view parameters ( $C_{k - 1}$ ). According to the pin-hole camera model, the b_i parameters can be defined as

\begin{array}{l} b_{1} = C_{1}^{2} + C_{2}^{2} - C_{3}^{2} - C_{4}^{2} \\ b_{2} = 2 (C_{2} C_{3} + C_{1} C_{4}) \\ b_{3} = 2 (C_{2} C_{4} - C_{1} C_{3}) \\ b_{4} = C_{5} \\ b_{5} = 2 (C_{2} C_{3} - C_{1} C_{4}) \\ b_{6} = C_{1}^{2} - C_{2}^{2} + C_{3}^{2} - C_{4}^{2} \\ b_{7} = 2 (C_{3} C_{4} - C_{1} C_{2}) \\ b_{8} = C_{6} \end{array}

The EKF update equation will provide a new estimation of $C_{k}$ , updated and refined as more samples become available. For this particular formulation, the matrix $R_{k}$ was chosen as

R_{k} = [\begin{matrix} \frac{1}{W_{k}} & 0 \\ 0 & \frac{1}{W_{k}} \end{matrix}]

where W_k is the relative weight given to k-th observation. The initial covariance matrix ( $P_{0}$ ) can be obtained by considering a set number of points which define a starting estimation of the view parameters so that

P_{0} = {[H^{T} W H]}^{- 1}

The equations involved in the formulation of the EKF based on visual marker observations have been previously presented.^43,44 Additionally, the CSM parameters can be derived using a previously described iterative method.⁴⁵ The work presented here offers a comparison between parameters obtained with an EKF and the traditional CSM algorithm.⁴⁶

Experimental platform

The following section describes the parallel robot utilized for the experiments.

An estimation of the view parameters and positioning tasks was implemented using a 3-DOF Delta-type parallel manipulator¹⁰ and a two-camera vision system (see Figure 2). The parallel manipulator used is a Delta Parallix LKF-2040 composed of three kinematic chains that maintain the orientation of the mobile platform, thus offering three translational DOFs for the positioning of the end-effector. The radius of the fixed and mobile platform are of 150 mm and 50 mm, respectively. The robot’s actuators are located on the fixed platform and each one drives a 200-mm long input link connected to a 400-mm long parallelogram. More information on this device can be found in the literature.⁴⁷

Figure 2.

Delta Parallix LKF-2040 robot.

A grid of $6 \times 3$ LEDs was added to the robot’s end-effector and served as a set of visual markers for the vision system. This manipulator was controlled using a PIC-SERVO SC board coupled to a PC running Windows 8. The PC was used to obtain the camera-space coordinates of the visual markers, compute the view parameters in a real-time using either the EKF or the traditional CSM algorithm, and to control the manipulator’s position. The vision system consists of two CCD uEye cameras (IDS Imaging Development Systems GmbH, Alemania) model $U i - 1640 - C$ with a resolution of $640 \times 480$ pixels and a 1/3" CCD sensor.

Forward kinematic model

Figure 3 shows a schematic view of a Delta-type parallel robot. $f_{0} : = (O_{0}, X_{0}, Y_{0}, Z_{0})$ is defined as the inertial reference frame of the robot as shown in the figure, the subscript $i = (1, 2, 3)$ refers to each of the three legs of the robot.

Figure 3.

Schematic view of the Parallix LKF-2040 Delta-type robot.

From Figure 3

v - R_{i} - L 1_{i} - r_{i} = L 2_{i}

where $v$ is the vector distance from O₀ to the center of the mobile platform ( $V$ ), $R_{i}$ is vector $O_{0} A_{i}$ , $L 1_{i}$ is vector $A_{i} B_{i}$ , $L 2_{i}$ is vector $B_{i} C_{i}$ , and $r_{i}$ is vector $C_{i} V$ . By developing equation (14) in components of f₀ and rearranging the terms we obtain

{(X_{v} - X_{i})}^{2} + {(Y_{v} - Y_{i})}^{2} + {(Z_{v} - Z_{i})}^{2} = L 2^{2}

The forward kinematic model (FKM) for the Delta Parallel robot is known and has been previously reported by Maya et al.⁴⁷ and Coronado et al.,³⁶ as well as the references therein. The FKM of this mechanism is based on a set of three closed chains which can be solved as the intersection of three spheres with radius L2 (15) with radius L2 and centered in (X_i , Y_i . Z_i ). Two points exist that solve this system of equations. From the two possible solutions, one is discarded for practical purposes as the end-effector is naturally attracted to the position closer to the ground due to the effect of gravity.

Positioning test

A series of positioning tests were carried out to evaluate the performance of the nonlinear view parameters ( $C$ ) obtained from both the EKF and traditional CSM approaches with flattened data.

Before conducting the experiment, a set of 410 end-effector positions and their corresponding camera-based coordinates were obtained. This data set is known as the preplanned data and is designed to include a large volume of workspace that is also within the cameras’ field of view. This set of points was used to generate a first estimate of the vision parameters. During the experiment, a laser beam was projected onto a target positioned in the workspace. The laser beam was used to create a non-permanent luminous marker, visible on both cameras, on the desired position for the tip of the end-effector. Using camera-space coordinates as input, the 3-D position of the target can be estimated using the camera parameters obtained previously. These values may not be exact but offer a first estimate of the $C$ parameters. Any additional point reached can be used as a new sample for the refinement of the parameters under either estimation technique. For each target, the manipulator’s end-effector was moved autonomously, and the positioning error was measured.

This positioning task was repeated three times for 10 randomly chosen target positions for a total of 30 data points. Additionally, as the end-effector approached the target, the view parameters were refined. Data used for refining the CSM parameters are obtained by calculating the end-effector 3-D position via joint angle measurements and the FKM, and its corresponding position measured on the cameras’ images. This can be performed in real time, along the approaching trajectory, as the necessary measurements and calculations are simple to perform. Ten intermediate sampling steps were used to refine the CSM parameters. Several strategies for assigning weights to a sample have been proposed.⁴⁶

Generally, the weight (W_k) of a sample increases as it nears the target. This yields locally valid CSM parameters that reduce the positioning error in the target point’s vicinity.²²

An empirical weighting scheme where the first intermediate point was assigned with weight of 10, the second one a weight of 20, the third one a weight of 30, and so on.

Results

The positioning errors obtained using parameters estimated based on EKF, and the traditional CSM method are shown in Figure 4 for the 30 experiments. This information is summarized in Table 1 as the average error and its standard deviation. There does not appear to be a bias error between both approaches. It should be noted that the parameters are locally valid due to the refinement performed by increasing the corresponding weight as the tool nears the target. In other words, errors are associated with the target directly and should not be a function of its position inside the volume. This is demonstrated in Figure 4, where the standard deviation of the errors is comparable with the error itself and is sub-millimeter. To further examine this, a t-test was performed using the data from the 30 positioning tasks and no significant difference was found between the accuracy of the robot when using vision parameters obtained using the traditional CSM identification and the proposed EKF $(p = 0.29)$ . However, as is also shown in Table 1, the EKF implementation of the parameter estimation is faster as compared to the traditional globally nonlinear approach. This allows for the possibility to obtain results while the model is updated based on actual measures making real-time control easily achievable. When no refinement was applied, that is, using the CSM parameters obtained only from the preplanned trajectory, the position error reaches 1.79 ± 0.34 mm for the traditional CSM. For comparison, the mean positioning error of the parallel robot was found to be 3.17 ± 1.14 mm without vision-based control, that is, when only the robot’s kinematic model and a proportional–integral–derivative (PID) position control is used.⁴⁸

Figure 4.

Measured error for positioning tasks using parameters estimated with the Kalman filter and traditional CSM methods. CSM: camera-space manipulation.

Table 1.

Positioning error and average computation time during the CSM parameter identification using the traditional algorithm and the EKF with data corrected via flattening.

	Traditional	EKF
Positioning error
Pos. error (mm)	1.26 ± 0.34	1.11 ± 0.35
Computation time
Image processing (ms)	200	200
Robot kinematics (ms)	40	40
CSM parameter update (ms)	85	2

CSM: camera-space manipulation; EKF: extended Kalman filter.

Discussion and conclusions

The results in Table 1, based on the number of positioning experiments mentioned in the previous section, show that using an EKF to estimate a camera’s view parameters can help to accurately and rapidly place a manipulator’s end-effector at a target position. This article aims to show the decrease in computation time when using the EKF for estimating CSM parameters with respect to the traditional method while maintaining a comparable performance. Note that the positioning error is directly linked to the quality of the CSM parameter estimation and not the estimation method. An in-depth study of the positioning error while using CSM is not presented here. However, interested readers are referred to the work of Rendon-Mancha et al.,²² who have studied the attained errors with and without the use of local samples, or that of Bonilla et al.,²⁷ where the use of CSM for path-tracking was reported.

It should be noted that the EKF does not guarantee an optimal solution to the nonlinear problem⁴²; however, the results given here show that it is able to converge and find a set of suitable CSM view parameters. The EKF guarantees convergence which is something that the traditional CSM estimation approach does not.³⁷ Furthermore, due to is sequential nature, the EKF has a lower computational cost evidenced by its smaller computation time. The computational cost advantage of the EKF may aid in achieving higher end-effector speeds.

Figure 5 compares the performance of other vision-based control methods. The position of each circle indicates the reported average error with respect cycle time. The radius of each circle is proportional to the error’s standard deviation. Note that the differences in hardware utilized for each set-up are very difficult to consider and can greatly influence the cycle time. Also, there is great variability as to what is shown in Figure 5. The authors referenced in the figure do not define what is included in the cycle time, that is, it is not explicitly stated if the reported cycle time includes: image processing, robot movement, communication processes, and so on. The large error reported by Trashlosheros¹⁶ may be due to reporting a tracking task, rather than a positioning one. The EKF implementation presented here appears to be an improvement with respect to the decision-making time.

Figure 5.

Comparison of different implementations of vision-based control in serial and parallel manipulators, including the traditional CSM method and the proposed EKF. CSM: camera-space manipulation; EKF: extended Kalman filter.

The evolution of the work presented in this article is focused on the implementation of CSM as a vision-based algorithm for the control of parallel robots to account for moving targets. This could have industrial applications where the CSM model would account for robot calibration errors. Additionally, an in-depth study of the positioning accuracy as a function the robot’s position inside the workspace and the preplan collection strategy would be beneficial to understanding the limitations of the CSM method for industrial applications.

Footnotes

Acknowledgement

The authors would like to thank the C. Christopher Cooney and Brian Jackman endowed Professorship.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Partial funding was provided by CONACyT under the Catedras 2016-972 project.

ORCID iD

Alejandro González

Mauro Maya

Appendix 1

References

Merlet

. Parallel Robots. Berlin: Springer, 2006.

Pandilov

Dukovski

. Several open problems in parallel robotics. Acta Technica Corvininesis – Bull Eng 2011; 4(3): 77–84.

Zhou

Chen

Liu

. A new forward kinematic algorithm for a general Stewart platform. Mech Mach Theory 2015; 87: 177–190.

Liu

Wang

Zhang

. Real-time solution of the forward kinematics for a parallel haptic device using a numerical approach based on neural networks. J Mech Sci Technol 2015; 29(6): 2487–2499.

Guglielmetti

Model-based control of fast parallel robots a global approach in operational space. PhD Thesis, EPFL, Lausanne, 1994.

Liu

Lewis

Lebret

. The singularities and dynamics of a Stewart platform manipulator. J Intell Robot Syst 1993; 8(3): 287–308.

Ibaraki

Okuda

Kakino

. Compensation of gravity-induced errors on a hexapod-type parallel kinematic machine tool. JSME Int J C Mech Syst Mach Elem Manuf 2004; 47(1): 160–167.

Šika

Hamrle

Valášek

. Calibrability as additional design criterion of parallel kinematic machines. Mech Mach Theory 2012; 50: 48–63.

Traslosheros

Sebastián

Torrijos

. An inexpensive method for kinematic calibration of a parallel robot by using one hand-held camera as main sensor. Sensors 2013; 13(8): 9941–9965.

10.

Clavel

. Conception d’un robot parallèle rapide à 4 degrés de liberté. PhD Thesis, EPFL, Lausanne, 1991.

11.

Merlet

JP.

Parallel robots: commande, http://www-sop.inria.fr/members/Jean-Pierre.Merlet/Commnde/biblio_espace.tml (accessed 20 December 2014).

12.

Özgür

Dahmouche

Andreff

. A vision-based generic dynamic model of PKMs and its experimental validation on the Quattro parallel robot. In: 2014 IEEE/ASME international conference on advanced intelligent mechatronics, Besacon, 8–11 July 2014, pp. 937–942. IEEE.

13.

Tsai

. An efficient and accurate camera calibration technique for 3D machine vision. In: Proceedings of IEEE conference on computer vision and pattern recognition, Miami Beach, FL, 22–26 June 1986, pp. 364–374.

14.

Tsai

. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J Robot Autom 1987; 3(4): 323–344.

15.

Tsai

Lenz

. A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans Robot Autom 1989; 5(3): 345–358.

16.

Traslosheros

Sebastian

Angel

. Visual servoing of a parallel robot system. In: 2007 IEEE international symposium on intelligent signal processing, Alacala de Henares, Spain, 3–5 October, pp. 1–6. IEEE.

17.

Bellakehal

Andreff

Mezouar

. Force/position control of parallel robots using exteroceptive pose measurements. Meccanica 2011; 46(1): 195–205.

18.

Siradjuddin

Behera

McGinnity

. Image-based visual servoing of a 7-DOF robot manipulator using an adaptive distributed fuzzy PD controller. IEEE/ASME Trans Mechatron 2014; 19(2): 512–523.

19.

Liu

Liang

. Autonomous path planning and experiment study of free-floating space robot for target capturing. J Intell Robot Syst 2008; 51(3): 303–331.

20.

Liang

. Autonomous target capturing of free-floating space robot: theory and experiments. Robotica 2009; 27(3): 425–445.

21.

Seelinger

Yoder

. Automatic visual guidance of a forklift engaging a pallet. Robot Auton Syst 2006; 54(12): 1026–1038.

22.

Rendon-Mancha

Cardenas

Garcia

. Robot positioning using camera-space manipulation with a linear camera model. IEEE Trans Robot 2010; 26(4): 726–733.

23.

González-Galván

Pazos-Flores

Skaar

. Camera pan/tilt to eliminate the workspace-size/pixel-resolution tradeoff with camera–space manipulation. Robot Comput Integr Manuf 2002; 18(2): 95–104.

24.

Cárdenas

Goodwine

Skaar

. Vision-based control of a mobile base and on-board arm. Int J Robot Res 2003; 22(9): 677–698.

25.

Baumgartner

. In-situ exploration of mars using rover systems. In: Proceedings of the AIAA space 2000 conference, Long Beach, CA, U.S.A., 19–21 September 2000, pp. AIAA–2000–5062. American Institute of Aeronautics & Astronautics.

26.

Dallej

Andreff

Martinet

. Contribution to generic modeling and vision-based control of a broad class of fully parallel robots. Robotica 2018; 36(12): 1874–1896.

27.

Bonilla

Mendoza

Gonzalez-Galván

. Path-tracking maneuvers with industrial robot manipulators using uncalibrated vision and impedance control. IEEE Trans Syst Man Cyber C (Appl Rev) 2012; 42(6): 1716–1729.

28.

Baumgartner

Schenker

. Autonomous image-plane robot control for martian lander operations. In: Proceedings of IEEE international conference on robotics and automation Vol 1, Minneapolis, MN, USA, 22–28 April 1996, pp. 726–731. IEEE.

29.

Seelinger

Yoder

Baumgartner

. High-precision visual control of mobile manipulators. IEEE Trans Robot Autom 2002; 18(6): 957–965.

30.

Karras

Loizou

Kyriakopoulos

. A visual-servoing scheme for semi-autonomous operation of an underwater robotic vehicle using an IMU and a laser vision system. In: 2010 IEEE international conference on robotics and automation, Anchorage, AK, USA, 3–7 May 2010, pp. 5262–5267. IEEE.

31.

Seelinger

Gonzalez-Galvan

Robinson

. Towards a robotic plasma spraying operation using vision. IEEE Robot Autom Mag 1998; 5(4): 33–38.

32.

Fan

. Industrial applications of camera space manipulation with structured light. PhD Thesis, University of Notre Dame, USA, 2003.

33.

Chen

Peng

Wang

. Pallet recognition and localization method for vision guided forklift. In: 2012 8th International conference on wireless communications, networking and mobile computing, Shangai, China, 21–23 September 2012, pp. 1–4.

34.

Zhou

Hao

Guo

. Visual servo control system of 2-DOF parallel robot. In: Wu

(ed) Software engineering and knowledge engineering: theory and practice. Berlin: Springer Berlin Heidelberg, pp. 425–433.

35.

Özgür

Dahmouche

Andreff

. High speed parallel kinematic manipulator state estimation from legs observation. In: 2013 IEEE/RSJ international conference on intelligent robots and systems, Tokyo, Japan, 3–7 November 2013, pp. 424–429. IEEE.

36.

Coronado

Maya

Cardenas

. Vision-based control of a delta parallel robot via linear camera-space manipulation. J Intell Robot Syst 2017; 85(1): 93–106.

37.

Gonzalez-Galvan

Skaar

Korde

. Application of a precision-enhancing measure in 3D rigid-body positioning using camera-space manipulation. Int J Rob Res 1997; 16(2): 240–257.

38.

Chen

. Kalman filter for robot vision: a survey. IEEE Trans Ind Electron 2012; 59(11): 4409–4420.

39.

Faugeras

Luong

. The geometry of multiple images. Cambridge: MIT Press, 2004.

40.

Szeliski

. Computer vision: algorithms and applications. Berlin: Springer-Verlag, 2011.

41.

Dementhon

Davis

. Model-based object pose in 25 lines of code. Int J Comput Vis 1995; 15(1): 123–141.

42.

Simon

. Optimal state estimation: Kalman H infinity, and non linear approaches. Hoboken: John Wiley & Sons, 2006.

43.

Crassidis

Junkins

. Optimal estimation of dynamic systems. Boca Raton, FL: CRC Press Taylor & Francis Group, Chapman and Hall/CRC, 2011.

44.

Jazwinski

. Stochastic processes and filtering theory. Mineola: Dover, 2007.

45.

Gonzalez-Galvan

Skaar

. Efficient camera-space manipulation using moments. In Proceedings of IEEE international conference on robotics and automation, Vol. 4, Minneapolis, Mn, USA, 22–28 April 1996, pp. 3407–3412. IEEE.

46.

Liu

Shi

Skaar

. Robust industrial robot real-time positioning system using VW-camera-space manipulation method. Ind Robot 2014; 41(1): 70–81.

47.

Maya

Castillo

Lomelí

. Workspace and payload-capacity of a new reconfigurable delta parallel robot. Int J Advan Rob Syst 2013; 10(1): 56.

48.

López

. Control basado en visión de un robot paralelo tipo delta por medio del método Manipulación en Espacio de Cámara. Master’s Thesis, Universidad Autónoma de San Luis Potosí, Mexico, 2017.

Estimation of camera-space manipulation parameters by means of an extended Kalman filter: Applications to parallel robots

Abstract

Keywords

Introduction

Methods

The pin-hole camera model approximated by means of an orthographic camera model

Target estimation

Flattening

Extended Kalman filter

Approximation of view parameters by means of an EKF

Experimental platform

Forward kinematic model

Positioning test

Results

Discussion and conclusions

Footnotes

Acknowledgement

Declaration of conflicting interests

Funding

ORCID iD

Appendix 1

References