Robust Visual Control of Parallel Robots under Uncertain Camera Orientation

Abstract

This work presents a stability analysis and experimental assessment of a visual control algorithm applied to a redundant planar parallel robot under uncertainty in relation to camera orientation. The key feature of the analysis is a strict Lyapunov function that allows the conclusion of asymptotic stability without invoking the Barbashin-Krassovsky-LaSalle invariance theorem. The controller does not rely on velocity measurements and has a structure similar to a classic Proportional Derivative control algorithm. Experiments in a laboratory prototype show that uncertainty in camera orientation does not significantly degrade closed-loop performance.

1. Introduction

The high stiffness and high speed of parallel robots make them attractive for several applications, including manufacturing lines [1], the design of climbing robots [2] and haptic devices [3]. Moreover, the application of Visual Servoing techniques for parallel robot control is a viable alternative to schemes based on solving forward kinematics, which may not have an analytical solution. Visual Servoing is an active research area [4], [5], [6], [7], [8], [9] and has many potential applications, including micromanipulation [10], medical robotics [11], [12], human-robot cooperation [13] and active fluid flow control [14].

There exist several reported works on Visual Servoing techniques applied to spatial parallel robots – including them [15], [16], [17], [18], [19], [20], [21], [22], [23] – which develop control laws where the control signal is a desired velocity command feeding a low level velocity controller. However, this approach does not take into account the dynamics of the low-level velocity controller. In contrast, [24], [25], [26], [27] consider the full robot dynamics for designing a controller producing torque signals; specifically, [24] uses fuzzy sliding mode algorithms and the controllers presented in [26], [27] resort to computed torque techniques.

Planar parallel robots are an important class of parallel robots [28] and have also been the focus of several publications. In many cases, the control algorithms applied to this kind of robot use the forward kinematics for estimating the end-effector position [29], [30], [31], [32], [33], [34], [35], [36], [37]. Alternatively, some approaches use a vision system in the feedback loop for the same purpose [38], [39], [40]. A number of Visual Servoing methodologies developed in the case of open-chain and parallel planar robot manipulators assume knowledge of the matrix that describes the rotation between the camera and the robot planes [41], [42], [43], [44]; however, this matrix may not be exactly known. Interestingly, [43] – which uses a linear model for the closed-loop system and the first Lyapunov method – shows that the control algorithm proposed there produces a closed-loop system which is robust in relation to camera rotation uncertainties.

For open-chain robots, several methods tackled the uncertainty problem by using adaptive techniques [45], [46], [47]. Despite being effective in this case, there is no publication applying these approaches to planar parallel robots; moreover, adaptive controllers are complex, difficult to tune and may generate transient responses with excessive overshoot.

An alternative to adaptive controllers explored here is the use of an algorithm proposed in [48], similar to the classic Proportional Derivative (PD) control law; that algorithm shares the main features of the PD controller (i.e., tuning is relatively straightforward, the derivative action shapes the transient response and the proportional action sets the closed-loop bandwidth). Moreover, the computational burden associated to this controller is small and it is well-suited for set point tracking tasks. [40] describes an approach closer to that presented in this work; there exist, however, several differences. First, the method described in [40] resorts to active joint position measurements for generating damping. Second, and as mentioned in the preceding paragraphs, it relies on an exact knowledge of the rotation matrix. Finally, the stability analysis uses the Barbashin-Krassovsky-LaSalle theorem for concluding asymptotic stability. The present work takes the algorithm described in [48] under the assumption that only an estimate of the rotation matrix is known. Moreover, instead of relying on the Barbashin-Krassovsky-LaSalle invariance theorem, the stability analysis presented here employs a strict Lyapunov function, while the second Lyapunov method allows the conclusion of the asymptotic stability of the closed-loop system. The paper is organized as follows. Section 2 presents the model of the robotic system constituted by the parallel robot model and the vision system. Section 3 describes the controller and presents the stability analysis. Section 4 shows the experiments conducted on a laboratory prototype. Finally, the concluding remarks at the end review the findings. In the sequel, the notation ‖ • ‖ stands for the Euclidean norm for matrices and vectors; the symbols λ_m(A) and λ_M(A) along with A ∈ℝ ^n×n denote, respectively, the smallest and highest eigenvalues of A.

2. Modelling Framework

2.1 Modelling of a redundant planar parallel robot

Figure 1 depicts the schematic of a redundant planar parallel robot including a reference frame. Three servo motors drive the joints A₁, A₂ and A₃; the joints P₁, P₂ and P₃ are passive – i.e., they are not actuated. The end-effector corresponds to the common joint whose position is denoted by X. The entries of the vectors q _a = [θ₁ θ₂ θ₃] ^T and q _p = [α₁ α₂ α₃] ^T stand, respectively, for the angles of the actuated and passive joints.

Figure 1.

Planar parallel robot.

According to [36], the Lagrange-D'Alembert formulation yields a simple scheme for computing the dynamics of redundant parallel robots; this approach uses an equivalent open-chain mechanism shown in Figure 2. Thus, the dynamics of the parallel robot are equivalent to the dynamics of the open-chain system and a set of loop constraints.

Figure 2.

Equivalent open-chain mechanism

The well-known Euler-Lagrange formalism [49] allows the modelling of the open-chain system when the parallel robot moves in the horizontal plane:

$\begin{array}{c}{M}_i\left[\begin{array}{c}{\ddot{\theta }}_i\\ {\ddot{\alpha }}_i\end{array}\right]+C_i\left[\begin{array}{c}{\dot{\theta }}_i\\ {\dot{\alpha }}_i\end{array}\right]=\left[\begin{array}{c}{\tau }_{ai}\\ {\tau }_{pi}\end{array}\right];\text{i=1},2,3\\ {\mathrm{M}}_{\mathrm{i}}=\left[\begin{array}{cc}{\mathrm{M}}_{{\mathrm{i}}_{11}}& {\mathrm{M}}_{{\mathrm{i}}_{12}}\\ {\mathrm{M}}_{{\mathrm{i}}_{21}}& {\mathrm{M}}_{{\mathrm{i}}_{22}}\end{array}\right]=\left[\begin{array}{cc}{\lambda }_{\mathrm{i}}\text{+2}{\beta }_{\mathrm{i}}\cos {\alpha }_{\mathrm{i}}& {\gamma }_{\mathrm{i}}+{\beta }_{\mathrm{i}}\cos {\alpha }_{\mathrm{i}}\\ {\gamma }_{\mathrm{i}}+{\beta }_{\mathrm{i}}\cos {\alpha }_{\mathrm{i}}& {\gamma }_{\mathrm{i}}\end{array}\right]\\ C_i=\left[\begin{array}{cc}{C}_{i_{11}}& C_{i_{12}}\\ C_{i_{21}}& 0\end{array}\right]=\left[\begin{array}{cc}-{\beta }_i{\dot{\alpha }}_i\sin {\alpha }_i& -{\beta }_i\left({\dot{\alpha }}_i+{\dot{\theta }}_i\right)\sin {\alpha }_i\\ {\beta }_i{\dot{\alpha }}_i\sin {\alpha }_i& 0\end{array}\right]\\ {\lambda }_i=I_{i1}+I_{i2}+m_{i1}{r}_{i1}^2+m_{i2}\left(L^2+r_{i2}^2\right)\\ {\beta }_i=m_{i2}{r}_{i2}L\\ {\gamma }_i=I_{i2}+m_{i2}{r}_{i2}^2\end{array}$

(1)

The model above assumes that all the links have the same length – i.e., L = a_i= b_i. The parameters I_ij, m_ij and r_ij correspond, respectively, to the inertia, mass and centre of mass of each link. As can be seen in Figure 2, τ_a = [τ_a1 τ_a2 τ _a3] ^T stands for the active joint torques and the vector τ_p = [τ_p1 τ_p2 τ_p3] ^T corresponds to the torques in the passive joints. The neglect of the friction in the passive joints permits the setting τ_p = 0.

The following is a compact representation of model (1):

$\begin{array}{l}\bar{M}\ddot{q}+\bar{C}\dot{q}=\tau \\ \bar{M}=\left[\begin{array}{cccccc}{M}_{1_{11}}& 0& 0& M_{1_{12}}& 0& 0\\ 0& M_{2_{11}}& 0& 0& M_{2_{12}}& 0\\ 0& 0& M_{3_{11}}& 0& 0& M_{3_{12}}\\ M_{1_{12}}& 0& 0& M_{1_{22}}& 0& 0\\ 0& M_{2_{12}}& 0& 0& M_{2_{22}}& 0\\ 0& 0& M_{3_{12}}& 0& 0& M_{3_{22}}\end{array}\right]\\ \bar{C}=\left[\begin{array}{cccccc}{C}_{1_{11}}& 0& 0& C_{1_{12}}& 0& 0\\ 0& C_{2_{11}}& 0& 0& C_{2_{12}}& 0\\ 0& 0& C_{3_{11}}& 0& 0& C_{3_{12}}\\ C_{1_{21}}& 0& 0& 0& 0& 0\\ 0& C_{2_{21}}& 0& 0& 0& 0\\ 0& 0& C_{3_{21}}& 0& 0& 0\end{array}\right]\\ \begin{array}{cc}q=\left[\begin{array}{c}{q}_a\\ q_p\end{array}\right],& \tau =\left[\begin{array}{c}{\tau }_a\\ {\tau }_p\end{array}\right]\end{array}\end{array}$

(2)

The terms M̄ and C̄ define the inertia and the Coriolis and centripetal force matrices respectively. Therefore, the open-chain system (2) obeying the holonomic constraints h(q) = h(q_a,q_p) =0 is equivalent to the original parallel robot.

It is worth mentioning that model (2) describes the parallel robot dynamics in terms of the passive and active joint coordinates. However, since the proposed control law uses visual measurements of the robot end-effector position X, it would be more convenient to have a model in terms of these coordinates. To this end, the following proposition shows a key relationship between the vector τ and the active joint torques τ_a. This equality is instrumental in obtaining a dynamic model of the parallel robot in terms of the end-effector coordinates X.

Proposition 1 Assume that τ is the joint torque of the equivalent open-chain system (2) for a given motion, satisfying the loop constraints h(q) = h(q_a,q_p) = 0 and that τ _a is the active joint torque of the redundant closed-chain system required to generate the same motion. Accordingly, both torques are related as follows:

$S^T{\tau }_a=W^T\tau$

(3)

The terms W = (∂q/∂X) and S = (∂q _a / ∂ X ) are Jacobian matrices.

Proof See [36].

The following expressions define the Jacobian matrices W and S:

$W=\left[\begin{array}{cc}\frac{\cos \left({\theta }_1+{\alpha }_1\right)}{L\sin {\alpha }_1}& \frac{\sin \left({\theta }_1+{\alpha }_1\right)}{L\sin {\alpha }_1}\\ \frac{\cos \left({\theta }_2+{\alpha }_2\right)}{L\sin {\alpha }_2}& \frac{\sin \left({\theta }_2+{\alpha }_2\right)}{L\sin {\alpha }_2}\\ \frac{\cos \left({\theta }_3+{\alpha }_3\right)}{L\sin {\alpha }_3}& \frac{\sin \left({\theta }_3+{\alpha }_3\right)}{L\sin {\alpha }_3}\\ -\frac{d_{1x}}{L\sin {\alpha }_1}& -\frac{d_{1y}}{L\sin {\alpha }_1}\\ -\frac{d_{2x}}{L\sin {\alpha }_2}& -\frac{d_{2y}}{L\sin {\alpha }_2}\\ -\frac{d_{3x}}{L\sin {\alpha }_3}& -\frac{d_{3y}}{L\sin {\alpha }_3}\end{array}\right]$

(4)

$\begin{array}{l}S=\left[\begin{array}{cc}\frac{\cos \left({\theta }_1+{\alpha }_1\right)}{L\sin {\alpha }_1}& \frac{\sin \left({\theta }_1+{\alpha }_1\right)}{L\sin {\alpha }_1}\\ \frac{\cos \left({\theta }_2+{\alpha }_2\right)}{L\sin {\alpha }_2}& \frac{\sin \left({\theta }_2+{\alpha }_2\right)}{L\sin {\alpha }_2}\\ \frac{\cos \left({\theta }_3+{\alpha }_3\right)}{L\sin {\alpha }_3}& \frac{\sin \left({\theta }_3+{\alpha }_3\right)}{L\sin {\alpha }_3}\end{array}\right].\\ d_{ix}=\cos {\theta }_i+\cos \left({\theta }_i+{\alpha }_i\right),\text{i=1},2,3.\hspace{0.17em}\\ d_{iy}=\sin {\theta }_i+\sin \left({\theta }_i+{\alpha }_i\right),\text{i=1},2,3.\end{array}$

(5)

Note that the Jacobian matrix W relates the velocities of the end-effector and the robot joints:

$\dot{q}=W\dot{X}$

(6)

The vector $\dot{q}={\left[\begin{array}{cccccc}{\dot{\theta }}_1& {\dot{\theta }}_2& {\dot{\theta }}_3& {\dot{\alpha }}_1& {\dot{\alpha }}_2& {\dot{\alpha }}_3\end{array}\right]}^T$ corresponds to the joint velocity vector and Ẋ = [ẋ ẏ] ^T represents the end-effector velocity. Taking the time derivative of the Jacobian relationship, (6) yields:

$\ddot{q}=\dot{W}\dot{X}+W\ddot{X}$

(7)

Substituting the input torque (2) into (3) leads to:

$W^T\left(\bar{M}\ddot{q}+\bar{C}\dot{q}\right)=S^T{\tau }_a$

(8)

Substituting (6) and (7) into (8) produces the following dynamic model:

$\begin{array}{l}M\ddot{X}+C\dot{X}=S^T{\tau }_a\\ M=W^T\bar{M}W\\ C=W^T\bar{M}\dot{W}+W^T\bar{C}W\end{array}$

(9)

The matrices M and C satisfy the following structural properties so long as matrix W is full rank [36]:

Property 1: Matrix M is symmetric and positive definite.

Property 2: Matrix Ṁ – 2C is skew-symmetric.

Property 3: There exists a positive constant k_C such that:

$\Vert C\Vert \le k_C\Vert \dot{X}\Vert$

(10)

2.2 Modelling of the vision system

Consider the redundant planar parallel robot described previously. Figure 3 shows the vision system configuration.

Figure 3.

Camera and robot workspace

Here, a camera provides an image of the robot workspace, including the robot end-effector. The camera optical centre is located at a distance z with respect to the robot coordinate frame x_R – y_R and the camera optical axis intersects this frame at the point O = [O_x O_y] ^T , located anywhere in the robot workspace (see Figure 4).

Figure 4.

Robot and camera coordinate frames

The variable β denotes the camera orientation measured clockwise with respect to the negative side of axis x _R ; this angle fulfils the restriction –π /2 ≤ β ≤ π/2. The camera sensor has a coordinate frame – called the image coordinate frame – with axes x_i – y_i; this frame is parallel to the robot coordinate frame. The visual feature of interest is the robot end-effector position given in pixels and defined within the image coordinate frame as X_i = [x_i y_i] ^T . Image-processing algorithms allow the estimation of X_i; this estimate feeds the control algorithm without further processing. This latter feature is common to all image-based Visual Servoing algorithms and permits the avoidance of camera calibration procedures.

Using the perspective camera model [4], [43] allows the description of the end-effector position X in the robot coordinate frame in terms of the image coordinate frame:

$X_i=\eta hR(\beta )[X-O]+C_i$

(11)

The vector C_i = [C _{x i} C _{y i} ] ^T is the image centre, η is a negative number called the scale factor whose units are pixels / m and h is the magnification factor defined as:

$h=\frac{f}{f-z}<0$

(12)

The parameter f is the camera focal length and R(β) ∈ SO(2) is the rotation matrix generated by the clockwise rotation of the camera about its optical axis by β rad:

$R(\beta )=\left[\begin{array}{cc}\cos \beta & -\sin \beta \\ \sin \beta & \cos \beta \end{array}\right]$

(13)

The time derivative of (11) gives the end-effector linear velocity in terms of the image coordinate frame as:

${\dot{X}}_i=\eta hR(\beta )\dot{X}$

(14)

The next relationship describes the desired end-effector position X^* _i = [x^*_i y_i*] ^T in the image coordinate frame in terms of the end-effector position X*, given in the robot coordinated frame by the following equation:

$X_i^{\ast }=\eta hR(\beta )[X^{\ast }-O]+C_i$

(15)

The visual distance between the end-effector image position and the desired end-effector image position (see Figure 5) defines the image position error X˜_i:

Figure 5.

Image position error

${\stackrel{\sim }{X}}_i=X_i^{\ast }-X_i$

(16)

Assuming a constant desired position and taking the time derivative of the image position error yields:

$\frac{d}{dt}{\stackrel{\sim }{X}}_i=-{\dot{X}}_i=-\eta hR(\beta )\dot{X}$

(17)

3. Visual Control using only position measurements

3.1 Problem statement

The following statement defines the control problem: given just those measurements provided by a vision system and the robot joints, design a control law generating active joint torques τ _a such that the robot end-effector reaches the desired image position under uncertainty of the rotation matrix R(β). In other, words, the control law must ensure that $\underset{t\to \infty }{\lim }{\stackrel{\sim }{X}}_i(t)=0$ when only an estimate of R(β) is known.

The next expressions permit the computing of the active torques τ _a using an auxiliary control signal u:

${\tau }_a={\left(S^T\right)}^{+}u$

(18)

The matrix ( S ^T )⁺ corresponds to the Moore-Penrose pseudo-inverse of S ^T :

${\left(S^T\right)}^{+}=S{\left(S^{T}S\right)}^{-1}$

(19)

The matrix ( S ^T )⁺ is well-defined if S is full rank. Matrix S looses rank if the parallel robot reaches a singular configuration; in the sequel, matrix S is assumed to have full rank.

Replacing τ_a given by (18) into (9) yields:

$M\ddot{X}+C\dot{X}=u$

(20)

Therefore, the control laws discussed below will be defined in terms of the control signal u while bearing in mind that the computation of the active torques τ_a will require the use of the relationship (18).

3.2 Proposed control law

First, consider the following visual servoing PD control law [48]:

$u=R{(\beta )}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\dot{\xi }\right]$

(21)

$\dot{\xi }=-2\alpha \xi +{\alpha }^2{X}_i$

(22)

Where k₁, k₂ and α are positive constants. The fact that the camera orientation β is unknown precludes the use of control law (21) and (22). Assume that only an estimate β of β is known, with –π /2 < β < π /2, this permits the rewriting of (21) and (22) as:

$u=R{({\beta }_e)}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\dot{\xi }\right]$

(23)

$\dot{\xi }=-2\alpha \xi +{\alpha }^2{X}_i$

(24)

Hence, (18) allows the computation of the control active torque τ _a as follows:

${\tau }_a={\left(S^T\right)}^{+}R{({\beta }_e)}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\dot{\xi }\right]$

(25)

$\dot{\xi }=-2\alpha \xi +{\alpha }^2{X}_i$

(26)

Substituting (23) into (20) leads to the following closed-loop dynamics:

$M\ddot{X}+C\dot{X}=R{({\beta }_e)}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\dot{\xi }\right]$

(27)

Adding and subtracting the right-hand-side of (21) to the right-hand-side of (27) yields:

$M\ddot{X}+C\dot{X}=R{(\beta )}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\dot{\xi }\right]+H$

(28)

The term H has the following expression:

$H={\left[R({\beta }_e)-R(\beta )\right]}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\stackrel{.}{\xi }\right]$

(29)

The first term in the right hand side of (28) corresponds to the nominal closed-loop system with known camera rotation. The vector H acts like a disturbance generated by the mismatch R(β_e) – R(β) in the camera orientation. Since β and its estimate β_e are constant, so is R(β _e ) – R(β); thus, it is natural to assume a bound on this last term, as follows:

$\Vert R({\beta }_e)-R(\beta )\Vert \le \rho$

(30)

3.3 Stability analysis

Defining the state vector $\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)$ allows for an alternative description of the closed-loop system (26) and (28), namely:

$\begin{array}{l}{\dot{\stackrel{\sim }{X}}}_i=-\eta hR(\beta )\dot{X}\\ \ddot{X}=M^{-1}\left\{R{(\beta )}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\dot{\xi }\right]-C\dot{X}+H\right\}\\ \dot{\xi }=-2\alpha \dot{\xi }+\eta h{\alpha }^{2}R(\beta )\stackrel{.}{X}\end{array}$

(31)

An equilibrium point for the closed-loop system (31) is $\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)=\left(0,0,0\right)$ and at this coordinate the term H in (29) vanishes. The following proposition states the main result of this section.

Proposition 2 Consider the controllers (25) and (26) in a closed-loop with the dynamic model of the parallel robot (9). Then, if α is sufficiently large and the bound ρ is small enough, the equilibrium point $\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)=\left(0,0,0\right)$ is asymptotically stable. The next expression describes a strict Lyapunov function to prove this result:

$\begin{array}{c}V\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)=\frac{\eta h}{2}{\dot{X}}^{T}M\dot{X}+\frac{{\alpha }^2{k}_1}{2}{\stackrel{\sim }{X}}_i^T{\stackrel{\sim }{X}}_i\\ +\frac{k_2}{2}{\dot{\xi }}^T\dot{\xi }-\frac{k_1}{\alpha k_2}{\stackrel{\sim }{X}}_i^{T}R(\beta )M\dot{X}\\ -\frac{1}{\alpha }{\dot{\xi }}^{T}R(\beta )M\dot{\xi }\end{array}$

(32)

Proof The following expression is an alternative writing of the Lyapunov function candidate (32):

$\begin{array}{c}V\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)=y^{T}My+z^{T}Mz\\ +\frac{{\alpha }^2{k}_1}{2}{\stackrel{\sim }{X}}_i^T{\stackrel{\sim }{X}}_i\\ -\frac{k_1^2}{{\alpha }^2\eta h{k}_2^2}{\stackrel{\sim }{X}}_i^{T}R(\beta )MR{\left(\beta \right)}^T{\stackrel{\sim }{X}}_i\\ +\frac{k_2}{2}{\dot{\xi }}^T\dot{\xi }-\frac{1}{{\alpha }^2\eta h}{\dot{\xi }}^{T}R(\beta )MR{\left(\beta \right)}^T\dot{\xi }\\ y=\frac{\sqrt{\eta h}}{2}\dot{X}-\frac{k_1}{\alpha k_2\sqrt{\eta h}}R{\left(\beta \right)}^T{\stackrel{\sim }{X}}_i\\ z=\frac{\sqrt{\eta h}}{2}\dot{X}-\frac{1}{\alpha \sqrt{\eta h}}R{\left(\beta \right)}^T\dot{\xi }\end{array}$

The above function is a radially unbounded positive definite, provided that the following terms:

$\frac{{\alpha }^2{k}_1}{2}{\stackrel{\sim }{X}}_i^T{\stackrel{\sim }{X}}_i-\frac{k_1^2}{{\alpha }^2\eta h{k}_2^2}{\stackrel{\sim }{X}}_i^{T}R(\beta )MR{(\beta )}^T{\stackrel{\sim }{X}}_i$

(33)

$\frac{k_2}{2}{\dot{\xi }}^T\dot{\xi }-\frac{1}{{\alpha }^2\eta h}{\dot{\xi }}^{T}R(\beta )MR{(\beta )}^T\dot{\xi }$

(34) are also radially unbounded positive definite functions in X˜_i and $\dot{\xi }$ , respectively. The terms (33) and (34) vanish at X˜_i = 0 and $\dot{\xi }=0$ respectively. Thus, it only remains to show that they are radially unbounded and positive for X˜_i ≠ 0 and $\dot{\xi }\ne 0$ .

Let us consider the following bounds:

$\begin{array}{l}\frac{K_1^2}{{\alpha }^2\eta h{k}_2^2}{\tilde{\mathbf{X}}}_{\mathbf{i}}^{T}R(\beta )\mathbf{\text{MR}}{(\beta )}^T{\tilde{\mathbf{X}}}_{\mathbf{i}}\le \frac{k_1^2{\lambda }_M\left\{\mathbf{M}\right\}}{{\alpha }^2\eta h{k}_2^2}{\Vert {\tilde{\mathbf{X}}}_{\mathbf{i}}\Vert }^2\hfill \\ \frac{1}{{\alpha }^2\eta h}{\dot{\xi }}^{T}R(\beta )\mathbf{\text{MR}}{(\beta )}^T\dot{\xi }\le \frac{{\lambda }_M\left\{M\right\}}{{\alpha }^2\eta h}{\Vert \dot{\xi }\Vert }^2\hfill \end{array}$

Hence, the terms (33) and (34) satisfy:

$\begin{array}{l}\frac{{\alpha }^2{k}_1}{2}{\stackrel{\sim }{X}}_i^T{\stackrel{\sim }{X}}_i-\frac{k_1^2}{{\alpha }^2\eta h{k}_2^2}{\stackrel{\sim }{X}}_i^{T}R(\beta )MR{\left(\beta \right)}^T{\stackrel{\sim }{X}}_i\ge {\kappa }_1{\Vert {\stackrel{\sim }{X}}_i\Vert }^2\\ {\kappa }_1=\frac{{\alpha }^2{k}_1}{2}-\frac{k_1^2{\lambda }_M\left\{M\right\}}{{\alpha }^2\eta h{k}_2^2}\\ \frac{k_2}{2}{\dot{\xi }}^T\dot{\xi }-\frac{1}{{\alpha }^2\eta h}{\dot{\xi }}^{T}R(\beta )MR{\left(\beta \right)}^T\dot{\xi }\ge {\kappa }_2{\Vert \dot{\xi }\Vert }^2\\ {\kappa }_2=\frac{k_2}{2}-\frac{{\lambda }_M\left\{M\right\}}{{\alpha }^2\eta h}\end{array}$

From the above, it is clear that for a value of α which is large enough, the terms (33) and (34) are positive definite functions.

Therefore, the Lyapunov function candidate (32) is radially unbounded and positive definite.

The following expression corresponds to the time derivative of (32) along the trajectories of the closed-loop system (31):

$\begin{array}{l}\dot{V}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)=\eta h{\dot{X}}^T\left\{R{(\beta )}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\stackrel{.}{\xi }\right]-C\dot{X}+H\right\}\\ +\frac{1}{2}\eta h{\dot{X}}^{T}M\dot{X}-\eta h{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i^{T}R(\beta )\dot{X}\\ +k_2{\dot{\xi }}^T\left(-2\alpha \dot{\xi }+\eta h{\alpha }^{2}R(\beta )\dot{X}\right)\\ -\frac{k_1}{\alpha k_2}{\stackrel{\sim }{X}}_i^{T}R(\beta )\left\{R{(\beta )}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\dot{\xi }\right]-C\dot{X}+H\right\}\\ -\frac{k_1}{\alpha k_2}{\stackrel{\sim }{X}}_i^{T}R(\beta )\dot{M}\dot{X}+\frac{\eta h{k}_1}{\alpha k_2}{\stackrel{.}{X}}^{T}M\dot{X}\\ -\frac{1}{\alpha }{\dot{\xi }}^{T}R(\beta )\left\{R{(\beta )}^T\left[{\alpha }^2{k}_1{\stackrel{\sim }{X}}_i-{\alpha }^2{k}_2\dot{\xi }\right]-C\dot{X}+H\right\}\\ -\frac{1}{\alpha }{\dot{\xi }}^{T}R(\beta )\dot{M}\dot{X}-\frac{1}{\alpha }{\left(-2\alpha \dot{\xi }+\eta h{\alpha }^{2}R(\beta )\dot{X}\right)}^{T}R(\beta )M\dot{X}\end{array}$

Using Property 2 and, after some simplifications, we yield:

$\begin{array}{c}\dot{V}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)=-\eta h\left(\alpha -\frac{k_1}{\alpha k_2}\right){\dot{X}}^{T}M\dot{X}\\ -\frac{\alpha k_1^2}{k_2}{\stackrel{\sim }{X}}_i^T{\stackrel{\sim }{X}}_i\\ -\alpha k_2^T{\dot{\xi }}^T\dot{\xi }\\ +2{\dot{\xi }}^{T}R(\beta )M\dot{X}\\ -\frac{k_1}{\alpha k_2}{\stackrel{\sim }{X}}_i^{T}R(\beta )C^T\dot{X}\\ -\frac{1}{\alpha }{\dot{\xi }}^{T}R(\beta )C^T\dot{X}\\ -\frac{k_1}{\alpha k_2}{\stackrel{\sim }{X}}_i^{T}R(\beta )H\\ +\eta h{\dot{X}}^{T}H\\ -\frac{1}{\alpha }{\dot{\xi }}^{T}R(\beta )H\end{array}$

(35)

Property 3 and (30) allows the bounding of the following terms:

$\begin{array}{c}-\frac{k_1}{\alpha k_2}{\stackrel{\sim }{X}}_i^{T}R(\beta )C^T\dot{X}-\frac{1}{\alpha }{\dot{\xi }}^{T}R(\beta )C^T\dot{X}\le \frac{k_C}{\alpha }\left[\frac{k_1}{k_2}\Vert {\stackrel{\sim }{X}}_i\Vert +\Vert \dot{\xi }\Vert \right]{\Vert \dot{X}\Vert }^2\\ H\le \rho {\alpha }^2{k}_1\Vert {\stackrel{\sim }{X}}_i\Vert +\rho {\alpha }^2{k}_2\Vert \dot{\xi }\Vert \end{array}$

The above bounds permit, in turn, the bounding of the time derivative (35) as follows:

$\begin{array}{l}\dot{V}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\le -\eta h{\lambda }_m\left\{M\right\}\left[\alpha -\frac{k_1}{\alpha k_2}\right]{\Vert \dot{X}\Vert }^2\\ -\frac{\alpha k_1^2}{k_2}{\Vert {\stackrel{\sim }{X}}_i\Vert }^2-\alpha k_2{\Vert \dot{\xi }\Vert }^2+2{\lambda }_M\left\{M\right\}\Vert \dot{\xi }\Vert \Vert \dot{X}\Vert \\ +\frac{k_C}{\alpha }\left[\frac{k_1}{k_2}\Vert {\stackrel{\sim }{X}}_i\Vert +\Vert \dot{\xi }\Vert \right]{\Vert \dot{X}\Vert }^2+\rho \frac{\alpha k_1^2}{k_2}{\Vert {\stackrel{\sim }{X}}_i\Vert }^2\\ +\rho \alpha k_2{\Vert \dot{\xi }\Vert }^2+\rho {\alpha }^2\eta h{k}_1\Vert {\stackrel{\sim }{X}}_i\Vert \Vert \dot{X}\Vert \\ +\rho {\alpha }^2\eta h{k}_2\Vert \dot{\xi }\Vert \Vert \dot{X}\Vert +2\rho \alpha k_1\Vert {\stackrel{\sim }{X}}_i\Vert \Vert \dot{\xi }\Vert \end{array}$

(36)

In order to obtain a compact writing of (36), let us define the vector $\psi ={[\Vert {\tilde{\mathrm{X}}}_i\Vert \hspace{0.17em}\hspace{0.17em}\Vert \stackrel{.}{\mathrm{X}}\Vert \hspace{0.17em}\hspace{0.17em}\Vert \stackrel{.}{\xi }\Vert ]}^T$ , the scalar $\mathrm{\Gamma }=\frac{k_C}{\alpha }\left[\frac{k_1}{k_2}\Vert {\stackrel{\sim }{X}}_i\Vert +\Vert \dot{\xi }\Vert \right]{\Vert \dot{X}\Vert }^2\hspace{0.17em}$ and the matrices:

$\begin{array}{l}G=\left[\begin{array}{ccc}\frac{k_1^2}{k_2}& 0& 0\\ 0& \eta h{\lambda }_m\left\{M\right\}\left[1-\frac{k_1}{{\alpha }^2{k}_2}\right]& \frac{{\lambda }_M\left\{M\right\}}{\alpha }\\ 0& \frac{{\lambda }_M\left\{M\right\}}{\alpha }& k_2\end{array}\right]\\ \\ D=\left[\begin{array}{ccc}\frac{k_1^2}{k_2}& \frac{\alpha \eta h{k}_1}{2}& k_1\\ \frac{\alpha \eta h{k}_1}{2}& 0& \frac{\alpha \eta h{k}_2}{2}\\ k_1& \frac{\alpha \eta h{k}_2}{2}& k_2\end{array}\right]\end{array}$

The matrix G is positive definite if α is large enough. Thus, rewriting (36) leads to:

$\dot{V}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\le -\alpha {\psi }^{T}G\psi +\mathrm{\Gamma }+\rho \alpha {\psi }^{T}D\psi$

(37)

At this point of the analysis, it is convenient to establish the next inequalities obtained from (32) and the definition of vector ψ:

$\begin{array}{l}\Vert {\stackrel{\sim }{X}}_i\Vert \le \frac{1}{\alpha }\sqrt{\frac{2}{k_1}}{V}^{1/2}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\\ \Vert \dot{\xi }\Vert \le \sqrt{\frac{2}{k_2}}{V}^{1/2}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\\ \Vert \dot{X}\Vert \le \Vert \psi \Vert \end{array}$

Hence, it is not difficult to show – using the above inequalities – that the term Γ has the following upper bound:

$\begin{array}{l}\mathrm{\Gamma }\le \frac{c_1}{\alpha }{V}^{1/2}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right){\Vert \psi \Vert }^2\\ c_1=k_C\left[\sqrt{\frac{2k_1}{\alpha k_2}}+\sqrt{\frac{2}{k_2}}\right]\end{array}$

Finally, using the above bound leads to the following inequality:

$\dot{V}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\le -b(\alpha ,\rho ,t){\Vert \psi \Vert }^2$

(38)

$b(\alpha ,\rho ,t)=\alpha {\lambda }_m\left\{G\right\}-\frac{c_1}{\alpha }{V}^{1/2}\left(t\right)-\rho \alpha {\lambda }_M\left\{D\right\}$

(39)

The term V(t) in (39) is a simplified writing of $V\left({\stackrel{\sim }{X}}_i(t),\dot{X}(t),\dot{\xi }(t)\right)$ . It is worth noting that c₁ and the entries of G do not increase if α increases, but αλ_M {D} could increase if α increases.

When ρ = 0 for a given initial state $\left({\stackrel{\sim }{X}}_i(0),\dot{X}(0),\dot{\xi }(0)\right)$ , assume a value α* of α high enough such that:

$b({\alpha }^{*},0,0)={\alpha }^{*}{\lambda }_m\left\{G\right\}=\frac{c_1}{{\alpha }^{*}}V{(0)}^{1/2}=b^{*}>0.$

(40)

Then, there exists a non-zero value ρ* small enough fulfilling:

$\begin{array}{l}b({\alpha }^{\ast },{\rho }^{\ast },0)={\alpha }^{\ast }{\lambda }_m\left\{G\right\}-\frac{c_1}{{\alpha }^{\ast }}V{(0)}^{½}\\ -{\rho }^{\ast }{\alpha }^{\ast }{\lambda }_M\left\{D\right\}>0\end{array}$

(41)

With b(α*, 0,0) >b(α*, ρ*, 0). Note also that the relationship $\Vert \psi \Vert =\Vert \left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\Vert$ yields:

$\dot{V}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\le -b(\alpha ,\rho ,t){\Vert \left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\Vert }^2$

(42)

Therefore, choosing the initial conditions for the Lyapunov function $V(\widetilde{{\mathrm{X}}_i}(0),\dot{\mathrm{X}}(0),\dot{\xi }(0))$ fulfilling the inequality (41) produces a decreasing Lyapunov function (32) – i.e., V(t) ≤ V(0); thus, b(α*, ρ*, 0) < b(α,ρ,t) ∀t > 0, so that:

$\dot{V}\left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\le -b({\alpha }^{\ast },{\rho }^{\ast },0){\Vert \left({\stackrel{\sim }{X}}_i,\dot{X},\dot{\xi }\right)\Vert }^2.$

Consequently, applying the second Lyapunov stability method [50] permits the conclusion that the equilibrium point $(\widetilde{{\mathrm{X}}_i},\dot{\mathrm{X}},\dot{\xi })=(0,0,0)$ of the closed-loop system (31) is asymptotically stable.

4. Experimental Results

The 3-RRR redundant parallel manipulator shown in Figure 6 serves as a test bench for evaluating the proposed visual controller. The link lengths of this prototype are L=15cm. Brushed servomotors from a Litton PolyScientific – model C34L80W40 – drive the active joints through timing belts with a 3.6:1 ratio. Pulse width modulation digital amplifiers from Copley Controls – model Junus 90 working in current mode – drive the motors. Absolute magnetic encoders from US Digital – model MA3 – with a resolution of 10 bits, supply the measurements of the robot active and passive joint angles θ_i and α _i that are used for computing ( S ^T )⁺. The experimental setup is an update of the architecture presented in [51]. A computer called the Vision Computer and endowed with a 2.5 – GHz Intel Core 2 Quad processor performs image acquisition and processing using a Dalsa Camera – model CA-1D-128A – which is connected to the vision computer through a National Instruments 1422 interface card. The Visual C++ language, the image processing library ICE and the DIAS environment [52] allow the coding of the image processing algorithms. A second computer – called the control computer and endowed with a 2.4 – GHz Intel Core 2 processor – executes the control algorithm and performs the data logging. This computer receives data from the vision computer through an RS-232 port at 115 Kbaud. Data acquisition is carried out through a data acquisition card from Servotogo. This card reads the voltage signals provided by the magnetic encoders and supplies control voltages to the power amplifiers. The Matlab/Simulink R2007a software under the Wincon 5.2 real-time environment generates the real-time code for executing the control algorithms. A counter in a dedicated custom-made timing card sets a sampling period of T_ie= 1 ms, which corresponds to the closed-loop system master clock; this sampling period also sets the sampling time for reading the magnetic encoders. The image sampling period is T_im =3 ms. During this time interval, the vision computer executes data acquisition and processing; this interval also includes the time required to send the centroid of the robot end-effector to the control computer through the RS-232 link.

Figure 6.

Laboratory prototype used in the experiments

In order to evaluate the performance of the proposed control laws (25) and (26) under uncertainty in relation to camera orientation, this section presents two sets of experiments. The first set considers β = 0 rad and β_e = 0 rad – i.e., the robot and camera coordinate frames are parallel (see Figure 7). In the second set of experiments, the real and estimated camera orientation are different; their respective values are β = π/6 rad and β_e = 0 rad (see Figure 8). The controller gains used for both sets of experiments are k₁ = 7.26 × 10³, k₂ = 1.46×10⁻⁴ and α = 150. The reference X^* _i = [x^*_i y*_i] ^T is set as follows. Signal x_i* is a square wave of 16 pixels of amplitude:

Figure 7.

Image of the robot workspace without camera rotation

Figure 8.

Image of the robot workspace with camera rotation

with a frequency of 0.1 Hz plus a constant signal of 80 pixels. Reference y*_i is also a square wave of 16 pixels of amplitude with a frequency of 0.1 Hz plus a constant signal of 75 pixels. The following linear filter:

$G(s)=\frac{20}{s+20}$

smooths out the step references for avoiding excessive control signal oscillation.

In the case of the first set of experiments, Figure 9 shows the time evolution of the end-effector coordinates and Figure 10 depicts the corresponding image position errors; Figure 11 portrays the active joint torque signals.

Figure 9.

Closed-loop system response for β = 0

Figure 10.

Image position errors for β = 0

Figure 11.

Active joint torques for β = 0

The above results show that the closed-loop response has almost no oscillations; the position errors remain around 1 pixel and the active torque signals do not exhibit a high level of high-frequency components.

For the second set of experiments, the time evolution of the end-effector coordinates depicted in Figure 12 has essentially the same behaviour as that observed in the first set of experiments; a similar observation may be made for the position errors shown in Figure 13 and the active joint torques portrayed in Figure 14. It is clear from the results of both sets of experiments that the uncertainty in the camera orientation does not have a significant impact on the closed-loop response, with only a slight increment in the position error.

Figure 12.

Closed-loop system response for β = π/6

Figure 13.

Image position errors for β = π/6

Figure 14.

Active joint torques for β = π/6

5. Conclusions

The theoretical and experimental results discussed in previous sections show that the visual control algorithm studied here – which essentially behaves like a classic PD algorithm – is able to withstand uncertainties in camera orientation. This algorithm does not rely on velocity measurements and has low computational requirements. On the theoretical side, a key element in the stability analysis is a strict Lyapunov function that includes the camera rotation matrix; this function allows the establishment of asymptotic stability without relying on the invariance set stability theorems. On the experimental side, the results obtained using a laboratory prototype show that the closed-loop performance remains essentially the same when the camera rotation is unknown for the controller.