Leader–follower formation with reduction of the off-tracking and velocity estimation under visibility constraints

Abstract

This article addresses the time-varying leader–follower formation control problem for nonholonomic mobile robots, under communication and visibility constraints. Although the leader–follower formation control under visibility constraints has been studied, the elimination of the off-tracking effect has not been widely addressed yet. In this work, a new method to eliminate the off-tracking effect, considering the time-invariant formation as a tractor–trailer system, for unknown and circular tractor paths, taking into account the visibility constraints, is proposed. For a time-varying formation with not circular tractor’s path, the proposed method significantly reduces the off-tracking. Only the relative position and the relative orientation, provided by the on board monocular camera, are required. Thus, both the leader robot’s absolute position and the leader robot’s velocities are not needed. Furthermore, to avoid explicit communication among the robots, an extended state observer is implemented to estimate both the translational and the rotational leader’s velocity. In this way, the desired tasks are executed and achieved in a decentralized manner. For a time-varying formation, with constant leader robot’s velocities, the proposed control strategy, based on the kinematic model, guarantees that the formation errors asymptotically converge to the origin. Based on the Lyapunov theory, the stability proof of the formation errors dynamics is shown. Simulation results, considering time-varying leader robot’s velocities, show the efficiency of the proposed scheme.

Introduction

The problem of convoy driving can be seen as a special case of group formation. ^1,2 Military applications of convoy driving are the most obvious, where a given number of autonomous vehicles follow each other while keeping a safe constant distance. The convoy-like vehicle can be seen as an emulation of the multi-steered general n-trailer with the difference that the physical link does not exist between the tractor/leader and the trailer/follower, but an additional error dynamics equation is introduced to virtually represent this physical constraint. ³ A tractor–trailer mobile robot (TTMR) is a mechanical system composed of a known number of trailers pulled by a tractor.

The off-tracking phenomenon is the major problem in TTMR. This term refers to the deviation of the path of each articulated vehicle from the paths of preceding vehicles. ⁴ The reduction or elimination of off-tracking will result in much improved performance in terms of safety during turns, cornering, overtaking other especially small cars, and backward motion. If both the tractor and the trailers can track an identical geometric path, the overall width of the system is only equal to that of the tractor or the trailer. ⁵ Thus, the robot can perform transport tasks in a narrow space which is the most important problem when finding an obstacle-free path. One way to reduce off-tracking is by using the proportional navigation guidance law, more precisely deviated pursuits. The guidance laws used in Fethi and Boumediene ¹ to model and control a robotic convoy are based on the geometry and the kinematics equations between two successive robots. With the deviated pursuits, the linear velocity of the follower robot is designed to keep a constant distance between robots. The rotational velocity is designed such as the follower robot moves pointing toward the leader robot but considering a small parameter in the relative angle to increase the curvature radius of the follower robot’s path. Since a constant value for that small parameter is used, and a formal analysis is missing, the off-tracking effects are not correctly reduced. To eliminate the off-tracking effect, authors’ ⁵ proposed approach is the generation of the desired full-state trajectory from a desired geometric path, in the Cartesian plane along a time-parameterized path. This technique is useful only when the desired path, given in the configuration space, is known. The trajectory generation process will be more complex according to each desired path. One way to eliminate the off-tracking on physical trailers, for circular paths only, is by using the sliding kingpin mechanism, proposed in Deligiannis et al. ⁶ According to this technique, the kingpin hitch of the trailer slides in a perpendicular direction to the longitudinal axle of the tractor when the tractor–trailer turns. The displacement of the kingpin hitched off-axle along the rear axle allows to increase the radius of curvature of the path that the trailer travels. Notice that, for implementing this technique, a mechanic device is required which increases its cost.

By the other hand, in the formation control approaches, it is assumed that each robot in the formation can obtain its accurate global position information by the use of some global positioning sensors. To address this issue, some researchers have focused on the use of alternate sensors on board (e.g. laser, cameras, and sound ranging technologies). Compared to other traditional sensors, the visual cameras (monocular, stereo and omnidirectional cameras, kinect device) can provide richer information at lower cost, making them a very popular option for formation control using only available relative on board sensing. ⁷ However, the practical drawbacks of incorporating additional sensors include increased cost, increased complexity, decreased reliability, and increased processing burden. ⁸ Taking into account the previous disadvantages, many authors have chosen visual servo strategies, using a monocular camera, that rely on analytic techniques to address the lack of depth information. Visual servoing is classified into two groups: position-based visual servoing (PBVS) and image-based visual servoing (IBVS). PBVS control methods ^{9 –12} use three-dimensional scene information that is reconstructed from image information. That is, the camera acts as a “Cartesian sensor,” where pose estimation algorithms use camera data to generate an error signal in Cartesian space. This error signal is then used in a feedback control law. The main advantage of the position based approach is to perfectly control the robot’s trajectory done in the Cartesian space using well-known path tracking techniques. As a bad consequence, certain trajectories defined in the Cartesian space can lead the target out of the field-of-view (FOV) of the camera. ¹³ In IBVS methods, ^{14 –18} the controlled states are image features of the target, this means that the image data are used directly in the control loop. The major drawback of these methods is that they can only regulate the pose of the camera with respect to a reference pose where some reference image was taken.

Related works

Dani etal. ¹⁰ proposed that control law requires only the knowledge of a single known length on the leader. The relative pose and the relative velocity are obtained using a geometric pose estimation technique and a nonlinear velocity estimation strategy, respectively. A Lyapunov analysis indicates asymptotic tracking of the leader vehicle. Xinwu et al. ¹⁹ proposed a time-invariant leader–follower formation tracking control scheme for nonholonomic mobile robots with on board perspective camera, in the image space. Measurements of the position and the translational velocity of the leader robot are not needed. An adaptive observer is used to estimate the linear velocity. The stability proof shows that the system is stable. The relative orientation angle between mobile robots is obtained by using a compass sensor. In Hasan et al., ¹² a time-invariant, state-feedback control law that allows one differential drive robot to follow another, at a constant relative distance, is presented. The proposed control law does not require measurement or estimation of the leader robot’s velocity and has tunable parameters that allows one to prioritize the error bounds of either the relative polar angle or the relative orientation. In Dimitra and Vijay ²⁰ andXiaomei et al., ²¹ to address the cooperative motion coordination of leader–follower formations of nonholonomic mobile robots in known polygonal obstacle environments, the visibility constraints are defined. In Jie et al., ¹³ an adaptive image-based visual servoing control strategy is proposed following the prescribed performance control methodology. Firstly, the leader–follower visual kinematics in the image plane and an error transformation with predefined performance specifications are presented. Moreover, the off-tracking effect is not addressed. In Qun et al., ²² a distributed leader–follower formation for nonholonomic mobile robots only using local interactions among the robots is proposed to solve the trajectory tracking problem. A distributed estimation strategy is presented for each follower robot to estimate the leader’s states, since the dynamic model is used, the leader velocity is also considered as a state. In Fabio et al., ²³ a method for estimating the relative distance between leader and followers by a reduced-order nonlinear observer is introduced. The leader mobile robot’s velocities are not estimated since both leader robot and follower robot’s velocities are considered as the control inputs of the system. Also, the off-tracking is not addressed. In Sida et al., ²⁴ a formation controller based on model predictive control scheme is proposed to assure the desired formation and the position consistency of followers. Afterward, a dynamic controller based on adaptive terminal sliding mode control scheme is developed for the leader to observe and then compensate the external disturbance as soon as possible. However, any vision strategy is addressed. González-Sierra and Aranda-Bricaire ⁴ proposed the emulation of the so-called kingpin mechanism to reduce the off-tracking effects exhibited by the standard and generalized n-trailer systems. In turn, both systems are emulated by a group of differential-drive mobile robots using the leader–follower scheme. It is assumed that the absolute robots’ position and the leader robot’s velocity are known. Also visibility constraints are not addressed.

The main advantage of this work is that the absolute posture of the leader robot, its velocities, and the knowledge of its path are not needed. Only a monocular camera is used to estimate, on board the follower robot, both the relative position and the relative orientation. Thus, the use of another sensor either IMU or compass is not necessary. In addition to this, an extended state observer is implemented to estimate both the translational leader robot’s velocity and the rotational leader robot’s velocity. Therefore, the proposed time-varying leader–follower formation scheme is decentralized, computationally inexpensive and simple to be implemented on board. Also, the main difference of this work compared to the related works that address the vision-based leader–follower formation control is that also the off-tracking effect is significantly reduced for a time-varying formation. Furthermore, communication constraints are considered since a velocity estimator of the leader robot is implemented.

This article presents the following contributions. First, a new method to eliminate the off-tracking effect for a time-invariant formation, considering a circular leader robot’s path, is proposed. With this method, the off-tracking is significantly reduced either for a time-varying formation or for a time-varying leader robot’s velocities. To guarantee that the time-varying formation control problem can be solved, the minimum curvature radius of the leader robot is defined. Also, this method considers both the visibility and the communication constraints. Second, a control strategy is proposed to solve the time-varying formation control problem with reduction of the off-tracking effect, using only both the relative measurements and the leader’s velocity estimation. Based on Lyapunov theory, the proposed control law guarantees that the formation errors asymptotically converge to the origin, for the time-varying formation with a circular leader robot’s path. To our knowledge, the reduction of the off-tracking for a time-varying leader–follower formation of nonholonomic mobile robots under communication and visibility constraints, using relative position/orientation measurements only, had not been studied.

The article is organized as follows: In the second section, the vision-based algorithm is described. The third section presents the formation kinematics, the visibility constraints, the minimum curvature radius condition, and the strategy to reduce the off-tracking. In the fourth section, the observer to estimate the leader robot’s velocities and the controller design are described. Simulation results are shown in the fifth section. Finally, some conclusions are mentioned in the sixth section.

Vision-based relative position/orientation reconstruction

In this section, the vision-based relative posture reconstruction algorithm is briefly described. For simplicity in the notation, consider one robot only, with an ideal perspective camera, without distortion, with reference frame $\left\{F_c\right\}=[X_c,Y_c,Z_c{]}^T$ , and one black and white rectangular pattern. It is assumed that the width and height of the pattern ( $W,H$ ) are known. The rectangular pattern’s vertexes are the four feature points of the pattern, which are expressed as $M_k=[X_k,Y_k,Z_k{]}^T$ , where $k=1,\mathrm{...},4$ , with respect to the camera’s frame. Based on the perspective projection model of the pinhole camera, the 3D points M_k are projected onto 2D points m_k through the optical center. Thus, all feature points in the image plane coordinates are expressed as $m_k=[u_k,v_k{]}^T$ . All sides of the rectangular pattern are projected into the image plane. From m_k , with $k=1,\mathrm{...},4$ , the corresponding two vertical lines projections, in the image plane, are obtained (see Figure 1). The length of each vertical line projection, in pixels, is given as $\Delta h_l$ , $\Delta h_r$ , for the left and right line, respectively. As was mentioned, the length of both vertical lines, in the configuration space, is H. Omitting the Y_c axis, consider the $P_{jA}$ and $P_{jB}$ points as the points which define the width of the pattern, left side and right side, respectively, (see Figure 2).

Figure 1.

Projection of the 3D feature points onto the image plane.

Figure 2.

Relative distances and relative orientation in the camera’s frame.

Similar to Hasan et al., ¹² from the Thales’ theorem, the distance of each vertical line that composes the pattern, along the camera’s optical axis (Z_c ), is given by $d_{z\zeta }=Hf{\kappa }_z/\Delta h_{\zeta }$ , with $\zeta =l,r$ , for the left and the right line, respectively. ${\kappa }_z$ is a scaling factor which is the number of pixels per unit distance in image coordinates [pixels/m], and f is the focal length [m]. The distance of each vertical line projection ( $d_{zl}$ , $d_{zr}$ ) corresponds to points, $P_{jA}$ and $P_{jB}$ , respectively.

From the Thales’ theorem as well, the lateral offset of points $P_{jA}$ and $P_{jB}$ , in the configuration space, along the camera’s X_c axis, is given by $H_{\zeta }=\Delta w_{\zeta }{d}_{z\zeta }/(f{\kappa }_x)$ , with $\zeta =l,r$ , for the left and right point, respectively. $\Delta w_l$ , $\Delta w_r$ , are the distances, in pixels, along the horizontal axis (u_c ) in the image plane, of each vertical line, respectively. ${\kappa }_x$ is the corresponding scaling factor [pixels/m]. Thus, the relative position of the pattern’s centroid, with respect to the camera’s frame, is given by

$d_z^c=(d_{zl}+d_{zr})/2,d_x^c=(H_l+H_r)/2$

1

The relative orientation between mobile robots is obtained from the knowledge of W, $d_{zl}$ , and $d_{zr}$ , that is $\gamma =ASIN($ $\Delta Z/W)$ , with $\Delta Z=d_{zr}-d_{zl}$ . Therefore, another sensor either IMU or compass sensor is not required to estimate the relative orientation. In this work, only a perspective camera is used.

Similar to Gans et al., ⁹ the presented method is also distinguished due to the knowledge of two geometric lengths is much less restrictive than the complete geometric knowledge of all lengths. Furthermore, initialization is not required, and errors due to large motions are not propagated because of measurements are computed from the present frame only.

Preliminaries and strategy to reduce the off-tracking

In this section, the leader–follower formation kinematics and the visibility constraints are described. Also, the measurement of the off-tracking, the minimum curvature radius condition, and the proposed strategy to reduce the off-tracking effect are presented.

Kinematics of the leader–follower formation

Let us consider a set of n unicycle mobile robots moving in a plane, $r_1\mathrm{,...},r_n$ , where r_i represents the ith nonholonomic unicycle mobile robot. The kinematics of the ith nonholonomic mobile robot is described by the following differential equations

${\dot{x}}_i=v_i\text{\hspace{0.17em} cos \hspace{0.17em}}{q}_i,{\dot{y}}_i=v_i\text{\hspace{0.17em} sin \hspace{0.17em}}{q}_i,{\dot{q}}_i={\omega }_i$

2

The coordinates $x_i(t)$ and $y_i(t)$ describe the position of the rear axle midpoint of the ith mobile robot, $P_i=(x_i,y_i)$ , with respect to the global coordinate frame $\left\{G\right\}=[X_G,Y_G,Z_G{]}^T$ , (see Figure 3). The orientation q_i is the angle between the heading of the ith robot and the X_G axis of the fixed coordinate frame $\left\{G\right\}$ . The translational velocity and rotational velocity are given by $v_i(t)$ and ${\omega }_i(t)$ , respectively; these are the control inputs of the ith mobile robot.

$r_i(i=1,\mathrm{...},n-1)$ is the follower robot of its local leader robot $r_j(j=i+1)$ . r_n is the global leader robot that has the knowledge of the desired trajectory or is assigned the task of the environment exploration. It is assumed that the camera’s optical center is located at the midpoint of the local follower (P_i ). The rectangular pattern is mounted on the leader robot such as the pattern’s centroid coincides with the local leader robot’s midpoint (P_j ) (see Figure 3).

The representation of separation-bearing parameters is often used by many formation control approaches. In addition, in many formation control approaches for mobile robots, the relative position between the mobile robots is defined with respect to the leader robot’s frame. As done in Xinwu et al. ¹⁹ and Consolini et al., ²⁵ the position of the leader robot, with respect to the follower robot’s frame $\left\{F_i\right\}=[X_i,Y_i,Z_i{]}^T$ , is chosen to describe the relative position between a pair of leader–follower robots. Then, the formation states that describe a pair of leader–follower robots are given as $x_i=[{\rho }_i,{\psi }_i,{\gamma }_i{]}^T$ , with $i=1,\mathrm{...},n-1$ ; where ${\rho }_i$ is the relative distance between P_i and P_j midpoints, ${\psi }_i$ is the relative bearing angle, ${\gamma }_i$ is the relative orientation between the mobile robots. From Figure 3, one can see that the separation between the mobile robots and the bearing angle, are, respectively given, with respect to the follower robot’s frame, by

${\rho }_i=\sqrt{{({\rho }_{ix})}^2+{({\rho }_{iy})}^2},{\psi }_i=\text{ATAN}2({\rho }_{iy}/{\rho }_{ix})$

3

with ${\rho }_{ix}=d_{iz}^c$ , ${\rho }_{iy}=-d_{ix}^c$ , where ATAN2 is a two-argument computational function that calculates a four-quadrant arc tangent with a range of $[-\pi ,\pi ]$ . $d_{iz}^c,d_{ix}^c,{\gamma }_i$ are measured by the on board camera on mobile robot i using the vision-based algorithm, described in the second section. The formation states, with respect to the global frame, are given by

$\begin{array}{l}{\rho }_i=\sqrt{{(x_j-x_i)}^2+{(y_j-y_i)}^2},\\ {\psi }_i=Q_i-q_i,{\gamma }_i=q_j-q_i,\end{array}$

4

$\text{with \hspace{0.17em}}{Q}_i=ATAN2\left(\frac{y_j-y_i}{x_j-x_i}\right)\mathrm{.}$

5

Therefore, from Figure 3, it is implied that

$x_j-x_i={\rho }_i\text{cos}({\psi }_i+q_i),y_j-y_i={\rho }_i\text{sin}({\psi }_i+q_i)$

6

Figure 3.

Leader–follower formation.

By differentiating (4) with respect to time, taking into account (2), (6), and using the following trigonometric identities: $\text{cos}(a\pm b)=\text{cos}(a)\text{cos}(b)\mp \text{sin}(a)\text{sin}(b)$ , $\text{sin}(a\pm b)=\text{sin}(a)\text{cos}(b)\pm \text{cos}(a)\text{sin}(b)$ , the kinematics of the leader–follower formation is given by

${\dot{\rho }}_i=-v_i\text{\hspace{0.17em} cos}({\psi }_i)+v_j\text{\hspace{0.17em} cos}({\gamma }_i-{\psi }_i)$

7

${\dot{\psi }}_i=\frac{v_i\text{\hspace{0.17em} sin}({\psi }_i)}{{\rho }_i}+\frac{v_j\text{\hspace{0.17em} sin}({\gamma }_i-{\psi }_i)}{{\rho }_i}-{\omega }_i$

8

${\dot{\gamma }}_i={\omega }_j-{\omega }_i$

9

where $i=1,\mathrm{...},n-1$ , for the local follower robot, and $j=i+1$ , for the local leader robot. It is important to note that the leader–follower kinematics (7)–(9) requires only the relative position, the relative bearing angle, and the relative orientation angle, thus both the absolute position and the absolute orientation of all mobile robots are not needed. Furthermore, only the instantaneous local leader robot’s velocities are used, also the leader robot’s path is not required. Additionally, any velocity information is sent from the local leader robot to the local follower robot. To estimate the translational velocity and rotational velocity of the local leader robot, an extended state observer is implemented. ^26,27 Therefore, the proposed vision-based time-varying leader–follower formation scheme is decentralized. Both the measurement of the off-tracking effect and the proposed strategy to reduce it will be addressed next.

Measurement of the off-tracking

For a unicycle mobile robot, the instantaneous curvature radius of its path is given by

$R(t)=v(t)/\omega (t)$

10

Remark 1

Notice that $R(t)\to \infty$ , when $\omega (t)\to 0$ , which implies that the path is a straight line. Off-tracking is defined as the deviation of the path of each vehicle from the path of preceding vehicle. It has been pointed out that when the leader robot (r_j ) is traveling along a circular path with a constant radius R_j , then the follower robot (r_i ) is traveling along another circular path with radius R_i with the same center, where $R_j>R_i$ , ²⁸ considering a constant separation. Complementary, Jing et al. ⁵ proved that if the leader robot tracks a circular path with a constant curvature radius R_j , the motion path of the follower robot will converge to a concentric circle with a radius given by

$R_{iD}=\sqrt{R_j^2-{\rho }_i^2}$

11

Note that $R_{i_D}$ is the minimum curvature radius of the follower robot when any reduction action of the off-tracking is implemented. Thus, similar to Jing et al., ⁵ Deligiannis et al., ⁶ and Bushnell et al., ²⁸ the measure of the off-tracking of a pair leader–follower robots is given by the difference between the curvature radius of each mobile robot, that is

$d_{iT}=|R_j(t)|-|R_i(t)|$

12

From (11), the maximum deviation, when any reduction action is implemented, is given by $D_{iT}=|R_j|-|R_{iD}|$ . Therefore, the off-tracking is eliminated and reduced, respectively, when the following conditions are satisfied

$d_{iT}=0,0<|d_{iT}|<|D_{iT}|$

13

Modeling of the visibility constraints

In Dimitra and Vijay ²⁰ and Xiaomei et al., ²¹ the visibility constraints are introduced. It is assumed that each follower robot is equipped with a fixed on board camera of limited angle of view $2\alpha <\pi$ . Furthermore, it is assumed that the local follower robot can reliably detect objects which lie within a limited region with respect to the forward-looking direction. The limited sensing region is modeled as a triangle of view for the follower, which essentially is an isosceles triangle in obstacle-free environments. Consequently, the follower’s camera has a visual detection range such that the leader robot can only be detected if and only if the leader is within the triangle of view, that is, if the following visibility constraints are satisfied

$|{\psi }_i|\le \alpha \&{\rho }_i\le \frac{L_{\text{max}}\text{\hspace{0.17em} cos}(\alpha )}{|\text{cos}({\psi }_i)|}$

14

where $L_{\text{max}}$ is the length of the equal sides of the triangle of view (see Figure 4).

It is important to note that, in the original work, both the relative orientation ( ${\gamma }_i$ ) and the minimum separation between the mobile robots are not taken into account. The main problem with this is that visibility constraints in (14) may be satisfied even when the relative orientation is ill-defined, for example, when $|{\gamma }_i|=\pi /2$ and ${\psi }_i=0$ , since the pinhole camera model is used (see Figure 4). Thus, the ${\gamma }_i$ state adds an extra DOF to the visibility constraints.

Figure 4.

Visibility constraints.

Therefore, in this work, visibility constraints are modified such as, furthermore, the size of the mobile robots and the maximum relative orientation are taken into account. Notice that the modifications in the visibility constraints are made not only for the proposed vision-based algorithm but also for any vision-based strategy that uses a perspective camera.

Assume that the camera’s FOV is given by $2\alpha <\pi$ , $L_{\text{max}}$ is the length of the equal sides of the triangle of view and W is the width of the leader’s pattern, for all mobile robots. Since an ideal pinhole camera model is used, notice that a singularity of the ${\gamma }_i$ state occurs when ${\gamma }_i=\pi /2$ , with ${\psi }_i=0$ , which implies that all three points ( $P_{jA},P_j,P_{jB}$ ) are line up. Hence, to avoid this singularity, the maximum relative orientation must be chosen as ${\gamma }_{iM}<\pi /2$ .

The minimum separation between a pair of mobile robots takes into account both the minimum separation under visibility constraints and the size of the mobile robots. Thus, the minimum separation between a pair of leader–follower robots is given by

${\rho }_{im}=l_{im}/|\text{cos}({\psi }_i)|,\text{\hspace{0.17em} with \hspace{0.17em}}{l}_{im}=\text{max}\{L_{\text{min}}\text{\hspace{0.17em} cos}(\alpha )\mathrm{,2}{S}_k\}$

15

where $L_{\text{min}}$ is the length of another smaller triangle of view, $S_k>W/2$ is the radius of the security zone of the mobile robots. So, the distance $L_{\text{min}}\text{cos}(\alpha )>W$ is the minimum distance, along the X_i axis, which guarantees that all three points ( $P_{jA},P_j,P_{jB}$ ) are within the limited sensing region, $\forall {\gamma }_i\in [0,{\gamma }_{iM}]$ . The maximum separation between a pair of mobile robots ( ${\rho }_{iM}$ ), is given similar to (14), that is

${\rho }_{iM}=L_{\text{max}}\text{\hspace{0.17em} cos}(\alpha )/|\text{cos}({\psi }_i)|$

16

Therefore, in this work, the leader robot’s pattern can only be detected if and only if the following visibility constraints are satisfied

$|{\psi }_i|\le \alpha \&|{\gamma }_i|\le {\gamma }_{iM}\&{\rho }_{im}\le {\rho }_i\le {\rho }_{iM}$

17

where ${\rho }_{im}$ and ${\rho }_{iM}$ are given by (15) and (16), respectively. Hence, the three states of the system, $x_i=[{\rho }_i,{\psi }_i,{\gamma }_i{]}^T$ , are well defined in the domain

$\begin{array}{l}\mathbb{D}=\{x_i|x_i\in {ℝ}^{+}\times {({\mathbb{S}}^1)}^2\cap {\rho }_{im}\le {\rho }_i\le {\rho }_{iM}\cap \\ |{\psi }_i|\le \alpha \cap |{\gamma }_i|\le {\gamma }_{iM}\}\end{array}$

18

Minimum curvature radius condition

Let us now consider the steady state of a time-invariant pair of leader–follower robots, it implies that, both the leader robot and the follower robot are moving along the same circular path defined by the constant curvature radius R_j , and with a desired constant separation, denoted as ${\rho }_{id}$ . This steady state is depicted in Figure 5. From the law of cosines, the desired relative orientation is given by

${\gamma }_{id}=\text{ACOS}\left(\frac{{\rho }_{id}^2-2R_j^2}{-2R_j^2}\right)$

19

According to the $\text{ACOS}$ function, omitting the case when ${\rho }_{id}=0$ which is impossible in a real application and without considering visibility constraints yet, the variable ${\gamma }_{id}$ is well defined in the domain $\mathbb{D}=\{{\rho }_{id}\in {ℝ}^{+}|0<{\rho }_{id}\le 2R_j\}$ . Then, the range of ${\gamma }_{id}$ is given as $\mathbb{I}=\{{\gamma }_{id}\in {\mathbb{S}}^1|0<{\gamma }_{id}\le \pi \}$ . Since ${\rho }_{id}$ is a given value, both the curvature radius of the local leader robot’s path and the desired relative orientation are restricted by the desired separation between the mobile robots.

Due to the visibility constraints, to guarantee that the formation control problem can be archived, the leader robot’s path is restricted by the minimum curvature radius condition. Thus, to define the minimum curvature radius condition taking into account visibility constraints, from Figure 5, substituting ${\gamma }_{id}$ and R_j by ${\gamma }_{iM}$ and $R_{j\text{min}}$ , respectively, from the law of cosines, ${\rho }_{id}^2=2R_{j\text{min}}^2-2R_{j\text{min}}^2\text{cos}({\gamma }_{iM}),$ and using the trigonometric identity: ${{\displaystyle \text{sin}}}^2(2a)=(1/2)[1-\text{cos}(a)]$ , the minimum curvature radius of the leader robot’s path, is given by

$R_{j\text{min}}=\frac{{\rho }_{id}}{2\text{sin}({\gamma }_{iM}/2)}$

20

Figure 5.

For the steady state, the follower robot moves along the leader robot’s path with a constant radius of curvature.

To solve the formation control problem considering the reduction of the off-tracking effect, the instantaneous curvature radius of the leader robot’s path (R_j ) must satisfy the following condition

$|R_j(t)|\ge R_{j\text{min}},\forall t\ge 0$

21

Remark 2

In this work, to guarantee that the time-varying formation control problem, with reduction of the off-tracking effect, can be solved, the minimum curvature radius of the local leader robot is defined. Visibility constraints only guarantee that the local leader’s pattern can be detected by the local follower’s camera. The minimum curvature radius condition guarantees that the pair of mobile robots can fit on the leader’s circular path. Otherwise, the off-tracking effect cannot be eliminated for a time-invariant formation in the steady state even using absolute positions.

Proposed strategy to reduce the off-tracking

For simplicity of notation, consider the auxiliary variable ${\theta }_{id}={\gamma }_{id}-{\psi }_{id}$ , as shown in Figure 5. In the steady state, the translational velocity of each mobile robot is tangent to the circular path, thus ${\beta }_i+{\psi }_{id}={\beta }_i+{\theta }_{id}=\pi /2$ , which implies that

${\psi }_{id}={\gamma }_{id}/2,\forall {\gamma }_i\in (0,\pi /2]$

22

Remark 3

Notice that there are two problems if equations (22) and (19) are used to compute ${\psi }_{id}$ :

(1) the desired curvature radius of the local leader robot ( $R_{jd}$ ) is required, thus the local leader robot’s path is needed, either parametrized by the time or in the configuration space, (2) otherwise, to obtain ${\gamma }_{id}$ , using the instantaneous curvature radius of the local leader robot ( $R_j(t)$ ) in (19), the desired bearing angle ( ${\psi }_{id}$ ), can be obtained, but when computing the corresponding time derivative ( ${\dot{\psi }}_{id}$ ), the time derivative of the local leader robot’s velocities ( ${\dot{v}}_j,{\dot{\omega }}_j$ ) are required, which cannot be estimated by the observer. Then, to avoid the use of both the knowledge of the desired local leader robot’s path and the time derivatives ( ${\dot{v}}_j,{\dot{\omega }}_j$ ), in this work, it is proposed not to use the desired relative orientation in the steady-state case (19). Instead, the desired relative bearing angle ( ${\psi }_{id}$ ) is chosen as a function of the instantaneous relative orientation between the mobile robots ( ${\gamma }_i$ ).

From (22), the desired bearing angle and its time derivative are chosen as

${\psi }_{id}={\gamma }_i/2,{\dot{\psi }}_{id}={\dot{\gamma }}_i/2=({\omega }_j-{\omega }_i)/2$

23

The formation errors of a pair of mobile robots are given by

$e_{\rho i}={\rho }_i-{\rho }_{id},e_{\psi i}={\psi }_i-{\psi }_{id}$

24

where ${\psi }_{id}$ is the corresponding desired value of ${\psi }_i$ , $e_{\rho i}$ is the separation error, and $e_{\psi i}$ is the bearing angle error. Notice that, since the desired value of the states ${\rho }_{id}$ and ${\psi }_{id}$ is time-varying, the formation control problem is translated into a pure tracking control problem.

The considered input vector of a pair of mobile robots is $u_i=[v_i,{\omega }_i{]}^T$ with output vector $y_i=[{\rho }_i,{\psi }_i{]}^T$ . Furthermore, the nonholonomic nature of the system makes it impossible to control all three states continuously. ¹² The system (7)–(9) has a vector relative degree ${[{\text{r}}_1,{\text{r}}_2]}^T{=[1,1]}^T$ . Since $\text{r}={\text{r}}_1+{\text{r}}_2=2$ , the system has a first-order dynamics associated with the relative orientation, ${\gamma }_i$ . Thus, the proposed control law guarantees that the left state ( ${\gamma }_i$ ) remains bounded. Additionally, if the global leader robot follows a desired trajectory, then the trajectory tracking problem of the time-varying formation is solved as well.

In this work, the following assumptions are considered:

Assumption 1

The desired separation between a pair of mobile robots, ${\rho }_{id}(t)$ , is $C^{\infty }$ and bounded. Thus, its time derivative is also $C^{\infty }$ and bounded; that is $|{\dot{\rho }}_{id}(t)|\le {\dot{\rho }}_{iM}$ , with $i=1,\mathrm{...},n-1$ , where ${\dot{\rho }}_{iM}\in {ℝ}^{+}$ .

Assumption 2

The control inputs, ( $v_k(t),{\omega }_k(t)$ ), are bounded $\forall t\ge 0$ , that is $v_{\text{max}}\ge v_k(t)\ge 0$ , and ${\omega }_{\text{max}}\ge |{\omega }_k(t)|$ , with $k=1,\mathrm{...},n$ , where $v_{\text{max}},{\omega }_{\text{max}}\in {ℝ}^{+}$ .

Assumption 3

The global leader robot is able to move in an obstacle-free environment. Furthermore, its velocities ( $v_n,{\omega }_n$ ) are constant, with v_n strictly positive $\forall t\ge 0$ , and are chosen arbitrarily such as the minimum curvature radius condition (21) is satisfied.

Assumption 4

The initial states ${[{\rho }_i(0),{\psi }_i(0),{\gamma }_i(0)]}^T$ , with $i=1,\mathrm{...},n-1$ , are given such as the visibility constraints (17) are satisfied.

Assumption 5

Each local follower can reliably detect its local leader which lies within a limited region with respect to the forward-looking direction.

Problem statement. Given an instantaneous global leader robot’s velocities ( $v_n,{\omega }_n$ ) which satisfies (21), design proper an observer to estimate the local leader robot’s velocities ( $v_j,{\omega }_j$ , with $j=2,\mathrm{...},n$ ) and a controller ( $u_i=[v_i,{\omega }_i{]}^T$ , with $i=1,\mathrm{...},n-1$ ), by using only feedback from the camera, such that the formation errors (24) asymptotically converge to the origin considering the time-varying formation with constant local leader robot’s velocities. Furthermore, the observer and the control law must guarantee that the visibility constraints (17) are satisfied $\forall t\ge 0$ , which implies that the relative orientation dynamics (9) also remains bounded $\forall t\ge 0$ . Additionally, the off-tracking must be eliminated for a time-invariant formation considering a circular leader robot’s path.

Remark 4

By using only the instantaneous local leader robot velocities, in the transient state for a time-invariant formation, this is, for a time-varying curvature radius of the leader robot, the local follower robot will try to converge to the instantaneous curvature radius of the leader, neglecting the path that was traveled by the leader robot, since the proposed method to eliminate the off-tracking is based on the steady state and the local leader robot’s path is omitted. Notice that, it is assumed that ${\gamma }_i$ is soft, since it is given by the mobile robot’s orientation. Therefore, the measurement of the off-tracking ( $|d_{iT}|$ ) depends on the rate of the curvature radius of the local leader robot, the leader robot’s accelerations, the desired separation between robots, and its time derivative. With this method, for a time-invariant formation, the off-tracking effect is eliminated in the steady state. Otherwise, in the transient state for a time-varying formation, the off-tracking effect is significantly reduced.

Observer and controller design

In this section, the extended state observer (ESO), implemented to estimate the local leader robot’s velocities, is described. Also, the proposed control law to solve the time-varying formation control problem with reduction of the off-tracking is shown.

Extended state observer

The ESO was first proposed in the context of the active disturbance rejection control (ADRC). ^26,27 The main ability of ESO is to estimate both the internal dynamics and external disturbances of the plant. Thus, in this work, an ESO is implemented to estimate the translational velocity and the rotational velocity of the local leader robot. Now, consider the $2(n-1)$ differential equations, given by the SI-SO first order dynamics (7) and (9), rewritten as

${\dot{\rho }}_i=-\text{cos}({\psi }_i)v_i+{\xi }_i(t)$

25

${\dot{\gamma }}_i=-{\omega }_i+{\zeta }_i(t)$

26

with $i=1,\mathrm{...},n-1$ , where ${\xi }_i(t)=v_j(t)\text{cos}({\gamma }_i-{\psi }_i)$ , with $j=i+1$ , is the external unknown disturbance of (25), ${\zeta }_i(t)={\omega }_j(t)$ is the unknown disturbance of (26). Hence, v_i and ${\omega }_i$ is the input of each subsystem, respectively, with ${\rho }_i$ and ${\gamma }_i$ as the output, respectively. Notice that, since the order of each subsystem is 1, for each subsystem the state and the output is the same variable. Due to a state observer is implemented, let ${\widehat{\rho }}_i$ and ${\widehat{\gamma }}_i$ be the estimation of the states ${\rho }_i$ and ${\gamma }_i$ , respectively. Furthermore, let ${\widehat{e}}_{\rho i}$ and ${\widehat{e}}_{\gamma i}$ be the state estimation errors, which are given by

${\widehat{e}}_{\rho i}={\rho }_i-{\widehat{\rho }}_i,{\widehat{e}}_{\gamma i}={\gamma }_i-{\widehat{\gamma }}_i$

27

Any state observer will estimate the state and the external disturbance since the latter is now a state in the extended state model. Such an observer is known as ESO. A particular ESO of (25) and (26) is, respectively, given as

${\dot{\widehat{\rho }}}_i=-\text{cos}({\psi }_i)v_i+{\widehat{\xi }}_i+{\lambda }_{1i}{\widehat{e}}_{\rho i}$

28

${\dot{\widehat{\xi }}}_i={\lambda }_{0i}{\widehat{e}}_{\rho i}$

29

${\dot{\widehat{\gamma }}}_i=-{\omega }_i+{\widehat{\zeta }}_i+{\kappa }_{1i}{\widehat{e}}_{\gamma i},{\dot{\widehat{\zeta }}}_i={\kappa }_{0i}{\widehat{e}}_{\gamma i}$

30

where ${\widehat{\xi }}_i$ and ${\widehat{\zeta }}_i$ , with $i=1,\mathrm{...},n-1$ , are the estimations of the corresponding disturbances, ${\lambda }_{0i},{\lambda }_{1i}>0$ and ${\kappa }_{0i},{\kappa }_{1i}>0$ are the observer gains to be chosen.

From the estimation of the disturbances, both the translational velocity and the rotational velocity of the local leader robot can be obtained. That is

${\widehat{v}}_j={\widehat{\xi }}_i/\text{cos}({\gamma }_i-{\psi }_i),{\widehat{\omega }}_j={\widehat{\zeta }}_i$

31

Then, the velocities estimation errors are given by

$\Delta v_j=v_j-{\widehat{v}}_j,\Delta {\omega }_j={\omega }_j-{\widehat{\omega }}_j\mathrm{.}$

32

By differentiating the state estimation error of ${\rho }_i$ , ${\dot{\widehat{e}}}_{\rho i}=\dot{\rho }-{\dot{\widehat{\rho }}}_i$ , and substituting (7) and (28), yields

${\dot{\widehat{e}}}_{\rho i}={\xi }_i-{\widehat{\xi }}_i-{\lambda }_{1i}{\widehat{e}}_{\rho i}$

33

By differentiating once (33), and using (29), the second-order differential equation is obtained

${\ddot{\widehat{e}}}_{\rho i}+{\lambda }_{1i}{\dot{\widehat{e}}}_{\rho i}+{\lambda }_{0i}{\widehat{e}}_{\rho i}={\dot{\xi }}_i$

34

In a similar manner for the other state estimation error ( ${\widehat{e}}_{\gamma i}$ ). In the domain of the frequency, the relationships between the disturbances and the state estimation errors are given by

${\widehat{e}}_{\rho i}(s)=\frac{s{\xi }_i(s)}{s^2+{\lambda }_{1i}s+{\lambda }_{0i}},{\widehat{e}}_{\gamma i}(s)=\frac{s{\zeta }_i(s)}{s^2+{\kappa }_{1i}s+{\kappa }_{0i}}$

35

where gains ${\lambda }_{0i},{\lambda }_{1i}$ , and ${\kappa }_{0i},{\kappa }_{1i}$ , with $i=1,\mathrm{...},n-1$ , are selected, such that the corresponding characteristic polynomial is Hurwitz. Then, the estimation errors are bounded by the time derivative of the disturbances.

Controller design

Similar to Ricardo et al. ¹¹ and Hasan et al., ²⁹ the proposed control law is based on a control strategy which linearizes the dynamics of the variables ( ${\rho }_i$ , ${\psi }_i$ ). Furthermore, the control law also takes into account their time derivative ( ${\dot{\rho }}_{id}$ , ${\dot{\psi }}_{id}$ ), and the proposed method to reduce the off-tracking, described in section “Proposed strategy to reduce the off-tracking.” Thus, the translational velocity and the rotational velocity of the local follower robot are, respectively, given by

$v_i=[{\widehat{v}}_j\text{\hspace{0.17em} cos}({\gamma }_i-{\psi }_i)+f_{\rho i}(e_{\rho i})-{\dot{\rho }}_{id}]/\text{cos}({\psi }_i)$

36

${\omega }_i=2\left[\frac{v_i\text{\hspace{0.17em} sin}({\psi }_i)}{{\rho }_i}+\frac{{\widehat{v}}_j\text{\hspace{0.17em} sin}({\gamma }_i-{\psi }_i)}{{\rho }_i}+f_{\psi i}(e_{\psi i})\right]-{\widehat{\omega }}_j$

37

$\text{with \hspace{0.17em} \hspace{0.17em}}{f}_{\rho i}(e_{\rho i})=K_{vi}\text{\hspace{0.17em} tanh}(K_{pi}{e}_{\rho i}),$

$f_{\psi i}(e_{\psi i})=K_{\omega i}\text{\hspace{0.17em} tanh}(K_{\psi i}{e}_{\psi i})$

38

where $K_{vi},K_{pi},K_{\omega i},K_{\psi i}$ , $i=1,\mathrm{...},n-1$ are positive gains and the pair ( ${\widehat{v}}_j,{\widehat{\omega }}_j$ ) is given by the ESO. Notice that $f_{\rho i}(e_{\rho i}),f_{\psi i}(e_{\psi i})\in \Omega$ , with $\Omega =\{f:ℝ\to ℝ|f(0)=0$ and $xf(x)>0,\forall x\in ℝ\}$ .

Unlike related works, since the reduction of the off-tracking is addressed, the proposed control law (36)–(37) requires not only the estimation of the local leader’s translational velocity ${\widehat{v}}_j$ but also the estimation of the local leader’s rotational velocity ${\widehat{\omega }}_j$ .

It is important to note that the proposed controller requires both the states of the system, $x_i=[{\rho }_i,{\psi }_i,{\gamma }_i{]}^T$ , and the local leader robot’s velocities, $(v_j,{\omega }_j)$ . In this work, the vision-based algorithm is implemented to compute the states of the system, thus, any other method to obtain the relative distance between the mobile robots, the relative bearing angle, and the relative orientation can be used with the proposed control law. Furthermore, an ESO is implemented to estimate the local leader robot’s velocities. Any other method to obtain leader robot’s velocities can be used. Also, for simplicity, explicit communication may be used, that is, the leader robot may send its velocities information to its follower robot. Notice also that, another control law could be implemented, for example, the one implemented in Dongyu et al., ³⁰ where all the elements of all the agent states reaching formation at the same time.

By differentiating once the formation errors (24) yields ${\dot{e}}_{\rho i}={\dot{\rho }}_i-{\dot{\rho }}_{id}$ , ${\dot{e}}_{\psi i}={\dot{\psi }}_i-{\dot{\psi }}_{id}$ , and taking into account the proposed assignment (23), yields

${\dot{e}}_{\rho i}=-v_i\text{\hspace{0.17em} cos}({\psi }_i)+v_j\text{\hspace{0.17em} cos}({\gamma }_i-{\psi }_i)-{\dot{\rho }}_{id}$

39

${\dot{e}}_{\psi i}=\frac{v_i\text{\hspace{0.17em} sin}({\psi }_i)}{{\rho }_i}+\frac{v_j\text{\hspace{0.17em} sin}({\gamma }_i-{\psi }_i)}{{\rho }_i}-\frac{1}{2}{\omega }_j-\frac{1}{2}{\omega }_i$

40

By substituting the proposed control signals (36)–(37) into (39) and (40), the time derivative of each formation error, in closed-loop, is given by

${\dot{e}}_{\rho i}=-f_{\rho i}(e_{\rho i})+\Delta v_j\text{\hspace{0.17em} cos}({\gamma }_i-{\psi }_i)$

41

${\dot{e}}_{\psi i}=-f_{\psi i}(e_{\psi i})+\Delta v_j[\text{sin}({\gamma }_i-{\psi }_i)]/{\rho }_i-\Delta {\omega }_j/2$

42

The necessary and sufficient conditions to solve the stated problem, using the proposed control law with the ESO, are summarized in the following lemma.

Lemma 1

Consider n unicycle mobile robots (2), which compose $n-1$ pairs of leader-follower formations (7)–(9), take into account Assumptions 1–5 and assume that a time-varying formation is required. Then, the ESO (28)–(30) with the control law (36)–(37), with ${\lambda }_{0i},{\lambda }_{1i}$ , ${\kappa }_{0i},{\kappa }_{1i}$ , $K_{vi},K_{\omega i}$ , $K_{\omega i}$ , $K_{\psi i}$ $>0$ , where $i=1,\mathrm{...},n-1$ , solve the problem statement, that is, ${{\displaystyle \text{lim}}}_{t\to \infty }{e}_{\rho i}\to 0$ , ${{\displaystyle \text{lim}}}_{t\to \infty }{e}_{{\psi }_i}\to 0$ .

Proof

Considering the Lyapunov function candidate given by $V(e_{\rho i},e_{\psi i})=\frac{1}{2}{e}_{\rho i}^2+\frac{1}{2}{e}_{\psi i}^2=V_{\rho }+V_{\psi }$ , with $i=1,\mathrm{...},n-1$ and substituting (41) and (42), the time derivative is given by $\dot{V}(e_{\rho i},e_{\psi i})={\dot{V}}_{\rho }+{\dot{V}}_{\psi }$ , where

${\dot{V}}_{\rho }=-e_{\rho i}{f}_{\rho i}(e_{\rho i})+e_{\rho i}\Delta v_j\text{\hspace{0.17em} cos}({\gamma }_i-{\psi }_i)$

43

$\begin{array}{l}{\dot{V}}_{\psi }=-e_{\psi i}{f}_{\psi i}(e_{\psi i})+e_{\psi i}\Delta v_j[\text{sin}({\gamma }_i-{\psi }_i)]/{\rho }_i\\ -e_{\psi i}\Delta {\omega }_j/2,\end{array}$

44

with $j=i+1$ . By using the ESO, under Assumption 3, constant leader robot’s velocities, and according to (35), the velocities estimations errors ( $\Delta v_j,\Delta {\omega }_j$ ) converge to zero after a transient state. Therefore, both ${\dot{V}}_{\rho }$ , and ${\dot{V}}_{\psi }$ are negative definite. Hence, when a circular leader robot’s path is required, even for a time-varying formation, the formation errors ( $e_{\rho i},e_{\psi i}$ ), with $i=1,\mathrm{...},n-1$ , asymptotically converge to the origin. Therefore, the control law ensures that the states of the system ( ${\rho }_i$ , ${\psi }_i$ ) converge to their desired values ( ${\rho }_{id}$ , ${\psi }_{id}$ ), either time-varying or time-invariant, which are defined by (17). Thus, the proof is concluded. $\blacksquare$

Remark 5

Since a time-varying formation is addressed ( ${\rho }_{im}\le {\rho }_{id}(t)\le {\rho }_{iM}$ ), from (17), the ${\gamma }_i$ state remains bounded ( $|{\gamma }_i(t)|\le {\gamma }_{iM}$ ). Hence, the equilibrium point of the relative orientation ( ${\gamma }_i$ ) cannot be obtained. However, the convergence of the control inputs of each local follower, when $e_{\rho i}\to 0$ and $e_{\psi i}\to 0$ , can be obtained by replacing ${\widehat{v}}_j=v_j$ , ${\psi }_i={\psi }_{id}={\gamma }_i/2$ , according to (23), in the proposed controller (36). Thus, the translational control input ( $v_i(t)$ ) is given by

$v_i(t)\to \frac{v_j\text{\hspace{0.17em} cos}({\gamma }_i/2)-{\dot{\rho }}_{id}(t)}{\text{cos}({\gamma }_i/2)}=v_j-{\dot{\rho }}_{id}(t)\text{\hspace{0.17em} sec}({\gamma }_i/2)$

45

Notice that, when following a straight line ( ${\gamma }_i=0$ ), the local follower’s translational velocity is given by the sum of the local leader’s translational velocity (v_j ), and the time derivative of the desired separation ( ${\dot{\rho }}_{id}$ ), as it is expected.

Now, according to the proposed controller (37), replacing ${\rho }_i={\rho }_{id}$ , ${\widehat{\omega }}_j={\omega }_j$ , and using (45), the rotational control input ( ${\omega }_i(t)$ ) is given by

${\omega }_i(t)\to \frac{4v_j\text{\hspace{0.17em} sin}({\gamma }_i/2)-2{\dot{\rho }}_{id}\text{\hspace{0.17em} tan}({\gamma }_i/2)}{{\rho }_{id}}-{\omega }_j$

46

Every particular case will be discussed next. Consider the time-invariant formation problem in the steady state, that is ${\dot{\rho }}_{id}=0$ . Thus, ${{\displaystyle \text{lim}}}_{t\to \infty }{e}_{\rho i}=0$ , ${{\displaystyle \text{lim}}}_{t\to \infty }{e}_{\psi i}=0$ , $\forall i=1,\mathrm{...},n-1$ . In this case, the relative orientation’s equilibrium ( ${\overline{\gamma }}_i$ ) can be obtained. Since ${\dot{\rho }}_{id}=0$ , from (45), yields $v_i\to v_j$ , independently of the curvature radius of the local leader robot. From (9), due to ${\dot{\gamma }}_i=0$ , ${\omega }_i\to {\omega }_j$ . Furthermore, from (46), the control input ${\omega }_i$ converges to

${\omega }_i=4v_j\text{\hspace{0.17em} sin}({\overline{\gamma }}_i/2)/{\rho }_{id}-{\omega }_j$

47

By substituting (47) into (9), yields

$-4v_j\text{\hspace{0.17em} sin}({\overline{\gamma }}_i/2)/{\rho }_{id}+2{\omega }_j=0$

48

Notice that (48) can be rewritten in a similar manner to (20), that is $R_j={\rho }_{id}/[2\text{sin}({\overline{\gamma }}_i/2)]$ . Hence, the relative orientation’s equilibrium is given by

${\overline{\gamma }}_i=2\text{ASIN}\left(\frac{{\rho }_{id}}{2R_j}\right)$

49

The measurement of the off-tracking of a time-invariant leader–follower formation is given by (12). Since $v_i\to v_j$ and ${\omega }_i\to {\omega }_j$ impl that $d_{iT}=|R_j|-|R_i|=0$ . Therefore, the off-tracking is eliminated for this particular case.

Finally, from (10) and (49), the convergence to zero for all three variables ( $e_{\rho i},e_{\psi i},{\gamma }_i$ ) is only possible if the local leader robot’s path is a straight line, that is ${\omega }_j\to 0$ .

Figure 6 shows the information flow of the proposed scheme.

Figure 6.

Control block diagram of the proposed decentralized scheme.

Simulation results

In this section, to validate the proposed time-varying leader–follower formation control scheme, simulation results are shown.

The simulation platform for the formation control of mobile robots is developed using Gazebo simulator on Robot Operating System (ROS) implemented by C++. ³¹ To study the performance of the control strategy presented in the fourth section as well as the vision-based pose estimation algorithm introduced in the second section, a simulation interface is designed as Figure 7, with two subwindows illustrating each camera view of two followers and a main window demonstrating the motion of three mobile robots in the environment. Thus, two pairs of leader–follower mobile robots are used. So, the red mobile robot (r ₁) is the local follower of the blue robot (r ₂). The blue mobile robot 2 (r ₂) is the follower of the global mobile robot in gray color (r ₃), which has the knowledge of the desired trajectory.

Figure 7.

Simulation interface.

Simulations were performed with three TurtleBot3 mobile robots, ³² using a Raspberry Pi Camera Board V2.1 on each follower robot, with a horizontal field of view ${62.2}^{\text{o}}$ ; thus $\alpha =0.543$ [rad]. Only the horizontal view limit is considered in this work. The OpenCV’s libraries were used in the image processing. The width and height of the rectangular patterns were chosen as $W=0.15$ [m] and $H=0.05$ [m], respectively. The lengths of the triangles of view were setup to $L_{\text{max}}=0.8$ [m] and $L_{\text{min}}=0.35$ [m]. The security zone of each mobile robot was $S_k=0.105$ [m], with $k=1,2,3$ . So, from (15), the $l_{im}$ distance was given as $l_{im}=0.299$ [m], with $i=1,2$ .

Two simulation experiments were developed. For both simulation experiments, the desired separation between robots 1 and 2 was chosen as

${\rho }_{1d}=\frac{({\rho }_{dM}-{\rho }_{dm})}{2}[\text{cos}(2\pi t/T_1)+1]+{\rho }_{dm}$

50

where ${\rho }_{dM}=0.55$ [m] and ${\rho }_{dm}=0.35$ [m] are the maximum and minimum desired separation, respectively, which satisfy the visibility constraints (17). $T_1=60$ [s] is the period of the cosine signal. The other desired separation was chosen as ${\rho }_{2d}=0.35$ [m] with ${\dot{\rho }}_{2d}=0$ . Furthermore, according to both the given desired time-varying separation and the FOV, we choose ${\gamma }_{iM}={\gamma }_M=0.8$ [rad], with $i=1,2$ . The settings of the parameters are ${\lambda }_{0i}={\kappa }_{0i}={\omega }^2=0.64$ , ${\lambda }_{1i}={\kappa }_{1i}=2\xi \omega =2.56$ , $K_{vi}=1$ , $K_{pi}=0.7$ , $K_{\omega i}=0.5$ , $K_{\psi i}=0.6$ , with $i=1,2$ .

Simulation 1

Similar to related works which address the vision-based formation control problem, the first experiment was carried out considering a circular path. ^21,29,33 In open-loop, the global leader robot’s velocities were given as $v_3=$ 0.1 [m/s] and ${\omega }_3=$ 0.1 [rad/s]. Hence, the trajectory’s period is given by $T_c=2\pi /{\omega }_3\approx 62.831$ [s]. The initial positions and the initial orientations of all mobile robots were chosen as $P_1(0)=(0.9,-0.9)$ [m], $q_1(0)=2$ [rad], $P_2(0)=(0.5,-0.5)$ [m], $q_2(0)=2.1$ [rad], $P_3(0)=(0.15,-0.2)$ [m], $q_3(0)=2.2$ [rad].

Figure 8 illustrates the trajectory of each mobile robot. As can be seen, the off-tracking phenomenon of the pair composed of robot 2 and robot 3, after a transient, is eliminated since both a time-invariant formation and a circular leader robot’s path are required. In Remark 5, it is demonstrated that for this particular case, $v_2\to v_3$ and ${\omega }_2\to {\omega }_3$ , which implies that the measure of the off-tracking is given by $d_{2T}=|v_3/{\omega }_3|-|v_2/{\omega }_2|=0$ , according to (12) and (13). In a similar manner, for the pair composed of robot 1 and robot 2, the measure of the off-tracking can be compute, using (11)–(13) and (45)–(46). From Figure 8, the off-tracking is significantly reduced, since a time-varying formation is required.

Figure 8.

Trajectories of all mobile robots (Simulation 1).

To show the efficiency of the proposed scheme, in Figure 9, the trajectories of all mobile robots without reducing the off-tracking, are depicted. To avoid the reduction of the off-tracking, it is necessary to consider ${\psi }_{id}=0$ , in (23).

Figure 9.

Trajectories of all mobile robots without reducing the off-tracking (Simulation 1).

In Figure 10, the formation errors and the relative orientation of each pair of robots are shown. As is demonstrated in Lemma 1, formation errors asymptotically converge to zero, independently whether a time-varying formation is required. Also, according to visibility constraints, the relative orientations are kept bounded by the maximum relative orientation, $|{\gamma }_i|\le {\gamma }_M$ , with $i=1,2$ . Furthermore, to the time-invariant leader–follower formation, the relative orientation ( ${\gamma }_2$ ) converges to its equilibrium point, given by (49), that is ${\overline{\gamma }}_2\approx 0.3518$ [rad]. To the pair composed of robot 1 and 2, the relative orientation is bounded in accordance with (17).

Figure 10.

Formation errors and relative orientations (Simulation 1).

Figure 11 shows the translational velocity of each mobile robot for the first simulation. Due to the desired separation between r ₂ and r ₃ is constant, v ₂ converges to the global leader robot’s velocity (v ₃). Since the desired separation between r ₁ and r ₂ is time-varying, the linear velocity of the follower robot 1 (v ₁) is oscillating around its local leader robot’s velocity (v ₂), according to (45).

Figure 11.

Translational velocities of the mobile robots (Simulation 1).

In Figure 12, the rotational velocity of each mobile robot, for simulation 1, is shown. In a similar manner to the translational velocities, the rotational velocity of the follower robot 2 ( ${\omega }_2$ ) converges to the global leader robot’s velocity ( ${\omega }_3$ ). Since a time-varying separation is required, between r ₁ and r ₂, the ${\omega }_1$ rotational velocity is oscillating around its local leader robot’s velocity ( ${\omega }_2$ ), according to (46).

Figure 12.

Rotational velocities of the mobile robots (Simulation 1).

Simulation 2

Unlike related works which address the vision-based formation control problem, the efficiency of the proposed scheme was tested on a difficult scenario, that is, not to consider only straight lines, ¹² circular/parabolic segments, ^{10 –12} and circular paths with constant radius of curvature. ^21,23,29,33 Similar to González-Sierra and Aranda-Bricaire, ⁴ where any camera is used and only the reduction of the off-tracking is addressed, to show that the time-varying formation control problem can be solved while significantly reducing the off-tracking effect, a lemniscate trajectory is considered. Additionally, to test many time-varying curvature radii until reach out the minimum radius, allowed by the visibility constraints, a spiral path for the global leader robot is chosen. Also, to test a sudden change between two different curvature radii, a straight line segment is added to connect both paths. According to the maximum relative angle ( ${\gamma }_M=0.8$ [rad]), from (20), the minimum radius that can be reached is $R_{\text{2min}}=R_{\text{3min}}\approx 0.447$ [m]. Then, the desired piecewise trajectory of the global leader robot is given by $X_d=2.4\text{sin}({\omega }_l\tau )$ , $Y_d=0.8\text{sin}(2{\omega }_l\tau )$ , for $t\in [0,T/2)$ , where ${\omega }_l=2\pi /T$ , with $T=140$ [s] as the period, $\tau =t+T/2$ as a temporal offset. For this part of the global leader robot’s path, a control law was required to solve the trajectory tracking problem of the global leader. After $T/2$ [s], in open-loop, the global leader robot’s velocities were given as $(v_3,{\omega }_3)=$ $(v_3\mathrm{,0})$ , for $T/2\le t<T/2+t_1$ , $(v_3,{\omega }_3)=$ $(v_3+a[t-T/2-t_1],-0.12)$ , for $T/2+t_1\le t\le t_2$ , where $v_3=0.13$ [m/s], $a=-0.001$ [m/s ${ }^2$ ], $t_1=10$ [s], $t_2=150$ [s]. The initial posture of each mobile robot is the same posture used in Simulation 1.

Figure 13 illustrates the trajectory of each mobile robot. As can be seen, although the off-tracking effects are significantly reduced during all test, the largest deviation between the paths occurs when the global leader robot suddenly changes its path from the straight line to the circular path (at 80 s). As was mentioned before, this situation cannot be eliminated by using the proposed scheme, since the proposed control law only requires the instantaneous velocities of the leader robot; the path traveled by the leader robot in the configuration space is unknown by the local follower robot. Furthermore, the proposed strategy to reduce the off-tracking is based on the steady state for a time-invariant leader–follower formation.

Figure 13.

Trajectories of the unicycle mobile robots (Simulation 2).

In Figure 14, both the formation errors and the relative orientation of the pairs of robots are shown. Formation errors converge to a small band within 20 s. As is mentioned in section “Extended state observer,” the estimation errors (27), will be bounded by the time derivative of the disturbances (35), which are the global leader robot’s velocities ( $v_3,{\omega }_3$ ), in this case. Therefore, the formation errors depends on the rate of the curvature radius of the desired trajectory, due to the global leader robot’s velocities are not constant. However, the relative orientations are kept bounded by the maximum relative orientation, $|{\gamma }_i|\le {\gamma }_M$ . Notice that at 80 s, when the global leader robot suddenly changes its curvature radius, the formations errors $e_{\psi 1}$ and $e_{\psi 2}$ are increased. This sudden change, in the curvature radius (R ₃), can be seen as an external perturbation that is compensated by the control law after a certain time.

Figure 14.

Formation errors and relative orientations (Simulation 2).

Figure 15 shows the translational velocity of each mobile robot, for Simulation 2. In Figure 16, the rotational velocity of each mobile robot is shown. Since the global leader robot’s velocities are not constant, from (35), the velocities estimation errors ( $\Delta {\omega }_2$ , $\Delta {\omega }_3$ ) are nonzero. Therefore, the velocities of the follower robot 2 ( $v_2,{\omega }_2$ ) do not converge to the global leader robot’s velocities ( $v_3,{\omega }_3$ ), even for the time-invariant formation. In a similar manner, to the time-varying formation composed of robot 1 and robot 2. The velocities estimation errors depends on the rate of change of the global leader robot’s velocities. Note that, after 80 s, the rotational velocity of the global leader robot ( ${\omega }_3$ ) was setup as a constant, thus the rotational velocity of the follower robot 2 ( ${\omega }_2$ ) converges to ${\omega }_3$ .

Figure 15.

Translational velocities of the mobile robots (Simulation 2).

Figure 16.

Rotational velocities of the mobile robots (Simulation 2).

Due to the reduced camera’s FOV, taking into account the visibility constraints, to guarantee that the time-varying formation problem can be solved, it is necessary to start either with a small relative orientation between the mobile robots and with a small formation errors or on a straight line.

From the simulation results, it can be concluded that the obtained development, in this time-varying leader–follower formation scheme with reduction of the off-tracking under communication and visibility constraints, is similar to the one that can be obtained by using both the absolute positions and the absolute orientation of the mobile robots with explicit communication. However, due to the reduced camera’s FOV, to guarantee that the time-varying formation control problem can be achieved, the desired trajectory of the global leader robot must satisfy the minimum curvature radius condition (21), which implies that the desired trajectory must be large enough compared to the desired separation between the mobile robots.

Conclusion

In this article, a time-varying leader–follower formation control scheme for unicycle mobile robots to reduce the off-tracking effect, using only a perspective camera, is presented. The off-tracking effect is eliminated in the steady state for a time-invariant leader–follower formation, using only feedback from the monocular camera, and it is significantly reduced for a time-varying formation. Based on Lyapunov theory, it is demonstrated that the formation errors converge to zero even for a time-varying formation, considering a circular leader robot’s path. Also, the relative orientation dynamics is stable. To guarantee that the formation control problem can be achieved, both the visibility constraints and the minimum curvature radius condition are defined. Simulation results, tested on a difficult path, with a time-varying curvature radius, show the efficiency of the proposed time-varying formation control with reduction of the off-tracking effect under communication and visibility constraints.