Planar in-hand manipulation via motion cones

Abstract

In this article, we present the mechanics and algorithms to compute the set of feasible motions of an object pushed in a plane. This set is known as the motion cone and was previously described for non-prehensile manipulation tasks in the horizontal plane. We generalize its construction to a broader set of planar tasks, such as those where external forces including gravity influence the dynamics of pushing, or prehensile tasks, where there are complex frictional interactions between the gripper, object, and pusher. We show that the motion cone is defined by a set of low-curvature surfaces and approximate it by a polyhedral cone. We verify its validity with thousands of pushing experiments recorded with a motion tracking system. Motion cones abstract the algebra involved in the dynamics of frictional pushing and can be used for simulation, planning, and control. In this article, we demonstrate their use for the dynamic propagation step in a sampling-based planning algorithm. By constraining the planner to explore only through the interior of motion cones, we obtain manipulation strategies that are robust against bounded uncertainties in the frictional parameters of the system. Our planner generates in-hand manipulation trajectories that involve sequences of continuous pushes, from different sides of the object when necessary, with 5–1,000 times speed improvements to equivalent algorithms.

1. Introduction

A motion cone is the set of feasible motions that a rigid body can follow under the action of a frictional push. We can think of it as a representation of the underactuation inherent to frictional contacts. Since contacts can only push, and since friction is limited, a contact can move an object only along a limited set of rays. The concept was introduced by Mason (1986) for a point contact in the context of a planar horizontal pushing task.

Motion cones are a practical geometric representation that abstracts the algebra involved in simulating frictional contact dynamics, and provides direct bounds on the set of feasible object motions. A contact force on the inside (or boundary) of the friction cone produces sticking (or slipping) behavior, and leads to motion rays on the inside (or boundary) of the motion cone. Lynch and Mason (1996) generalized the construction of motion cones to line contacts in a horizontal plane. They used them to plan stable pushing trajectories without having to deal explicitly with the complexities of the complementarity formulations of contact dynamics (Chavan-Dafle and Rodriguez, 2017; Posa et al., 2014; Stewart and Trinkle, 1996). Since then, motion cones have been the basis of several efficient planning and control strategies for planar manipulations on a horizontal support surface (Dogar and Srinivasa, 2010; Erdmann, 1998; Hogan and Rodriguez, 2016; Zhou and Mason, 2017).

This article studies the construction of motion cones for a broader set of planar pushing tasks, moving beyond the horizontal plane, and including the effect of gravity. In particular, we highlight the case of prehensile manipulation in the vertical plane where gravity, or other possible external forces, alter the dynamics of the contact interactions between an external pusher and a grasped object. In this article, we show that motion cones are an effective representation to capture the coupled gripper–object–environment mechanics, which is critical for efficient simulation, planning, and control.

We present four main contributions.

• Mechanics of motion cones for planar tasks in the gravity plane. We show that the motion cone is defined by a set of low-curvature surfaces, intersecting at a point and pairwise in lines. We propose a polyhedral approximation to the motion cone for efficient computation.

• Experimental validation of the boundaries of the motion cones in a planar prehensile pushing task. We record the object motion and the stick/slip of the pusher for thousands of prehensile pushes in three different grasp configurations. The motion cones characterize the feasible object motions with $\ge 80\%$ average accuracy. These numbers are considerably larger when taking conservative approximations of the coefficient of friction.

• Application of motion cones in a sampling-based planning framework for planar in-hand manipulation using prehensile pushes (see Figure 1). This yields significant speed improvements with respect to our prior work (Chavan-Dafle and Rodriguez, 2017, 2018b). Figure 2 shows an example of a prehensile pushing trajectory to change the grasp on a T-shaped object. The trajectory is a sequence of continuous stable pushes, during which the object sticks to an external pusher. The proposed planning algorithm obtains these trajectories consistently in less than a second.

• Robustness of prehensile pushes to variations in friction coefficients and gripping force. We show that conservative approximations of motion cones, and a careful selection of the orientation of the task with respect to gravity, yield increased reliability, and, in some cases, invariance.

Fig. 1.

(Top) Example friction cone and motion cone of an object moving in the vertical plane. The pusher can move the object along any direction $[V_x,V_z,{\omega }_y]$ inside the motion cone. (Bottom) A plan via motion cones. Motion cones capture local reachability. A path in the tree of motion cones generates a pushing strategy to move an object.

Fig. 2.

Manipulating a T-shaped object in a parallel-jaw grasp by pushing it against features in the environment. The manipulation is shown from a side view.

The generalization of motion cones to interactions with external forces such as gravity opens the door for efficient and robust planning of in-hand manipulation that respects, and exploits, the basic principles of frictional rigid-body contact interactions: Newton’s second law, Coulomb’s friction law, the principle of maximal dissipation, and the rigidity of rigid bodies.

1.1. Structure of the article

In this article, we describe the mechanics and algorithms involved in using motion cones. When possible, we use both the languages of algebra (for implementation) and geometry (for intuition) to describe them.

The article can be divided into four main parts: (i) mechanics of planar pushing; (ii) construction, computation, and validation of motion cones; (iii) using motion cones to plan in-hand manipulation; (iv) different mechanisms to increase the reliability of motion cones.

1.1.1. The mechanics of planar pushing

This is the basis of the mechanics analysis in the article and is reviewed in detail in Section 3. We show how the motion of a pushed object is governed by three key laws (described in three subsections):

1. the frictional interaction at the contact between an object and a supporting plane, the limit surface;

2. the frictional interaction at the contact between the object and the pusher; when the contact is a point, Coulomb’s law defines a friction cone; when the contact is a line or more complex, it defines a generalized friction cone;

3. the force balance imposed by Newton’s second law.

These three constraints define the set of possible force equilibria between the gripper, object, and environment.

1.1.2. Construction, computation, and validation of motion cones

In Section 4 we show how a particular choice of pusher motion, alongside the principle of maximal dissipation, i.e., relation between the friction and velocity of a sliding object, determines the motion of the pushed object. The set of object motions for all possible pusher motions defines the motion cone. Section 4 describes how to compute motion cones for the cases of an object pushed in the horizontal plane (review) and in the vertical plane with gravity (contribution). In Section 5 we provide an analytic expression for the motion cone as well as an approximation in the form of a polyhedral motion cone, which will be key to using them in a fast planning framework. At the end of this section we detail efforts to verify experimentally the validity of motion cones for three different task configurations, i.e., three different combinations of grasps and prehensile pushes.

1.1.3. Motion cones for planning in-hand manipulation

In Section 6 we describe how to use motion cones to manipulate grasped objects by pushing them against contacts in the environment. Motion cones directly describe the set of directions along which a grasped object can be displaced for a given external contact. In practice, we compute motion cones in real-time and use them to efficiently sample feasible regrasps in an RRT*-based planning framework. In Section 7 we show different examples of planned and executed planar in-hand regrasps.

1.1.4. Robustness of motion cones

Finally, we explore different ways to improve the reliability of motion cones. Section 8 shows that either a conservative approximation of motion cones, or a careful selection of the orientation of the task with respect to gravity, are sufficient to yield mechanical robustness with respect to variations in the available friction at the contacts and the gripping force.

We provide a video summarizing our approach at https://youtu.be/tVDO8QMuYhc.

1.2. Assumptions of the approach

This article presents a model-based approach to planar in-hand manipulation of grasped objects. The approach relies on the following assumptions and representations.

• Objects: three-dimensional objects grasped along parallel faces. The manipulation plane is parallel to the grasped faces. The outline of the object in the manipulation plane can be non-convex. We assume we know the geometry and mass of the object.

• Gripper: a parallel-jaw gripper with force control. The gripping force remains constant during the manipulation. The friction coefficient between the gripper fingers and the objects is known, with some bounded uncertainty.

• Environment: a rigid environment with geometric features (e.g., flat faces, edges, corners) to push against. These provide affordances of controlled patch, line, or point contacts. We know the location and geometry of the features with precision and their friction coefficient with some bounded uncertainty.

• Regrasp examples: planar regrasps are in the plane orthogonal to the axis between the fingers of the parallel-jaw grasp. The configuration space, i.e., the space of object grasps, is three-dimensional; the object can translate and rotate in the manipulation plane. The desired regrasp is specified by the initial and final object pose in the grasp, as illustrated in Figure 2.

The planning framework proposed in this article uses the mechanics of pushing, captured in the form of motion cones, to generate a sequence of external pushes that moves the object from the initial to the final grasp. In our implementation, the external pushes are executed by a robot forcing the grasped object against fixed features in the environment. These pushes could also abstract the interactions with a second robot arm or with extra fingers in a multi-finger gripper.

Figure 3 shows the schematic from the planner’s perspective: in the planning phase, we imagine the alternate perspective where the parallel-jaw gripper is fixed in the world and the environment is a “virtual” pusher with full three-degree-of-freedom (3DOF) mobility in the manipulation plane pushing the grasped object. Planning external pushes is then equivalent to planning the motion of the virtual pusher to achieve the desired regrasp. The “virtual” pusher motion is then rendered in the real world by instead moving the gripper.

Fig. 3.

(Left) An object held in a parallel-jaw gripper is pushed against a fixed feature in the environment. (Right) The planner models this interaction as if the object is held by a fixed gripper and pushed by a moving environment, i.e., a pusher.

2. Related work

Planning and control through contact is a central topic in robotic manipulation research. A rigid-body contact is modeled as a series of constraints on the possible motions and forces at contact. A particular contact mode, either sticking or sliding, invokes a specific set of constraints on the motions and forces at contact. Stewart and Trinkle (1996) and Trinkle et al. (1997) showed that a general rigid-body dynamics problem with hybrid sticking/slipping dynamics at contact interactions can be modeled as a complementarity problem.

Recent work on trajectory optimization and manipulation planning has shown that it is possible to reason about hybrid stick/slip contact modes and plan trajectories through continuous contact with complementarity constraints (Chavan-Dafle and Rodriguez, 2017; Posa et al., 2014). An alternative approach is to replace the hard complementarity constraints by optimization-friendlier soft-constraints, which yields a faster planning and control framework (Kumar et al., 2014; Todorov et al., 2012). These methods, though broad in their scope of application, often have to compromise between computational efficiency and the realism of contact dynamics (Kolbert et al., 2016).

In contrast, a pivotal theme in the non-prehensile manipulation literature is to identify application-specific and compact representations of contact constraints and the mechanics of the task. Goyal (1989) introduced the concept of a limit surface, a compact mapping from the friction wrench between an object and its support surface and the sliding twist at contact. Mason (1986) studied the mechanics of pushing and proposed the concept of the motion cone. These two fundamental geometric constructions provide direct force–motion mappings for contact interactions and have facilitated efficient planning and control techniques in non-prehensile manipulation (Dogar and Srinivasa, 2010, 2011; Hogan and Rodriguez, 2016; Lynch et al., 1992; Lynch and Mason, 1996; Zhou and Mason, 2017).

Lynch and Mason (1996) extended the idea of motion cones to a line pusher on a horizontal surface and showed its application for planning robust pushing strategies for which the pusher contact sticks to the object. Dogar and Srinivasa (2011) demonstrated the application of motion cones in the push-grasping framework to plan pushes that capture an object before grasping it. Hogan and Rodriguez (2016) and Zhou and Mason (2017) exploit the direct action–effect mapping in motion cones to develop efficient planning and control techniques for pushing an object on a horizontal plane.

Some recent work focuses on other types of manipulation primitives. Shi et al. (2015) demonstrated dynamic in-hand manipulation planning in a parallel-jaw grasp by exploiting inertial forces. With a pre-defined contact mode sequence at the fingers and a limit surface approximation for the force-motion interaction at the fingers, they derive a control law that can move the object to the goal grasp through a sequence of fast accelerations and decelerations. Similar approaches are explored for planning and controlling in-hand manipulations by actively using gravity (Viña B et al., 2016) or dynamic motions (Holladay et al., 2015; Hou, 2017; Sintov and Shapiro, 2016).

Sundaralingam and Hermans (2017) proposed a purely kinematic approach for in-hand manipulation based on trajectory optimization with a multi-finger gripper. They assumed that the fingers on the object do not slip and impose soft constraints that encourage the minimization of finger slip. By assuming all finger contacts to be sticking, they bypassed the need for modeling the dynamics of contacts and obtained fast kinematic plans.

In recent work (Chavan-Dafle and Rodriguez, 2018b), we presented a sampling-based planning framework for in-hand manipulations with prehensile pushes, where the pusher contact is forced to stick to the object using the mechanics of the task. The plans are discrete sequences of continuous pushes that respect friction, contact, and rigid-body constraints.

These promising results on in-hand manipulation, which are limited to certain types of manipulation primitives, motivate us to extend the concept of motion cones to more general pushing tasks. Motion cones provide direct bounds on the set of possible object motions and can be used in sampling-based frameworks to guide sampling, or as a set of direct constraints in a trajectory optimization framework.

3. Mechanics of planar pushing

Figure 4 shows four different cases of planar pushing. In case (a), the pusher force is the only external force on the object in the manipulation plane. However, in the other cases, a component of gravity is also present. The concept of motion cones, as originally studied by Mason (1986) and Lynch and Mason (1996), is limited to case (a). The extension of the mechanics of motion cones that we present in this article is valid for all cases in Figure 4.

Fig. 4.

Pushing an object (a) on a horizontal surface, (b) on an inclined surface, (c) in a grasp in the gravity plane, and (d) in a grasp in a tilted plane.

In this section we discuss the mechanics of pushing an object in a plane. First, we review the fundamental concepts, namely, the limit surface (Goyal, 1989), and the generalized friction cone (Erdmann, 1994). These tools will serve as building blocks for modeling the force–motion interaction at the contacts involved in pushing manipulations. Table 1 lists the notation used within the rest of the article.

3.1. Limit surface

The limit surface is a common construction to model the frictional interaction at a planar patch contact. In this article, we use the limit surface to model the force–motion interaction between an object and a support contact. In cases (a) and (b) in Figure 4, the surface on which the object rests is the support contact. In cases (c) and (d), the gripper fingers are the support contacts. As summarized in Table 1, for object motions in the manipulation plane, the resultant friction wrench ${\mathit{w}}_{\mathrm{s}}$ offered by the support contact on the object is composed of the frictional forces $(f_{\mathrm{x}},f_{\mathrm{z}})$ in the plane and the frictional torque $(m_{\mathrm{y}})$ about the normal to the plane.

Table 1.

(Left) Schematic of a prehensile pushing scene with important reference frame labeled. (Right) Mathematical notation used throughout the article.

	$q=[x,y,\theta ]\in \mathrm{SO}(2)$	Configuration of the object
	FW	World frame fixed to the gripper.
	FS	Support frame, fixed to the support surface and coinciding with the world frame, FW. Located at the center of pressure at the contact.
	FO	Object frame, moving with the center of mass of the object
	FPi	Pusher contact frame, moving with the ith point contact modeling the pusher
	${\mathit{w}}_{\mathrm{support}}\in [f_{\mathrm{x}},f_{\mathrm{z}},m_{\mathrm{y}}{]}^{\mathrm{T}}$	Wrench exerted on the object by the support contact (in frame FO)
	${\mathit{w}}_{\mathrm{pusher}}\in [f_{\mathrm{x}},f_{\mathrm{z}},m_{\mathrm{y}}{]}^{\mathrm{T}}$	Wrench exerted on the object by the pusher contact (in frame FO)
	${\mathit{w}}_{\mathrm{s}}\in [f_{\mathrm{x}},f_{\mathrm{z}},m_{\mathrm{y}}{]}^{\mathrm{T}}$	Wrench at the support contact (in frame FS).
	${\mathit{f}}_{\mathrm{p}}\in [f_{\mathrm{n}1},f_{\mathrm{t}1},f_{\mathrm{n}2},f_{\mathrm{t}2},\dots {]}^{\mathrm{T}}$	A vector of forces at pusher point contacts in $F_{P1},F_{P1},\dots$ respectively
	${\mathit{v}}_{\mathrm{obj}}\in [v_{\mathrm{x}},v_{\mathrm{z}},{\omega }_{\mathrm{y}}{]}^{\mathrm{T}}$	Object twist, i.e., generalized velocity, in FO
	${\mathit{v}}_s\in [v_{\mathrm{x}},v_{\mathrm{z}},{\omega }_{\mathrm{y}}{]}^{\mathrm{T}}$	Object twist at the support contact in FS
	${\mathit{J}}_{\mathrm{s}}\in {\mathbb{R}}^{3x3}$	Jacobian that maps ${\mathit{v}}_{\mathrm{obj}}$ to ${\mathit{v}}_s$
	${\mathit{J}}_{\mathrm{p}}=\left[\begin{array}{c}{\mathit{J}}_{P1}\hfill \\ \multicolumn{1}{c}{{\mathit{J}}_{P2}}\\ \multicolumn{1}{c}{⋮}\end{array}\right]\in {\mathbb{R}}^{2\mathrm{nx}3}$	Jacobian that maps ${\mathit{v}}_{\mathrm{obj}}$ to the velocity at the n pusher point contacts in the pusher contact frames $\left(F_{P1},F_{P2},\dots \right)$ black

Goyal (1989) defined the boundary of the set of all possible friction wrenches that a contact can offer as the limit surface. Each wrench on the boundary of the limit surface corresponds to a different sliding motion of the object on the support plane. Based on the principal of maximum dissipation, the perpendicular to the limit surface at a given wrench provides the corresponding generalized velocity of the object (Goyal, 1989).

Related to the limit surface, the gravito-inertial wrench cone (GIWC) is a geometric construct in legged locomotion literature. However, the embedding of the principal of maximum dissipation into the limit surface construction provides the information of the sliding velocity at the contact, which GIWC cannot. ¹

The limit surface is smooth and convex if the support force distribution is finite everywhere. Howe and Cutkosky (1996); Xydas and Kao (1999) showed that an ellipsoidal approximation allows for a simpler representation of the limit surface geometry. In this article we assume an ellipsoidal approximation of the limit surface, which has been shown to be computationally efficient for simulating and planning pushing motions (Dogar and Srinivasa, 2011; Lynch and Mason, 1996; Shi et al., 2015; Zhou and Mason, 2017).

Let ${\mathit{w}}_{\mathrm{s}}=[f_{\mathrm{x}},f_{\mathrm{z}},m_{\mathrm{y}}]$ be the frictional wrench on the object from the support contact in the support contact frame, ${\mathcal{F}}_S$ . A mathematical representation of the ellipsoidal limit surface is given by

$${\mathit{w}}_{\mathrm{s}}^{\mathrm{T}}\mathit{A}{\mathit{w}}_{\mathrm{s}}=1$$

(1)

where, for isotropic friction,

$$A=\left[\begin{array}{ccc}\frac{1}{{({\mu }_{\mathrm{s}}N)}^2}& & 0\\ & \frac{1}{{({\mu }_{\mathrm{s}}N)}^2}& \\ 0& & \frac{1}{{(k{\mu }_{\mathrm{s}}N)}^2}\end{array}\right]$$

such that ${\mu }_{\mathrm{s}}$ is the friction coefficient between the contact and the object, N is the normal force at the contact, and k is an integration constant. For a circular patch contact with a uniform pressure distribution at the contact, $k\approx 0.6r$ where r is the radius of the contact (Shi et al., 2015; Xydas and Kao, 1999).

Note that the geometry of the limit surface is defined by the area and pressure distribution at the contact and scaled by normal force and friction coefficient at the contact.

When the object slides on the support contact, the friction wrench ( ${\mathit{w}}_{\mathrm{s}}$ ) at the contact intersects the limit surface, as shown in Figure 5. The direction of object twist in the contact frame ( ${\mathit{v}}_{\mathrm{s}}$ ) is given by the normal to the limit surface at the intersection point (Goyal, 1989) and can be computed as

$${\mathit{v}}_{\mathrm{s}}\propto \frac{\partial }{\partial {\mathit{w}}_{\mathrm{s}}}({\mathit{w}}_{\mathrm{s}}^{\mathrm{T}}\mathit{A}{\mathit{w}}_{\mathrm{s}})\propto \mathit{A}{\mathit{w}}_{\mathrm{s}}$$

(2)

Fig. 5.

(Left) Ellipsoidal approximation of the limit surface at the finger contact. For a wrench ${\mathit{w}}_{\mathrm{s}}$ on the boundary of the limits surface, the twist at the contact ${\mathit{v}}_{\mathrm{s}}$ is normal to the limit surface. (Right) The wrench ${\mathit{w}}_{\mathrm{s}}$ and the corresponding object motion is shown in the manipulation scene.

Conversely, if the object twist at the contact ( ${\mathit{v}}_{\mathrm{s}}$ ) is known, we can find the friction wrench as

$${\mathit{w}}_{\mathrm{s}}=\frac{1}{\lambda }{\mathit{A}}^{-1}{\mathit{v}}_{\mathrm{s}}$$

(3)

Since ${\mathit{w}}_{\mathrm{s}}$ lies on the ellipsoidal limit surface, from (1), we can solve for $\lambda$ :

$$\begin{array}{c}{\left(\frac{1}{\lambda }{\mathit{A}}^{-1}{\mathit{v}}_{\mathrm{s}}\right)}^T\mathit{A}\left(\frac{1}{\lambda }{\mathit{A}}^{-1}{\mathit{v}}_{\mathrm{s}}\right)=1\hfill \\ {{\mathit{v}}_{\mathrm{s}}}^T{\mathit{A}}^{-T}\mathit{A}{\mathit{A}}^{-1}{\mathit{v}}_{\mathrm{s}}={\lambda }^2\hfill \\ {{\mathit{v}}_{\mathrm{s}}}^T{\mathit{A}}^{-T}{\mathit{v}}_{\mathrm{s}}={\lambda }^2\hfill \\ \sqrt{{{\mathit{v}}_{\mathrm{s}}}^T{\mathit{A}}^{-T}{\mathit{v}}_{\mathrm{s}}}=\lambda \hfill \end{array}$$

Given that $\mathit{A}$ is a diagonal matrix, $\lambda =\sqrt{{{\mathit{v}}_{\mathrm{s}}}^{\mathrm{T}}{\mathit{A}}^{-1}{\mathit{v}}_{\mathrm{s}}}$ . Rewriting (3):

$${\mathit{w}}_{\mathrm{s}}=\frac{{\mathit{A}}^{-1}{\mathit{v}}_{\mathrm{s}}}{\sqrt{{{\mathit{v}}_{\mathrm{s}}}^{\mathrm{T}}{\mathit{A}}^{-1}{\mathit{v}}_{\mathrm{s}}}}=\frac{{\mu }_{\mathrm{s}}N{\mathit{B}}^{-1}{\mathit{v}}_{\mathrm{s}}}{\sqrt{{{\mathit{v}}_{\mathrm{s}}}^{\mathrm{T}}{\mathit{B}}^{-1}{\mathit{v}}_{\mathrm{s}}}}={\mu }_{\mathrm{s}}N{\overline{\mathit{w}}}_{\mathrm{s}}$$

(4)

Here, $\mathit{B}=\mathrm{Diag}(1,1,k^{-2})$ and ${\overline{\mathit{w}}}_{\mathrm{s}}=[{\overline{f}}_{\mathrm{x}},{\overline{f}}_{\mathrm{z}},{\overline{m}}_{\mathrm{y}}{]}^{\mathrm{T}}$ is the wrench on a unit limit surface ( ${\overline{\mathit{w}}}_{\mathrm{s}}^{\mathrm{T}}B{\overline{\mathit{w}}}_{\mathrm{s}}=1$ ), which is scaled by ${\mu }_{\mathrm{s}}N$ to produce the net frictional wrench ${\mathit{w}}_{\mathrm{s}}$ .

The force–motion mapping we have discussed so far is in the support contact frame. From (2) and (4), the direction of the object twist in the object frame ( ${\mathit{v}}_{\mathrm{obj}}$ ) for a given wrench on the limit surface can be computed as

$${\mathit{v}}_{\mathrm{obj}}\propto {\mathit{J}}_{\mathrm{s}}^{-1}\mathit{B}{\overline{\mathit{w}}}_{\mathrm{s}}$$

(5)

where ${\mathit{J}}_{\mathrm{s}}^{-1}$ maps the object velocity from the support contact frame ${\mathcal{F}}_S$ to the object frame ${\mathcal{F}}_O$ .

3.2. Generalized friction cone

Coulomb’s friction law is the basis for many of the models used to simulate frictional interaction between two rigid bodies. A Coulomb friction cone establishes the relationship between the normal force and the maximum friction force at a point contact.

Erdmann (1994) introduced the concept of the generalized friction cone ( $\mathit{W}$ ) as the friction cone of the contact expressed in the reference frame of the object. The generalized friction cone for a line/patch contact modeled with multiple point contacts is the convex hull of the generalized friction cones for each constituent contact (Erdmann, 1993) as shown in Figure 6.

Fig. 6.

(Left) The line pusher contact in this example is modeled with two point contacts. The pusher contact frames and the object frame are drawn in the figure. (Right) The generalized friction cone of the pusher is the object frame representation of the set of forces the two point contacts can offer collectively.

We model the friction between the pusher and the object by constructing the generalized friction cone:

$${\mathit{W}}_{\mathrm{pusher}}=\{{\mathit{w}}_{\mathrm{pusher}}={\mathit{J}}_{\mathrm{p}}^{\top }·{\mathit{f}}_{\mathrm{p}}|{\mathit{f}}_{\mathrm{p}}\in F{C}_{\mathrm{pusher}}\}$$

(6)

where ${\mathit{J}}_{\mathrm{p}}^{\top }$ is the Jacobian that maps the contact forces ${\mathit{f}}_{\mathrm{p}}$ from the friction cone $F{C}_{\mathrm{pusher}}$ at the pusher point contacts to the object frame. Since the Coulomb friction cone represents the set of feasible forces a point contact can offer, the generalized friction cone captures the set of feasible wrenches the pusher can impose on the object. If the contact between the pusher and the object is maintained (sticking), the pusher can impose any wrench on the interior of the generalized friction cone. However, if the object slips with respect to the pusher, the pusher wrench on the object will be on the boundary of the generalized friction cone.

We model the interaction between support-and-object (limit surface) and between pusher-and-object (generalized friction cone) differently due to the different geometries and expected pressure distributions. For the support–object interaction, the ellipsoidal approximation to the limit surface provides a compact force-motion relationship at a support contact where we expect a constant normal force distribution during the manipulation. Whereas, the generalized friction cone is a better approach for modeling the friction at the pusher contact where we can have varying normal force distribution for every instantaneous push.

Now, with the chosen models for frictional contact, we formulate the mechanics of pushing in a plane.

3.3. Mechanics of Pushing

The motion of the pushed object is in accordance with the net wrench acting on the object. Under the quasi-static assumption, which is appropriate for slow pushing operations, the inertial forces on the object are negligible and force balance requires:

$${\mathit{w}}_{\mathrm{support}}+{\mathit{w}}_{\mathrm{pusher}}+m\mathit{g}=0$$

(7)

Here, Equation (7) is written in the object frame located at the center of gravity, ${\mathit{w}}_{\mathrm{support}}$ is the friction wrench provided by the support contact, ${\mathit{w}}_{\mathrm{pusher}}$ is the wrench exerted by the pusher, m is the mass of the object, and $\mathit{g}$ is the gravitational component in the plane of motion. The Jacobian ${\mathit{J}}_{\mathrm{s}}^{\top }$ maps the support contact wrench from the support contact frame (usually located at the center of pressure) of the support surface to the object frame. Thus, ${\mathit{w}}_{\mathrm{support}}={\mathit{J}}_{\mathrm{s}}^{\top }·{\mathit{w}}_{\mathrm{s}}$ and (7) becomes

$${\mathit{J}}_{\mathrm{s}}^{\top }·{\mathit{w}}_{\mathrm{s}}+{\mathit{J}}_{\mathrm{p}}^{\top }·{\mathit{f}}_{\mathrm{p}}+m\mathit{g}=0$$

(8)

Equations (8), (6), and (5) define the system of equations to solve to determine the velocity of the object ${\mathit{v}}_{\mathrm{obj}}$ for any possible vector for pusher forces ${\mathit{f}}_{\mathrm{p}}$ . Solving the mechanics of pushing refers to solving (8) at every time instant while considering the possibility of sticking/sliding at the pusher–object and support–object contact interactions. Based on the sticking or sliding mode at the support and pusher contact, the wrenches at these contacts will be either inside or on the boundary of the limit surface and the generalized friction cone, respectively, as discussed in Sections 3.1 and 3.2. We can formulate the hybrid dynamics of pushing as a rigid-body dynamics problem which will take the form of a complementarity problem (Chavan-Dafle and Rodriguez, 2017).

Generating a motion plan for a given pushing manipulation task, e.g., moving an object on a ground or in a grasp, involves building a continuous series of instantaneous feasible object and pusher motions that satisfy the mechanics of pushing at every instant. In a trajectory optimization framework, this translates to including the complementarity constraints from the dynamics of pushing into the optimization problem and optimizing for the series of object–pusher states. In a sampling-based framework this translates to sampling instantaneous pusher–object motions that comply with (8), (6), and (5). Checking their feasibility involves solving the mechanics of pushing and keeping track of the pusher–object state including the possibility of sliding between them.

Either way, solving the mechanics of pushing to predict the pusher–object motion and slipping/sticking at contacts is computationally expensive (Hogan and Rodriguez, 2016; Posa et al., 2014). A direct action–effect mapping, i.e., the relationship between the pusher motion and object motion, avoids the burden of solving the hybrid mechanics of pushing. The existence of such a mapping is the key motivation of this article, described by the motion cone construction that we study in the following section.

4. Motion cones for planar pushing

In this section, we present the general formulation for defining a motion cone for planar pushing. We discuss the nature of the constraints that define the motion cone for pushing on a horizontal surface (Figure 4(a)) and for more general pushing scenarios (Figure 4(b)–(d)). We highlight some of the peculiarities of the case of pushing an object in the gravity plane, which forms the basis for the computation and application of the motion cones in the following sections.

The motion cone is the set of instantaneous object velocities that a pusher can impose on the object using the frictional interaction between them. If the pusher moves along a twist inside the motion cone, the contact between the object and the pusher sticks and the object follows the pusher motion. If the pusher moves along a twist outside the motion cone, there is sliding at the pusher–object interface and the object moves along a twist on the boundary of the motion cone. While the pusher has full control to move with any velocity, it can impose only a set of object velocities defined by the motion cone.

Mathematically, a motion cone is the largest set of object motions for which the net required wrench for the object motion can be balanced by a wrench on the interior of the generalized friction cone of the pusher. It is the set of object velocities for which constraint (8) holds true while ${\mathit{f}}_{\mathrm{p}}\in F{C}_{\mathrm{pusher}}$ :

$$\begin{array}{c}{\mathit{J}}_{\mathrm{s}}^{\top }·{\mathit{w}}_{\mathrm{s}}+{\mathit{J}}_{\mathrm{p}}^{\top }·{\mathit{f}}_{\mathrm{p}}+m\mathit{g}=0,{\mathit{f}}_{\mathrm{p}}\in F{C}_{\mathrm{pusher}}\hfill \end{array}$$

Using (4) and (6), we can rewrite the previous equation as

$$\begin{array}{c}{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}+k{\overline{\mathit{w}}}_{\mathrm{pusher}}+m\mathit{g}=0\hfill \\ \multicolumn{1}{c}{{\overline{\mathit{w}}}_{\mathrm{pusher}}\in {\mathit{W}}_{\mathrm{pusher}},k\in {\mathbb{R}}^{+}}\end{array}$$

(9)

where k is the magnitude of the pusher force and ${\overline{\mathit{w}}}_{\mathrm{pusher}}$ is a unit pusher wrench and ${\overline{\mathit{w}}}_{\mathrm{s}}$ is a unit support wrench. To find the set of instantaneous object motions that satisfy the above equation, we rearrange it as

$$\begin{array}{c}{\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}=\frac{k}{-{\mu }_{\mathrm{s}}N}{\overline{\mathit{w}}}_{\mathrm{pusher}}+\frac{m}{-{\mu }_{\mathrm{s}}N}\mathit{g}\hfill \\ \multicolumn{1}{c}{{\overline{\mathit{w}}}_{\mathrm{pusher}}\in {\mathit{W}}_{\mathrm{pusher}},k\in {\mathbb{R}}^{+}}\end{array}$$

(10)

Using (5) we can map the support contact unit wrench ${\overline{\mathit{w}}}_{\mathrm{s}}$ to the object twist ${\mathit{v}}_{\mathrm{obj}}$ . Hence, to find a motion cone, we first find the set of support contact wrenches ( ${\overline{\mathit{w}}}_{\mathrm{s}}$ ) that satisfy (10) and then map this wrench set ( ${\mathit{W}}_{\mathrm{s}}$ ) to the set of object twists using (5). We denote this set of object twists, i.e., the motion cone, by ${\mathit{V}}_{\mathrm{obj}}$ . Mason (1986) defined the motion cone in terms of pusher twists which is a linear Jacobian transform of the motion cone ${\mathit{V}}_{\mathrm{obj}}$ . However, we will keep it in the object twist space because it is a more natural representation for planning object trajectories.

4.1. Motion cone for pushing on a horizontal surface

The presence of an external force other than the pusher force (such as the gravitational force) in the plane of motion complicates the mechanics and the structure of the motion cone. To explain this effect in detail, we will first consider the case of pushing an object on a horizontal surface where there is no such additional force.

For the case of pushing on a horizontal plane, $\mathit{g}=0$ . For an object on a flat support surface with uniform pressure distribution on the support (as is Figure 4, case (a) and (b)), the contact frame coincides with the object frame, ${\mathit{J}}_{\mathrm{s}}=\mathit{I}$ (identity matrix), because the support is infinite and spatially invariant. Then we can write (9) as

$${\mu }_{\mathrm{s}}N{\overline{\mathit{w}}}_{\mathrm{s}}+k{\overline{\mathit{w}}}_{\mathrm{pusher}}=0,{\overline{\mathit{w}}}_{\mathrm{pusher}}\in {\mathit{W}}_{\mathrm{pusher}},k\in {\mathbb{R}}^{+}$$

$${\overline{\mathit{w}}}_{\mathrm{s}}=\frac{-k{\overline{\mathit{w}}}_{\mathrm{pusher}}}{{\mu }_{c}N},{\overline{\mathit{w}}}_{\mathrm{pusher}}\in {\mathit{W}}_{\mathrm{pusher}}$$

(11)

The set of valid support contact wrenches that satisfy (11) is the negative of the generalized friction cone of the pusher, i.e., ${\mathit{W}}_{\mathrm{s}}=-{\mathit{W}}_{\mathrm{pusher}}$ . By mapping ${\mathit{W}}_{\mathrm{s}}$ through (5), we obtain the motion cone ${\mathit{V}}_{\mathrm{obj}}$ .

Note that for the case of pushing on a horizontal surface, ${\mathit{W}}_{\mathrm{s}}$ and ${\mathit{V}}_{\mathrm{obj}}$ are both convex polyhedral cones like ${\mathit{W}}_{\mathrm{pusher}}$ . Moreover, they are independent of the support normal force, i.e., the weight of the object $m\mathit{g}$ , and friction at the support surface ${\mu }_{\mathrm{s}}$ .

For more general pushing tasks, however, $\mathit{g}\ne 0$ . From (10) we can see that, unlike for horizontal pushing, the system parameter ( ${\mu }_{\mathrm{s}}$ ) and force magnitudes (k and N) influence the direction vectors of the wrench set ${\mathit{W}}_{\mathrm{s}}$ and motion cone ${\mathit{V}}_{\mathrm{obj}}$ . In the next section we focus on the case in Figure 4(c): pushing an object in a parallel-jaw grasp in the plane of gravity. There the gravitational force is not zero in the plane of motion and the Jacobians ${\mathit{J}}_{\mathrm{s}}$ are not always identity matrices since, in general, the support (finger) contact location changes in the object frame as the object is pushed in the grasp.

Note that the case in Figure 4(b) can be modeled in the same way as (c) except the Jacobian ${\mathit{J}}_{\mathrm{s}}=\mathit{I}$ and only a part of the gravitational force is in the plane of motion. The case in Figure 4(d) can be modeled in the same way as (c) except that only a fraction of the gravitational force acts in the plane of motion. The results derived for the case of pushing an object in the gravity plane then extend naturally to the cases in Figure 4(b) and (d). However, the experimental validation necessary to certify this claim and to evaluate the accuracy of the underlying assumptions for cases (a) and (b) is out of scope of this article.

4.2. Motion cone in the gravity plane

For a case similar to Figure 4(c), but in a gravity-free world, we can simplify Equation (10) by omitting the gravity term. We can then compute a convex polyhedral motion cone similar to that in the horizontal pushing case, but while taking the Jacobian ${\mathit{J}}_{\mathrm{s}}$ into consideration. We refer to this motion cone as the gravity-free motion cone in the later discussions. This cone, similar to that for the case of pushing on the horizontal plane, is invariant to the friction at the finger contacts, the mass of the object, and the support normal force, i.e., the grasping force.

By rearranging (9), we see that an instantaneous object motion is feasible if the net wrench required for the object motion falls inside the generalized friction cone of the pusher:

$$\begin{array}{c}-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}-m\mathit{g}\in {\mathit{W}}_{\mathrm{pusher}}\hfill \end{array}$$

(12)

The gravitational wrench $m\mathit{g}$ on the object can pull the net required wrench in or out of the generalized friction cone of the pusher. Some of the motions in the gravity-free motion cone may not be stable pushes in the gravity plane, while for some object motions outside the gravity-free motion cone, the gravitational force can make them feasible.

Proposition 1. Any object motion inside the gravity-free motion cone can also be made feasible in the gravity plane by increasing the grasping force above a minimum threshold.

Proof. For a motion inside a gravity-free motion cone, the support/grasp wrench direction lies on the interior the generalized friction cone of the pusher, i.e., ${\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}\in {\mathit{W}}_{\mathrm{pusher}}$ . Because the gravitational wrench is of fixed magnitude, we can always scale the support/grasp wrench direction such that the net wrench (the vector sum of the gravitational wrench and the support/grasp wrench) is inside the generalized friction cone of the pusher. Figure 7 shows the graphical interpretation. In general, for a given ${\mu }_{\mathrm{s}}$ , we can analytically find the bounds on the normal force N needed to pull the net wrench inside the generalized friction cone of the pusher. ■

Fig. 7.

To make a push inside the gravity-free motion cone also stable in a scenario with gravity, the unit grasp wrench can be scaled such that the net pusher wrench required for the desired push falls inside/on the generalized friction cone of the pusher.

A similar argument can be made for determining a minimum friction threshold at the finger contacts. For a given normal force N, we can analytically find the bounds on ${\mu }_{\mathrm{s}}$ needed to pull the net wrench inside the generalized friction cone of the pusher.

Corollary 1.1. With increased grasping force, more motions in the gravity-free motion cone are feasible in the gravity plane. For infinite grasping force, the gravity-free motion cone is equal to the motion cone in the gravity plane.

Proof. From (12) and Proposition 1, as the grasping force increases, the net wrench for more twists or velocities from the gravity-free motion cone fall inside the generalized friction cone of the pusher.

In the gravity-free case, ${\mathit{J}}_{\mathrm{s}}^{\top }·{\mathit{W}}_{\mathrm{s}}=-{\mathit{W}}_{\mathrm{pusher}}$ .

As $N\to \infty$ ,

$$-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}-m\overrightarrow{g}\to -{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}$$

Then, for all ${\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}\in {\mathit{J}}_{\mathrm{s}}^{\top }·{\mathit{W}}_{\mathrm{s}}$ with

$$-N{\mu }_{\mathrm{s}}{\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}-m\overrightarrow{g}\in {\mathit{W}}_{\mathrm{pusher}}$$

Thus, the set of object motions ${\mathit{V}}_{\mathrm{obj}}$ corresponding to ${\mathit{W}}_{\mathrm{s}}$ , which is the gravity-free motion cone, are also stable in the gravity case. ■

We see that, in theory, we can make any object motion in the gravity-free motion cone feasible under gravity by increasing the grasping force. With increased grasping force, more motions in the gravity-free motion cone are stable under gravity and become part of the motion cone. In practice, however, grippers have limited grasping force and often lack online force control. Moreover, by limiting to the gravity-free motion cone, we omit object motions outside the gravity-free motion cone that otherwise would be feasible when considering the gravitational force on the object. Therefore, both for practical reasons and for capturing the rich mechanics of prehensile pushing, we need to find the motion cone under gravity for a given grasping force.

Note that for the case of pushing the object on a horizontal plane, the motion cone is invariant to the support normal force. However, for pushing on a support plane at an angle, as in Figure 4(b), a component of gravitational force acts in the manipulation plane and we do not have a freedom to change the support normal force, so computing the motion cone for a fixed support normal force is essential for these non-prehensile manipulation cases as well.

5. Computation and experimental validationof motion cone

In this section we present a closed-form computation of the object motion cone in the gravity case for a given grasping force and friction parameters. We discuss the graphical intuition for the mechanics and the computation of the motion cone. A large set of pushing experiments in different object–pusher configurations demonstrate that the analytically found motion cones characterize the set of feasible object motions with an average of $\ge 80\%$ accuracy.

5.1. Analytical computation

For a known ${\overline{\mathit{w}}}_{\mathrm{pusher}}\in {\mathit{W}}_{\mathrm{pusher}}$ , Equation (10) is a set of three linear equalities with four unknowns, ${\overline{\mathit{w}}}_{\mathrm{s}}\in {\mathbb{R}}^3$ and $k\in {\mathbb{R}}^{+}$ . We also know that ${\overline{\mathit{w}}}_{\mathrm{s}}=[{\overline{f}}_{\mathrm{x}},{\overline{f}}_{\mathrm{z}},{\overline{m}}_{\mathrm{y}}{]}^{\mathrm{T}}$ is a unit wrench that satisfies the ellipsoidal limit surface constraint:

$$\frac{{\overline{f}}_{\mathrm{x}}^2}{1}+\frac{{\overline{f}}_{\mathrm{z}}^2}{1}+\frac{{\overline{m}}_{\mathrm{y}}^2}{{(\mathrm{rc})}^2}=1$$

(13)

Constraints (10) and (13) can be solved together analytically to find ${\overline{\mathit{w}}}_{\mathrm{s}}$ and k. Specifically, after substituting ${\overline{f}}_{\mathrm{x}}$ , ${\overline{f}}_{\mathrm{z}},$ and ${\overline{m}}_{\mathrm{y}}$ from (10) into (13), Equation (13) becomes a quadratic polynomial equation in k. Solving this quadratic equation with a constraint $k\in {\mathbb{R}}^{+}$ gives a unique solution for k. Substituting this value for k in (10) turns it into a set of three linear equalities with three unknowns $[{\overline{f}}_{\mathrm{x}},{\overline{f}}_{\mathrm{z}},{\overline{m}}_{\mathrm{y}}]$ , which can be solved for a unique solution. For prehensile pushing in the gravity plane, the relationship between ${\mathit{W}}_{\mathrm{s}}$ and ${\mathit{W}}_{\mathrm{pusher}}$ is not linear as in the gravity-free case. To find the wrench set ${\mathit{W}}_{\mathrm{s}}$ , we need to sweep ${\overline{\mathit{w}}}_{\mathrm{pusher}}$ over the boundary of ${\mathit{W}}_{\mathrm{pusher}}$ and solve (10) and (13) iteratively.

Figure 8 illustrates the process for solving the constraints (10) and (13) together to compute the wrench cone ${\mathit{W}}_{\mathrm{s}}$ and further map the wrench set ${\mathit{W}}_{\mathrm{s}}$ to the motion cone ${\mathit{V}}_{\mathrm{obj}}$ , using Equation (5). Intuitively we can read Figure 8 from left to right as follows.

1. Force modeling step involves formulating the set of forces present in the manipulation plane using the contact modeling techniques discussed in Sections 3.1 and 3.2. The limit surface represents the set of friction wrenches that the support contacts can offer to prevent the motion between the object and the support surface. The generalized friction cone of the pusher is the set of wrenches the pusher can impose on the object. Together with other external forces in the manipulation plane, e.g., the gravitational component, it represents the set of external wrenches that act on the object.

2. Force resolution refers to solving for the quasi-static force balance using (10) and (13). Solving for force balance means finding the subset ${\mathit{W}}_{\mathrm{s}}$ of the support limit surface that balances the set of external wrenches.

3. Motion resolution interprets the set of object twists that corresponds to the force-balance solution set using (5). Graphically, this represents drawing normals to the subset ${\mathit{W}}_{\mathrm{s}}$ computed in the earlier step, to obtain the corresponding set of object twists ${\mathit{V}}_{\mathrm{obj}}$ . This set is the motion cone.

Fig. 8.

Graphical illustration of the motion cone construction procedure applied to two different planar pushing systems: (top) pushing an object on a horizontal plane with a point pusher and (bottom) pushing an object in a grasp in the gravity plane. From left to right, the construction steps portray modeling of the forces involved in the task (force modeling), solving for the set of force-balance solutions in wrench space (force resolution), and finally computing the set of object twists corresponding to the force-balancing solutions (motion resolution). This set of object twists is the motion cone.

Figure 8(top) shows the computation of the motion cone for pushing an object on a horizontal support plane with a point pusher. As discussed in Section 4.1, from (11), when there is no external wrench other than the pusher wrench acting on the object, the motion cone computation simplifies and ${\mathit{W}}_{\mathrm{s}}$ and ${\mathit{V}}_{\mathrm{obj}}$ are polyhedral cones. In Figure 8(top) we observe that the motion cone is a polyhedral cone bounded by two twists ${\mathit{v}}_1$ and ${\mathit{v}}_2$ .

In more general cases where additional external forces, such as the gravitational force, are present in the manipulation plane, ${\mathit{W}}_{\mathrm{s}}$ and ${\mathit{V}}_{\mathrm{obj}}$ are not polyhedral cones as shown in Figure 8(bottom). In a general planar pushing task, they can be well characterized as cones defined by low-curvature surfaces that intersect all in a point and pairwise in lines. Figure 9 shows the wrench cone ${\mathit{W}}_{\mathrm{s}}$ and motion cone ${\mathit{V}}_{\mathrm{obj}}$ computed analytically for one particular grasp–pusher configuration.

Fig. 9.

The analytically computed wrench cone ( ${\mathit{W}}_{\mathrm{s}}$ ), motion cone ( ${\mathit{V}}_{\mathrm{obj}}$ ), and polyhedral approximation to the motion cone ( ${\overline{\mathit{V}}}_{\mathrm{obj}}$ ) for the configuration in Figure 3 and 45 N grasping force. Note that the surfaces defining the motion cone ( ${\mathit{V}}_{\mathrm{obj}}$ ) are curved.

5.2. Polyhedral approximation to the motion cone

As an object is pushed in a grasp, the position of the finger contacts in the object frame change, and consequently ${\mathit{J}}_{\mathrm{s}}$ and the motion cone ${\mathit{V}}_{\mathrm{obj}}$ also change. We need to compute the motion cone iteratively as the object moves in the grasp, which requires computational efficiency.

As the boundary surfaces of the motion cone have low curvatures, we propose a polyhedral approximation ( ${\overline{\mathit{V}}}_{\mathrm{obj}}$ ) of the motion cone for efficient computation. Each edge of the polyhedral approximation of the motion cone is the object motion corresponding to an edge of the generalized friction cone of the pusher. Figures 8 and 9 show example polyhedral approximations of the motion cone.

Procedure to compute polyhedral motion cone

1. Solve (10) and (13) simultaneously to get ${\overline{\mathit{w}}}_{\mathrm{s}}$ for ${\overline{\mathit{w}}}_{\mathrm{pusher}}$ corresponding to every edge of ${\mathit{W}}_{\mathrm{pusher}}$ .

2. Define the set of ${\overline{\mathit{w}}}_{\mathrm{s}}$ computed in step 1 as the generators/edges of the support/grasp wrench cone ${\overline{\mathit{W}}}_{\mathrm{s}}$ .

3. Map ${\overline{\mathit{W}}}_{\mathrm{s}}$ to the object twist space using (5) to obtain the polyhedral approximation ${\overline{\mathit{V}}}_{\mathrm{obj}}$ of the motion cone.

5.3. Experimental validation of the motion cone

Figure 10 shows our manipulation platform equipped with an industrial robot arm (ABB IRB120), a parallel-jaw gripper (WEISS WSG32), two geometric features in the environment that act as pushers, and a Vicon system (Bonita) for accurate object tracking.

Fig. 10.

Experimental setup used for the data collection for motion cone validation as well as for the regrasp examples in Sections 7 and 8.

The theory from Section 4, states that the pusher sticks to the object for the pusher twists inside the motion cone while the twists outside the motion cone will result in slipping at the pusher contact. To evaluate the experimental validity of the polyhedral approximation of the motion cone, we collected data for the slip observed at the pusher contact for a variety of experimental settings.

Using the rectangular prism object listed in Table 3, we conducted six experiments across three configurations as shown in the leftmost column of Table 2. The first two configurations use the side pusher and the grasp is offset from the center of mass of the object by $0$ mm and $-10$ mm respectively in the X direction. The third configuration uses a bottom pusher and the grasp is at the center of mass of the object.

Table 2.

We experimentally validated the polyhedral approximation of the motion cone across three configurations by characterizing thousands of random prehensile pushes by the slip observed at the pusher contact. The pushes that did slip are shown in gray, while those that did not are shown in black. For each configuration, we utilized a grasping force of $35$ N (top) and $45$ N (bottom). The percentages denote how many pushes lied inside the motion cone that did not slip and how many pushes outside the cone that did slip.

Experimental setup	Grasp Force	Inside the cone	Outside the cone	Prediction accuracy
				No slip	Slip
	35 N			90.2%	77.9%
	45 N			90.9%	77.6%
	35 N			82.9%	76.1%
	45 N			89.1%	73.0%
	35 N			76.5%	88.0%
	45 N			73.7%	85.0%

For each configuration, we conducted two experiments, first with a grasping force of $35$ N and then with $45$ N. In Table 2, the results are shown on the top and bottom, respectively, for each configuration. For each experiment we executed 1,000–1,500 randomly sampled pushes and labeled each push as “slipping” if the object moved with respect to the pusher/environment by more than 1.5 mm.

Owing to kinematic constraints of the robot and the workspace, we limit the sampled pushes to the space of $[0,10]\mathrm{mm}$ , $[-5,5]\mathrm{mm}$ , $[-35,35]$ degree displacements in $[X,Z,{\theta }_Y]$ when using the side pusher and $[-10,10]\mathrm{mm}$ , $[-8,0]\mathrm{mm}$ , $[-35,35]$ degree when using the bottom pusher.

For each experiment, Table 2 lists two accuracy percentages. The first denotes percentage of pushes that lie inside the motion cone that did indeed stick, as predicted. The second denotes the percentage of pushes that lie outside the motion cone that did slip, as predicted. The analytically computed polyhedral approximation to the motion cone classifies the stick/slip at the pusher contact and characterizes the feasibility of the object motions with an average accuracy of $\ge 80\%$ .

We observe that the mislabeled pushes are close to the boundaries of the motion cones. There could be a few reasons for these inaccurate predictions. The variation in the measured friction at the fingers or pusher contacts and variation in the grasping force can make the predictions close to the boundaries of the motion cone inaccurate. The motion cone construction builds upon the approximations of friction models at finger as well as pusher contacts and rigid-body assumptions. Therefore, any unmodeled dynamics and compliance at the contacts affect its validity, especially close to the boundary of the motion cone. Section 8 explores different strategies to make the predictions more reliable.

6. Planning in-hand manipulations viamotion cones

Motion cones abstract the algebra involved in solving the mechanics of pushing and provide direct bounds on the feasible object motions with pushing. For pusher motions inside the motion cone, the contact between the pusher and the object remains sticking, and the object follows the pusher. For pusher motions outside the motion cone, there is slipping at the pusher contact and the object motion is on the boundary of the motion cone. This mapping between the motions of the pusher and the object along with sticking/slipping contact mode resolution can be used for fast and efficient planning of pushing tasks.

In this section, we demonstrate the application of motion cones for planning in-hand manipulations with prehensile pushes. In particular, we constrain our planner to build manipulation strategies for which the pusher contact sticks to the object. Lynch and Mason (1996) studied pushing motions for which the pusher sticks to the object when pushing on a horizontal surface, which they referred to as stable pushing. We call our equivalent prehensile version stable prehensile pushing (Chavan-Dafle and Rodriguez, 2018b).

6.1. Problem formulation

An object is grasped in a parallel-jaw gripper and manipulated in the gravity plane by pushing it against contacts in the environment.

For the problem setup, we assume the following information about the manipulation system.

• Object geometry and mass.

• Initial and goal grasp of the object, specified by the locations of the finger contacts.

• Gripping force.

• Set of possible pusher contacts in the environment, specified by their geometries and locations in the object frame.

• Coefficients of friction of all contacts.

6.2. Planner framework

Our proposed planning framework works at two levels. At the high level, a T-RRT^*-based planning architecture samples the configuration space of different grasps, similar to earlier work in Chavan-Dafle and Rodriguez (2018b). At the low level, the planning tree is grown in the direction of the sampled grasp poses using the knowledge of locally reachable grasp poses in the form of motion cones.

For selective exploration, the T-RRT^* framework relies on a transition test that filters the sampled configurations to prefer exploration in low-configuration-cost regions (Devaurs et al., 2016; Jaillet et al., 2010). We define the configuration cost as the distance of the grasp from the goal. The transition test guides the stochastic exploration towards the goal grasp, while allowing the flexibility to explore high-cost transitions if they are necessary to get the object to the goal.

For effective connections, the T-RRT^* algorithm uses the underlying RRT^* (Karaman and Frazzoli, 2011) framework to make and rewire the connections in the tree at every step such that the local cost of the nodes is reduced when possible. We define the cost of a node as the sum of the cost of the parent node and the cost of the push to reach the sampled node from the parent node. We set the cost of a push to $0.1$ if the parent node uses the same pusher as the child and $1$ if there is a pusher switch-over. With our node cost definition, the planner generates pushing strategies that prefer fewer pusher switch-overs to push the object to the desired pose.

Let q denote the configuration of the object, i.e., the pose of the object in the gripper frame, which is fixed in the world. In this article, we consider planar manipulations in a parallel-jaw grasp, so the configuration space $\mathcal{C}$ is $[X,Z,{\theta }_y]\in {\mathbb{R}}^3$ , i.e., the object can translate in the grasp plane (XZ) and rotate about a perpendicular (Y) to the grasp plane.

Algorithm 1 describes our in-hand manipulation planner. Let $q_{\mathrm{init}}$ and $q_{\mathrm{goal}}$ be an initial and desired pose of the object in the gripper frame, respectively. The planner initiates a tree T with $q_{\mathrm{init}}$ and generates motion cones at $q_{\mathrm{init}}$ .

Algorithm 1. In-Hand Manipulation Planner
$\mathrm{input}:q_{\mathrm{init}},q_{\mathrm{goal}}$
$\mathrm{output}:$ tree $\mathcal{T}$
$\mathcal{T}\leftarrow \mathrm{initialize}\mathrm{tree}(q_{\mathrm{init}})$
$\mathrm{generate}\mathrm{\_}\mathrm{motionCones}(\mathcal{T},q_{\mathrm{init}})$
while $q_{\mathrm{goal}}\notin \mathcal{T}$ or cost( $q_{\mathrm{goal}})>\mathrm{cost}\mathrm{threshold}$ do
$q_{\mathrm{rand}}\leftarrow \mathrm{sample}\mathrm{random}\mathrm{configuration}(\mathcal{C})$
$q_{\mathrm{parent}}\leftarrow \mathrm{find}\mathrm{nearest}\mathrm{neighbor}(\mathcal{T},q_{\mathrm{rand}})$
$q_{\mathrm{sample}}\leftarrow \mathrm{take}\mathrm{unit}\mathrm{step}(q_{\mathrm{parent}},q_{\mathrm{rand}})$
if $q_{\mathrm{sample}}\notin \mathcal{T}$ then
if transition test $(q_{\mathrm{parent}},q_{\mathrm{sample}},\mathcal{T})$ then
$q_{\mathrm{new}}\leftarrow \mathrm{motionCone}\mathrm{\_}\mathrm{push}(q_{\mathrm{parent}},q_{\mathrm{sample}})$
if transition test $(q_{\mathrm{parent}},q_{\mathrm{new}},\mathcal{T})$ and
grasp maintained $(q_{\mathrm{new}})$ then
$q_{parent}^{\ast }\leftarrow \mathrm{optimEdge}(\mathcal{T},q_{new},q_{parent})$
$\mathrm{add}\mathrm{new}\mathrm{node}(\mathcal{T},q_{\mathrm{new}})$
$\mathrm{add}\mathrm{new}\mathrm{edge}(q_{parent}^{\ast },q_{new})$
$\mathrm{generate}\mathrm{\_}\mathrm{motionCones}(\mathcal{T},q_{\mathrm{new}})$
$\mathrm{rewire}\mathrm{tree}(\mathcal{T},q_{new},q_{parent}^{\ast })$

While the desired object pose is not reached within some cost threshold, a random configuration $q_{\mathrm{rand}}$ is sampled. A nearest configuration $q_{\mathrm{parent}}$ to $q_{\mathrm{rand}}$ in the tree T is found and a unit-step object pose $q_{\mathrm{sample}}$ from it towards $q_{\mathrm{rand}}$ is computed. Using the transition test, the planner evaluates if moving in the direction of $q_{\mathrm{sample}}$ from $q_{\mathrm{parent}}$ is beneficial. If it is, the motionCone_push routine computes an object configuration $q_{\mathrm{new}}$ closest to $q_{\mathrm{sample}}$ that can be reached using the motion cones at $q_{\mathrm{parent}}$ . It is further checked if moving towards $q_{\mathrm{new}}$ is beneficial and if $q_{\mathrm{new}}$ is an object configuration at which the grasp on the object is maintained. If both criteria are satisfied, $q_{\mathrm{new}}$ is added to the tree such that the local costs of $q_{\mathrm{new}}$ and the nodes near $q_{\mathrm{new}}$ are reduced. The motion cones are generated online for every new node added to the tree.

Two important routines in Algorithm 1, particularly for this article, are generate_motionCones and motionCone_ push.

generate_motionCones computes the polyhedral motion cones for a given object configuration in the grasp for all possible external pushers using the procedure listed in Section 5.2. At every node, we will have the same number of motion cones as possible pushers and each motion cone represents the set of object configurations that can be reached from the current configuration using a specific pusher.

motionCone_push finds an object pose closest to the desired sampled pose $q_{\mathrm{sample}}$ that is feasible to reach with pushing. This computation is done using the motion cones at the parent node $q_{\mathrm{parent}}$ . If the object twist needed from the parent node pose to the sampled pose is already inside any of the motion cones at $q_{\mathrm{parent}}$ , the sampled pose can be reached. If the required object twist is outside all the motion cones at $q_{\mathrm{parent}}$ , a twist that is closest to the desired twist and inside one of the motion cones is selected by projecting the sampled twist on the motion cones as shown in Figure 11.

Fig. 11.

Since the sampled grasp configuration is outside the motion cone, it projected onto the motion cone. The projected configuration is selected to grow the planning tree.

Motion cones define the set of feasible pushes or, in other words, the set of reachable object configurations. Using motion cones as reachability bounds directly in the configuration space allows us to rapidly explore the configuration space of different object poses in the grasp and generate dynamically feasible pushing strategies for the desired in-hand regrasp.

7. Regrasp examples and experimental results

We evaluate the performance of our proposed planner with examples of a parallel-jaw gripper manipulating a variety of objects listed in Table 3. The initial pose of an object in the gripper is treated as $[X,Z,{\theta }_Y]=[0,0,0]$ . Table 4 lists the goal poses (in [millimeters, millimeters, degrees]) for different examples. While there are no comparable available algorithms that can solve the type of regrasps we are interested in, we provide comparisons with our own implementations of the same high-level planner paired with different approaches to solve the mechanics of prehensile pushing. These include sampling with rejection by a feasibility check for stable prehensile pushing (Chavan-Dafle and Rodriguez, 2018b), and a more classical complementarity formulation (MNCP) that allows both sticking and slipping at the pusher contact (Chavan-Dafle and Rodriguez, 2017). We compare the performance in terms of planning time and the quality of the solutions. The planning times in Table 4 are the median times over 10 trials. All the computations are done in MATLAB R2017a on a computer with Intel Core i7 2.8 GHz processor.

Table 3.

Physical properties of the experimental objects.

Shape	Material	Dimensions $[L,B,H]$ (mm)	Mass (g)
Square prism	Al 6061	100, 25, 25	202
Rectangular prism	Delrin	80, 25, 38	113
T-shaped	ABS	70, 25, 50	62

Table 4.

Planning times (seconds) for approaches using motion cone, stable check (Chavan-Dafle and Rodriguez, 2018b), and MNCP (Chavan-Dafle and Rodriguez, 2017) for unit-step propagation.

Manipulation	Goal [ $X,Z,{\theta }_Y$ ]	Motion cone	Stable check	MNCP
Horizontal offset	20, 0, 0	0.45	2.83	592.8
$[X,Z,{\theta }_Y]$ regrasp	15, −13, 45	0.67	2.54	17,684
T-shaped	25, 17.5, 0	0.54	0.82	32,657

In all of the following examples, we assume three line pushers on the object, one on each of the side faces of the object parallel to the Z axis and one under the object parallel to the X axis. We use high friction line pushers, except in the first example, where we use low friction line pushers (Figure 12).

Fig. 12.

Simulation and experimental run for a pushing strategy to regrasp the aluminum object with low-friction pushers. In the simulation figure (top), the finger and pusher contacts are shown in green and magenta color, respectively.

7.1. Regrasping an object offset to the center

In this example, the goal is to regrasp the square prism horizontally 20 mm offset from the center. We use low-friction pushers first. Kolbert et al. (2016) showed that for a similar setting, if the object is pushed horizontally in the grasp, it slides down as it moves sideways in the grasp. For low-friction pushers, our planner generates a strategy where the object is first pushed up using the bottom pusher and then the side pusher is used to virtually keep the object stationary while the fingers slide up and along the length of the object as seen in Figure 12. This plan is similar to that found in Chavan-Dafle and Rodriguez (2017, 2018b).

When we replace the pushers with high-friction pushers (pushers with rubber coating), the planner estimates that the desired horizontal object motion lies inside the motion cone for the side pusher at the initial and all the subsequent grasp poses until the goal, i.e., simply pushing from the side is a valid pushing strategy as shown in Figure 13.

Fig. 13.

Simulation and experimental run for a pushing strategy to regrasp the aluminum object with high-friction pushers.

7.2. Regrasp in [ $X,Z,{\theta }_Y$ ]

The goal in this example is to regrasp the rectangular prism requiring a net object motion in all three dimensions [ $X,Z,{\theta }_Y$ ]. Similar to Chavan-Dafle and Rodriguez (2018b), our planner finds a strategy to achieve the regrasp using only one pusher. In fact, as we can see in Figure 14, the pushing strategy our planner generates is smoother and seems to avoid unnecessary object motions seen in the strategy shown in Chavan-Dafle and Rodriguez (2018b: Figure 1).

Fig. 14.

A pushing strategy for [ $X,Z,{\theta }_Y$ ] regrasp. In simulation, the direction of gravity remains constant in the pusher frame, because in real experiments, the pushers are fixed features in the environment.

7.3. Manipulating a non-convex object

In this example, the goal is to regrasp a T-shaped object. The goal pose is such that a greedy approach to push the object directly towards the goal will result in losing the grasp on the object. As shown in Figure 15, our planner comes up with a pushing strategy that respects the geometric constraints of the problem and moves the object to the desired pose without loosing the grasp.

Fig. 15.

Simulation and experimental run to manipulate a T-shaped object. Snapshots of the experimental run are as shown in Figure 2.

8. Robust in-hand manipulations viamotion cones

Sections 6 and 7 describe an application of motion cones to plan in-hand manipulations with stable prehensile pushes. In this section, we discuss constraining the planning tree to propagate through a conservative interior of motion cones for increased robustness. We show that we can exploit the structure of motion cones to derive manipulation strategies that are robust against uncertainties in the friction parameters at the finger and pusher contacts, grasping force, and mass of the object.

8.1. Robustness to friction variation at the pusher–object contact

For a given object mass, given friction at the finger contacts, and given grasping force, the motion cone is a function of the friction at the pusher. For a higher friction coefficient, the generalized friction cone is wider and so is the motion cone. There is a monotonic relationship between the friction at the pusher and the volume of the motion cone. This relationship can be exploited to derive pushing strategies robust to uncertainty in the friction at the pusher.

Proposition 2. Any object velocity inside the motion cone for a given pusher friction coefficient is inside the motion cone for a higher-friction coefficient at the pusher.

Proof. Let ${\mathit{W}}_{\mathrm{pusher}}$ be the generalized friction cone of a pusher and let ${\mathit{W}}_{\mathrm{pusher}}^{+}$ be the generalized friction cone for the same pusher but with a higher coefficient of friction. From the definition of the generalized friction cone in (6), it is a linear mapping of the Coulomb friction cone(s) at the constituent contact(s). A Coulomb friction cone for a given coefficient of friction is a subset of a friction cone with a higher coefficient of friction. Therefore, the generalized friction cone is also the subset of a generalized friction cone for a higher coefficient of friction:

$${\mathit{W}}_{\mathrm{pusher}}\subset {\mathit{W}}_{\mathrm{pusher}}^{+}$$

(14)

If an object motion is inside the motion cone, the net required wrench for the object motion lies inside the generalized friction cone of the pusher, i.e., repeating (12):

$$-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\mathrm{T}}·{\overline{\mathit{w}}}_{\mathrm{s}}-m\mathit{g}\in {\mathit{W}}_{\mathrm{pusher}}$$

(15)

From (14) and (15),

$$-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\mathrm{T}}·{\overline{\mathit{w}}}_{\mathrm{s}}-m\mathit{g}\in {\mathit{W}}_{\mathrm{pusher}}^{+}$$

(16)

Since the net required wrench lies inside the friction cone of the pusher, the object motion is inside the motion cone.

If there is uncertainty in the friction coefficient at the pusher, we can obtain a robust strategy by planning with the lower bound on the coefficient. The resulting strategy will produce the same net object motion in the grasp even if the true friction coefficient at the pusher is higher.

This property is also verified by our experiments from Section 5.3. For our experiment where we offset the grasp $10$ mm in the x-direction from the center of mass of the object and use a grasping force of $45$ N, our estimated friction coefficient at the pusher is $0.5$ . Given this coefficient, as stated in Table 2, $89.1\%$ of the pushes within the computed motion cone stick, as predicted. If we compute our motion cone with a coefficient of $0.25$ , then $95.6\%$ of the pushes lie within this reduced cone.

8.2. Robustness to friction variation at the gripper–object contact

For a given friction coefficient at the pusher, the motion cone is a function of the object mass and the friction coefficient at the finger contacts, and the grasping force.

Unlike the monotonic relationship of the motion cone to the friction at the pusher, the dependence of the motion cone on the friction force at the fingers is not straightforward. For increased friction at the fingers, either by increasing the grasping force or by increasing the coefficient of friction, we do not necessarily obtain a wider motion cone. Instead the motion cone shifts towards the gravity-free motion cone as shown in Figure 16. At infinite friction at the gripper the motion cone is equal to the gravity-free motion cone as shown in Corollary 1.1. Therefore, some of the object motions inside the motion cone, particularly those that are feasible by exploiting gravity, may no longer be feasible.

Fig. 16.

The motion cone shifts towards the gravity-free motion cone as the friction at the gripper increases due to the increased grip force. The intersection of the gravity-free motion cone and the motion cone is shown by the dotted region for the two grip forces. The motions inside the motion cone that are outside the gravity-free motion cone are feasible only by exploiting the gravitational force on the object (in Z direction). For higher friction at the gripper than expected, these motions may no longer be feasible. One of such motions that is shown in red falls outside the motion cone when the grip force is increased. All the motions inside the intersection however, will always be feasible for the higher than expected friction at the gripper.

We propose two approaches to address this problem and to achieve robust pushing strategies under the uncertainty in the friction at the fingers.

8.2.1. Modified motion cone

In this approach, we find a subset of a motion cone such that any object motion inside it will always be feasible with any friction at the gripper higher than a chosen value.

Proposition 3. Any motion inside the intersection of the gravity-free motion cone and the motion cone, for a given friction at the gripper, will always be inside the motion cone for a higher friction at the fingers and/or a higher grasping force.

Proof. If an object motion is inside the motion cone, from (12), the net required wrench is inside the generalized friction cone of the pusher:

$$-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }{\overline{\mathit{w}}}_{\mathrm{s}}-m\mathit{g}\in {\mathit{W}}_{\mathrm{pusher}}$$

(17)

If the object motion is inside the gravity-free motion cone:

$$-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }{\overline{\mathit{w}}}_{\mathrm{s}}\in {\mathit{W}}_{\mathrm{pusher}}$$

or, put simply,

$$-{\mathit{J}}_{\mathrm{s}}^{\top }{\overline{\mathit{w}}}_{\mathrm{s}}\in {\mathit{W}}_{\mathrm{pusher}}$$

(18)

Let ${\mu }_{\mathrm{s}}^{+}$ be a higher friction coefficient at the fingers and let $N^{+}$ be a higher grasping force. The net wrench required for the same object motion under the higher finger friction conditions is

$$-{\mu }_{\mathrm{s}}^{+}{N}^{+}{\mathit{J}}_{\mathrm{s}}^{\top }{\overline{\mathit{w}}}_{\mathrm{s}}-m\mathit{g}$$

which can be rewritten as

$$-k{\mathit{J}}_{\mathrm{s}}^{\top }{\overline{\mathit{w}}}_{\mathrm{s}}-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }{\overline{\mathit{w}}}_{\mathrm{s}}-m\mathit{g},k\in {\mathbb{R}}^{+}$$

(19)

From (17), (18), and (19):

$$-k{\mathit{J}}_{\mathrm{s}}^{\mathrm{T}}{\overline{\mathit{w}}}_{\mathrm{s}}-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\mathrm{T}}{\overline{\mathit{w}}}_{\mathrm{s}}-m\mathit{g}\in {\mathit{W}}_{\mathrm{pusher}}$$

Since the net required wrench lies inside the friction cone of the pusher, the object motion is inside the motion cone.

In the case of uncertainty in the friction at the gripper, to generate robust pushing strategies, we can constrain the planner to the intersection motion cones computed with lower bounds on the friction coefficients and the gasping force. Such a pushing strategy will produce the same regrasp outcome for the expected variation in the friction at the gripper.

8.2.2. Gravity-aligned pusher

In recent work (Chavan-Dafle and Rodriguez, 2018a), we showed that we can generate robust prehensile pushes in the presence of gravitational force by careful selection of pusher contact placement. Specifically, we demonstrated that by aligning the pusher normal along the gravitational force on the object, the dynamics of pushing and the motion cone becomes invariant to all the system parameters except for the friction at the pusher.

Proposition 4. The motion cone for gravity-aligned pushers is invariant to the object mass, grasping force, and friction coefficient at the finger contacts.

Proof. If the pusher is aligned such that the pusher contact normal is along the direction of gravity, the gravitational force on the object is entirely balanced by the part of the normal force at the pusher. Then, for ${\overline{\mathit{w}}}_{\mathrm{pusher}}\in {\mathit{W}}_{\mathrm{pusher}}$ , the dynamic check for a feasible prehensile push (9) becomes

$$\begin{array}{c}-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}-m\mathit{g}=k{\overline{\mathit{w}}}_{\mathrm{pusher}}\hfill \\ -{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}-k_2{\overline{\mathit{w}}}_{\mathrm{pusher}\mathrm{\_}\mathrm{n}}=k{\overline{\mathit{w}}}_{\mathrm{pusher}}\hfill \\ -{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\top }·{\overline{\mathit{w}}}_{\mathrm{s}}=k{\overline{\mathit{w}}}_{\mathrm{pusher}}+k_2{\overline{\mathit{w}}}_{\mathrm{pusher}\mathrm{\_}\mathrm{n}}\hfill \end{array}$$

where $k,k_2\in {\mathbb{R}}^{+}$ and ${\overline{\mathit{w}}}_{\mathrm{pusher}\mathrm{\_}\mathrm{n}}$ is the unit normal wrench at the pusher contact. Since ${\overline{\mathit{w}}}_{\mathrm{pusher}\mathrm{\_}\mathrm{n}}\in {\mathit{W}}_{\mathrm{pusher}}$ , we can rewrite the previous constraint as

$$-{\mu }_{\mathrm{s}}N{\mathit{J}}_{\mathrm{s}}^{\mathrm{T}}·{\overline{\mathit{w}}}_{\mathrm{s}}=k_3{\overline{\mathit{w}}}_{\mathrm{pusher}},k_3\in {\mathbb{R}}^{+}$$

where ${\overline{\mathit{w}}}_{\mathrm{pusher}}$ is different from before, but also in ${\mathit{W}}_{\mathrm{pusher}}$ . As ${\mu }_{\mathrm{s}}$ and N are scalar constants, we further simplify it as

$$-{\mathit{J}}_{\mathrm{s}}^{\mathrm{T}}·{\overline{\mathit{w}}}_{\mathrm{s}}=k_4{\overline{\mathit{w}}}_{\mathrm{pusher}},k_4\in {\mathbb{R}}^{+}$$

(20)

Now, we can write the dynamics condition for a feasible prehensile pushing with gravity balancing pushers as

$$-{\mathit{J}}_{\mathrm{s}}^{\mathrm{T}}·{\overline{\mathit{w}}}_{\mathrm{s}}\in {\mathit{W}}_{\mathrm{pusher}}$$

(21)

Equations (20) and (21) show that the dynamics of pushing, and the consequent motion cone for the gravity-aligned pusher, are invariant to the object mass, grasping force, and the friction coefficient at the fingers.

As discussed in Section 8.1, we can further make the pushing strategies robust to the uncertainty at the pusher contact by planning with lower bounds on the expected coefficient of friction at the pusher.

Figures 17, 18, and 19 show the regrasps in Table 4 planned with the constraint of using only the bottom pusher which is aligned along the direction of gravity.

Fig. 17.

Simulated motion of the object and snapshots of the experimental run for a pushing strategy to offset the object in the grasp using low-coefficient pushers.

Fig. 18.

Simulated motion of the object and snapshots of the experimental run for a general regrasp in [ $X,Z,{\theta }_y$ ].

Fig. 19.

Simulated motion of the object and snapshots of the experimental run for the T-shaped object.

To validate the invariance in the outcome of the regrasp actions planned when the friction at the fingers is changed, we varied the grasping force and executed the same pushing strategy. As the grasping force changes, the friction force at the fingers also change. However, as shown in the plots in Figures 20 and 21, the variation in the outcome of the pushing strategy for the three different grasping forces is negligible. This supports the expected invariance. Moreover, the minimal slip observed at the pusher contacts confirms that the pushing actions selected by the planner result in stable prehensile pushes.

Fig. 20.

Displacement of the object with respect to the environment for the regrasp action shown in Figure 17. As expected, during pushing, the object sticks to the environment and moves by a negligible amount as the fingers slide on it.

Fig. 21.

Displacement of the object with respect to the environment for the regrasp action shown in Figure 18.

It should be noted that using conservative friction values at the pusher as proposed in Section 8.1 or modified motion cones as in Section 8.2.1 provides robustness, however, it reduces the volume of the effective motion cones, i.e., the dynamically reachable configuration space. This can adversely affect planning time, especially for problems with “narrow corridors.” The method proposed in this section, however, does not suffer from such an effect since the volume of the motion cones in this case does not change significantly. We observe negligible (fractions of a second) difference in the planning times for the examples considered in this section compared with those in Section 7.

9. Discussion

A motion cone is the maximal set of dynamically feasible velocities that a pusher can transmit to a pushed object. It also describes the set of motions of the pusher that yield sticking contact with the object during the push. It abstracts away the dynamics of frictional pushing and provides direct bounds on the reachable space for pushing tasks.

In this article, we have extended the concept of motion cones to a general set of planar pushing tasks with external forces such as the gravitational force in the plane of motion. We have shown that the motion cone for a general planar push is defined as a cone with low-curvature facets, and have proposed a polyhedral approximation for efficient computation.

We have demonstrated the use of motion cones as the propagation step in a sampling-based planner for in-hand manipulation. Combining a T-RRT^*-based high-level planning framework and a motion cone-based dynamics propagation, the planner builds in-hand manipulation strategies with sequences of continuous prehensile pushes in a fraction of a second. Furthermore, we have shown that by constraining the planner to propagate the planning tree through a conservative interior of the motion cones and/or with specific selection of the pusher alignment, we can generate pushing strategies that are robust against uncertainty in the physical parameters of the system.

Throughout this article, we have emphasized the experimental validation of the theoretical contribution and planning framework. Thousands of pushes in different object–grasp–pusher configurations corroborate the analytical computation of motion cones. With a variety of regrasp experiments, we have shown that the pushing strategies generated by our planner result in motions with minimal slip at the pusher contact and negligible variation in the outcome of the regrasp.

The motion cone provides direct knowledge of the set of reachable configurations. It allows us to explore the connectivity of the configuration space for planning through regions/volumes of the configuration space for contact-rich manipulations (Brock and Kavraki, 2001; Morales et al., 2007; Shkolnik and Tedrake, 2011; Shkolnik et al., 2009). Moreover, motion cones, as bounds on pusher actions or as bounds on the object motions, have an adequate form to be incorporated into trajectory optimization frameworks to plan pushing strategies. We believe that the extension and application of motion cones to more general settings provides new opportunities for fast and robust manipulation through contact.