Sage Journals: Discover world-class research

Abstract

The design of a robot system for assembling numerous parts, including screws, washers, nuts, and other irregular parts, is important in manufacturing. Although a robot can flexibly manipulate various parts with the aid of vision and force sensors, some challenges remain, for example, precise orientation and precise insertion when uncertainty exists in the positioning and manipulation of irregular objects. In this work, we present a method to design the orienting strategy for positioning the irregular parts based on the constraints formed by the fixture, thus the irregular parts such as the motor can be precisely localized only with the robot squeezing actions. We then develop a peg-in-hole insertion strategy using the force constraints formed by the environment, where fewer robot actions are required to insert the peg, thus enabling efficient and precise peg-in-hole insertions. Finally, we developed a robot assembly system with three robots from Universal Robots to perform all the assembly tasks, including picking, holding, placing, and inserting. Experiments on the assembly of a belt drive unit reveal the efficiency of the proposed method.

Keywords

Assembly object reorientation force control

Introduction

Robot assembly shows excellent performance in manufacturing because it reduces the proportion of defective products.¹ In the past 30 years, robot assembly strategies combining vision and force sensors have been widely studied.

In a robot assembly system, vision is usually employed to estimate the object’s pose for picking. For example, Liu et al.¹ calculated the target position by applying a fast directional chamfer matching algorithm; hence, the target was selected based on visual information. Dupuis et al.² discussed the three-dimensional vision-based picking of a randomized connecting rod. Sansonia et al.³ developed a laser vision system for picking up piled objects. Su et al.⁴ developed a vision-based bin-picking system for picking and reorienting piled objects and constructed a constraint region to design the reorienting strategy. Domae et al.⁵ created a segment-based stereovision approach based on a versatile volumetric vision system for assembly. This approach was used to determine and monitor the location and orientation of items with curved surfaces. Although many vision-based positioning algorithms have been proposed to guide robot manipulation in complex environments, problems still exist in assembly tasks. For example, the current pose of the object is possibly not suitable for insertion. Thus, the robot must reorient the object into the expected goal pose. Although passive compliant motion strategies, such as regrasping,⁶ reorienting,⁴ or feeding,⁷ have been proposed to deal with the reorientation problem, some of them need to generate sequential robot postures for iterative object reorientation, which is time-consuming.

Once the object’s pose has been obtained, researchers have employed a force-based controller to guide the robot’s motions. Xie et al.⁸ described the assembly of a miniaturized gear using a hybrid vision–force control method. They employed visual feedback to coarsely locate the parts and guide grasping and then developed a fuzzy proportional–integral–derivative controller to control the contact force. Zheng et al.⁹ achieved a peg-in-hole assembly using hybrid vision–force guidance, where the rough adjustment of the robot was guided by an eye-to-hand vision system, and the position and orientation were precisely adjusted as per the force and torque information, respectively. They experimented with a maximum clearance of 0.5 mm between the peg and hole. Ma et al.¹⁰ studied the assembly of two small cylindrical components, where two microscopic cameras were employed to provide visual information in the aligning stage for assembly, and a micro force sensor was used to perceive the contact forces in the insertion stage. Gorjup et al.¹¹ calculated the target position using one camera and a three-dimensional computer-aided design model and then employed a force controller to compensate for errors in calibration or positioning. Kim et al.¹² presented a vision–force-guided assembly system using a force sensor and two cameras. The target pose was estimated from a two-dimensional image, and the object was assembled with the proper force. Wu et al.¹³ used an implementation approach fusing signals of vision detection and fuzzy force to realize peg-in-hole insertion. YOLO v.3 was applied as the visual detection network for rough alignment, and a semi-supervised learning network was used to optimize the hole-searching routine.

Irregular objects, which have special shapes and are difficult to fix or orient to the desired pose, are widely used in manufacturing. For example, the “L shape” holder plate and the eccentric shaft motor shown in Figure 1 are typical irregular parts. In assembly, orienting an irregular object such as an eccentric shaft motor to the desired pose for robot insertion is a difficult task. Although various reorienting methods had been proposed to deal with the reorientation problem, most of them needed to generate sequential robot postures for iterative object reorientation. And, in the inserting stage, force-based robot control methods usually perform the insertion by precisely adjusting the object’s position and orientation as per the force and torque information, where the complex adjustments are time-consuming and suffer from the resolution of the force sensor and the noise on the sensor. To solve the problems,

We present a method to design the orienting strategy for positioning the irregular part using the constraints formed by the potential energy of the fixture, thus the irregular part such as the motor can be precisely localized only with the robot squeezing actions and the fixture. That is, the fixture can reorient the motor to a determined pose that corresponds to the minimum of the constraints.

We develop a peg-in-hole insertion strategy using the force constraints formed by the environment, where several simple robot actions are required to insert the peg. The force constraints describe the relationships between the contact forces and peg-hole states, and the minimum of the constraint region relates to the state in which the peg has been inserted into the hole. Thus, we can precisely and quickly insert the peg into the hole by pushing the peg-hole state approach to the bottom of the force constraints with fewer robot actions and coarse sensing.

We design a robot assembly system with three robots to perform precision tasks, including peg-in-hole insertion, screw fixing, and irregular plate fixing.

The remainder of this paper is organized as follows. The second section describes the development of the assembly system. The third section introduces the object reorientation strategy. The fourth section presents the assembly strategy that combines compliant and sensor-based motions. The fifth section provides the experimental results of the assembly of a belt drive unit. The sixth section presents directions for future studies.

System overviews

Problem statement

The design of a flexible robot system for performing complex assembly tasks in which various parts including irregular parts were automatically assembled by robots remains challenging. The belt drive unit shown in Figure 1 was the challenging task in WRS 2020 competition,¹⁴ which consisted of small parts and irregular parts such as threaded screws, soft parts, gear units, washers, and nuts. The parts tray on the left supplied the parts used in the task, while the screws were placed on the right tray. The pose of all parts should be estimated by the camera mounted on the robot. The holder plate should be firstly fixed on the workbench. Then one robot arm should hold the plate, while another robot arm used the screwdriver to fix it on the workbench screws. The motor was required to be fixed on the plate using six screws. Thus, a small positioning error or an excessive force caused by misalignment in the assembly would damage both the parts and the robot system, even causing the failure of the task. To be concluded, the difficulties of the assembly of the belt drive unit were (a) a variety of parts with different shapes and different characteristics needed to be oriented and assembled precisely, and complex manipulations such as screwing screws and assembling belts should be performed. (b) The assembly required more than one robot arm to cooperate with the manipulations. (c) The assembly task was with high-precision requirements, for example, the clearance between the peg and the hole is 0.03 mm.

Figure 1.

(a) shows the parts for assembling, and (b) shows the assembled belt drive unit.

2.2 System design

We designed a flexible assembly system using three universal robots to perform the assembly of the belt drive unit, as illustrated in Figure 2. The assistant robot (UR-1) was used to pick the workpiece. The assembly robot (UR-2) was equipped with a force/torque sensor and used to insert the peg into the hole. The screwing robot (UR-3) was used to fasten the screw with the help of electric screwdrivers. Note that we choose to use UR robots as they are lightweight robotic arms and use less electricity than traditional industrial robots. And, UR robots are re-deployable and fast to reprogram, giving us the flexibility to change setup. Thus, we select the UR robots to perform the assembly in this work. Of course, other kinds of robots also be used to assemble the task with the proposed method. Charge-coupled device (CCD) cameras were used to capture images of the worktable and then send them to the host computer. Thus, the pose of the target workpiece on the worktable can be estimated by using the template-matching method.

Figure 2.

Assembly system with three UR robots: (a) model of the system and (b) real system.

A robot-layered software structure was designed for assembly. It contained five layers, as shown in Figure 3. The hardware layer provided the hardware components, including the three UR robots, electric screwdrivers, three types of grippers, an air pump, cameras, and a force sensor. The driver layer provided the drivers of the hardware, which included RTDE and URX libraries for the driving commands of the robots from Universal Robots. In addition, drivers for cameras, force sensors, and Robotiq grippers were presented. The planning layer contained control and path-planning algorithms, used to achieve robot position and compliance control. A multiarm cooperative planner lies in this layer. It was used to plan the trajectory of the three robots and prevent collisions. The algorithm layer included the template-matching algorithm for target recognition, task information for specific assembly task recognition, and the force control algorithm for performing force-guided compliant motion in the constraint region. The user layer was realized based on Qt, which was used for convenient human–computer interaction to complete different assembly tasks.

Figure 3.

Software architecture.

Object pose reorientation

Initial pose estimation using vision

The first important step of the flexible assembly was to locate the objects with the image and then detect the geometry properties of the parts. The vision system not only needed to identify each part on the worktable but also needed to precisely estimate the position of the hole on the part. That is, the vision system had the requirement of the field of view and the detection precision. Thus, we employed two types of cameras, where CCD cameras mounted on the end-effector of the robot arm the system was employed to detect the pose and size of the parts. Once the part’s pose had been detected, the position of the holes on the part for screwing also can be obtained. In this work, we used the shape information to identify the parts from the image and used the pyramid-based image matching method^15–17 to speed up the search process. The estimating process is given in Figure 4.

Figure 4.

Pose estimation using pyramid-based image matching method.

The pose estimation algorithm first built the tree structure of the pyramid model to layer the template. A template that achieved a certain similarity was classified as a class, which was denoted by “aspect.” We stored the “aspect” on the next floor of the pyramid and then calculated the image similarity again until the final pose of the target was obtained on the top of the pyramid. Based on the pyramid-based image matching method, we matched the input image with the generated hierarchical shape template. The hierarchical matching method would improve the matching efficiency. Figure 5 gives some examples of shape-matching results, where the target objects were, respectively, matched to their shape template in the library. Note that the time cost for the pose estimation of the “L shape” holder plate and other shapes is about 100 ms.

Figure 5.

The results of shape matching.

Pose reorientation using fixture constraints

In some cases, the current pose of the object was not the desired pose for the robot’s next task; hence, we needed to reorient the pose of the object. Given the current and desired poses of an object, the goal of reorientation is to determine a sequence of robot manipulations that change the object from its current pose to the desired pose. Reorientation is an important skill when a robot cannot directly pick an object with the desired pose owing to kinematic constraints and collisions. In our previous work,¹⁸ we developed grasping algorithms to eliminate the uncertainties of the object inside the gripper by applying the constraints formed by the gripper. However, orienting an object to the desired pose in only one grasp was difficult. As shown in Figure 6, the gripper has difficulty grasping an eccentric shaft motor so that it attained a pose in which all six holes of the motor should be aligned with the corresponding holes on the holder.

Figure 6.

Eccentric shaft motor fixed on the plate, where six threaded screws were used to fasten the motor on the holder.

In this work, we designed a fixture and a robot action sequence to reorient the motor to the desired pose using the constraints formed by the potential energy of the fixture. Note that we also designed another fixture to reorient the “L shape” holder. We used a vision system to capture the image of the motor and the holder, estimated the initial pose of the motor using a shape detection method, and identified the hole of the holder using an ellipse detection method. The gripper was then guided to pick up the target part from the worktable. The key problem in this stage was to grasp the motor in the desired position for alignment with the hole on the holder. Therefore, the motor should be reoriented to the desired position for insertion. In this section, we discussed a method for reorienting an eccentric shaft motor using the constraints formed by the fixture. We used the information about the distance between the two parallel jaws upon closure to distinguish the orientation. We briefly described the design of the fixture used to reorient the eccentric shaft motor.

As shown in Figure 7(a), the eccentric shaft of the motor was supported by the supporting point, where the gravitational force decreases the pitch angle, $β$ . The motor can rotate around its eccentric shaft under squeeze forces F ₁ and F ₂, as shown in Figure 7(b); thus, the rotational angle, α, should be decreased.

Figure 7.

(a) Eccentric shaft motor, where d denoted the height of the center of the motor, and $α \in [0, π / 6]$ is the angle between the shaft of the motor and the support plate. (b) Orthographic projection of the motor, where F ₁ and F ₂ are the squeezing forces of the fixture, and $β \in [- π / 6, π / 6]$ is the angle of the motor around its eccentric shaft.

A potential energy function was defined as

E_{p} = R \cdot \frac{(1 - cos β)}{cos α} \cdot G

where R was the radius of the motor, G was the gravitational force on the motor, $α \in [0, π / 6]$ was the angle between the shaft of the motor and the support plate, and $β \in [- π / 6, π / 6]$ was the angle of the motor around its eccentric shaft.

By applying (1), we can build a graph of E_p , as shown in Figure 8. When squeeze and gravitational forces were applied to the eccentric shaft motor, the state of the motor approached the bottom of the graph. As the orientation of the eccentric shaft motor was fixed at the bottom of the graph, the eccentric shaft can be precisely inserted into the hole on the holder, as shown in Figure 9(d). In this case, the six threaded holes of the eccentric shaft motor aligned with the related hole on the holder and then were fastened by using threaded screws. Figure 9(a) to (c) shows the process of picking the motor, placing it on the fixture, and squeezing the fixture by using the jaws of the two-fingered gripper.

Figure 8.

Graph of the potential energy function of the motor inside the reorienting device.

As shown in Figure 9, the support base was composed of a shaft supporter and two semicircular squeezing plates. The supporter was used to locate the position of the eccentric shaft, and blocks were applied to eliminate the pose uncertainties of the motor, as shown in Figure 9(a). When the gripper squeezes the blocks, the rotation and movement of the eccentric shaft motor were restrained, as shown in Figure 9(b).

Figure 9.

(a) Robot picking and placing the part in the reorienting device, (b) motor support point device for reorienting the motor, (c) gripper squeezing the blocks for fixing the motor, and (d) motor aligned with the hole.

Figure 9(b) to (d) shows the process of placing the motor on the fixture, reorienting the motor, and picking the block to restrain the motor. We were able to use vision to determine whether the six holes on the motor aligned with the six corresponding holes on the holder.

Precise alignment using force control

We previously developed a robot assembly strategy using active and passive compliant motions when the initial pose of the shaft was unknown.¹⁹ However, the assembly process with the two compliant motions was time-consuming because the controlled motion was designed in two sub-configuration spaces, and the insertion direction was downward. In this work, we discuss a more general case in which the direction of insertion was not perpendicular to the earth.

Once the rough pose of the peg was estimated visually, we applied an active compliant control method to perform robot insertion precisely and efficiently. We designed a force-guided compliant motion in the constraint region, in which only several contact states are estimated from the force information to guide the robot’s motion. Thus, we can quickly and precisely insert the nut into the hole with fewer robot actions and coarse sensing (Figure 10).

Figure 10.

A nut was inserted into the hole with fewer robot actions and coarse sensing.

We analyzed the contact forces and frictional forces at the two-point contact states, as shown in Figure 11. We assumed that the two symmetric points on the end surface of the peg are C ₁ and C ₂, $F_{N_{1, 2}}$ were, respectively, the supporting forces on the peg that point to O_p at contact points $C_{1, 2}$ , and F_f was the frictional force. In addition, we assumed that r is the radius of the hole, and $r_{1, 2}$ were, respectively, the distances from contact points $C_{1, 2}$ to the X_h -axis. Here, O_h was the center of the upper surface of the peg, O_p was the center of the end surface of the peg, $O_{p}^{′}$ was the projection of O_p on the $X_{h} O_{h} Y_{h}$ -plane, and (d) showed the contact forces between the peg and hole.

Figure 11.

(a) Coordinate frame built on the hole, where O_h was the center of the upper surface of the hole and O_p was the center of the end surface of the peg, (b) contact forces between the peg and the hole, (c) projection of the end surface of the peg, where the red circle was the end surface of the peg, black circle was the upper surface of the hole, dotted red elliptic was the projection of the end surface of the peg on the $X_{h} O_{h} Y_{h}$ -plane, O_h was the center of the black circle, O_p was the center of the elliptic, and $O_{p}^{′}$ was the projection of O_p on $X_{h} O_{h} Y_{h}$ -plane, and (d) projection of contact forces.

If the active force was applied by the robot along the −Z_h -axis, we can obtain

F_{z} = μ (F_{N_{1}} + F_{N_{2}}),

F_{x} = F_{N_{1}} cos α_{1} + F_{N_{2}} cos α_{2},

F_{y} = - F_{N_{1}} sin α_{1} + F_{N_{2}} sin α_{2},

where $μ$ was the friction coefficient between the peg and the hole.

We assumed that the pitch angle of the peg related to the Z_h -axis was $θ$ , the position of O_p on the $X_{h} O_{h} Y_{h} Z_{h}$ frame is ( $x_{o p}, y_{o p}, z_{o p}$ ), and the position of $O_{p}^{'}$ on the $X_{h} O_{h} Y_{h}$ -plane was ( $x_{o p}, y_{o p}$ ). Thus, the two contact points, denoted by ( $x_{1}, y_{1}$ ) and ( $x_{2}, y_{2}$ ), can be calculated as

{\begin{matrix} x^{2} + y^{2} = r^{2} \\ \frac{{(x - x_{o p})}^{2}}{{(r cos θ)}^{2}} + \frac{{(y - y_{o p})}^{2}}{r^{2}} = 1 \end{matrix} .

Here, $θ$ can be set to a small value because the initial pitch angle of the peg is small. It can be adjusted by performing a rough alignment guided by vision. We can compute $α_{1}$ and $α_{2}$ as

α_{1} = {tan}^{- 1} (| y_{1} - y_{o p} | / | x_{1} - x_{o p} |) and α_{2} = {tan}^{- 1} (| y_{2} - y_{o p} | / | x_{2} - x_{o p} |) .

Hence,

cos α_{1} = | x_{1} - x_{o p} | / r and cos α_{2} = | x_{2} - x_{o p} | / r .

Substituting (7) into (3) yields

F_{x} = F_{N_{1}} \frac{| x_{1} - x_{o p} |}{r} + F_{N_{2}} \frac{| x_{2} - x_{o p} |}{r} .

For simplicity, we assumed that $F_{N_{1}} and F_{N_{2}}$ were constant values and then defined the contact force function as

F_{x} = g_{x} (x_{o p}, y_{o p}) and F_{y} = g_{y} (x_{o p}, y_{o p}) .

Because $θ$ was set to a small value, $cos θ \approx 1$ , and we can calculate $x_{1, 2}$ from (5). We assumed that $F_{N_{1}} = F_{N_{2}} = 1 N$ ; thus

F_{x} = g_{x} (x_{o p}, y_{o p}) = \frac{| x_{1} - x_{o p} |}{r} + \frac{| x_{2} - x_{o p} |}{r} .

The graph of $| F_{x} |$ relating to $x_{o p}$ and $y_{o p}$ was displayed in Figure 12. It formed a force constraint region along the X_h -axis. In other words, the contact force along the X_h -axis approaches its minimum when $O_{p}^{'}$ was on the X_h -axis ( $x_{o p} = 0$ ).

Figure 12.

Graph of $| F_{x} |$ related to $x_{o p}$ and $y_{o p}$ .

Similarly, we can establish the following relationship

F_{y} = - F_{N_{1}} sin α_{1} + F_{N_{2}} sin α_{2} = - \frac{| y_{1} - y_{o p} |}{r} + \frac{| y_{2} - y_{o p} |}{r} .

The graph of $| F_{y} |$ relating to $x_{o p}$ and $y_{o p}$ was presented in Figure 13.

Figure 13.

Graph of $| F_{y} |$ related to $x_{o p}$ and $y_{o p}$ .

Figures 12 and 13 show that forces F_x and F_y decrease if O_p approaches O_h . In Figure 12, F_x reached its minimum when $x_{o p}$ equaled zero, indicating that the uncertainties of the peg position along the X_h -axis had been eliminated. Similarly, in Figure 13, F_y reached its minimum when $y_{o p}$ equaled zero, indicating that the uncertainties of the peg position along the Y_h -axis had been eliminated. Therefore, we can design a simple force controller to achieve this goal using the force constraints.

Figure 14.

Force-based assembly process.

We considered each Cartesian variable independently so that the control input to the robot can be written as follows

{\begin{matrix} x (t + 1) = x (t) + sign (F_{x} (t)) . σ x, i f | F_{x} (t) | > F_{x d} (t) \\ y (t + 1) = y (t) + sign (F_{y} (t)) . σ y, i f | F_{y} (t) | > F_{y d} (t) \end{matrix},

where $x (t + 1) and y (t + 1)$ were, respectively, the control outputs to the robot end effector along the X_h - and Y_h -axes, small values of $σ x < 0$ and $σ y < 0$ are selected, and $F_{x d}$ and $F_{y d}$ were the desired measure forces related to the bottom of the constraint region.

Based on (12), the force-guide assembly process is provided in Figure 14.

Experiments

Figures 15 to 18 showed the entire process of the assembly of the belt drive unit using the developed assembly system. The vision system first identified the target part from the worktable and then estimated its pose in the robot coordinate frame. In this study, the shape information was used to identify the parts from the image, and the pyramid-based image-matching method was employed to accelerate the search process.^15,16 Estimating the pose of the rectangular parts was easy because they have some typical features, such as edges and circles. In this work, a camera mounted on UR3-1 was used to determine whether the threaded holes on the L-shaped part and the peg were aligned. Then, UR3-2 mounted on the screwdriver was used to turn the screws one by one.

As shown in Figure 15(a), the vision system mounted on the robot end effector identified and estimated the position of the screw holes; then, UR3-1 holds the plate, and another UR3-2 uses the screwdrivers to fasten the screws. In Figure 15(b), the eccentric shaft motor was assembled using the method described in the third section. UR3-1 inserts the shaft motor into the holder, and UR3-2 uses an electric screwdriver to fasten the screw.

Figure 15.

(a) Fixing the plate on the workbench with two robots; (b) fixing the motor on the plate with two robots.

As displayed in Figure 16(a) and (b), UR3-1 inserts the nut into the hole and then UR3-2 uses screwdrivers to fasten the screws under the guidance of the vision system. In this step, the force controller proposed in Figure 14 was used to achieve precise alignment. Note that the clearance between the nut and hole is approximately 0.03 mm and is hard to be inserted only with a position-control approach.

Figure 16.

(a) Inserting the nut into the hole; (b) fixing the nut on the plate with screws.

The contact force measured in the assembly of the peg is shown in Figure 17, where the green, orange, and blue lines represent the measured forces along x -, y-, and z-directions, respectively. For example, the contact force in the X_h -the direction was initially large, indicating that the large position error on the X_h -axis should be decreased. Thus, the peg moves along the $- X_{h}$ -axis to decrease the position error based on (12) until the contact force was less than the preset threshold.

The positions of the peg on the coordinates of the hole are shown in Figure 17(b) and (c). The peg moved along the X_h - or Y_h -direction according to the measured forces, that is, the peg moved along −X_h if $F_{x} > F_{x d}$ and along −Y_h if $F_{y} > F_{y d}$ . Similarly, the peg moved along X_h if $F_{x} < - F_{x d}$ and along Y_h if $F_{y} < - F_{y d}$ , where $F_{x d}$ and $F_{y d}$ were preset thresholds. The solid line was the edge of the hole, and the dotted and dashed lines represent the positions of the peg before and after adjustment, respectively. Note that the proposed method with fewer steps to insert the peg because the adjustments of the peg’s pose only need coarse force information feedback while the constraint of the hole will eliminate the pose uncertainties of the peg in adjustment.

Figure 17.

(a) Measured forces in the assembly, (b) peg movement along X_h -axis, and (c) peg movement along $- Y_{h}$ -axis.

Figure 18(a) and (b) show the insertion of a peg using the proposed force controller, which is similar to the assembly of the nut described in Figure 16.

Figure 18.

(a) Inserting one long peg into the hole; (b) inserting another long peg into the hole.

Conclusion

To make industrial robots programmable universal machines that can be easily and quickly configured and reconfigured into systems without incurring a high cost for system integration, we developed an assembly strategy using reorientation and sensor-based motion control methods. Then, we designed a flexible robot assembly system to perform the challenging assembly task.

The system included three UR robots, two CCD cameras, one force/torque sensor, two Robotiq grippers, and three specially designed electric screwdrivers. The three robots were employed to perform different tasks: UR-1 was used to pick the workpiece, UR-2 was used with a force/torque sensor to insert the peg into the hole, and UR-3 was used to turn the screw with the help of electric screwdrivers. CCD cameras were used to capture an image of the worktable, and a force controller combined with the constraint formed by the environment was designed to achieve precision assembly tasks. Minor visual errors were compensated for by force control. The proposed force-based assembly method enabled the system to achieve a high accuracy and success rate during task execution. The designed quick-change grippers and reorientation strategies maximized their versatility in performing different manufacturing tasks. The effectiveness of the designed system was verified in the WRS2020 competition.

In our future work, we will discuss the assembly of more complex parts, such as gears and cables. We plan to design a more flexible force-guided alignment algorithm to control the compliant motion of the robot.

Footnotes

Acknowledgment

The authors thank Yan Meng and Lili Wang for their help on the simulations and experiments of the work.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by NSFC under Grant Number 62273343, supported by the Beijing Natural Science Foundation under Grant Number L201019, and supported by the Key Laboratory of Spaceflight Dynamics Technology Foundation under Grant Number KGJ6142210210311.

ORCID iD

Jianhua Su

Appendix

We have performed a comparative experiment using the classic impedance control method. The impedance model used in force tracking is normally selected as a linear mass-spring-damper system; hence, the dynamic relationship between the robot movement error and the external force perturbation can be written as^20,21

with

where M, $K,$ and D are the full mass, stiffness, and damping matrices, respectively; $x_{d}$ represents the reference position; $x_{t}$ is the actual movement of the robot end-effector; t denotes time; f_t is the actual force; and f_d is the desired force.

Figure 19 shows the force tracking results of the impedance controller, where the solid lines represent the desired forces and the dotted lines represent the measured forces, where we set $K = 0, M = 0.35, B = 20$ . Figure 17 shows our results, where the peg poses are adjusted to make the contact force approach to the bottom of the constraint region. It is known that the proposed method can finish the insertion task with fewer robot actions (40 steps) than with the impedance control method (80 steps). Note that the impedance controller should adjust the object’s pose as per the force and torque information to follow the predefined force trajectories.

References

Liu

Tuzel

Veeraraghavan

, et al. Fast object localization and pose estimation in heavy clutter for robotic bin picking. Int J Robot Res 2012; 31(8): 951–973.

Dupuis

Léonard

Baumann

, et al. Two-fingered grasp planning for randomized bin-picking. Proc. of the Robotics: Science and Systems 2008 Manipulation Workshop - Intelligence in Human Environments 2008; pp. 25–28.

Sansonia

Bellandia

Leonia

, et al. Optoranger: a 3D pattern matching method for bin picking applications. Opt Lasers Eng 2014; 54:222–231.

Liu

Qiao

, et al. Pose-estimation and reorientation of pistons for robotic bin-picking. Ind Rob 2016; 43(1): 22–32.

Domae

Okuda

Taguchi

, et al. Fast graspability evaluation on single depth maps for bin-picking with general grippers. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May 2014–07 June 2014, pp. 1997–2004. IEEE.

Wan

Igawa

Harada

, et al. A regrasp planning component for object reorientation. Auton Robot 2019; 43:1101–1115.

Zhang

Goldberg

. Gripper point contacts for part alignment. IEEE Trans Robot 2002; 18(6): 902–910.

Xie

Chen

Sun

, et al. Hybrid vision-force control for automatic assembly of miniaturized gear system. In: Proceedings of the 2005 IEEE international conference on robotics and automation, Barcelona, Spain, 18–22 April 2005, pp. 1368–1373. IEEE.

Zheng

Zhang

Chen

, et al. Peg-in-hole assembly based on hybrid vision/force guidance and dual-arm coordination. In: 2017 IEEE international conference on robotics and biomimetics (ROBIO), Macau, Macao, 05–08 December 2017, pp. 418–423, doi: 10.1109/ROBIO.2017.8324453.

10.

Zhou

, et al. Automatic precision robot assembly system with microscopic vision and force sensor. Int J Adv Robot Sys 2019; 16(3). doi:10.1177/1729881419851619.

11.

Gorjup

Gao

Dwivedi

, et al. Combining compliance control, CAD based localization, and a multi-modal gripper for rapid and robust programming of assembly tasks. In: 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS), Las Vegas, NV, USA, 24 October 2020–24 January 2021, pp. 9064–9071. IEEE.

12.

Kim

Choi

Son

, et al. Vision-force guided precise robotic assembly for 2.5D components in a semistructured environment. Autom Constr 2021; 41(2): 200–207.

13.

Zhou

Guo

, et al. Peg-in-hole assembly in live-line maintenance based on generative mapping and searching network. Rob Auton Syst 2021; 143: 103797. https://doi.org/10.1016/j.robot.2021.103797.

14.

Rules and Regulations for WRS2020, The industrial robotics competition committee, August 27, 2021. https://wrs.nedo.go.jp/en/

15.

Cyr

Kimia

. A similarity-based aspect-graph approach to 3D object recognition. Int J Comput Vis 2004; 57:5–22.

16.

Ulrich

Wiedemann

Steger

. Combining scale-space and similarity-based aspect graphs for fast 3D object recognition. IEEE Trans. on Pattern Analysis and Machine Intelligence 2012; 34(10): 1902–1914.

17.

Ulrich

Wiedemann

Steger

. Combining scale-space and similarity-based aspect graphs for fast 3D object recognition. IEEE Trans Pattern Anal Mach Intell 2012; 34(12): 1902–1914.

18.

Qiao

Liu

, et al. Grasping objects: the relationship between the cage and form-closure grasp. IEEE Robot Autom Mag 2017; 24(3): 84–96.

19.

Liu

. Robot precision assembly combining with passive and active compliant motions. IEEE Trans Ind Electron 2022; 69(8): 8157–8167.

20.

Seraji

Colbaugh

. Force tracking in impedance control. In: Proceedings 1993 IEEE international conference on robotics and automation, Atlanta, GA, USA, 02–06 May, 1993, pp. 499–506. IEEE.

21.

Duan

Gan

Chen

, et al. Adaptive variable impedance control for dynamic contact force tracking in uncertain environment. Rob Auton Syst 2018; 102: 54–65.

Design of a robot system for reorienting and assembling irregular parts

Abstract

Keywords

Introduction

System overviews

Problem statement

2.2 System design

Object pose reorientation

Initial pose estimation using vision

Pose reorientation using fixture constraints

Precise alignment using force control

Experiments

Conclusion

Footnotes

Acknowledgment

Declaration of conflicting interests

Funding

ORCID iD

Appendix

References