Vision-based calibration/compensation technique for automatic stiffener bonder

Abstract

This study developed a novel error compensation method aimed at eliminating placement error caused by hand–eye calibration and pick-and-place tool motions in automatic stiffener bonder for flexible printed circuit. Using the transformation of homogeneous coordinates to develop an error model of the system describing the coupling of errors among various coordinate systems, the least squares method is used to calculate the unknown model parameters. The experiment results demonstrate that this error compensation method reduced placement error by an order of magnitude. The mounting precision throughout the entire work area was ±0.046 mm at 3sigma, and for flexible printed circuit products with a specification limit of 0.1 mm, the process capability index of the automatic stiffener bonder in this study was 2.19. This represents that the system is capable of fully satisfying the precision requirements of flexible printed circuit stiffener bonding. The proposed system employing a vibrating feeder bowl and machine vision–aided target positioning is applicable to a variety of stiffeners, which enhances production flexibility. The proposed error model considers the complex coupling effect of the errors among multiple coordinate systems in hand–eye calibration, without the need of detecting and calculating the calibration error item by item, and takes into account the errors produced by the rotation and downward pressing motions of the pick-and-place tool.

Introduction

Flexible printed circuit (FPC) boards require stiffeners to increase rigidity in specific areas; ¹ however, inaccuracy in the placement of stiffeners can lead to defects in the resulting product. ² Machine vision can be used to assist the automatic stiffener bonders (ASBs) in precise positioning in order to achieve high-speed, precision bonding; however, this approach requires calibration to eliminate errors.^3–5 First, laser interferometers, step gauges, and double ballbars are commonly used to measure and calibrate geometrical errors in motion mechanisms, in order to enhance precision in control.^6–10 Second, hand–eye calibration is also required to rectify errors in the transformation between coordinate systems, which is equivalent to solving matrix equations of the form AX = XB. Obtaining the unique solution of the equation necessitates the inclusion of specific constraints during the calibration process.^11,12 For example, the robots must perform three mutually orthogonal translations. ¹³ Zhuang proposed a hand–eye calibration method for the Selective Compliance Assembly Robot Arm (SCARA) robot with 4 degrees of freedom (DOFs). That study demonstrated that decreasing the number DOFs that can be controlled makes it impossible to obtain the position vector about the Z-axis in the solution space. ¹⁴ Furthermore, nonlinear optimization is required to mitigate the effects of noise and measurement error during calibration.^15,16 The varying number of DOFs among various devices has prevented the development of a unified approach to solving problems associated with hand–eye calibration, and the large number of constraints further complicates calculations.

Hand–eye calibration has been simplified in practical planar operations that are similar to mounting processes by focusing on problems associated with the transformation between two two-dimensional (2D) coordinate systems.^17,18 Surface mount technology (SMT) uses equipment similar to that of ASBs in which motion mechanisms comprise Cartesian coordinate robot. Zupancic ¹⁹ used linear displacement transducers to measure positional error in one-dimensional motion for the calibration of SMT systems. Shih and Ruo ²⁰ devised a rapid hand–eye calibration approach for SMT systems without the need for additional measurement equipment. However, this method fails to deal with the errors resulting from manual operations in which pick-and-place tools are moved to the location indicated by the fiducial marks. First, the robot is manually operated to move the tip of the stylus mounted on the tool to the center of the fiducial mark, in order to obtain the coordinate of the fiducial mark.^17,18,21 Second, as the calculation of the stylus coordinate is based on the calibration of the tool coordinate system, ²² the calibration error of the tool coordinate system results in the measurement error and the measurement error is propagated to the hand–eye calibration results. In addition, the use of cameras to measure positional errors caused by pick-and-place tool rotation does not consider the influence of pressing motion associated with the pick-and-place tool.^23,24

Accompanying the expanding scope of FPC applications is the increasing diversity in stiffener materials. At present, most automatic stiffener bonders for flexible printed circuits (FPCASBs) use easy-to-punch materials such as polyimide (PI) film. ²⁵ The feed systems in conventional ASB equipment punch out the stiffeners directly. The position and direction of the punching mold are fixed, such that the total placement error caused by the errors in hand–eye calibration is fixed. Thus, the error can be eliminated simply by setting a corresponding offset value.

For other types of stiffeners that are not easy to punch, the bonding process is entirely manual, which makes it nearly impossible to guarantee precision and quality. To address this issue, this study proposed a system employing vibrating feeder bowl and turntable as the feeding component; this makes it suitable for a variety of stiffener materials, including PI, metals, and epoxy glass. This system helps to bridge a gap in the FPCASB industry, but brings some new technological issues affecting the placement precision. First, the feeding component developed in this study increased the variation in rotation angle of stiffeners during the mounting, so that the variation in rotation angle caused the changes in placement error. The coupling effect of the errors from multiple coordinate systems increases the complexity associated with the elimination of the error. Second, the use of conductive adhesives in the bonding of stiffeners means that when a stiffener is mounted, the pick-and-place tool must heat and press down on it. This strong downward force can increase positional error caused by rotation of the pick-and-place tool; this positional error is difficult to measure directly.

This study developed a novel error compensation method aimed at enhancing mounting precision by eliminating placement error caused by hand–eye calibration and the motions of the pick-and-place tool in ASB. The flowchart of the compensation procedure is shown in Figure 1. Homogeneous transformation coordinate is used to develop a kinetic model associated with the robot and the machine vision component. ²⁶ Using transformation to describe the coordinate of the stiffener after rotation and its placement location in different coordinate systems, the motion pathway is calculated for the controller. The placement error model of the ASB system is developed and the placement errors are compensated in the system, without the need of detecting and calculating the calibration error item by item. The effectiveness of the proposed model was verified in the experiments. Unlike conventional FPCASB equipment, the proposed system can be adapted to various types of stiffener, thereby enhancing flexibility in production. The proposed error model considers the complex coupling of hand–eye calibration errors in multiple coordinate systems and takes into account the errors produced by the rotation and downward pressing motions of the pick-and-place tool.

Figure 1.

Flowchart of the compensation procedure.

ASB calibration method

The ASB system designed in this study comprises three components, as shown in Figure 2. In the first feeding component, the stiffener is screened and palletized using a turntable conveyor. The second component is the motion mechanism with a Cartesian coordinate robot. The stiffener is handled by a pick-and-place tool, which moves along the X-axis and rotates around the Z-axis. A vacuum platform moving along the Y-axis is used to grasp the FPCs by suction. The X–Y plane of the robot base frame is used as the working plane, which is on the same plane as the vacuum platform. The third component is the machine vision system, which includes two high-resolution cameras. A moving camera positions the location of the stiffener mounted on the FPC, and an upward facing stationary camera positions the original location of the stiffener (attached to the pick-and-place tool).

Figure 2.

Structure of ASB system.

The ASB system proposed in this article adopts GTS-800-PV(G)-PCI series motion controller to control the MINAS A5-series AC Servo Motor & Driver produced by Panasonic. The software is developed using C-sharp (C#); the programs associated with camera calibration and image processing are accomplished using Halcon software; ²⁷ this part of the code is replaced with open source code (OpenCV) in the subsequent development.

In order to achieve high-speed, precision bonding in the ASB system, first, to ensure the accuracy and repeatability in positioning, the motion precision of the robot (e.g. rotation angle around the Z-axis) must reach a given standard. ²⁸ Second, the camera calibration and hand–eye calibration must be implemented for the motion pathway calculation.

Camera calibration

Eliminating the errors in the vision system, such as camera distortion, requires a calibration board to estimate the nonlinear intrinsic and extrinsic parameters using a pinhole camera model; the coordinate of the target in the image frame $X_{\mathrm{img}}$ is transformed to the camera frame, noted as $X_{\mathrm{cam}}$ . ²⁹

Hand–eye calibration

In the calculation of motion pathway, $X_{\mathrm{cam}}$ is needed to transform into the robot base frame coordinate; this requires hand–eye calibration to derive the homogeneous coordinate transformation matrixes ( ${{}^{base}H}_{\mathrm{scam}}$ and ${{}^{base}H}_{\mathrm{mcam}}$ ), through selecting a fiducial mark (fiducial point) and obtaining its coordinates in various coordinate systems. As the stiffeners are mounted on the same plane, the hand–eye calibration in this study is implemented based on the transformation between 2D coordinate systems. ¹⁸ Unlike the calibration between three-dimensional (3D) coordinate systems, ¹⁴ this method has no extra constraints.

Principles for the transformation of the stationary camera and the base

The transformation between stationary camera frame and robot base frame frames is calculated as given in equation (1)

$$X_{\mathrm{sbase}}={{}^{base}H}_{\mathrm{scam}}·X_{\mathrm{scam}}$$

(1)

In the calibration of the stationary camera, the rotation center of the pick-and-place tool (the origin of the tool coordinate system) is selected as the fiducial mark, and the coordinates of the fiducial mark in two coordinate systems ( ${\overline{X}}_{\mathrm{sbase}}$ and ${\overline{X}}_{\mathrm{scam}}$ ) are obtained in this experiment to calculate the hand–eye matrix ( ${{}^{base}H}_{\mathrm{scam}}$ ). First, moving the pick-and-place tool and reading its location provide the coordinates of the fiducial mark ( ${\overline{X}}_{\mathrm{sbase}}$ ). Second, the rotation center of the pick-and-place tool has no unique image features, and therefore, the location of the rotation center ( ${\overline{X}}_{\mathrm{scam}}$ ) should be calculated indirectly using the dots on the calibration board. As shown in Figure 3(a), the calibration board is attached securely to the plane at the bottom of the pick-and-place tool. After the stationary camera captures the image of a dot (the coordinate of the dot noted as $X_0$ ), the pick-and-place tool is rotated by 180°, whereupon another coordinate of the same dot is taken ( $X_{180}$ ). The midpoint along the line segment connecting the two dot locations represents the rotation center of the pick-and-place tool ( ${\overline{X}}_{\mathrm{scam}}$ ), as shown in equation (2)

$${\overline{X}}_{\mathrm{scam}}=0.5(X_0+X_{180})$$

(2)

Figure 3.

Schematics of hand–eye calibration: (a) stationary camera and (b) moving camera.

In this manner, the ${\overline{X}}_{\mathrm{scam}}$ is obtained. Thus, equation (3) is used to derive ${{}^{base}H}_{\mathrm{scam}}$

$$\sum {\Vert {\overline{X}}_{\mathrm{sbase}}-{{}^{base}H}_{\mathrm{scam}}·{\overline{X}}_{\mathrm{scam}}\Vert }^2=\mathrm{minimum}$$

(3)

Principles for the transformation of the coordinate systems of the moving camera and base

Equation (4) illustrates the relative positions and orientations of the moving camera frame and base frame, which changes with the location of the pick-and-place tool. However, the relative positions and orientations of the tool frame and the moving camera frame do not change. Therefore, ${{}^{\mathrm{tool}}H}_{\mathrm{mcam}}$ is a fixed value and can be obtained with the hand–eye calibration, and calculating the ${{}^{base}H}_{\mathrm{mcam}}$ using ${{}^{\mathrm{tool}}H}_{\mathrm{mcam}}$ is capable of transforming the coordinate of stiffener between various coordinate systems

$$\begin{array}{cc}\hfill X_{\mathrm{mbase}}\hfill & \hfill ={{}^{base}H}_{tool}·{{}^{tool}H}_{\mathrm{mcam}}·X_{\mathrm{mcam}}\hfill \\ \multicolumn{1}{c}{}& \hfill ={{}^{base}H}_{\mathrm{mcam}}·X_{\mathrm{mcam}}\hfill \end{array}$$

(4)

First, as shown in Figure 3(b), the calibration board is fixed to the working plane, and the central dot of the calibration board is set as the fiducial mark. The robot is manually operated to move the tip of the stylus mounted on the end-effector to the center of the fiducial mark, to measure the coordinate of the fiducial mark in robot base frame ( ${\overline{X}}_{\mathrm{mbase}}$ ). Second, the moving camera is used to capture images of the fiducial mark as it moves to different locations; meanwhile, ${{}^{base}H}_{tool}$ is calculated by reading each location of the pick-and-place tool. The images provide the coordinates of the fiducial mark in the moving camera frame ( ${\overline{X}}_{\mathrm{mcam}}$ ). Thus, ${{}^{\mathrm{tool}}H}_{\mathrm{mcam}}$ can be derived using equation (5)

$$\sum {\Vert {{}^{base}H}_{tool}^{-1}·{\overline{X}}_{\mathrm{mbase}}-{{}^{base}H}_{\mathrm{mcam}}·{\overline{X}}_{\mathrm{mcam}}\Vert }^2=\mathrm{minimum}$$

(5)

In summary, this section illustrated the process of obtaining the homogeneous coordinate transformation ${{}^{base}H}_{\mathrm{scam}}$ and ${{}^{\mathrm{tool}}H}_{\mathrm{mcam}}$ . Equations (1) and (4) are used to transform the initial locations and placement locations of the stiffeners obtained from the two cameras ( $X_{\mathrm{scam}}$ and $X_{\mathrm{mcam}}$ ) into coordinates for the base coordinate system, thereby deriving $X_{\mathrm{sbase}}$ and $X_{\mathrm{mbase}}$ . This makes it possible to calculate the motion pathway for use by the motion controller, as shown in Figure 1.

Error analysis and compensation in ASB system

Placement error in the ASB originates primarily from the errors associated with hand–eye calibration and the motions of the pick-and-place tool. With regard to the former, the coordinates of the fiducial mark in the base frame ( ${\overline{X}}_{\mathrm{sbase}}$ and ${\overline{X}}_{\mathrm{mbase}}$ ) can be derived through direct or indirect measurement by manual operations, wherein the errors in measurement can lead to coupling among the positional errors of the origins of multiple coordinate systems. For the latter, although precision is guaranteed in the control of the rotation angle of the pick-and-place tool, the location of the stiffener on the pick-and-place tool still changes somewhat following the rotating and pressing motions. The error in the placement of the system is derived using the images of the bonded stiffeners ( ${\mathrm{errX}}_{\exp }$ ), which enable to calculate the parameters ( ${\theta }_{\mathrm{inc}}$ and ${\theta }_{\mathrm{com}}$ ) used in an error model ( ${\mathrm{errX}}_{\mathrm{inc}}$ and ${\mathrm{errX}}_{\mathrm{com}}$ ) capable of revising the original motion pathways ( ${\mathrm{disX}}_{\mathrm{ori}}$ ), as shown in Figure 1.

Incomplete error model based on hand–eye calibration error

In the calculation of the motion pathway, homogeneous coordinates’ transformation is used to obtain the relationship between the location of the stiffener and the rotation angle of the pick-and-place tool. Then the stationary camera provides the initial location of the stiffener ( $X_{\mathrm{sbase}}$ ); based on the rotation angle $\alpha$ required for each stiffener, the new location of the stiffener following rotation ( $X{\prime }_{\mathrm{sbase}}$ ) can be obtained, as shown in equation (6)

$$X{\prime }_{\mathrm{sbase}}={{}^{base}H}_{\mathrm{tool}}·\mathrm{Rot}(Z,\alpha )·X_{\mathrm{tool}}{·}^{\mathrm{tool}}{H}_{\mathrm{base}}·X_{\mathrm{sbase}}$$

(6)

$${{}^{base}T}_{tool}=\left[\begin{array}{ccc}\hfill 1\hfill & \hfill 0\hfill & \hfill x_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{0}& \hfill 1\hfill & \hfill y_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{0}& \hfill 0\hfill & \hfill 1\hfill \end{array}\right]$$

(7)

$$\mathrm{Rot}(Z,\alpha )=\left[\begin{array}{ccc}\hfill \cos \alpha \hfill & \hfill -\sin \alpha \hfill & \hfill 0\hfill \\ \multicolumn{1}{c}{\sin \alpha }& \hfill \cos \alpha \hfill & \hfill 0\hfill \\ \multicolumn{1}{c}{0}& \hfill 0\hfill & \hfill 1\hfill \end{array}\right]$$

(8)

$$X_{\mathrm{sbase}}^{\prime }=\left[\begin{array}{ccc}\hfill \cos \alpha \hfill & \hfill -\sin \alpha \hfill & \hfill x_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{\sin \alpha }& \hfill \cos \alpha \hfill & \hfill y_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{0}& \hfill 0\hfill & \hfill 1\hfill \end{array}\right]\left[\begin{array}{c}\hfill x_{\mathrm{sbase}}-x_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{y_{\mathrm{sbase}}-y_{\mathrm{rot}}}\\ \multicolumn{1}{c}{1}\end{array}\right]$$

(9)

The moving camera provides the stiffener placement location $X_{\mathrm{mbase}}$ ; the motion displacement $\mathrm{disX}$ is obtained in equation (10)

$${\mathrm{disX}}_{\mathrm{ori}}=X_{\mathrm{mbase}}-X_{\mathrm{sbase}}^{\prime }$$

(10)

Hand–eye calibration can result in an offset in the origin of multiple coordinate systems. If each error term caused by the offset is considered as a fixed value (e.g. ${\mathrm{dX}}_{\mathrm{mbase}}$ and $\mathrm{dX}{\prime }_{\mathrm{sbase}}$ ), then the placement error is ${\mathrm{errX}}_{\mathrm{inc}}$ , as shown in equation (11)

$$\begin{array}{cc}\multicolumn{1}{c}{{\mathrm{errX}}_{\mathrm{inc}}}& ={\mathrm{dX}}_{\mathrm{mbase}}-{\mathrm{dX}}_{\mathrm{sbase}}^{\prime }=Y_{\mathrm{inc}}·{\theta }_{\mathrm{inc}}\hfill \\ \multicolumn{1}{c}{}& =\left[\begin{array}{c}\hfill {\mathrm{dx}}_{\mathrm{mbase}}\hfill \\ \multicolumn{1}{c}{{\mathrm{dy}}_{\mathrm{mbase}}}\end{array}\right]-\left[\begin{array}{ccc}\hfill \cos \alpha \hfill & \hfill -\sin \alpha \hfill & \hfill {\mathrm{dx}}_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{\sin \alpha }& \hfill \cos \alpha \hfill & \hfill {\mathrm{dy}}_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{0}& \hfill 0\hfill & \hfill 1\hfill \end{array}\right]\left[\begin{array}{c}\hfill {\mathrm{dx}}_{\mathrm{sbase}}-{\mathrm{dx}}_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{{\mathrm{dy}}_{\mathrm{sbase}}-{\mathrm{dy}}_{\mathrm{rot}}}\\ \multicolumn{1}{c}{1}\end{array}\right]\hfill \end{array}$$

(11)

$Y_{\mathrm{inc}}$ and ${\theta }_{\mathrm{inc}}$ are the regressor matrix and parameter matrix of the incomplete error model, respectively, as shown in equation (12)

$$Y_{\mathrm{inc}}·{\theta }_{\mathrm{inc}}=\left[\begin{array}{cccc}\hfill 1\hfill & \hfill -\cos a\hfill & \hfill 0\hfill & \hfill \sin a\hfill \\ \multicolumn{1}{c}{0}& \hfill -\sin a\hfill & \hfill 1\hfill & \hfill -\cos a\hfill \end{array}\right]\left[\begin{array}{c}\hfill {\mathrm{dx}}_{\mathrm{mbase}}-{\mathrm{dx}}_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{{\mathrm{dx}}_{\mathrm{sbase}}-{\mathrm{dx}}_{\mathrm{rot}}}\\ \multicolumn{1}{c}{{\mathrm{dy}}_{\mathrm{mbase}}-{\mathrm{dy}}_{\mathrm{rot}}}\\ \multicolumn{1}{c}{{\mathrm{dy}}_{\mathrm{sbase}}-{\mathrm{dy}}_{\mathrm{rot}}}\end{array}\right]$$

(12)

Using the placement errors ( ${\mathrm{errX}}_{\exp }$ ) derived from the experiments in conjunction with the least squares method, ${\theta }_{\mathrm{inc}}$ is calculated as given in equation (13)

$$\sum {\Vert {\mathrm{errX}}_{\exp }-Y_{\mathrm{inc}}·{\theta }_{\mathrm{inc}}\Vert }^2=\mathrm{minimum}$$

(13)

Complete error model based on errors from calibration and tool motion

The incomplete error model only considers the error caused by hand–eye calibration. As shown in Figure 4, the rotation center associated with the pick-and-place tool changes with $\beta$ , the rotation angle of axis Z. Following the pressing motion, the tool presents an elliptic trajectory on the work plane $Z(\beta )$ , as shown in equation (14), where $r_x$ and $r_y$ are the lengths of the ellipse axes and $\theta$ is the direction angle

$$\begin{array}{c}\multicolumn{1}{c}{\mathrm{Zx}(\beta )=r_x\cos \beta \cos \theta +r_y\sin \beta \sin \theta }\\ \multicolumn{1}{c}{\mathrm{Zy}(\beta )=-r_x\cos \beta \sin \theta +r_y\sin \beta \sin \theta }\end{array}$$

(14)

Figure 4.

Error associated with pick-and-place tool.

As a result, the pick-and-place tool creates new errors in $\mathrm{Rot}\prime (Z,\alpha )$ , as shown in equation (15)

$$\mathrm{Rot}\prime (Z,\alpha )=\left[\begin{array}{ccc}\hfill \cos \alpha \hfill & \hfill -\sin \alpha \hfill & \hfill \mathrm{Zx}(\beta )\hfill \\ \multicolumn{1}{c}{\sin \alpha }& \hfill \cos \alpha \hfill & \hfill \mathrm{Zy}(\beta )\hfill \\ \multicolumn{1}{c}{0}& \hfill 0\hfill & \hfill 1\hfill \end{array}\right]$$

(15)

Equation (16) reveals that the resulting new placement error is ${\mathrm{dX}}_{\mathrm{sbase}}^{″}$ , as shown in equation (17)

$$X_{\mathrm{sbase}}^{″}={{}^{base}H}_{\mathrm{tool}}·\mathrm{Rot}\prime (Z,\alpha )·X_{\mathrm{tool}}$$

(16)

$${\mathrm{dX}}_{\mathrm{sbase}}^{″}=\left[\begin{array}{ccc}\hfill \cos \alpha \hfill & \hfill -\sin \alpha \hfill & \hfill {\mathrm{dx}}_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{\sin \alpha }& \hfill \cos \alpha \hfill & \hfill {\mathrm{dy}}_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{0}& \hfill 0\hfill & \hfill 1\hfill \end{array}\right]\left[\begin{array}{c}\hfill {\mathrm{dx}}_{\mathrm{sbase}}-{\mathrm{dx}}_{\mathrm{rot}}\hfill \\ \multicolumn{1}{c}{{\mathrm{dy}}_{\mathrm{sbase}}-{\mathrm{dy}}_{\mathrm{rot}}}\\ \multicolumn{1}{c}{1}\end{array}\right]+\left[\begin{array}{c}\hfill \mathrm{Zx}(\beta )\hfill \\ \multicolumn{1}{c}{\mathrm{Zy}(\beta )}\\ \multicolumn{1}{c}{1}\end{array}\right]$$

(17)

In summary, the final placement errors caused by hand–eye calibration and motions of the pick-and-place tool are ${\mathrm{errX}}_{\mathrm{com}}$ , as shown in equation (18)

$$\begin{array}{cc}\multicolumn{1}{c}{{\mathrm{errX}}_{\mathrm{com}}}& ={\mathrm{dX}}_{\mathrm{mbase}}-{\mathrm{dX}}_{\mathrm{sbase}}^{″}\hfill \\ \multicolumn{1}{c}{}& ={\mathrm{errX}}_{\mathrm{inc}}-Z(\beta )=Y_{\mathrm{com}}·{\theta }_{\mathrm{com}}\hfill \end{array}$$

(18)

$$\begin{array}{cc}\multicolumn{1}{c}{Y_{\mathrm{com}}}& =\left[\begin{array}{cccccccc}\hfill 1\hfill & \hfill -\cos \alpha \hfill & \hfill 0\hfill & \hfill \sin \alpha \hfill & \hfill -\cos \beta \hfill & \hfill -\sin \beta \hfill & \hfill 0\hfill & \hfill 0\hfill \\ \multicolumn{1}{c}{0}& \hfill -\sin \alpha \hfill & \hfill 1\hfill & \hfill -\cos \alpha \hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill -\cos \beta \hfill & \hfill -\sin \beta \hfill \end{array}\right]\hfill \\ \multicolumn{1}{c}{}\\ \multicolumn{1}{c}{{\theta }_{\mathrm{com}}}& =[\begin{array}{cccc}\hfill {\mathrm{dx}}_{\mathrm{mbase}}-{\mathrm{dx}}_{\mathrm{rot}}\hfill & \hfill {\mathrm{dx}}_{\mathrm{sbase}}-{\mathrm{dx}}_{\mathrm{rot}}\hfill & \hfill {\mathrm{dy}}_{\mathrm{mbase}}-{\mathrm{dy}}_{\mathrm{rot}}\hfill & \hfill ...\hfill \end{array}\hfill \\ \multicolumn{1}{c}{}& {\begin{array}{ccccc}\hfill {\mathrm{dy}}_{\mathrm{sbase}}-{\mathrm{dy}}_{\mathrm{rot}}\hfill & \hfill \cos \theta {\mathrm{dr}}_x\hfill & \hfill \sin \theta {\mathrm{dr}}_y\hfill & \hfill -\sin \theta {\mathrm{dr}}_x\hfill & \hfill \cos \theta {\mathrm{dr}}_y\hfill \end{array}]}^T\hfill \end{array}$$

${\theta }_{\mathrm{com}}$ is obtained as given in equation (19). In the subsequent production process, ${\mathrm{errX}}_{\mathrm{com}}$ is achieved in order to calculate the revised motion pathway ${\mathrm{disX}}_{\mathrm{rev}}$

$$\sum {\Vert {\mathrm{errX}}_{\exp }-Y_{\mathrm{com}}·{\theta }_{\mathrm{com}}\Vert }^2=\mathrm{minimum}$$

(19)

Experiment results

To compensate the placement error of the ASB system, the incomplete and complete error model is developed, and the compensation results are compared between the two models. The flowchart of the experiment procedure is shown in Figure 5.

Figure 5.

Flowchart of the experiment procedure.

Hand–eye calibration results of ASB

A calibration board was used to calibrate the moving camera and the stationary camera, ²⁷ the calibration board is made from glass plate and its size is $12\times 12{\mathrm{mm}}^2$ , as shown in Figure 6. Through shifting the tool and measurement by steel rulers, the zero position of the tool in the base frame (26.0 mm, 360.4 mm) is derived. Thus, ${{}^{base}H}_{tool}$ and ${\overline{X}}_{\mathrm{sbase}}$ can be calculated using the tool’s zero position. The calibration board is fixed to the working plane, and the tip of the stylus is moved to the center of fiducial mark on the calibration board, to calculate the coordinate of fiducial mark in robot base frame ( ${\overline{X}}_{\mathrm{mbase}}=(150.0\mathrm{mm},101.5\mathrm{mm})$ ). As the measurement by manual operations brings errors to these values, these measurement errors should be eliminated through compensation with error model in the subsequent process. Through the multiple images captured by the two cameras, ${\overline{X}}_{\mathrm{scam}}$ and ${\overline{X}}_{\mathrm{mcam}}$ are derived, as shown in Figure 6. The matrixes ( ${{}^{\mathrm{tool}}H}_{\mathrm{mcam}}$ and ${{}^{base}H}_{\mathrm{mcam}}$ ) are obtained for pathway calculation using equations (3) and (5). The elements of the matrixes (a_ij) are displayed in Table 1.

Figure 6.

Images taken by (a) moving camera and (b) stationary camera during hand–eye calibration.

Table 1.

Results of homogeneous coordinate transformation matrix through hand–eye calibration.

Matrix element	Moving camera: ${{}^{\mathrm{tool}}H}_{\mathrm{cam}}$	Stationary camera: ${{}^{base}H}_{\mathrm{cam}}$
a 11	0.9999080	0.0915938
a 12	−0.0011441	−0.9957554
a 13	0.0134916	0.0090286
a 14	−0.104280	0.4995807
a 21	0.0011612	0.9957909
a 22	0.9999990	0.09156
a 23	−0.0012631	−0.0040965
a 24	0.0375423	0.3649793
a 31	−0.0134901	0.0032525
a 32	0.0012787	0.0093658
a 33	0.9999080	0.9999507
a 34	−0.1574190	−0.183458

Development of system error model

To obtain an accurate error model, the FPC is replaced with a film that is designed to high-precision patterns according to the shape of standard stiffeners, as shown in Figure 7(b). The stiffeners were mounted on the film, and the images of the mounted stiffeners were collected to derive the placement error and calculate the parameters of the error model.

Figure 7.

Experiment images: (a) FPCASB, (b) film pieces used to replace the FPCs and (C) FPC products.

As the rotation angle of the stiffeners is related to the mechanical structure of the feeding component (e.g. a rectangular slot is used for feeding on the turntable), the rotation angle of the stiffeners during the mounting is approximately −90° and 0°. To derive the unique solution of equation (19) and increase the accuracy of the model parameters, the stiffeners are placed manually on the turntable to approximately −90°, 0°, 90°, and 180°, which caused the stiffeners to be picked up (by the pick-and-place tool) at various angles, as shown in Figure 8. This was repeated 20 times for each angle, which resulted in a total of 80 records pertaining to the rotation angle required for each stiffener ( $\alpha$ ), and the rotation angle of the Z-axis following each placement ( $\beta$ ). In addition, the angle of the Z-axis is set at 3° intervals before each placement ( ${\beta }_{\mathrm{initial}}$ ), such that $\beta$ was evenly distributed between −180° and 180° after mounting, as given in equation (20)

$$\beta ={\beta }_{\mathrm{initial}}+\alpha$$

(20)

Figure 8.

Stiffeners held at different angles in the initial location: (a) images on turntable and (b) images on pick-and-place tool.

Figure 9 shows the images of the stiffeners mounted at the same location. The offset between the top centers of the stiffeners ( $X_a$ ) and the top centers of the corresponding patterns on the film ( $X_b$ ) is calculated to obtain the placement errors ( ${\mathrm{errX}}_{\exp }$ ), as shown in equation (21) and Figure 13

$${\mathrm{errX}}_{\exp }=X_a-X_b$$

(21)

Figure 9.

Images of stiffeners mounted at the same location without error compensation.

As shown in Figures 10 and 11, different rotation angles can lead to different coupling situations, resulting in different values pertaining to placement error. The offset of the stiffeners in the X- and Y-directions was obvious, revealing the edges of the film pattern in the images. The placement errors were significant (exceeding 0.3 mm), far exceeding the precision threshold associated with mounting.

Figure 10.

Placement error in X-direction without error compensation.

Figure 11.

Placement error in Y-direction without error compensation.

To contrast the effects of error compensation, the complete and incomplete error models are developed using the ${\mathrm{errX}}_{\exp }$ , and the model parameters ( ${\theta }_{\mathrm{com}}$ and ${\theta }_{\mathrm{inc}}$ ) are calculated using equations (13) and (19), which are displayed in Table 2.

Table 2.

Error model parameters.

${\theta }_{\mathrm{com}}$	${\theta }_{\mathrm{inc}}$
4.58E−04	4.58E−04
3.71E−05	3.96E−05
3.38E−04	3.38E−04
1.30E−06	4.30E−06
(8.90E−06	N/A
1.36E−05	N/A
4.70E−06	N/A
1.01E−05	N/A

N/A: not available.

Results of error compensation

To provide a reasonable description of placement precision, a process capability index (CPK) is employed to serve as a standard, ³⁰ as given in equation (22), where sigma denotes the root mean square error (RMSE), $\mu$ is the mean error, and SL refers to the specification limit. The 3sigma value is used as placement precision

$$\mathrm{CPK}=\frac{\mathrm{SL}-\mu }{3\mathrm{sigma}}$$

(22)

Compensation with incomplete error model

Using the same placement location, the stiffeners are rotated to four directions 80 times. An incomplete error model (without consideration for the errors in tool motions) is used for error compensation. The images of the mounted stiffeners and placement error results are presented in Figures 12–15. As can be seen, after compensation, the top centers of the stiffeners and the film patterns are almost identical, and the effects of error compensation are obvious. A comparison with the mounting results without this compensation shows that the errors after compensation were decreased by an order of magnitude. Based on the value of 3sigma in equation (22), the mounting precision of the system in the X- and Y-directions is (0.043 mm, 0.049 mm), noted as ±0.049 mm at 3sigma.

Figure 12.

Images of stiffeners mounted at the same location with compensation using incomplete error model.

Figure 13.

(a) Film pattern and mounted stiffener images (b) before and (c) after compensation using incomplete error model.

Figure 14.

Placement error in X-direction with compensation using incomplete error model.

Figure 15.

Placement error in Y-direction with compensation using incomplete error model.

Compensation with complete error model

Using a complete error model, the experiment is repeated, the placement results of which are presented in Figures 16 and 17. The error after compensation with complete model is more evenly distributed and less affected by the rotation angle; comparing with the incomplete model, the complete model enables to eliminate the errors from motion associated with the pick-and-place tool. The mounting precision of the system became (0.032 mm, 0.038 mm), noted as ±0.038 mm at 3sigma. Based on the mounting precision after compensation with the two models, this comparison shows that using complete error model results in further 28% reduction in placement error.

Figure 16.

Placement error in X-direction with compensation using complete error model.

Figure 17.

Placement error in Y-direction with compensation using complete error model.

With regard to the placement error caused by pick-and-place tool motion, the complete model not only considers the affect of the tool rotation but also considers the strong downward force after tool pressing down which can increase positional errors of the rotation center of pick-and-place tool. Through rotating the tool every 30° without pressing down and capturing the images using the stationary camera, the error trajectories of the rotation center are obtained with a similar method of deriving ${\overline{X}}_{\mathrm{scam}}$ in hand–eye calibration. The obtained error trajectories are nearly circular, as shown in Figure 18. Clearly, the error trajectories simulated are elliptical and greater than those obtained through direct observation. This demonstrates that the observed error trajectories projected to the work plane are included as part of the simulated trajectories, and the error model simultaneously considers two factors that produce the errors: the downward pressing motions and rotation of the pick-and-place tool.

Figure 18.

Error trajectories of pick-and-place tool associated with rotating and pressing down.

Compensation in the entire workspace

To determine the mounting precision of the system throughout the entire work area, 80 stiffeners are mounted onto multiple placement locations and used the complete error model for compensation. The results are presented in Figures 19–21. As can be seen, an increase in the work area led to an increase in the placement error caused by motions in the mechanism, becoming (0.041 mm, 0.046 mm), noted as ±0.046 mm at 3sigma. In actual FPC production, the SL of stiffener mounting generally ranges from 0.1 to 0.15 mm, and the CPK of the qualifying mounting equipment must be greater than or equal to 1.33. This study adopted a higher precision requirement (SL = 0.1 mm), as shown in Figure 9(c). The calculations of mounting precision throughout the entire work area of the ASB system in this study presented a CPK of 2.19, clearly demonstrating the effectiveness of the proposed system. Several tests indicated that the finished products also fulfilled precision requirements.

Figure 19.

Stiffeners mounted at different locations.

Figure 20.

Placement error in X-direction over the entire work area.

Figure 21.

Placement error in Y-direction over the entire work area.

Conclusion

The experiment results show that error coupling and placement error vary with the rotation angle of the stiffener; the errors were exceeding 0.3 mm before compensation. Therefore, an error model is used to compensate for placement error in the system, thereby reducing placement error by an order of magnitude. The complete error model considers the errors caused by motions associated with the pick-and-place tool, further reducing the error by 28%. As for the errors in the rotation center of the pick-and-place tool, the simulation by error model and direct observation presented different results, demonstrating that the error model takes into account the influence of downward pressing motion associated with the tool. For the same mounting location, the mounting precision of the system is ±0.038 mm at 3sigma, whereas the precision of the system in the entire work area is ±0.046 mm at 3sigma. For FPC products with an SL of 0.1 mm, the CPK of the FPCASB in this study is 2.19, which indicates good mounting capabilities, capable of fully satisfying the high-precision requirements of FPC stiffener bonding. The approach used in this study is suitable for other types of workpiece that vary randomly in pick-and-place angle. This method is also applicable to applications in visually guided high-precision pick-and-place systems with 4 DOFs. Furthermore, based on the proposed error model, the effect of temperature on the placement error in the ASB system will be considered for future work.