Sage Journals: Discover world-class research

Abstract

Laser stripe vision sensors have limited application when the whole surface of an object has to be measured; therefore, a galvanometric laser scanner is presented using a one-mirror galvanometer element as its mechanical device to drive the light stripe to sweep along the object. A flexible plane-constraint together with a look-up table calibration algorithm by viewing a planar calibration target from unknown orientations is presented to calibrate the proposed galvanometric laser scanner. The plane-constraint method generates a larger number of control points by imposing a plane constraint on the control points. Compared with the invariance of double cross-ratio, the plane-constraint-based calibration method generates the same number of control points without error propagation. Contrast experiments are conducted to verify the efficiency and accuracy of the proposed calibration method.

Keywords

A galvanometer laser scanner calibration control points plane-constraint method three-dimensional metrology

Introduction

Laser stripe vision sensors have been gaining wide acceptance in industry inspection due to fast acquisition time, a very simple structure, low cost, and robust nature. The inspection principle of the laser stripe vision sensor is shown in Figure 1. In Figure 1, a laser projector projects a light stripe plane onto the object’s surface, forming a light stripe. The light stripe is modulated by the depth of the object’s surface. Consequently, the distorted light stripe contains three-dimensional (3D) characteristic information of the object’s surface. The 3D characteristic information of the object’s surface can be derived from two-dimensional (2D) distorted light stripe images. However, a laser stripe vision sensor can only obtain the point cloud on one light stripe, so it has limited applications when the whole surface of the object must be measured.¹ Consequently, a galvanometric laser scanner is presented using a one-mirror galvanometer element as its mechanical device to drive the light stripe to sweep along the object to gather the point cloud on all the light stripes. The proposed galvanometric laser scanner consists of two main parts: a laser stripe vision sensor and a galvanometric element. Therefore, calibration of the laser stripe vision sensor is the basis of the proposed laser scanner. There are two ways to complete the laser stripe vision sensor calibration process. One is based on the homography model² between the light stripe plane and the image plane, which is called the one-step calibration method, and the other is based on the simple triangular methodology,¹ which is the two-step calibration method. Unfortunately, an effective and highly accurate extraction of calibration points (control points) for a laser stripe vision sensor has been an obstacle to calibration.

Figure 1.

The inspection principle of the laser stripe vision sensor.

Tremendous efforts have been devoted to the field of extracting the control points to calibrate the laser stripe vision sensor, and a wide range of methods have been developed. Xu et al.³ and Huynh et al.² separately present a novel method which is based on the invariance of cross-ratio in the perspective transformation. The novel method is used to get the coordinates of control points with respect to the world coordinate system (WCS). A 3D calibration target is used in both methods, because the one-step calibration method does not calibrate the camera and there are no extrinsic parameters of the camera during the calibration process. The number of the control points is limited for the invariance of the cross-ratio method, and many more control points are essential to enhance calibration accuracy. Wei et al.⁴ proposes an approach based on the invariance of double cross-ratio to estimate an arbitrary number of control points on the light stripe plane from at least three known world points with a 3D calibration target. However, accuracy of the control points is limited by error propagation, because the invariance of double cross-ratio uses the invariance of cross-ratio on the light stripe again, and the laser stripe plane is not an ideal plane and the light stripe projected on the calibration targets is not an ideal line. The 3D calibration target is very difficult to manufacture accurately, and it is also very hard to capture a good image by viewing the different planes of the 3D calibration target at once. There are also some methods^5,6 where the 3D calibration target is constituted by a 2D calibration target and a movable platform. The calibration process is very inconvenient in operation and time-consuming. Zhou et al.⁷ present a flexible approach where the invariance of cross-ratio is employed to generate a large number of control points on the light stripe plane by viewing a planar calibration target from unknown orientation. The number of control points is limited at each location of the planar calibration target; the planar calibration target has to be moved many times to gain a large number of control points. Sun⁸ obtains many control points by the homography transformation of calibration target planes and camera image plane. However, the homography transformation of calibration target plane and camera image plane is estimated separately via the Levenberg–Marquardt optimization algorithm, and the estimated homography transformation is less accurate and is not needed when camera calibration is performed. As to the structure and calibration method of the 3D laser scanner, Li et al.^9–11 designed a laser scanner by mounting the laser stripe vision sensor to the end-effect of a robot. The accuracy of the laser scanner is limited by the absolute accuracy of the robot. The laser stripe vision sensor can also be fixed to the end of a coordinate measuring machine to form a highly accurate laser scanner;¹² however, the coordinate measuring machine is not suitable to install on the production line. There is another form of laser scanner where the position of the projector with respect to the camera coordinate system (CCS) can be determined by additional reference frames^13,14 or physical constraints.^15,16 Ren et al.^17,18 proposed a linear laser scanner by mounting the laser stripe vision sensor to a linear rail; the linear rail moves the vision sensor to perform linear scanning. The shortcoming of the linear laser scanner is that its relatively large size and heavy load of the linear rail will limit its industrial applications. Li and Xi¹⁹ and Xiao et al.²⁰ presented a rotational laser scanner by mounting the laser stripe vision sensor to a turntable. This has the same drawbacks as the linear laser scanner. The galvanometric laser scanner using the galvanometric element as its mechanical device can solve the aforementioned drawbacks. The galvanometric element consists of a precision motor and a reflective mirror. The precision motor guarantees the accuracy of the galvanometric laser scanner. The load of the precision motor is the lightweight mirror, and it is much lighter than the aforementioned laser scanners.^17–20 The galvanometric element is mounted inside the laser scanner, so the size of the galvanometric laser scanner is relatively smaller. Manakov et al.²¹ designed a two-mirror galvanometric laser scanner with a point-mark laser projector and proposed a model-driven calibration method. However, the rotational axis that lays on the reflective mirror is assumed in the mathematical model, and initial estimations of the intrinsic and extrinsic parameters of the scanning system are calibrated separately. Wissel et al.²² presented a data-driven learning method for calibrating the two-mirror galvanometric laser scanner; however, a large amount of data is needed to train the data-driven model. The point-mark laser projector along with the two-mirror galvanometric element spends a lot of time to sweep through the surface of the object, and consequently, the two-mirror galvanometric laser scanner is not suitable for measuring the shape of the object on the production line. Recently, Chi et al.²³ proposed a laser line auto-scanning system for underwater 3D reconstruction; it overcomes the time-consuming drawback of the point-mark laser scanner.²¹ However, the mathematical model²³ assumes that the galvanometer rotation axis should completely coincide with the line intersected by the laser plane and the mirror of the galvanometer. The assumption is relatively harsh for machining and assembly accuracy, because the direction of the laser stripe plane is very hard to measure, and consequently, it is impossible to adjust the incoming laser stripe plane to hit the galvanometer rotation axis. In summary, available mathematical models^21,23 of the galvanometric laser scanner make some assumptions that require the system to be manufactured and assembled well enough; thus, the available mathematical models are limited in actual applications.

In this article, a galvanometric laser scanner is designed using a one-mirror galvanometric element as its mechanical device. A flexible method is proposed to generate an arbitrary number of control points based on plane constraint by viewing a planar calibration target from multiple orientations. Due to the high repeatability of the galvanometric element, the pre-defined set of light stripe planes in the calibration process can be used in the measuring process. Combined with the look-up table (LUT), the plane-constraint-based calibration method can be used to calibrate the proposed laser scanner. Compared with the invariance of double cross-ratio, the plane-constraint-based calibration approach extracts the same number of control points without error propagation. The remaining sections are organized as follows. Section “System construction” introduces the construction and inspection principle of the proposed laser scanner; section “Mathematical model of a laser stripe vision sensor” presents the mathematical model of the two-step calibration method for a laser stripe vision sensor. The mathematical model consists of the camera model, the projector model and 3D reconstruction in a two-step calibration method. In section “Plane-constraint-based calibration method through planar calibration target,” a plane-constraint-based calibration method using a planar calibration target is described. It includes camera calibration, extracting control points based on plane constraint, transforming from a local WCS to a global CCS, and projector calibration using an orthogonal regression method. To validate the efficiency and accuracy of the proposed method, contrast experiments are conducted in section “Experiments,” and this article ends with concluding remarks in section “Conclusion.”

System construction

The galvanometric laser scanner is designed using a one-mirror galvanometer element as its mechanical device to drive the light stripe sweep along the object as shown in Figure 2. The galvanometric laser scanner comprised a charge-coupled device (CCD) camera (Basler acA1300/30 µm) with a 16 mm lens, a laser line projector (wavelength of 730 nm, beam width ≤1 mm), and a one-mirror galvanometer element which was made by ourselves using a reflective mirror and a stepper motor (Suruga KRE04360 angular resolution is 0.008°). Here, there are 23 stripe lines in the view of the laser scanner, and each pose of the stripe line is fixed in the CCS owing to the high repeatability of the stepper motor. The LUT method consists of two parts: (1) storing calibration matrices for 23 pre-defined laser planes in an LUT and (2) usage of the calibration matrix at a pre-defined rotational angle for 3D reconstruction. The working principle is as follows: an incoming laser stripe plane, which is projected by a laser line projector, hits the reflective mirror and the light stripe formed by intersecting the outgoing laser stripe plane; the object’s surface is captured by the fixed camera. The 3D characteristic information of the object’s surface can be derived from the 2D distorted light stripe images after 3D reconstruction. The camera is triggered to acquire pictures continuously when the stepper motor gives a hardware trigger signal. The hardware trigger signal is given when the stepper motor rotates across its zero position. The reflected laser stripe plane scans across the object’s surface along with the rotation of the galvanometer element; the 3D information of the object’s whole surface can be reconstructed. The measuring process is shown in Figure 3. In Figure 3(a), the edge points of the circle are extracted in the region(s) of interest (ROI) when the laser line scans across the work piece. Figure 3(b) shows the 3D point cloud of the measured circle after 3D reconstruction.

Figure 2.

Schematic diagram of the proposed galvanometric laser scanner.

Figure 3.

The measuring process of the work piece feature: (a) the processed gray image and (b) the corresponding 3D point cloud.

Mathematical model of a laser stripe vision sensor

The one-step calibration method is based on the homography model without considering the lens distortion. The proposed calibration method is similar to the two-step calibration method, and consequently, the basic mathematical model in the two-step calibration method for the laser stripe vision sensor is discussed in the following. The two-step calibration method calibrates the camera first and then calibrates the projector since the laser stripe vision sensor is composed of a camera and a projector.

Camera model

The camera has been extensively studied in the past, and its modeling and calibration techniques are very mature.^24–27 As shown in Figure 4, $P_{ω} = {[\begin{matrix} X & Y & Z \end{matrix}]}^{T}$ is a point in the WCS (O-XYZ), and its coordinates in the CCS (o-xyz) are expressed by $P_{c} = {[\begin{matrix} x & y & z \end{matrix}]}^{T}$ . The transformation relationship from $P_{ω}$ to $P_{c}$ is given by

P_{c} = R \cdot P_{ω} + T

(1)

where R and T are the rotation and translation matrices, respectively.

Figure 4.

Perspective projection model of camera in the laser stripe vision sensor.

Let $P_{n}$ be the projection of $P_{c}$ onto the normalized image plane, which is parallel to the image plane and is unit distance to the lens center. Then, $P_{n}$ can be expressed by

P_{n} = [\begin{matrix} x_{n} \\ y_{n} \end{matrix}] = [\begin{matrix} \frac{x}{z} \\ \frac{y}{z} \end{matrix}]

(2)

Due to the influences of the radial and tangential lens distortion on $P_{n}$ , the distorted projection $P_{d}$ on the normalized image plane is given by²⁵

\begin{array}{l} P_{d} = f_{d} (P_{n}, K) = [\begin{matrix} x_{d} \\ y_{d} \end{matrix}] = P_{n} + (k_{1} r^{2} + k_{2} r^{4}) P_{n} \\ + [\begin{matrix} 2 k_{3} x_{n} y_{n} + k_{4} (r^{2} + 2 x_{n}^{2}) \\ k_{3} (r^{2} + 2 y_{n}^{2}) + 2 k_{4} x_{n} y_{n} \end{matrix}] \end{array}

(3)

where $r^{2} = x_{n}^{2} + y_{n}^{2}$ and $K = [\begin{matrix} \begin{matrix} k_{1} & k_{2} \end{matrix} & \begin{matrix} k_{3} & k_{4} \end{matrix} \end{matrix}]$ represent the lens distortion coefficient. The last two terms in the right side of equation (3) stand for the radial and tangential lens distortion, respectively.

The projection point $P_{i}$ on the image plane coordinate frame can be expressed as follows

P_{i} = [\begin{matrix} u \\ v \end{matrix}] = [\begin{matrix} f_{u} x_{d} + u_{0} \\ f_{v} y_{d} + v_{0} \end{matrix}]

(4)

where $f_{u}$ and $f_{v}$ represent the horizontal and vertical focal length, respectively, and $u_{0}$ and $v_{0}$ are the coordinates of the principle point with respect to the image plane coordinate frame.

According to equations (1)–(4), the ideal camera model can be written as follows using homogeneous coordinates

μ {\tilde{P}}_{i} = {{}^{c^{i}}T}_{C} {{}^{C}H}_{w} {\tilde{P}}_{w} = [\begin{matrix} f_{u} & 0 & u_{0} & 0 \\ 0 & f_{v} & v_{0} & 0 \\ 0 & 0 & 1 & 0 \end{matrix}] [\begin{matrix} R_{3 \times 3} & T_{3 \times 1} \\ 0_{1 \times 3} & 1 \end{matrix}] {\tilde{P}}_{w}

(5)

where $\tilde{P_{ω}} = {[\begin{matrix} X & Y & \begin{matrix} Z & 1 \end{matrix} \end{matrix}]}^{T}$ is a homogeneous coordinate of $P_{ω}$ , similar to $\tilde{P_{i}}$ . The transformation matrices ${{}^{c^{i}}T}_{C}$ and ${{}^{C}T}_{w}$ relate the camera coordinate frame to the normalized image plane frame and the world coordinate frame to the camera coordinate frame, respectively. $μ$ is an arbitrary scale factor.

Equations (3) and (5) show the real perspective projection model of a camera. All parameters in the camera model can be estimated by camera calibration.²⁶

Projector model

The projector model is accurately represented using the equation of a plane, which is called the light stripe plane. To complete 3D reconstruction, the light stripe plane parameters must be described in the camera coordinate frame. The equation of the light stripe plane is described by

a x_{c} + b y_{c} + c z_{c} + d = 0

(6)

where $(\begin{matrix} a & \begin{matrix} b & \begin{matrix} c & d \end{matrix} \end{matrix} \end{matrix})$ are the estimated plane parameters and $(\begin{matrix} a & \begin{matrix} b & c \end{matrix} \end{matrix})$ are the normal vector of the plane. The point $(\begin{matrix} x_{c} & \begin{matrix} y_{c} & z_{c} \end{matrix} \end{matrix})$ is expressed in the CCS. The plane parameters can be estimated using at least three non-collinear control points in the CCS.

The 3D reconstruction

Equations (3), (5), and (6) describe a complete real mathematical model of the laser stripe vision sensor. According to the mathematical model, 3D reconstruction is completed as follows. Here, 3D reconstruction is derived using the intersection of a line and a plane instead of the triangular-based method. Equation (5) is rewritten to describe the transformation from the camera coordinate frame to the normalized image plane frame

μ {\tilde{P}}_{i} = {{}^{c^{i}}T}_{C} {\tilde{P}}_{c} = [\begin{matrix} f_{u} & 0 & u_{0} & 0 \\ 0 & f_{v} & v_{0} & 0 \\ 0 & 0 & 1 & 0 \end{matrix}] {\tilde{P}}_{c}

(7)

where $\tilde{P_{c}} = {[\begin{matrix} x & y & \begin{matrix} z & 1 \end{matrix} \end{matrix}]}^{T}$ is a homogeneous coordinate of $P_{c}$ in the CCS. The column rank of ${{}^{c^{i}}T}_{C}$ is deficient, so there is no left inverse matrix ${{}^{c^{i}}T}_{C}$ . ${{}^{c^{i}}T}_{C}$ which represents the pinhole model of the camera can be considered as a 3D spatial line in the CCS. The point $P_{c}$ is laid on the light stripe plane, so an additional equation is given in the following

a x_{c} + b y_{c} + c z_{c} + d = 0

(8)

Equations (7) and (8) are combined as follows

μ {\tilde{P}}_{i 0} = μ [\begin{matrix} u \\ v \\ 1 \\ 0 \end{matrix}] = {{}^{c^{i}}H}_{C} {\tilde{P}}_{c} = [\begin{matrix} f_{u} & 0 & u_{0} & 0 \\ 0 & f_{v} & v_{0} & 0 \\ 0 & 0 & 1 & 0 \\ a & b & c & d \end{matrix}] [\begin{matrix} x_{c} \\ y_{c} \\ z_{c} \\ 1 \end{matrix}]

(9)

where ${{}^{c^{i}}T}_{C}$ is an invertible matrix due to the structure of the laser stripe vision sensor and $P_{c}$ is very easy to calculate. The first two rows in matrix ${{}^{c^{i}}T}_{C}$ describe a 3D spatial line in the CCS and the last row represents a 3D plane in the same coordinate system. The intersection of the spatial line and plane is derived from equation (9).

Plane-constraint-based calibration method through planar calibration target

A flexible method is proposed to generate an arbitrary number of control points based on plane constraints by viewing a planar calibration target from multiple orientations. It is a very effective method; at least two different poses of the planar calibration target must be viewed by the camera. The plane-constraint-based calibration approach extracts the same number of control points as the invariance of double cross-ratio, but it does not suffer from error propagation. Combined with LUT, the proposed galvanometer laser scanner is calibrated using the plane-constraint-based calibration method.

Camera calibration

Camera calibration is conducted to estimate intrinsic parameters which reflect the optical characteristics of the camera and extrinsic parameters that express the pose of the local WCS relative to the CCS. Zhang²⁶ presents a flexible camera calibration algorithm in which a planar calibration target is observed by a camera from multiple points of view. The camera is fixed, the planar calibration target is freely moved, and the relative calibration target poses are unknown. After camera calibration, the pose of each calibration target with respect to the CCS is determined.

Transformation from local WCS to CCS

As indicated in Figure 5, any points expressed in the local WCS can be transformed into the global CCS

{}^{C}{\tilde{P}}_{i} = {{}^{C}T}_{Wi} \cdot {}^{Wi}{\tilde{P}}_{i}

(10)

where $\tilde{{{}^{C}P}_{i}} = {[\begin{matrix} \begin{matrix} x_{i} & y_{i} \end{matrix} & \begin{matrix} z_{i} & 1 \end{matrix} \end{matrix}]}^{T}$ is the homography coordinate expressed in the global CCS $({C})$ . The transformation matrix ${{}^{C}T}_{Wi}$ represents the pose of the ith local WCS $({W_{i}})$ with respect to ${C}$ . $\tilde{{{}^{W_{i}}P}_{i}} = {[\begin{matrix} \begin{matrix} X_{i} & Y_{i} \end{matrix} & \begin{matrix} Z_{i} & 1 \end{matrix} \end{matrix}]}^{T}$ is the homography coordinate of ith calibration marker expressed in ${W_{i}}$ . Similarly, any calibration markers on the jth local WCS $({W_{j}})$ can also be expressed in the same coordinate system ${C}$ . Transformation matrices ${{}^{C}T}_{Wi}$ and ${{}^{C}T}_{Wj}$ can be easily derived using the extrinsic parameters of the camera. All calibration markers on both the ith and jth planar calibration target are expressed in ${C}$ . The multiple poses of the planar calibration targets can be considered as a virtual 3D calibration target using the above transformation technique.

Figure 5.

The diagrammatic sketch of transforming the multiple local world coordinate system into the camera coordinate system.

Computing camera coordinates of control points

Without loss of generality, the local WCS is defined as follows: its original point is located at the upper left corner of the planar calibration target, its X axis is along one side of the target, and its Y axis is along the other side. As a result, the Z coordinates of any points located on the planar target is 0

Z = 0

(11)

Equation (5) can be expanded as

μ {\tilde{P}}_{i} = {{}^{c^{i}}T}_{C} {{}^{C}H}_{w} {\tilde{P}}_{w} = [\begin{matrix} t_{11} & t_{12} & t_{13} & t_{14} \\ t_{21} & t_{22} & t_{23} & t_{24} \\ t_{31} & t_{32} & t_{33} & t_{34} \end{matrix}] {\tilde{P}}_{w}

(12)

where ${{}^{c^{i}}T}_{C}$ and ${{}^{C}T}_{w}$ contain intrinsic and extrinsic parameters separately, so they are known after camera calibration. However, ${{}^{c^{i}}T}_{C} \cdot {{}^{C}T}_{w}$ is a $3 \times 4$ transformation matrix with row rank r = 3, so there is no left inverse matrix of ${{}^{c^{i}}T}_{C} \cdot {{}^{C}T}_{w}$ . As a result, the world control point $\tilde{P_{ω}}$ cannot be derived only from equation (12). However, all the world control points are constrained on the surface of the planar target, so there is an additional equation (Z = 0) for all the world control points. Equations (11) and (12) are combined to form the plane-constraint-based calibration method for extracting world control points

μ [\begin{matrix} {\tilde{P}}_{i} \\ 0 \end{matrix}] = T {\tilde{P}}_{w} = [\begin{matrix} t_{11} & t_{12} & t_{13} & t_{14} \\ t_{21} & t_{22} & t_{23} & t_{24} \\ t_{31} & t_{32} & t_{33} & t_{34} \\ 0 & 0 & 1 & 0 \end{matrix}] {\tilde{P}}_{w}

(13)

where the transformation matrix T is a 4 ×4 invertible matrix, and the fourth row of matrix T indicates that all the world control points are constrained on the OXY plane of the local WCS. The local world control points can be transformed into the global CCS by equation (10). The planar calibration target can be placed in multiple poses, so the proposed method easily extracts a large number of control points.

Projector calibration

A large number of control points $(\begin{matrix} \begin{matrix} x_{i} & y_{i} \end{matrix} & z_{i} \end{matrix})$ (i = 1, …, m) are expressed in the CCS and can easily be extracted using the proposed method. The objective function is established by minimizing the sum of the squares distance from control points to the fitted plane

{\begin{matrix} f = \sum_{i = 1}^{m} d_{i}^{2} \\ s . t . a^{2} + b^{2} + c^{2} = 1 \end{matrix}

(14)

where $d_{i} = | a x_{i} + b y_{i} + c z_{i} + d |$ and $(\begin{matrix} a & \begin{matrix} b & c \end{matrix} \end{matrix})$ is the normal vector of the fitted plane.

A nonlinear least square method is used by Zhou et al.⁷ to solve equation (14). The orthogonal regression method is employed to solve the problem.²⁸ The plane equation can be written as

N \cdot (X - A) = 0

(15)

where $N = (\begin{matrix} a & \begin{matrix} b & c \end{matrix} \end{matrix})$ is the normal vector of the fitted plane, and A is a point on the fitted plane.

Define $X_{i} = (\begin{matrix} \begin{matrix} x_{i} & y_{i} \end{matrix} & z_{i} \end{matrix})$ to be the control point, and then

X_{i} = A + λ_{i} N + ρ_{i} N_{i}^{⊥}

(16)

where $λ_{i} = N \cdot (X_{i} - A)$ is the orthogonal distance from the control point $X_{i}$ to the fitted plane. $ρ_{i} N_{i}^{⊥}$ is a vector pointing from A to the projection of $X_{i}$ onto the fitted plane.

The sum of the squares distance from control points to the fitted plane is

E (\begin{matrix} A & N \end{matrix}) = \sum_{i = 1}^{m} λ_{i}^{2}

(17)

Define $Y_{i} = (X_{i} - A)$ and $λ_{i} = N \cdot Y_{i} = N^{T} Y_{i} = Y_{i}^{T} N$ is the inner product of N and $Y_{i}$ . As a result, there are two alternates for the objective function

E (\begin{matrix} A & N \end{matrix}) = \sum_{i = 1}^{m} (Y_{i}^{T} N) (Y_{i}^{T} N)^{T} = \sum_{i = 1}^{m} Y_{i}^{T} (N N^{T}) Y_{i}

(18)

and

E (\begin{matrix} A & N \end{matrix}) = \sum_{i = 1}^{m} (N^{T} Y_{i}) (N^{T} Y_{i})^{T} = N^{T} (\sum_{i = 1}^{m} Y_{i} Y_{i}^{T}) N

(19)

Take the derivative of equation (18) with respect to A to obtain

\frac{\partial E}{\partial A} = - 2 (N N^{T}) \sum_{i = 1}^{m} Y_{i}

(20)

When $\sum_{i = 0}^{m} Y_{i} = 0$ , the partial derivative is zero. Then, A is

A = \frac{1}{m} \sum_{i = 1}^{m} X_{i}

(21)

Given A, the symmetric matrix $\sum_{i = 0}^{m} Y_{i} Y_{i}^{T}$ is known. Then, the singular value decomposition (SVD) of the symmetric matrix is

\sum_{i = 1}^{m} Y_{i} Y_{i}^{T} = US U^{T}

(22)

where U and S are the eigenvectors and the eigenvalues of the matrix $\sum_{i = 0}^{m} Y_{i} Y_{i}^{T}$ , respectively. Equation (19) reaches its minimum when N equals the eigenvector corresponding to the smallest eigenvalue.

Summary

To sum up, the procedure of the proposed calibration method is described in Figure 6 as follows:

The planar calibration target is located at several different poses, and each pose of the planar calibration target is captured by the camera. The last two poses are captured twice in two cases, one with no laser stripe line projected onto the planar for camera calibration and the other with laser stripe line for projector calibration.

The captured images without laser stripe line are used to calibrate the camera and both intrinsic parameters and extrinsic parameters are determined.

Extract all the image coordinates of the control points in captured images.

Correct the distortion for image coordinates according to equation (3).

Transform the image coordinates of the control points to the local WCS using both camera intrinsic parameters and extrinsic parameters according to equation (13).

Transform the local world coordinate of control points to the global CCS according to equation (10).

Calibrate the projector according to equations (21) and (22).

Each laser stripe plane is calibrated and an LUT is established to complete the calibration of the proposed galvanometric laser scanner.

Figure 6.

Flowchart of the system calibration procedures.

Experiments

The proposed calibration method is compared with three other calibration methods: the one-step calibration method with the invariance of cross-ratio,² the one-step calibration method with the invariance of double cross-ratio,⁵ and the two-step method with the invariance of cross-ratio.⁷ Four calibration methods use the same calibration images to complete the calibration procedure. Experiments are conducted to verify the efficiency and accuracy of the proposed calibration method.

Calibration procedure

The planar calibration target is located at 20 different poses and the last two poses are captured twice in two cases, one with no laser stripe line projected onto the planar for camera calibration and the other with laser stripe line for projector calibration. There are 20 images for camera calibration and 2 images for each laser stripe plane calibration.

Figure 7 shows the last two images of the planar calibration target in the camera calibration procedure. The coordinate (i, j) in Figure 7 indicates that the center of the circle is located at the ith row and the jth column. The procedure of determining the coordinate is modeled as Single Source Shortest Paths (SSSP) and solved by the Bellman–Ford algorithm. The planar target contains a pattern of 7 ×7 circles. The distance between the adjacent two circles is 3.75 mm in both the horizontal and the vertical directions. Figure 8 shows the four captured images with strip lines lying on the planar target. Figure 8(a) shows the first strip line lying on the 19th pose of the planar target, and Figure 8(b) shows the first strip line lying on the 20th pose of the planar target. Figure 8(c) and (d) shows the 23rd strip line lying on the 19th and 20th pose of the planar target, respectively. Knowing the poses of the last two planar targets, the first and the 23rd laser stripe plane can be calibrated from Figure 8. All 23 laser stripe planes are calibrated and sorted in an LUT for the measuring process.

Figure 7.

The last two images of the planar target.

Figure 8.

Stripe line lying on the planar calibration target: (a) the first stripe line lying on the 19th planar calibration target; (b) the first stripe line lying on the 20th planar calibration target; (c) the 23rd stripe line lying on the 19th planar calibration target; (d) the 23rd stripe line lying on the 20th planar calibration target.

Table 1 shows the calibrated parameters of the proposed method and the other three methods above; only the first laser stripe plane parameters are shown in Table 1. As indicated in Table 1, the proposed method and the Zhou’s method are the two-step calibration methods. The camera parameters are the same for these two methods, while the projector parameters differ. There is a translation (0.2 mm) along the norm vector direction of the plane between the two laser stripe plane equations. However, the rest of the two methods are the one-step calibration method. The homography matrices $(T_{3}, T_{4})$ are different from each other.

Table 1.

Calibration results of the proposed laser scanner.

Method	$f_{x}, f_{y}$	$u_{0}, v_{0}$	K = $K = [\begin{matrix} k_{1} & k_{2} \\ p_{1} & p_{2} \end{matrix}]$	$[\begin{matrix} a & b \\ c & d \end{matrix}]$	T
Proposed method	4462.08 4462.95	642.539 501.843	$[\begin{matrix} - 0.2647 & 2.4354 \\ - 0.0014 & - 0.0001 \end{matrix}]$	$[\begin{matrix} 0.0038 & - 0.8511 \\ 0.5250 & - 74.6248 \end{matrix}]$	T₁
Zhou et al. ⁷	4462.08 4462.95	642.539 501.843	$[\begin{matrix} - 0.2647 & 2.4354 \\ - 0.0014 & - 0.0001 \end{matrix}]$	$[\begin{matrix} 0.0038 & - 0.8502 \\ 0.5264 & - 74.8130 \end{matrix}]$	T₂
Huynh et al. ²	−	−	−	−	T₃
Forest ⁵	−	−	−	−	T₄

where

\begin{matrix} T_{1} = [\begin{matrix} 4462.08 & 0 & \begin{matrix} 642.539 & 0 \end{matrix} \\ 0 & 4462.95 & \begin{matrix} 501.843 & 0 \end{matrix} \\ \begin{matrix} 0 \\ 0.0038 \end{matrix} & \begin{matrix} 0 \\ 0.8511 \end{matrix} & \begin{matrix} \begin{matrix} 1.00000 & 0 \end{matrix} \\ \begin{matrix} 0.5250 & 74.6248 \end{matrix} \end{matrix} \end{matrix}] \\ T_{2} = [\begin{matrix} 4462.08 & 0 & \begin{matrix} 642.539 & 0 \end{matrix} \\ 0 & 4462.95 & \begin{matrix} 501.843 & 0 \end{matrix} \\ \begin{matrix} 0 \\ 0.0038 \end{matrix} & \begin{matrix} 0 \\ 0.8502 \end{matrix} & \begin{matrix} \begin{matrix} 1.00000 & 0 \end{matrix} \\ \begin{matrix} 0.5264 & 74.8130 \end{matrix} \end{matrix} \end{matrix}] \\ T_{3} = [\begin{matrix} 4531.69 & - 5.71 & \begin{matrix} 653.17 & 4.88 \end{matrix} \\ 0.91 & 4522.27 & \begin{matrix} 510.39 & - 47.90 \end{matrix} \\ 0.0052 & - 0.0066 & \begin{matrix} 1.0166 & 0.0072 \end{matrix} \end{matrix}] \\ T_{4} = [\begin{matrix} 4526.75 & - 8.52 & \begin{matrix} 652.72 & 4.92 \end{matrix} \\ 0.57 & 4518.70 & \begin{matrix} 510.19 & - 47.76 \end{matrix} \\ 0.0031 & - 0.0087 & \begin{matrix} 1.0161 & 0.0072 \end{matrix} \end{matrix}] \end{matrix}

Experiment results

The radius of a standard sphere is measured to verify the efficiency and accuracy of the proposed calibration method. As shown in Figure 9, the designed galvanometer laser scanner is mounted at the end of the robot to construct a robot-based inspecting system. Coordinates of the spherical centers obtained by the laser scanner are used to calibrate the inspecting system. The standard sphere (bearing steel ball and coated with matt material) is measured by the sensor at six different robot positions. The sphere is inspected on a coordinate measuring machine (CMM; Thome, 2 + (L/350) μm) by sampling 15 points evenly on its surface; the standard deviation of distance errors is approximately 12.1 μm, and the diameter is 25.4160 mm. In Figure 10(a), the surface points of the sphere are extracted in the ROI and the corresponding 3D point clouds are obtained after calibrating the laser scanner. Figure 10(b) shows the 3D point cloud and the fitted sphere by nonlinear least-squares sphere fitting algorithm (TLS_A).²⁹ The measurement radius and the root mean square error (RMSE) among four methods are given in Table 2. As indicated in Table 2 and in Figure 11, the RMSE of the one-step calibration method is much larger than the other three two-step calibration methods. This is because the one-step method is based on the homography model, which neglects the lens distortion when the camera is modeled. The RMSE of the plane-constraint-based calibration method is smaller than that of Zhou’s method, because there are only 14 control points to fit each laser stripe plane in the calibration process. The influence of noise contained in the control points will greatly affect the accuracy of the fitted plane when control points are limited in quantity.⁴ The 7×7 dot array calibration target should be placed at many positions to extract a large number of control points via Zhou’s calibration method, and it is inconvenient to implement. The plane-constraint-based method theoretically extracts all control points on the calibration target by imposing a plane constraint (Z = 0) on them, so it estimates the laser stripe plane effectively and accurately.

Figure 9.

The experimental setup in the automobile production line.

Figure 10.

The measuring process of the sphere: (a) the processed gray image and (b) the fitted sphere by TLS_A.

Table 2.

The measurement radius among four methods.

Robot pose	Proposed method (mm)	Zhou et al.⁷ (mm)	Huynh et al.² (mm)	Forest⁵ (mm)
1	12.7148	12.725	12.758	12.767
2	12.705	12.718	12.749	12.758
3	12.716	12.729	12.762	12.772
4	12.722	12.734	12.769	12.779
5	12.701	12.714	12.746	12.766
6	12.700	12.713	12.742	12.750
RMSE	0.0084	0.0162	0.0473	0.0581

RMSE: root mean square error.

Figure 11.

The radius error among four methods.

Conclusion

In this article, a galvanometric laser scanner is proposed using a one-mirror galvanometer element as its mechanical device to drive the light stripe to sweep along the object. A flexible plane-constraint-based calibration method together with an LUT calibration method is proposed to calibrate the galvanometric laser scanner. The proposed method only needs the laser scanner to view a planar calibration target located at least two different orientations; orientations of the planar target need not be known, so it is very flexible and easy to implement. Compared with the invariance of double cross-ratio, the plane-constraint-based calibration method generates the same number of control points without error propagation. The experiments conducted in the automobile production line reveal that the proposed approach is more accurate than conventional methods and is effective in the vision inspection system. In addition, the idea of a plane-constraint-based calibration method can be used for reference when other forms of laser stripe vision sensors are calibrated.

Footnotes

Handling Editor: Crinela Pislaru

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is supported by the National Natural Science Foundation of China (nos 51175386 and 51405350).

ORCID iD

Yang Mao

References

Gan

Tang

. Visual sensing and its applications: integration of laser sensors to industrial robots. Hangzhou, China: Zhejiang University Press, 2011.

Huynh

Owens

Hartmann

. Calibrating a structured light stripe system: a novel approach. Int J Comput Vision 1999; 33: 73–86.

Liu

Zeng

et al . A new method of calibration in 3D vision system based on structure-light. Chinese J Comput 1995; 18: 450–456.

Wei

Zhang

. Calibration approach for structured-light-stripe vision sensor based on the invariance of double cross-ratio. Opt Eng 2003; 42: 2956–2966.

Forest Collado

. New methods for triangulation-based shape acquisition using laser scanners. Girona: Universitat de Girona, 2004.

Niola

Rossi

Savino

et al . A method for the calibration of a 3-D laser scanner. Robot Cim-Int Manuf 2011; 27: 479–484.

Zhou

Zhang

Jiang

. Constructing feature points for calibrating a structured light vision sensor by viewing a plane from unknown orientations. Opt Laser Eng 2005; 43: 1056–1070.

Sun

. Universal method for calibrating structured-light vision sensor on the spot. J Mech Eng 2009; 45: 174–177.

Guo

Zhu

et al . Large depth-of-view portable three-dimensional laser scanner and its segmental calibration for robot vision. Opt Laser Eng 2007; 45: 1077–1087.

10.

Zhu

Guo

et al . Calibration of a portable laser 3-D scanner used by a robot and its use in measurement. Opt Eng 2008; 47: 017202.

11.

Chen

Jin

et al . Calibration of a multiple axes 3-D laser scanning system consisting of robot, portable laser scanner and turntable. Optik 2011; 122: 324–329.

12.

Xie

Wang

Chi

. Simultaneous calibration of the intrinsic and extrinsic parameters of structured-light sensors. Opt Laser Eng 2014; 58: 9–18.

13.

Abzal

Varshosaz

Saadatseresht

. Development of a new laser triangulation system based on an optical frame of reference. Photogramm Rec 2011; 26: 293–306.

14.

Zhang

Lin

. Building a 3D scanner system based on monocular vision. Appl Optics 2012; 51: 1638–1644.

15.

Furukawa

Kawasaki

. Laser range scanner based on self-calibration techniques using coplanarities and metric constraints. Comput Vis Image Und 2009; 113: 1118–1129.

16.

Ozan

Gumustekin

. Calibration of double stripe 3D laser scanner systems using planarity and orthogonality constraints. Digit Signal Process 2014; 24: 231–243.

17.

Ren

Yin

Zhu

. Calibration technology in application of robot-laser scanning system. Opt Eng 2012; 51: 114204–114204.

18.

Yin

Ren

Guo

et al . Development and calibration of an integrated 3D scanning system for high-accuracy large-scale metrology. Measurement 2014; 54: 65–76.

19.

. Free and global pose calibration of a rotating laser monocular vision sensor for robotic 3D measurement system. In: International conference on optics in precision engineering and nanotechnology (icOPEN2013), Singapore, 22 June 2013, 87690I.

20.

Xiao

et al . A new three-dimensional laser scanner design and its performance analysis. Optik 2015; 126: 701–707.

21.

Manakov

Seidel

Ihrke

. A mathematical model and calibration procedure for galvanometric laser scanning systems. In: Vision, modeling, and visualization workshop 2011, Berlin, Germany, 4–6 October 2011, pp.207–214.

22.

Wissel

Wagner

Stuber

et al . Data-driven learning for calibrating galvanometric laser scanners. IEEE Sens J 2015; 15: 5709–5717.

23.

Chi

Xie

Chen

. A laser line auto-scanning system for underwater 3D reconstruction. Sensors 2016; 16: 1534.

24.

Tsai

. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE T Robotic Autom 1987; 3: 323–344.

25.

Heikkila

Silven

. A four-step camera calibration procedure with implicit image correction. In: IEEE computer society conference on computer vision & pattern recognition, San Juan, Puerto Rico, 17–19 June 1997, pp.1106-1112. New York: IEEE.

26.

Zhang

. A flexible new technique for camera calibration. IEEE T Pattern Anal 2000; 22: 1330–1334.

27.

Chen

Jin

et al . Accurate calibration for a camera–projector measurement system based on structured light projection. Opt Laser Eng 2009; 47: 310–319.

28.

Eberly

. Least squares fitting of data. Chapel Hill, NC: Magic Software, 2000.

29.

Sun

Hill

Mcbride

. An investigation of the robustness of the nonlinear least-squares sphere fitting method to small segment angle surfaces. Precis Eng 2008; 32: 55–62.

Plane-constraint-based calibration method for a galvanometric laser scanner

Abstract

Keywords

Introduction

System construction

Mathematical model of a laser stripe vision sensor

Camera model

Projector model

The 3D reconstruction

Plane-constraint-based calibration method through planar calibration target

Camera calibration

Transformation from local WCS to CCS

Computing camera coordinates of control points

Projector calibration

Summary

Experiments

Calibration procedure

Experiment results

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References