3D Wide FOV Scanning Measurement System Based on Multiline Structured-Light Sensors

Abstract

Structured-light three-dimensional (3D) vision measurement is currently one of the most common approaches to obtain 3D surface data. However, the existing structured-light scanning measurement systems are primarily constructed on the basis of single sensor, which inevitably generates three obvious problems: limited measurement range, blind measurement area, and low scanning efficiency. To solve these problems, we developed a novel 3D wide FOV scanning measurement system which adopted two multiline structured-light sensors. Each sensor is composed of a digital CCD camera and three line-structured-light projectors. During the measurement process, the measured object is scanned by the two sensors from two different angles at a certain speed. Consequently, the measurement range is expanded and the blind measurement area is reduced. More importantly, since six light stripes are simultaneously projected on the object surface, the scanning efficiency is greatly improved. The Multiline Structured-light Sensors Scanning Measurement System (MSSS) is calibrated on site by a 2D pattern. The experimental results show that the RMS errors of the system for calibration and measurement are less than 0.092 mm and 0.168 mm, respectively, which proves that the MSSS is applicable for obtaining 3D object surface with high efficiency and accuracy.

1. Introduction

Structured-light 3D vision measurement is becoming increasingly important owing to its noncontact, high precision, and good system flexibility. A basic structured-light sensor is usually composed of one camera and one laser projector together forming an active stereo pair. The laser projector projects a light plane onto an object. When the light plane reaches the target surface, it forms a distorted stripe of illuminated points due to the irregular surface. Then the camera records the illuminated target. If the sensor is properly calibrated, the 3D coordinates of the illuminated points can be calculated by the well-known triangulation method.

According to the number of structured-light sensors, the existing structured-light 3D surface capturing system can be currently classified into two categories. One is single structured-light 3D surface capturing system which means that it contains only one structured-light sensor and projects a single light stripe on the object surface. This kind of system is easy to implement due to its simple structure, but its major defects are quite obvious: measurement range is limited and there are blind measurement areas for complicated objects. As a result, the single structured-light 3D surface capturing system is only applied to measure objects with simple structure and no hiding surface [1–4]. The other category is the double structured-light 3D surface capturing system, which is composed of two structured-light sensors. For such a system, the measured object is scanned from two different angels, so the measurement range is expanded and the dead zone is reduced relatively. However, the scanning efficiency of the double structured-light 3D surface capturing system is low, because the number of light stripes is small and it only covers a small area on the object surface.

Some vision measurement systems based on structured-light have already been successfully applied to industrial inspection. For example, Alippi et al. developed a detection system for track profiles of railways based on laser scanning and image analysis [5]; Loh and Lu presented a solder joint inspection system using structured-light [6]; Haug and Pritschow proposed a robust laser-stripe sensor system for automated weld-seam-tracking in shipbuilding industry [7].

All the preceding inspection systems have a common point that they are all constructed on the basis of a single structured light sensor, as single structured-light system is low cost and easy to implement due to the simple structure. However, if the single structured-light sensor is adopted in the 3D surface measurement system, three obvious problems are inevitable: limited measurement range, blind measurement area, and low scanning efficiency. As a result, the single structured-light 3D surface measurement system is only applicable to the object with simple structure and no hidden surface [8].

Aiming at overcoming these problems, we design a 3D wide FOV scanning measurement system, which adopts two multiline structured-light sensors to construct the system. Each sensor is composed of one digital CCD camera and three line-structured-light projectors. While the object is moving at a certain speed, the system projects six parallel light stripes onto the object surface and scans from the two sides of the measured object. Therefore the FOV and measurement range had been expanded, dead zone is effectively reduced, and, most importantly, the scanning efficiency is greatly improved.

To design and use a MSSS system, three main aspects need to be made efforts on: the measurement model, the extraction and arrangement of light stripes, and system calibration.

Firstly, establishing a reasonable measurement model is the first requirement to construct a structured-light 3D surface measurement system. Because, in the scanning process, all the measurement data is calculated according to the measurement model, many existing measurement models have limitations. The model described in [9] requires the camera optical axis to be parallel to the datum axis, while, in the model established in [10], the structured-light plane should be perpendicular to the measurement platform. Both of the two models have restrictions due to the spatial arrangement limitations of their system components, which increased the difficulty of system implementation.

Secondly, subpixel image coordinates of the light stripes need to be localized in order to guarantee the sensor calibration accuracy and the measurement accuracy. A larger number of extraction methods at subpixel level have been presented in previous works, and all of these methods require certain conditions [11–14]. However, most of them are not robust enough to prevent suffering from considerable noise and the blaze of light stripe. Meanwhile, the arrangement of light stripes is also important as each light stripe is required to be matched to the corresponded projector, so that the 3D coordinates of the illuminated points can be calculated.

Thirdly, the structured-light sensor calibration, which directly affects the measurement accuracy, is the key to the 3D measurement results. The structured-light sensor calibration includes camera calibration and projector calibration. The camera calibration is to estimate the world-to-image perspective transformation matrix and the distortion coefficients of the camera lens. The projector calibration is to determine the coefficients of the light plane equation in space. There are many calibration algorithms for reference: Dewar [15] proposed a calibration method based on self-generated targets; Duan et al. [16] used a toothed target and 1-D mobile stage to locate the projecting light plane of the sensor in the camera coordinate and calculated the parameters with a restrained penalty function method; Huynh et al. [17] proposed the cross-ratio invariance calibration method. However, in our case, the data fusion and the spacial relationship between the two sensors should also be considered, making those methods inconvenient. In addition, the calibration methods described in [18, 19] require stringent standard calibration patterns with high positioning accuracy, which make the calibration process costly and difficult to implement.

In this paper, the potential of a MSSS is assessed. Emphases are put on the three main aspects mentioned above. Moreover, the following three aspects should be noted. First, the mathematical model of the MSSS should fit the structure which is fabricated and assembled without particular spatial constrains. Second, the light stripes should be extracted at subpixel level and arranged in an appropriate order. Third, the system calibration should be simple and easy to implement without requiring complex calibration patterns or high-precision auxiliary equipment. The rest of contents of this paper are organized as follows. In Section 2, the structure of the MSSS is introduced. The measurement model of the system is built and explained in Section 3. In Section 4, the extraction and arrangement methods of light stripes are given. In Section 5, the system calibration approach for the MSSS is described in detail. Calibration experiments and the experimental measurement results are presented in Section 6. And the paper ends with conclusions in Section 7.

2. Construct of the MSSS

The MSSS shown in Figure 1 mainly consists of two multiline structured-light sensors, a glass platform, a motorized linear motion stage, a stepping motor and its controller, a grating ruler, and a computer. Each sensor is composed of one digital CCD camera and three line-structured-light projectors. The block diagram of the system is shown in Figure 2.

Figure 1:

Schematic diagram of the MSSS scanning measurement system.

Figure 2:

Block diagram of the system.

The measured object is placed on the glass platform, and the six laser projectors generate six distorted light stripes on the object surface, which contain the 3D characteristics information of the object surface since the stripes are modulated by the depth changing. Two CCD cameras capture the images of the distorted light stripes in two different angles. After calculating the 3D coordinates of points on the light stripes from the 2D images on the basis of parameters of cameras and light plane equations, the profiles of the six light plane sections can be obtained. During the measurement process, two sensors are driven by the stepping motor along one direction to scan the object at a certain speed; thus, a series of profiles can be acquired. At the same time, the separation distances between the profiles are read from the grating ruler. By building up the profiles according to the separation distances, the entire 3D surface of the measured object can be obtained.

3. Measurement Model

The measurement model of the MSSS is shown in Figure 3. The coordinate systems in Figure 3 are defined as follows: o_c1 − x_c1y_c1z_c1 is camera 1 coordinate system, o_c2 − x_c2y_c2z_c2 is camera 2 coordinate system, o_u1 − x_u1y_u1 is nondistortion image coordinate system of camera 1, o_u2 − x_u2y_u2 is nondistortion image coordinate system of camera 2, and o _w − x _w y _w z _w is world coordinate system. Those coordinate systems are relatively static during the scanning process. In addition, O_m1 and O_m2 are the principal points of the two cameras, π _s ^l is the sth light plane in sensor 1, and π _s ^r is the sth light plane in sensor 2. The following description only takes the light plane π₁ ^l in sensor 1 as an example because the model of either sensor is similar.

Figure 3:

The model of the MSSS scanning measurement system.

For any illuminated point P on the light stripe formed by light plane π₁ ^l and the object surface, its coordinate in o _w − x _w y _w z _w is denoted by ${\mathit{P}}_w={\left(x_w,y_w,z_w\right)}^T$ and its coordinate in o_c1 − x_c1y_c1z_c1 is denoted by ${\mathit{P}}_c={\left(x_c,y_c,z_c\right)}^T$ . The Euclidian transformation between o _w − x _w y _w z _w and o_c1 − x_c1y_c1z_c1 is specified by a rotation matrix and a translation vector as follows:

${\mathit{P}}_c=\mathit{R}{\mathit{P}}_w+\mathit{T},$

(1)

where R is the 3 × 3 rotation matrix and T is the 3 × 1 translation vector. R and T are called the external parameters.

Point p on the image plane is the projection of P. The homogeneous coordinate of p in o_u1 − x_u1y_u1 is denoted by ${\mathit{P}}_u={\left(x_u,y_u,\mathrm{1}\right)}^T$ . According to the pinhole camera model, the transformation from the camera coordinate to the 2D undistorted image coordinate is expressed as

$\rho {\mathit{P}}_u=\mathit{A}{\mathit{P}}_c,$

(2)

where $\mathit{A}=\left[\begin{array}{ccc}{f}_x& \mathrm{0}& u_{\mathrm{0}}\\ \mathrm{0}& f_y& v_{\mathrm{0}}\\ \mathrm{0}& \mathrm{0}& \mathrm{1}\end{array}\right]$ is the internal parameters matrix. f _x and f _y are the effective focal length of the camera in the x and y directions, (u₀,v₀) is the coordinate of the principal point in o_u1 − x_u1y_u1, and ρ is a scale factor.

Real lenses do not satisfy the pinhole model and usually exhibit some degree of geometric distortion. Thus, considering the first two terms of the radial distortion for the camera lens, we have the following equations for each point in each image:

$\begin{array}{c}{x}_u=x_d\left[\mathrm{1}+k_{\mathrm{1}}\left(x_d^{\mathrm{2}}+y_d^{\mathrm{2}}\right)+k_{\mathrm{2}}{\left(x_d^{\mathrm{2}}+y_d^{\mathrm{2}}\right)}^{\mathrm{2}}\right],\\ y_u=y_d\left[\mathrm{1}+k_{\mathrm{1}}\left(x_d^{\mathrm{2}}+y_d^{\mathrm{2}}\right)+k_{\mathrm{2}}{\left(x_d^{\mathrm{2}}+y_d^{\mathrm{2}}\right)}^{\mathrm{2}}\right],\end{array}$

(3)

where ${\mathit{P}}_d={\left(x_d,y_d,\mathrm{1}\right)}^T$ is the distortion image coordinates of p.

In camera 1 coordinate system, the space equation of light plane π₁ ^l is described by

$a{x}_c+b{y}_c+c{z}_c+d=\mathrm{0}.$

(4)

The measurement coordinate system o _m − x _m y _m z _m is established at the position of the world coordinate system where the first image is captured during the scanning process. It should be noted that the measurement coordinate system is fixed and does not move with the translation of the sensors. Since the sensors are moved linearly, the transformation relationship between the measurement coordinate system o _m − x _m y _m z _m and the world coordinate system o _w − x _w y _w z _w is only translation. Assume that the coordinate of point P in o _m − x _m y _m z _m is ${\mathit{P}}_m={\left(x_m,y_m,z_m\right)}^T$ , and then the translation can be described by

${\mathit{P}}_m={\mathit{P}}_w+d\mathit{V},$

(5)

where V is a 3 × 1 unit translation vector representing the scanning direction in the world coordinate system, and d is the translation distance read from the grating ruler (starting at 0).

Equations (1)–(5) are the complete measurement model of the system. In sensor 1, the 3D measurement coordinates ${\left(x_m,y_m,z_m\right)}^T$ of an illuminated point on the light stripe formed by light plane π₁ ^l can be derived from the corresponding 2D image coordinate. The process is as follows.

1. Get the image coordinates ${\left(x_d,y_d\right)}^T$ of the center position of the light stripe corresponding to the light plane π₁ ^l by image processing.

2. Calculate the nondistortion image coordinate ${\left(x_u,y_u\right)}^T$ from (3) to correct the radial distortion of the camera lens.

3. Combine (2) and (4) to get the 3D coordinates in camera frame ${\left(x_c,y_c,z_c\right)}^T$ .

4. Convert the camera coordinates to world coordinates ${\left(x_w,y_w,z_w\right)}^T$ according to (1).

5. Transform the world coordinates ${\left(x_w,y_w,z_w\right)}^T$ to measurement coordinates ${\left(x_m,y_m,z_m\right)}^T$ by (5).

The 3D measurement coordinates of the illuminated points on the other light stripes can also be calculated by the same method.

4. Light Stripes Extraction and Arrangement

In the MSSS, the captured light stripes are usually not continuous, as shown in Figure 4(a), which is caused by the uneven surface and the irregular surface material. However, according to the measurement model, the 3D coordinates of the points on the light stripes can be correctly calculated as long as the points correspond to the light plane equation. Therefore, it is necessary to find the one-to-one correspondence relationship between the light stripes and the light. The best solution is to arrange the light stripes in an appropriate order. We divided the extraction and arrangement of light stripes into four steps.

Figure 4:

Light stripes extraction and arrangement. (a) Image with light stripes. (b) Extracted light stripes. (c) Light stripes after disturbances elimination. (d) Linked light stripes. (e) Light stripes arrangement.

Firstly, extract light stripe, which means locating the center of the light stripe. We adopt a method presented by Steger [20]. For each pixel, the second order Taylor polynomial is computed by convolving the image with the derivatives of a Gaussian smoothing kernel. Line points are required to have a vanishing gradient and high curvature in the direction perpendicular to the line. The resulting filter generates a single response for each line. The line position can be determined with subpixel accuracy and the algorithm scales to lines with arbitrary width. The extracted results of light stripes are shown in Figure 4(b).

Secondly, eliminate the disturbances. The disturbances come from two aspects: one is the small thorns at the ends of the light stripes; the other is the short segment that is obviously out of the area covered by light stripes. Those disturbances can be eliminated easily by setting a threshold for the length of the light stripes. The length is calculated as the sum of the Euclidian distances between the successive points on light stripe as follows:

$L=\stackrel{m-\mathrm{2}}{\sum _{i=\mathrm{0}}}{\parallel {\mathit{P}}_{i+\mathrm{1}}-{\mathit{P}}_i\parallel }_{\mathrm{2}},$

(6)

where P _i = (x _i ,y _i ) ^T is the image coordinate of the ith point, P_{i + 1} = (x_{i + 1},y_{i + 1}) ^T is the image coordinate of the (i + 1)-th point, and m is the total number of the points on the light stripe. Figure 4(c) shows the light stripes after disturbances elimination, in which the length threshold is 20, and we consider all of them are effective.

Thirdly, link the light stripes. The purpose of this step is to tie up the light stripes that belong to the same light plane and prepare for next arrangement. For any two light stripes in Figure 4(c), if the difference between the abscissas of endpoints is less than the preset threshold, the two light stripes can be linked. Figure 4(d) is the light linked stripes from Figure 4(c) and the preset threshold is 30. It should be noted that this step is just to tie up the light stripes that belong to the same light plane, instead of adding light points at the gaps.

Finally, arrange the linked light stripes. The arrangement principle is based on the distance between the straight line determined by the left edge of the image and the light stripes: the light stripe with shorter distance is more close to the left. The distance between a straight line and a light stripe is defined as the minimum of distances from the points on the light stripe to the line:

$d=\underset{\mathrm{1}\le i\le m}{\min }\left|a{x}_i+b{y}_i+c\right|,$

(7)

where P _i = (x _i ,y _i ) ^T is the image coordinates of the ith point on the light stripe, m is the total number of points, ax + by + c = 0 is the equation of straight line on the image plane, and, here, the straight line is x = 0. The ordered light stripes are shown in Figure 4(e).

5. System Calibration

Calibration is the third major problem to be solved in MSSS. The parameters in the MSSS and their physical meanings are listed in Table 1 and all of them are determined by calibration.

Table 1:

System parameters.

Category	Parameters	Physical meaning	Remarks
Internal parameters	(f x ,f y )	Focal length in the x andy directions	One each sensor
	(u0,v0)	Main point coordinates	One each sensor
	(k1,k2)	Radial distortion parameters	One each sensor
External parameters	R	Rotation matrix	One each sensor
	T	Translation vector	One each sensor
Light plane equation	(a,b,c,d)	Light plane equation coefficients	Three each sensor
Unit translation vector	V	Scanning direction	Only sensor 1

5.1. Parameters of Camera and Light Plane Equations

The internal and external parameters of the two cameras can be calibrated by Zhang's method described in [21].

The calibration of light plane equation can be obtained by using a 2D planar target. Take sensor 1 as an example. The calibration model of line-structured-light sensor is shown in Figure 5. The world coordinate system o _w − x _w y _w z _w is established as shown in Figure 5: o _w − x _w y _w is the target plane and the z component is 0. Light stripe L, projected as l in the image plane, is the intersection of the structured-light plane π₁ ^l and the target plane.

Figure 5:

Calibration model of line structured-light sensor.

Owing to the 2D projective transformation between the target plane and the image plane, the coordinates of the points on the image plane can be transformed from the coordinates of the points on the target plane through a 3 × 3 homography matrix:

$s{\mathit{P}}_u=\mathit{H}{\mathit{P}}_w^{\prime },$

(8)

where H is the homography matrix; ${\mathit{P}}_u={\left(x_u,y_u,\mathrm{1}\right)}^T$ is the image coordinate of the feature points on the 2D target and ${\mathit{P}}_w^{\prime }={\left(x_w,y_w,\mathrm{1}\right)}^T$ is the local 2D world coordinate of the corresponding points; s is a nonzero scale factor.

Matrix H has 8 degrees of freedom and it can be obtained with a scale factor from (8) requiring at least four noncollinear corresponding points. Since the matrix H obtained by the linear approach is susceptible to noise interference, we establish an objective function to optimize the matrix H in order to get an optimal solution. The objective function is the sum of the squares of the distance between two coordinates:

$\stackrel{m}{\sum _{i=\mathrm{1}}}{\parallel {\mathit{P}}_{ui}-\mathit{F}\left(\mathit{H},{\mathit{P}}_{wi}^{\prime }\right)\parallel }^{\mathrm{2}},$

(9)

where P _ui is the image coordinate of the ith feature point on the planar target; F (H,P _wi ′) is the coordinate that P _wi ′ transformed to the image coordinate system through H matrix. P _wi ′ is the local 2D world coordinate of the ith feature point on the target, and m is the number of the feature points on the target. The nonlinear optimization problem is solved by Levenberg-Marquardt optimization algorithm, and initial value can be got from the linear solution.

Since matrix H is a nonsingular matrix, we can calculate the local 2D world coordinates of points on target plane from the corresponding image coordinate through the inverse matrix of H :

${\left(x_w,y_w,\mathrm{1}\right)}^T=s{\mathit{H}}^{-\mathrm{1}}{\left(x_u,y_u,\mathrm{1}\right)}^T.$

(10)

In summary, the camera coordinates of the points on the light stripe L can be obtained in the following three steps. Firstly, get the image coordinates of the feature points on the 2D planar target and calculate the homography matrix H according to (8)–(9). Secondly, get the image coordinates ${\left(u,v,\mathrm{1}\right)}^T$ of the points in the light stripe l, and use (10) to calculate the local 2D world coordinate ${\left(x_w,y_w,\mathrm{1}\right)}^T$ , whose corresponding 3D world coordinate is ${\mathit{P}}_w={\left(x_w,y_w,\mathrm{0}\right)}^T$ . Finally, convert ${\left(x_w,y_w,\mathrm{0}\right)}^T$ to camera 1 coordinate system to get the camera coordinates of points on the light stripe L by (1).

After placing the planar target at several different positions, coordinates of a sufficiently large number of points in the camera coordinate system on the light plane π₁ ^l can be calculated using the above method, after which the space equation of light plane π₁ ^l can be fitted. Similarly, the space equations of the other light planes can also be obtained in the same way.

In practical, before calculating the space equation of light planes, the one-to-one corresponding relationship of the light planes and the straight light stripes in the images must be determined in advance. The simplest solution is to arrange the light stripes from left to right using the method described in Section 4.

5.2. Unit Translation Vector

As shown in Figure 6, the position of the world coordinate system is fixed while camera 1 is driven to move along the direction of the vector V .

Figure 6:

Mathematical model of the sensor translation.

For a point F in the space, its world coordinates are denoted by ${\mathit{P}}_w={\left(x_w,y_w,z_w\right)}^T$ and its camera coordinates before and after translation are denoted by ${\mathit{P}}_c^{\prime }={\left(x_c^{\prime },y_c^{\prime },z_c^{\prime }\right)}^T$ , ${\mathit{P}}_c^{\mathrm{\prime \prime }}={\left(x_c^{\mathrm{\prime \prime }},y_c^{\mathrm{\prime \prime }},z_c^{\mathrm{\prime \prime }}\right)}^T$ . According to the rotation and translation relationship between world coordinate system and camera 1 coordinate system, we have

$\begin{array}{c}{\mathit{P}}_c^{\prime }={\mathit{R}}_{\mathrm{1}}{\mathit{P}}_w+{\mathit{T}}_{\mathrm{1}}.\\ {\mathit{P}}_c^{\mathrm{\prime \prime }}={\mathit{R}}_{\mathrm{2}}{\mathit{P}}_w+{\mathit{T}}_{\mathrm{2}}.\end{array}$

(11)

R ₁, T ₁ are the rotation matrix and translation vector between camera coordinate system and world coordinate system before translation, while R ₂, T ₂ are the rotation matrix and translation vector after translation. The translation vector V in camera 1 coordinate system is described by

$\mathit{V}={\mathit{P}}_c^{\mathrm{\prime \prime }}-{\mathit{P}}_c^{\prime }=\left({\mathit{R}}_{\mathrm{2}}-{\mathit{R}}_{\mathrm{1}}\right){\mathit{P}}_w+{\mathit{T}}_{\mathrm{2}}-{\mathit{T}}_{\mathrm{1}}.$

(12)

To facilitate the calculation, the world coordinate system origin P _w = (0, 0, 0) ^T is selected as the point F and then the translation vector in camera 1 coordinate system is transformed into V = T ₂ − T ₁. After converting it to world coordinate system and vector unitization, the unit translation vector can be obtained.

Above describes the process of obtaining 3D coordinates in the world coordinate system for sensor 1. The process for sensor 2 is similar. Therefore, merging left and right scanners’ 3D data is done by transforming them from camera 1 and camera 2 to the world coordinate system, respectively.

6. Experimental Results

6.1. System Calibration Setup

The MSSS is illustrated in Figure 7.

Figure 7:

The MSSS scanning measurement system. (1) Camera 1. (2) Camera 2. (3) Glass platform. (4) Linear motion stage. (5) Grating ruler. (6) Laser projectors of sensor 1. (7) Laser projectors of sensor 2. (8) Stepping motor.

The conFiguration information of every part is listed in Table 2.

Table 2:

Configuration information of MSSS.

Digital CCD camera
Quantity	Interface	Resolution	Pixel size	Frame rate	Optic size
2	USB2.0	1280 (H) × 1024 (V)	5.2 (μm) × 5.2 (μm)	15 fps	1/1.8 inch
Lens
Quantity	Focal length	Horizontal angle of view		Dimensions
2	16 mm	22.72°		29.5 mm (D) × 32 mm (L)
Laser projector
Quantity	Size	Wavelength	Operating voltage	Output power	Fan angle
6	ϕ8 × 26 mm	650 nm	DC 5 V	2.5 mW	50°
Stepping motor
Length	Rated current		Step angle
51 mm	1 A		1.8°
Stepping motor controller
Input voltage		Output current		Support RS232
24–48 VDC		0.1–5 A (peak)		RS232, CAN bus
Grating ruler
Operating voltage	Length	Resolution	Max speed	Interface
5 V ± 5% DC	500 mm	5 μm	≥ 60 m/min	USB2.0

We designed a planar pattern with 10 × 11 circles for calibration, shown in Figure 8. The centers of the circles on the pattern are used as the calibration feature points. The diameter of the big circle is 15 mm and that of the small one is 9 mm. In the vertical direction, the distance between two adjacent centers is 15 mm, and the distance of two adjacent centers in the horizontal direction is 15 mm except that the fifth and the sixth column are 30 mm.

Figure 8:

Target and the projected light stripes.

The system calibration process is as follows.

1. Place the planar pattern randomly at 10 positions within the FOV of each camera and each camera obtains an image at each position. Extract the image coordinates of the feature points on the pattern from the images. Then calibrate the internal parameters of the two cameras by Zhang's method.

2. Place the planar pattern at the public area in the FOV of the two cameras and two cameras obtain an image, respectively. Extract the image coordinates of the feature points on the pattern from the two images. Calibrate the camera external parameters by Zhang's method to obtain the rotation and translation relationship between the camera and the world coordinate systems.

3. Fix the pattern within the FOV of camera 1, start the stepping motor, and drive the sensors to move forward. Camera 1 captures an image before and after the movement, respectively. As the world coordinate system plane o _w − x _w y _w coincides with the pattern plane, the location of the world coordinate system is fixed. Then calibrate the unit translation vector V using the two images as mentioned in Section 5.2.

4. Open the line-structured-light projectors, and project the planar lights on the pattern. Randomly move the pattern to three positions within the FOV of the two cameras, and each camera captures an image for each position. Calculate the camera coordinates of the points on the light stripe on the pattern. Fit the six light plane equations of the two sensors in each camera coordinate system according to the method described in Section 5.1.

6.2. Experimental Results

6.2.1. Calibration Results

The calibration results of the MSSS are shown in Table 3.

Table 3:

Calibration results of the MSSS scanning measurement system.

	Sensor 1	Sensor 2
(f x ,f y )/pixels	(3129.509, 3128.652)	(3043.739, 3045.095)
(u0, v0)/pixels	(614.430, 536.804)	(632.633, 554.142)
(k1, k2)	(−0.1857, −0.0143)	(−0.1870, 0.1157)
Rotation matrix ( R )	$\begin{array}{ccc}-\mathit{9.375}\times {\mathit{10}}^{-\mathit{1}}& \mathit{1}.\mathit{422}\times {\mathit{10}}^{-\mathit{1}}& \mathit{3.177}\times {\mathit{10}}^{-\mathit{1}}\\ -\mathit{1}.\mathit{127}\times {\mathit{10}}^{-\mathit{3}}& -\mathit{9.139}\times {\mathit{10}}^{-\mathit{1}}& \mathit{4.058}\times {\mathit{10}}^{-\mathit{1}}\\ \mathit{3.481}\times {\mathit{10}}^{-\mathit{1}}& \mathit{3}.\mathit{801}\times {\mathit{10}}^{-\mathit{1}}& \mathit{8.569}\times {\mathit{10}}^{-\mathit{1}}\end{array}$	$\begin{array}{ccc}\mathrm{9.495}\times {\mathrm{10}}^{-\mathrm{1}}& \mathrm{1.373}\times {\mathrm{10}}^{-\mathrm{1}}& -\mathrm{2.823}\times {\mathrm{10}}^{-\mathrm{1}}\hspace{0.17em}\\ \mathrm{3.787}\times {\mathrm{10}}^{-\mathrm{3}}& \mathrm{8.942}\times {\mathrm{10}}^{-\mathrm{1}}& \mathrm{4.476}\times {\mathrm{10}}^{-\mathrm{1}}\\ \mathrm{3.139}\times {\mathrm{10}}^{-\mathrm{1}}& -\mathrm{4}.\mathrm{260}\times {\mathrm{10}}^{-\mathrm{1}}& \mathrm{8.485}\times {\mathrm{10}}^{-\mathrm{1}}\end{array}$
Translation vector ( T )	$\begin{array}{ccc}\mathrm{74.612}& \mathrm{71.063}& \mathrm{228.107}\end{array}$	$\begin{array}{ccc}-\mathrm{92.646}& -\mathrm{66.043}& \mathrm{299.890}\end{array}$
Light plane equation coefficients	8.793 × 10−3,1.444 × 10−4, − 3.315 × 10−3,1.000 9.456 × 10−3,2.054 × 10−4, − 3.504 × 10−3,1.000 9.974 × 10−3,1.280 × 10−4, − 3.788 × 10−3,1.000	− 1.046 × 10−2, − 2.670 × 10−4, − 3.810 × 10−3,1.000 − 9.878 × 10−3, − 2.885 × 10−4, − 3.672 × 10−3,1.000 − 9.026 × 10−3, − 2.965 × 10−4, − 3.445 × 10−3,1.000
Translation union vector ( V )	$\begin{array}{ccc}\mathrm{9.384}\times {\mathrm{10}}^{-\mathrm{1}}& -\mathrm{5.897}\times {\mathrm{10}}^{-\mathrm{3}}& -\mathrm{3.454}\times {\mathrm{10}}^{-\mathrm{1}}\end{array}$

6.2.2. Accuracy Evaluation Results

The line-structured-light sensor calibration accuracy is verified by comparing two values: one is the standard value, which is the camera coordinate of the intersection point of the ray O_c1p and the target plane, as shown in the Figure 5, and the other is the measured value, which is the camera coordinate of the intersection point of O_c1p and the light plane. Taking light plane π₁ ^l for example, two of the images used for structured-light calibration are chosen to evaluate the sensor calibration accuracy. We selected six points on the light stripe in each image and evaluated the sensor calibration accuracy. The calibration accuracy evaluation data is shown in Table 4, which shows that the RMS errors in x, y, and z directions are 0.004 mm, 0.008 mm, and 0.092 mm, respectively.

Table 4:

Calibration accuracy evaluation data for π₁ ^l in sensor 1.

	Image coordinates	Intersection points of line and    target plane			Intersection points of line and    light plane			Errors of coordinates
	(u,v)/pixels	x/mm	y/mm	z/mm	x/mm	y/mm	z/mm	Δx/mm	Δy/mm	Δz/mm
Position 1	(663.495, 20.000)	4.918	−51.840	312.263	4.920	−51.856	312.356	−0.002	0.016	−0.093
	(628.546, 216.800)	1.372	−31.127	303.753	1.372	−31.137	303.852	0.000	0.010	−0.099
	(593.597, 413.600)	−1.969	−11.649	295.753	−1.970	−11.653	295.837	0.001	0.003	−0.083
	(558.648, 610.400)	−5.138	6.780	288.186	−5.139	6.782	288.237	0.001	−0.001	−0.051
	(523.699, 807.200)	−8.160	24.324	280.985	−8.160	24.324	280.989	0.000	0.000	−0.004
	(488.750, 1004.000)	−11.059	41.118	274.094	−11.057	41.110	274.038	−0.002	0.008	0.056
Position 2	(583.124, 20.000)	−2.935	−48.434	291.751	−2.934	−48.422	291.678	−0.001	−0.012	0.074
	(546.201, 216.800)	−6.200	−29.084	283.792	−6.202	−29.091	283.855	0.001	0.007	−0.064
	(509.277, 413.600)	−9.289	−10.886	276.313	−9.294	−10.892	276.449	0.005	0.005	−0.136
	(472.353, 610.400)	−12.230	6.337	269.241	−12.237	6.340	269.395	0.007	−0.004	−0.154
	(435.429, 807.200)	−15.047	22.736	262.514	−15.054	22.746	262.639	0.007	−0.011	−0.124
	(398.506, 1004.000)	−17.761	38.439	256.079	−17.165	38.447	256.133	0.004	−0.008	−0.054
RMS errors (mm)								0.004	0.008	0.092

We placed the planar pattern at another random position, and calculate the errors of the light plane π₁ ^r of the calibrated system in the same way. Those data can serve as the evaluation for sensor measurement accuracy since they are not involved in calibration process. The RMS errors for the system measurement accuracy in each axis are RMSx = 0.006 mm, RMSy = 0.015 mm, and RMSz = 0.168 mm, as shown in Table 5.

Table 5:

Measurement accuracy evaluation data for π₁ ^r in sensor 2.

Image coordinates	Intersection points of line and target      plane			Intersection points of line and light     plane			Errors of coordinates
(u,v)/pixels	x/mm	y/mm	z/mm	x/mm	y/mm	z/mm	Δ x/mm	Δ y/mm	Δ z/mm
(449.701, 20.000)	−19.302	−56.272	319.259	−19.310	−56.297	319.400	0.008	0.024	−0.140
(475.684, 216.800)	−15.959	−34.264	308.722	−15.955	−34.256	308.647	−0.003	−0.008	0.075
(501.667, 413.600)	−12.870	−13.800	298.917	−12.862	−13.791	298.723	−0.008	−0.009	0.194
(527.650, 610.400)	−9.998	5.357	289.732	−9.990	5.352	289.501	−0.008	0.004	0.232
(553.633, 807.200)	−7.311	23.409	281.070	−7.306	23.393	280.870	−0.005	0.017	0.201
(579.616, 1004.000)	−4.782	40.528	272.851	−4.780	40.511	272.737	−0.002	0.017	0.113
RMS errors (mm)							0.006	0.015	0.168

6.3. Measurement Results

A mouse is scanned by the MSSS to demonstrate the applicability. Figures 9(a) and 9(b) show the captured mouse image pair with light stripes projected on. From Figures 9(a) and 9(b), we can see, with single sensor, that there is blind area, where 3D coordinates are unable to obtain. The whole mouse is covered when 2 sensors are adopted, complementing each other's blind area. Figure 9(c) shows the 3D scanning result of the mouse, in which the scanning speeding is 7 mm/s and the time consumption is 16 seconds. Compared to the existing single or double structured-light 3D surface measurement system, our MSSS greatly improved scanning efficiency and obtained richer 3D surface data at the same scanning speeding and time consumption. In order to compare with the single line scan method, we used only one laser generator in each sensor. Figures 10(a) and 10(b) show the single stripe image captured by cameras 1 and 2, respectively, and Figure 10(c) shows the 3D reconstruction results. Note that here we use the same scan speed and frame rate for both multiline and single-line experiments. Because, with the same scan speed and frame rate, multiline approach provides more features in one frame than that of single-line approach; therefore, it gains richer information of the target.

Figure 9:

Measurement experiment of a mouse. (a) Light stripe image of the mouse captured by sensor 1. (b) Light stripe image of the mouse captured by sensor 2. (c) 3D reconstruction results of the mouse.

Figure 10:

Comparison experiment using single scan line. (a) Single stripe image of the mouse captured by camera 1. (b) Single stripe image of the mouse captured by camera 2. (c) 3D reconstruction results of the mouse.

7. Conclusions

A novel approach is presented to capture the 3D surface based on multiline structured-light sensors. There are three superiorities for this method compared with the existing methods.

1. The disadvantages of single structured-light 3D surface measurement system are overcome, such as limited measurement range, blind measurement area, and low scanning efficiency.

2. The mathematical model of the MSSS is established without particular spatial relationship, making the system easy to implement.

3. The system calibration is completed by a 2D planar pattern. It does not require complex calibration patterns or high-precision auxiliary equipment, which reduces the difficulty of the system calibration and ensures the measurement accuracy simultaneously.

In addition, note that the method proposed in this paper is not only applicable for the multi-line structured-light sensor with three light stripes, but also suitable for the sensor with more light stripes, which can further improve the scanning efficiency.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.