Sage Journals: Discover world-class research

Abstract

Outdoor surveillance and security robots have a wide range of industrial, military, and civilian applications. In order to achieve autonomous navigation, the LiDAR-camera system is widely applied by outdoor surveillance and security robots. The calibration of the LiDAR-camera system is essential and important for robots to correctly acquire the scene information. This article proposes a fast calibration approach that is different from traditional calibration algorithms. The proposed approach combines two independent calibration processes, which are the calibration of LiDAR and the camera to robot platform, so as to address the relationship between LiDAR sensor and camera sensor. A novel approach to calibrate LiDAR to robot platform is applied to improve accuracy and robustness. A series of indoor experiments are carried out and the results show that the proposed approach is effective and efficient. At last, it is applied to our own outdoor security robot platform to detect both positive and negative obstacles in a field environment, in which two Velodyne-HDL-32 LiDARs and a color camera are employed. The real application illustrates the robustness performance of the proposed approach.

Keywords

Surveillance and security robots LiDAR-camera system calibration

Introduction

Outdoor surveillance and security robots are widely used to implement desired tasks without human guidance in certain environments, such as public area patrol, border control, anti-terrorism, mine monitoring, sea area patrol, event security, and so on.^1

–4 In order to implement the above tasks, robots are commonly required to have the function of autonomous navigation. In order to navigate more safely, multiple sensors are usually employed together on the surveillance and security robot to acquire the scene information. Typical sensors are cameras, LiDARs, IMUs, and so on. For instance, camera sensor provides rich color and feature information. On the other hand, with recent advancements in technologies, LiDAR sensors are more and more popular to be applied to provide much accurate distance information.^5–
7 Thus, sensor fusion technology has been greatly researched in the field of robotics and computer vision for autonomous navigation, object recognition, posture estimation, and so on.

For outdoor surveillance and security robots, the more environmental information they perceived, the better they would navigate and implement tasks.⁸ As analyzed above, both camera and LiDAR have their own advantages. One common way for outdoor surveillance and security robots is that both camera and LiDAR are fixed on the robot platform and the information that different sensors collected are fused according to some algorithm.

In order to achieve the information fusion of different kinds of sensors, the calibration between sensors is the first and most important prerequisite. Lots of algorithms for calibration between camera and LiDAR have emerged in decades.^9

–12 The core task of calibration is to build the relationship between 2-D points on the image and 3-D points on the point cloud. However, manually establishing correspondences for their mapping relationship is laborious and inaccurate because it requires multiple matches. Therefore, many methods have been put forward to solve this problem by using kinds of checkerboard or other tools. For instance, a special designed calibration between camera and LiDAR was introduced by Gong et al.¹³; an ordinary box was employed by Pusztai and Hajder¹⁴; a polygonal planar board was applied by Park et al.¹⁵ Furthermore, several automatic extrinsic calibration algorithms were also proposed.^16,17 For example, the reflex value generated by the LiDAR and the gray-scale intensity value generated by the camera, which were automatically detected, were employed to calibrate the relationship between those two sensors by Pandey et al.¹⁸ Besides, the idea of camera motion estimation by using sensor fusion odometry was adopted by Ishikawa et al.¹⁹ to get the calibration between two sensors automatically.

This article also proposes a calibration approach for building the relationship between camera sensor and LiDAR sensor, which are fixed on an outdoor surveillance and security robot platform. Compared with these traditional algorithms mentioned above, the main idea of the proposed approach is that the calibration can be achieved by two independent steps successively: firstly, calibrating the camera (or LiDAR) coordinate into the robot platform coordinate (or an arbitrarily fixed coordinate); secondly, building the relationship between the camera and the LiDAR through the robot platform coordinate (or the predefined coordinate). This approach is easy to implement and needs no external cost, since both the calibration of camera and the calibration of LiDAR are essential once these two sensors are fixed on the robot platform. The proposed approach is applied on our own outdoor surveillance and security robot platform (Figure 1) to detect both positive and negative obstacles in field environment, and experimental results illustrate its effective and efficient performance.

Figure 1.

The proposed approach is applied on our own outdoor surveillance and security robot platform to calibrate the following sensors: two compact HDL-32 LiDARs are mounted on the two sides of the platform top, and a color camera is mounted on the middle of the platform front.

The remainder of this article is organized as follows. The second section reviews some related algorithms of calibration between the camera and the LiDAR. In the third section, a mathematical model of the proposed approach is deduced, and some details of the approach are described. In the fourth section, experimental results show that the introduced approach is robust and stable. The fifth section concludes the article.

Related work

Calibration of LiDAR-camera systems for robotics has been widely studied in decades.^13,14,18 The core of the calibration is to build the relationship between the coordinates of the two sensors. In general, the main idea of these emerged algorithms is that a series of corresponding pairs of points, or lines, or other features are chosen manually or automatically. Then, the relationship between the coordinates of the two sensors can be solved by calculating the nonlinear least squares (NLS) problem of these corresponding pairs. These existed algorithms can be broadly classified into two categories: one that needs the calibration tool and one that does not need the calibration tool.

Algorithms that need the calibration tool

In the work of Gong et al.,¹³ a special object or likeness scene was required for their algorithm, since planar constraint of these three sides was used in its calibration process. The relative transformation between the two sensors was changed into the result of a NLS problem. In that paper, two kinds of sensors were required to be individually calibrated in each configuration firstly.

Similar work was carried out by Pusztai and Hajder,¹⁴ in which a cardboard box was applied to implement their calibration. The size of the box was measured manually and carefully in advance. During the calibration process, three sides of the box should be clearly visible both in point cloud and in image. As the paper mentioned, the point cloud can be heavily affected by noise when it detects three planes. Thus, in their algorithm, the step of outlier removal is essential.

In the study by Li et al.,²⁰ a special right-angled triangular checkerboard was designed to find the edges of the checkerboard in point cloud, and the corresponding edges in image were also detected by edge detection algorithm. Similarly, a special polygonal planar board was used as the calibration tool by Park et al.¹⁵ and a plane with a triangular hole was used as the calibration tool by Ha.²¹

Geiger et al.²² also presented a toolbox with web interface for fully automatic camera-to-camera and camera-to-LiDAR calibration. According to their method, several checkerboards were placed all over the room, with different directions. Features of checkerboards are detected by the camera and LiDAR, respectively, and the features captured by the two sensors were matched. The biggest advantage of this article is that its checkerboard corner detector algorithm is significantly better than other algorithms.

Similar method was introduced by Wang et al.²³ Wang et al.²³ also employed a normally printed chessboard for calibration. The proposed method was based on the 3-D corner estimation of the chessboard from the sparse point cloud generated by one frame scan of the LiDAR. To estimate the corners, it formulated a full-scale model of the chessboard and fitted it into the segmented 3-D points of the chessboard. The model was fitted by optimizing the cost function under constraints of correlation between the reflex intensity of laser and the color of the chessboard patterns. Once the corners of the checkerboard in the 3-D point cloud were estimated, the extrinsic calibration of the two sensors was converted to a traditional 3D–2D matching problem.

Unlike other works which required at least three checkerboard poses, Zhou et al.²⁴ reduced the minimal number of poses to one by combining 3-D line and plane correspondences. Thus, only one checkerboard was required in their method. Besides, it is also considered that parallel planar targets with parallel boundaries provided the same constraints in their algorithm.

Algorithms that do not need the calibration tool

The idea of the algorithms that do not need the calibration tool is usually as follows: Firstly, some features are designed and extracted manually or automatically from both camera data and LiDAR data in the natural scene. Secondly, those extracted features are manually or automatically used to establish the corresponding relationship between camera data and LiDAR data. Finally, the relative transformation between these two sensors is changed into the result of a NLS problem according to the corresponding relationship of those features.

A data-driven automatic extrinsic calibration method was presented by Pandey et al.¹⁸ The main idea of this article is that under the correct rigid-body transformation, the correlation between the LiDAR reflectivity and camera intensity was maximized. The surface reflectivity values reported by the LiDAR and the gray-scale intensity values reported by the camera were employed to calibrate the modalities of the two sensors.

In the study by Zachary Taylor and Johnson,¹⁷ the orientations of gradients formed from the two candidate sensors were employed, and the sensor’s extrinsic parameters were obtained by minimizing the misalignment of the gradients. Thus, in this article, the calibration problem was changed into the minimalism problem between the orientations of gradient maps of the two sensors.

Another automatic extrinsic calibration of a camera-LiDAR system was mentioned by Irie et al.²⁵: a dependence estimator called bagged least-squares mutual information was described to address the calibration parameter between two sensors. Their algorithm had two parts: a kernel-based dependence estimator and the noise reduction by bootstrap aggregating, which can handle richer features and robustly estimate dependence. It is thought that their algorithm would overcome drawbacks of outdoor environment, such as noise and non-smoothness.

A motion estimated algorithm by sensor fusion was introduced by Ishikawa et al.,¹⁹ in which the calibration process also needed no-special calibration tool and could be implemented automatically. The proposed method only required the 3-D point cloud and the camera image and did not need other information such as reflex of LiDAR and the initial extrinsic parameter.

Taylor et al.¹⁶ also presented a camera-LiDAR calibration method that required no markers to be placed in the scene. In this method, both the intensity data from LiDAR and the images from camera were processed into orientation magnitude map. According to the camera model, LiDAR scan map was projected onto the camera’s map. Particle swarm optimization was used to find the optimal parameters for this model.

Unlike the above mentioned papers, this article gives up the traditional algorithms to calibrate LiDAR and camera. Instead of finding corresponding points in two coordinates, the proposed approach firstly calibrates the camera (LiDAR) coordinate into the robot platform coordinate (or a predefined fixed coordinate), then calculates the relationship between the camera coordinate and the LiDAR coordinate through the robot platform coordinate (or the predefined fixed coordinate). Experimental results show that the presented approach is effective and robust.

The description of the proposed approach

Theory and basic equations

What we will discuss in this article is the calibration of the extrinsic parameters of LiDAR and camera. Camera and LiDAR are mounted commonly as shown in Figure 2. The relationship between two coordinates can be presented by equation (1)

Figure 2.

(a) The relationship between image coordinate and robot platform coordinate; (b) the relationship between LiDAR coordinate and robot platform coordinate.

[\begin{matrix} u \\ v \\ 1 \end{matrix}] = K K \times [^{C} R_{L}^{C} T_{L}] \times [\begin{matrix} x \\ y \\ z \\ 1 \end{matrix}]

where $(u, v)$ represents the position in image coordinate, $(x, y, z)$ represents the position in LiDAR coordinate, $K K$ is a $3 \times 3$ matrix means the intrinsic parameter of camera, $^{C} R_{L}$ is a $3 \times 3$ rotation matrix from LiDAR coordinate to image coordinate, and $^{C} T_{L}$ is a $3 \times 1$ translation vector from LiDAR coordinate to image coordinate. $^{C} R_{L}$ and $^{C} T_{L}$ present the relationship between two coordinates, which should be calculated during the calibration process.

In traditional methods, a series of corresponding points are detected manually or automatically both in image and in LiDAR data, which are used to calculate R and T according to equation (1). Furthermore, this problem can be solved by changing it into equation (2), a general cost function as follows

arg min ∥ P \times (R + t) - x ∥^{2}

where P is the projection operation from 3-D to 2-D in image coordinate, X represents points in LiDAR coordinate, and x represents points in image coordinate.

On the other hand, this problem in equation (1) can be divided into two independence parts, which are represented by equation (3) and in equation (4)

[\begin{matrix} t \times u \\ t \times v \\ t \end{matrix}] = K K \times [^{C} R_{V} \times [\begin{matrix} X \\ Y \\ Z \end{matrix}] +^{C} T_{V}]

Equation (3) is a classical problem of camera’s calibration to the robot platform coordinate, when a camera is fixed on a robot platform. $^{C} R_{V}$ is a $3 \times 3$ rotation matrix from robot platform coordinate to image coordinate, and $^{C} T_{V}$ is a $3 \times 1$ translation vector from robot platform coordinate to image coordinate. $(X, Y, Z)$ represents the position in robot platform coordinate, and t is a operating coefficient

[\begin{matrix} X \\ Y \\ Z \end{matrix}] =^{V} R_{L} \times [\begin{matrix} x \\ y \\ z \end{matrix}] +^{V} T_{L}

Equation (4) is the expression of mapping LiDAR coordinate into robot platform coordinate, in which $^{V} R_{L}$ is a $3 \times 3$ rotation matrix from LiDAR coordinate to robot platform coordinate, and $^{V} T_{L}$ is a $3 \times 1$ translation vector from LiDAR coordinate to robot platform coordinate.

In the application of our outdoor surveillance and security robot platform, it is found that the relationship between LiDAR and camera is fixed once they are mounted on the platform. Thus, the calibration can be generally undertaken only once, and the parameters will not change anymore. Unlike many previously reported methods, the approach proposed in this article first calibrates the two sensors independently according to equations (3) and (4), and then builds the relationship by combining these two equations.

Approach description

The calibration of camera is well researched, and many open source calibration tools are public. In our application, a classical calibration toolbox proposed by Bouguet.²⁶ is employed in our approach, which is mainly used to calibrate both intrinsic parameter and extrinsic parameter of the camera to achieve the equation (3). Calibration methods for LiDARs are also widely researched,^27
–29 and we expand our previous work³⁰ to the implementation of the calibration in this application. The proposed algorithm is described as in Algorithm 1.

Algorithm 1.

The proposed calibration approach between LiDAR and camera.

Figure 3.

The process to calibrate LiDAR coordinate into the robot platform coordinate.

Thus, input the parameter of $(X, Y, Z)$ in equation (4) into equation (3), the relationship between LiDAR and camera can be expressed as equation (5)

[\begin{matrix} t \times u \\ t \times v \\ t \end{matrix}] = K K \times [^{C} R_{V} \times^{V} R_{L} \times [\begin{matrix} x \\ y \\ z \end{matrix}] +^{C} R_{V} \times^{V} T_{L}] +^{C} T_{V}

When the parameter $K K$ , $^{C} R_{V}$ , $^{C} T_{V}$ , $^{V} R_{L}$ , and $^{V} T_{L}$ are estimated by the proposed approach in Algorithm 1, the calibration relationship between two sensors is achieved.

Experimental results and analysis

In order to evaluate and verify the proposed approach, experiments are designed and carried out both in indoor environments and field environments. Outdoor experiments are composed of three parts: first, a test of indoor qualitative analysis is carried out to verify the proposed approach; second, a series of experiments, where LiDAR and camera are under different poses are designed to quantitatively analyze the accuracy of the proposed approach, which is compared to the state of the art; at last, the proposed approach is applied on our own outdoor surveillance and security robot platform for calibrating the relationship among two LiDARs, a camera and the robot platform itself. The outdoor robot platform is employed to detect both positive and negative obstacles in field environment, and the results show the proposed approach is effectiveness.

Indoor qualitative analysis

A special platform, as shown in Figure 4(b), which can be rotated around two axes is designed specially to carry out our experiments. A Velodyne HDL-32 LiDAR is mounted on the platform, and a camera is also mounted beside the LiDAR. The platform is mounted on a shelf, which is used to instead of a robot platform (Figure 4(a)), and a camera is also mounted beside the LiDAR. Indoor room environment is chosen, since there are three orthogonal planes: ground and two orthogonal walls (Figure 4(c)) to support the proposed calibration approach. A box is placed in front of the platform to check the accuracy of the calibration (Figure 4(c)). A checkerboard is also employed to calculate both the intrinsic parameter and the extrinsic parameter of the camera.

Figure 4.

Platform of our indoor experiments. (a) The indoor scenario, in which a shelf simulated as a robot platform is placed and a camera and a LiDAR are mounted on it. A box is placed in front the robot platform to show the corresponding relationship between the LiDAR and the image; (b) a special designed platform that LiDAR is fixed on it, the platform can be rotated around two different axes. (c) The indoor scenario that three orthogonal planes are existed for the proposed approach, and a checkerboard is also placed for calibrating both the intrinsic parameter and the extrinsic parameter of the camera.

According to the proposed approach, the calibration is composed of two parts: the calibration of LiDAR coordinate to robot platform coordinate (according to equation (4)) and the calibration of camera coordinate to robot platform coordinate (according to equation (3)). The checkerboard is employed to calibrate both the intrinsic parameter and the extrinsic parameter of the camera according to the study by Bouguet.²⁶ The process of calibrating the LiDAR is described in Algorithm 1, in which three orthogonal planes are employed to improve the accuracy. Some typical results of mapping LiDAR’s data into the image are shown in Figure 5.

Figure 5.

(a to e) Some typical results that mapping the LiDAR data of the person and the box into the image to show the effectiveness of the proposed approach.

In order to show the accuracy more clearly, only those LiDAR points, which are located on the person, the checkerboard and the box are mapped into the image in Figure 5. From Figure 5, it is obvious that the proposed approach is effective.

Indoor quantitative analysis

Thanks to our special platform, which can rotate around two axes, many groups of data sets generated by LiDAR under different poses are collected, indicating different relationships between LiDAR and camera, and verifying the robustness and the accuracy of the proposed calibration approach. Among these groups of data sets, four groups that are obtained by the LiDAR under several typical poses are chosen to quantitatively analyze the proposed approach. Figure 6(a) shows these four poses of the equipped LiDAR. From top down, the first pose shows the LiDAR is vertically equipped (labeled as Scene1). The second pose shows that LiDAR rotates around X-axis (labeled as Scene2). The third and the forth poses show that LiDAR is rotated around both X-axis and Z-axis (labeled as Scene3 and Scene4). Figure 6(b) shows the original point cloud data of the LiDAR corresponding to these scenes. During the process, the pose of camera is kept unchanged.

Figure 6.

Experiment results under four typical poses between the LiDAR and the camera. (a) Four scenes that the LiDAR is rotated in different poses while the camera is kept unchanged. (b) The original point cloud in LiDAR coordinate. (c) The LiDAR data that located on the box under different poses. (d) The mapping result in image of those LiDAR data.

The experiment in this part is designed as follows: the pose of camera is kept unchanged, and four different poses of LiDAR are rotated and labeled as $S c e n e 1, \dots, S c e n e 4$ . Thus, four groups of relationships between LiDAR and camera are listed. A box is placed in front of the platform to quantitative analyze the accuracy of the proposed approach, and the steps are as follows: mapping these LiDAR points that located on the box into image coordinate, and calculating the number manually to show how many points are mapped correctly. Figure 6(c) shows the LiDAR points located on the box under that four cases. Figure 7 shows the target range of the box (marked as the red rectangle) that the data in Figure 6(c) have to map into it. The number of LiDAR points on the box is labeled as N_L , and the corresponding number in image is labeled as N_I . They are calculated manually. Let $R = \frac{N_{I}}{N_{L}} \times 100 %$ to measure the accuracy.

Figure 7.

Quantitative analysis to estimate the accuracy of mapping LiDAR data into image.

The results of this experiment are shown in Figure 6(d) and are compared with the state-of-the-art algorithm presented by Unnikrishnan and Hebert⁹ is employed for comparison. The comparison results are listed in Table 1. Compared with the state-of-the-art algorithm, the proposed approach has higher calibration accuracy and easier operation, since it acquires at least nine different poses of planes to achieve a good result according to the study by Unnikrishnan and Hebert,⁹ but only three orthogonal walls are needed.

Table 1.

The quantitative analysis of the calibration compared with the study by Unnikrishnan and Hebert.⁹

Scene	N_L	N_I (this article)	R, % (this article)	N_I (Unnikrishnan and Hebert⁹)	R, % (Unnikrishnan and Hebert⁹)
Case 1	678	474	69.91	480	70.8
Case 2	655	515	78.63	499	76.18
Case 3	753	506	67.2	478	63.48
Case 4	773	510	65.98	469	60.67

In Table 1, it is found that the N_I is much smaller than N_L both in our approach and in the compared one. The main reason is that the bottom of the box is out of the image.

Outdoor real application

The proposed calibration approach has been applied in our own outdoor surveillance and security robot platform (shown in Figure 1). The task of our outdoor surveillance and security robot platform is to patrol and monitor the field area. There are various positive obstacles and negative obstacles on the patrol road. In order to achieve the safer autonomous patrol, both positive obstacle detection and negative obstacle detection is necessary. Thus, the LiDAR-camera system fixed on our outdoor surveillance and security robot platform is mainly used for detecting those positive and negative obstacles in field environment. As we know, the patrol roads in field environment are bumpy. Besides, sensors work under different rates. Therefore, it is a test for the calibration effectiveness of the robot platform when it drives on such a bumpy road.

There are two Velodyne HDL-32 LiDARs mounted on the two sides of the top of the robot platform, and a camera mounted on the middle of the robot platform front as shown in Figure 1. These LiDARs and camera are employed to find positive and negative obstacles in field roads. Thus, the relationship between those sensors are prerequisite and important. In this application, the LiDAR-camera system, the LiDAR-robot platform system, and the camera-robot platform system are calibrated accurately by the proposed approach. Figures 8 and 9 show the experimental results. In order to display the corresponding points more clearly, only these LiDAR points that of potential negative obstacles are mapped into image to verify the correctness.

Figure 8.

(a to c) Typical field experimental results show the calibration of the proposed algorithm is effective in different scenes (I).

Figure 9.

(a to c) Typical field experimental results show the calibration of the proposed algorithm is effective in different scenes (II).

In those experimental results, to show the corresponding relationship more clearly, four parts are presents in the interface window. They are (from left to right) LiDAR coordinate map, grid map in robot platform coordinate, the control platform, and image coordinate map. The detected obstacles are marked both in the LiDAR map, the grid map, and the image map according to their relationship. The results indicate that these three coordinates are well calibrated, since all of these obstacles are painted correctly. The distance between the robot platform and the obstacle at the current moment is also marked beside the obstacle to show the accuracy of the calibration results. Experimental results illustrate the high performance of the proposed calibration approach.

Conclusion

This article introduces a fast calibration approach for LiDAR-camera system. The theory and mathematical equations of the relationship among LiDAR, camera, and robot platform are deduced. On this basis, this article proposes a fast approach that combines two independent calibration processes, which are the calibration of LiDAR and the camera to robot platform, so as to build the relationship between LiDAR and camera. A novel approach to calibrate LiDAR to robot platform is applied to improve the accuracy. Indoor and outdoor experiments are both carried out to verify the effectiveness and robustness. Finally, the proposed approach is applied on our own outdoor robot platform to detect negative obstacles in field environment. Two Velodyne HDL-32 LiDARs and a camera are used on the robot platform to detect negative obstacles. The relationship between these three sensors and the robot platform are established by the proposed approach. Both indoor experiments and outdoor real application illustrate the effective and efficient performance of the proposed approach.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the National Natural Science Foundation of China under grant 61803380,61790565.

ORCID iD

Shang Erke

Zhu Qi

References

Shang

, et al. Lidar based negative obstacle detection for field autonomous land vehicles. J Field Robot 2016; 33(5): 591–617.

Cui

, et al. A feature matching and fusion-based positive obstacle detection algorithm for field autonomous land vehicles. Int J Adv Robot Syst 2017; 14(2): 1729881417692516.

Shang

, et al. Robust unstructured road detection: the importance of contextual information. Int J Adv Robot Syst 2013; 10(3): 1–8.

Xiao

Wang

Dai

, et al. Hybrid conditional random field based camera-Lidar fusion for road detection. Inf Sci 2018; 432: 543–558.

Zhang

Gui

Wang

, et al. Hierarchical topic model based object association for semantic slam. IEEE Trans Vis Comput Graph 2019; 25(11): 3052–3062.

Zhang

Liu

Yin

, et al. Intelligent collaborative localization among air-ground robots for industrial environment perception. IEEE Trans Ind Electron 2019; 66(12): 9673–9681.

Wang

Zhang

Chen

, et al. Robust high accuracy visual-inertial-laser SLAM system. In: IEEE/RSJ international conference on intelligent robots and systems, 2019. IROS 2019, Macau, China, 3–8 November 2019, pp. 123–128. IEEE.

Xiao

Dai

Liu

, et al. CRF based road detection with multi-sensor fusion. In: 2015 IEEE Intelligent Vehicles Symposium (IV), Seoul, South Korea, 28 June–1 July 2015, pp. 192–198.

Unnikrishnan

Hebert

. Fast extrinsic calibration of a laser rangefinder to a camera. Robotics Institute, Pittsburgh, PA, Tech Rep CMU-RI-TR-05-09, 2005.

10.

Liao

Chen

Liu

, et al. Extrinsic calibration of Lidar and camera with polygon. In: 2018 IEEE international conference on robotics and biomimetics (ROBIO), Kuala Lumpur, Malaysia, 12–15 December 2018, pp. 200–205. IEEE.

11.

Heng

Pollefeys

. CamOdoCal: automatic intrinsic and extrinsic calibration of a rig with multiple generic cameras and odometry. In: 2013 IEEE/RSJ international conference on intelligent robots and systems, Tokyo, Japan, 3–7 November 2013, pp. 1793–1800. IEEE.

12.

Xiao

Dai

Liu

, et al. Monocular road detection using structured random forest. Int J Adv Robot Syst 2016; 13(3): 101.

13.

Gong

Lin

Liu

. 3D Lidar-camera extrinsic calibration using an arbitrary trihedron. Sensors 2013; 13(2): 1902–1918.

14.

Pusztai

Hajder

. Accurate calibration of Lidar-camera systems using ordinary boxes. In: 2017 international conference on computer vision, Venice, Italy, 22 October 2017–29 October 2017, pp. 394–402. IEEE.

15.

Park

Yun

Won

, et al. Calibration between color camera and 3D Lidar instruments with a polygonal planar board. Sensors 2014; 14(3): 5333–5353.

16.

Taylor

Nieto

Johnson

. Automatic calibration of multi-modal sensor systems using a gradient orientation measure. In: 2013 IEEE/RSJ international conference on intelligent robots and systems, Tokyo, Japan, 3–7 November 2013, pp. 1293–1300. IEEE.

17.

Zachary Taylor

Johnson

. Multi-modal sensor calibration using a gradient orientation measure. J Field Robot 2015; 32(5): 675–695.

18.

Pandey

Mcbride

Savarese

, et al. Automatic extrinsic calibration of vision and Lidar by maximizing mutual information. J Field Robot 2015; 32(5): 696–722.

19.

Ishikawa

Oishi

Ikeuchi

. LiDAR and camera calibration using motions estimated by sensor fusion odometry. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), Madrid, Spain, 1 October 2018, pp. 7342–7349. IEEE.

20.

Liu

Dong

, et al. An algorithm for extrinsic parameters calibration of a camera and a laser range finder using line features. In: 2007 IEEE/RSJ international conference on intelligent robots and systems, San Diego, CA, USA, 29 October–2 November 2007, pp. 3854–3859. IEEE.

21.

. Extrinsic calibration of a camera and laser range finder using a new calibration structure of a plane with a triangular hole. Int J Control Autom 2012; 10(6): 1240–1244.

22.

Geiger

Moosmann

Car

, et al. Automatic camera and range sensor calibration using a single shot. In: 2012 IEEE international conference on robotics and automation (ICRA), St Paul, MN, USA, 14–18 May, 2012, pp. 3936–3943. IEEE.

23.

Wang

Sakurada

Kawaguchi

. Reflectance intensity assisted automatic and accurate extrinsic calibration of 3D Lidar and panoramic camera using a printed chessboard. Remote Sens 2017; 9(8): 851.

24.

Zhou

Kaess

. Automatic extrinsic calibration of a camera and a 3D Lidar using line and plane correspondences. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), Madrid, Spain, 1–5 October 2018, pp. 5562–5569. IEEE.

25.

Irie

Sugiyama

Tomono

. Target-less camera-Lidar extrinsic calibration using a bagged dependence estimator. In: 2016 IEEE international conference on automation science and engineering, Worth, TX, USA, 21 August 2016, pp. 1340–1347. IEEE.

26.

Bouguet

. Camera calibration toolbox for Matlab, 2004.

27.

Atanacio-Jiménez

González-Barbosa

Hurtado-Ramos

, et al. Lidar Velodyne HDL-64e calibration using pattern planes. Int J Adv Robot Syst 2011; 8(5): 70–82.

28.

Gao

Spletzer

. On-line calibration of multiple Lidars on a mobile vehicle platform. In: 2010 IEEE international conference on robotics and automation (ICRA), Anchorage, Alaska, 3–8 May 2010, pp. 279–284. IEEE.

29.

Jiao

Liao

Zhu

, et al. A novel dual-Lidar calibration algorithm using planar surfaces. arXiv preprint arXiv:190412116, 2019.

30.

Shang

Shi

, et al. An efficient calibration approach for arbitrary equipped 3-D Lidar based on an orthogonal normal vector pair. J Intell Robot Syst 2015; 79: 21–36.

A fast calibration approach for onboard LiDAR-camera systems

Abstract

Keywords

Introduction

Related work

Algorithms that need the calibration tool

Algorithms that do not need the calibration tool

The description of the proposed approach

Theory and basic equations

Approach description

Experimental results and analysis

Indoor qualitative analysis

Indoor quantitative analysis

Outdoor real application

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References