Sage Journals: Discover world-class research

Abstract

The high demand of efficient large-scale machining operations by concurrently decreasing operating time and costs has led to an increasing usage of mobile robotic systems. This paper introduces a mobile robotic system which is consisted of a hybrid robot named TriMule on an automated guided vehicle, and a fringe-projection-based measurement system. TriMule exhibits desirable performance in terms of rigidity, accuracy, work envelop and reconfigurability. It is therefore suitable to be built on an autonomous platform for multi-station manufacturing in situ. In order to increase the absolute accuracy of the mobile robotic system, the fringe-projection-based measurement system obtains high accuracy and high density cloud to measure the position and orientation of the robot and workpiece in relation to each other. This system is suitable for large-scale manufacturing in situ, drilling, riveting and high-speed milling for example.

Keywords

Vision-based measurement system pose estimation 5-DOF hybrid robot mobile robotic systems large-scale manufacturing

Introduction

Large-volume manufacturing companies, for example, aeronautics, astronautics and shipping, are looking for cost-effective flexible solutions to meet the ever-increasing customer demands toward high speed and high quality.^1,2 Large-scale components such as aircraft skin, space capsule are the kernel of large airplanes, aerospace equipment. Usually, this kind of component has the characteristics of large size, high length/thickness ratio, complex profile, etc., and the requirements of the dimensional accuracy and surface quality of finished workpiece are very high. For example, Airbus A220-300 aircraft has a length of 38 m and a wingspan of 35.1 m. At the same time, the number of rivets/bolt fasteners involved in the skin-assembly is as high as millions. It requires the assembly position and orientation errors are less than ±0.1 mm and ±0.5°, respectively. The strict requirements of the machining quality arise great challenges to the manufacturing equipment and process of large-scale components.

Compared with CNC machine tools, one of the important development trends in the field of large-scale machining operations in suit is the use of a mobile robotic system.³ The idea is to install the robot or workpiece on a single- or omni-directional mobile platform to move the platform to the area where the robot can complete various processing work. In the A380 wing flexible manufacturing system at the Spain Airbus plant, fast movements over a 70 m length are achieved by mounting two sets of face-to-face Tricept 9000 hybrid robot modules on a linear guide.⁴ By cooperating with a dedicated end-effector and a measurement control system, high efficiency and high precision drilling and riveting can be complete on the left and right wing panels. KUKA has developed an omni-directional mobile robot system (Omnimove mobile platform + Quantec shelf-mounted robot) for machining large turbine blades.⁵ The Southwest research institute of the United States develops the MR ROAM 2 system for commercial (military) aircraft spraying. The system consists of a Motoman industrial robot on an omnidirectional mobile platform. It can be seen that the use of mobile robotic system is an effective way to achieve precise processing and assembling distributed in multiple areas of large-scale structural parts.

Instead of traditional and most frequent solution to machining large-scale parts with serial robots, a five-degree-of-freedom (5-DOF) hybrid robot (which is made up of a 3-DOF parallel manipulator and a 2-DOF serial manipulator) shows a huge potential to meet high precision requirements.⁶ Some types of hybrid robots have been proposed over the last decade. Invented by Karl-Erik NeuMann, the Tricept robot^7,8 is the most popular hybrid robot, and over 300 have been put into production in Boeing, EADS and Bombardier. Another hybrid robot is Exechon^9,10 also invented by NeuMann. Aiming at manufacturing process of sophisticated and individualized large components, a novel 5-DOF hybrid robot named TriMule¹¹ was designed in our previous work. As a plug-and-play module, TriMule can be applied in the large-scale machining systems for local processing and be fixed on a mobile platform for multi-station machining.

To realize the high precision machining for large-scale structural parts, the robotic system requires not only high precision of the robots, but also corresponding measurement systems which can measure and control the precision of the robot in time.^12,13 Various measurement schemes have been proposed for improving the robotic manufacturing accuracy.^14–17 In general, these measurement schemes are mainly divided into two types, the first one improves the absolute positioning accuracy of the robot through measuring devices outside the robot, and the second one measures and compensates errors through the measuring devices that are mounted on the end-effector of robot. The first type usually uses external measuring devices to measure the pose of the target mounted on the robot. In the mobile robot processing system proposed by Fraunhofer IFAM,¹⁸ the laser tracker was used to determine the position of the workpiece relative to the moving platform, and a stereo camera was used to capture the marks mounted on the robot effector in real time to improve the positioning accuracy of the robot. Qu¹⁹ developed a robot-aided aircraft assembly drilling system based on tracker closed-loop feedback to improve the accuracy of the position and orientation of the robotic end tool. The second uses measuring devices on the robot effector to measure the machining datum or profile of the workpiece. Frommknecht²⁰ presented a multi-sensor measurement system for robotic drilling, which consists of a 2D camera system, laser distance sensors and contactless high accuracy distance measurement sensors for referencing and orthogonal alignment. Rao²¹ proposed a method to realize robust, accurate, and online normal direction measurement and optimization to improve the surface smoothness after rivet installation by using a structural-light-based three-dimensional measurement. In contrast, the latter is more flexible due to the movement of the end-effector of the robot, and can consider the errors caused by deformations of the workpiece. In recent years, three-dimensional reconstruction technology has been widely used in the industrial field. Commonly used three-dimensional measurement methods include fringe projection technology,²² binocular vision,²³ and laser radar.²⁴ Compared to other methods, fringe projection technology has high resolution, real-time operability and robustness.²⁵ The measurement system can perform on-line measurement of the workpiece profile and measure the position and orientation of the robot and workpiece in relation to each other, which makes robotic machining more flexible.

This paper introduces a mobile robotic system, which consists of a hybrid robot mounted on a mobile platform and a fringe-projection-based measurement system. The measurement system enables fast measurement of the machined surface profile and provides dense and high-precision 3D point cloud. In order to determine the positional relationship between the reference coordinate system and the robot coordinate system efficiently, a vision-based method is proposed to measure the position of the reference features. To solve the problem that the actual profile of the workpiece does not conform to the CAD model due to machining error, stress release and uneven workpiece clamping force, the system can estimate the profile error by matching and comparing the actual surface profile and the CAD model to reduce the effects of workpiece deformation. Due to the absolute positioning error of the robot, the deformation of the workpiece and other factors, the normal direction of drilled surface are required to be measured online. The system enables to measure the profile of the drilled surface and determines the normal direction at the drilled point. In this paper, the process for aligning the robotic tool to the measured surface is also studied.

The remainder of the paper is organized as follows. Section 2 describes the mobile robot processing system and focuses on the robot and the fringe-projection-based measurement system. In Section 3, some measurement tasks that the system can achieve are introduced. In Section 4, the experimental setup and results achieved are presented. Finally, conclusions are drawn in Section 5.

Descriptions of the mobile robotic system

As shown in Figure 1, the mobile robotic system mainly consists of a hybrid robot named TriMule, a fringe-projection-based measurement system, an automated guided vehicle (VGA) and a flexible fixture, etc. TriMule is mounted on the VGA that greatly extend the workspace. The measurement system is mounted on the robot’s end-effector for high-precision 3D measurement in the work area. Combined with the high-speed electric spindle and tools on the end-effector, the system can complete the high-efficiency and high-precision drilling, riveting and milling for large-scale structural parts in situ.

Figure 1.

Conceptual design of the mobile robotic system.

The technical scheme of the mobile robotic system has the following advantages: (1) the system has low-cost and strong reconfigurability, which can be rapidly rearranged according to the task requirements to build a multi-robot collaborative manufacturing system; (2) the robot equipped with various end-effectors is convenient to quickly reach the optimal working position, which has stronger adaptability to the machining of large curvature surface; (3) the robot is flexible in motion and suitable for working in narrow space, and furthermore the collaboration of multi-robot can significantly improve the work efficiency; (4) the accurate metrology information allows the robot to perform tasks such as measuring the shape of an irregular surface, which would have been very difficult to automate.

TriMule

TriMule^11,26 is a novel 5-DOF hybrid robot that is composed of an over-constrained 1T2R (T, translation; R, rotation) parallel mechanism and a 2-DOF wrist, as shown in Figure 2. The parallel mechanism comprises an actuated UPS limb and a stand-alone 1T1R planar linkage that contains two actuated RPR limbs and a properly constrained passive RP limb in-between. The base link of the 1T1R planar linkage is elaborately designed into a three-in-one part that locates the rear R joints of the two actuated RPR limbs, and the R and P joints of the RP limb, and at the same time it is connected by a pair of R joints with the machine frame. R, P, U, and S stand, respectively, for revolute, prismatic, universal, and spherical joints where the underlined joint represents the actuator. This feature brings a special issue that the compliance compatibility conditions amongst three planar limbs must be taken into account in the semi-analytic stiffness modeling of the parallel mechanism. In brief, TriMule is an overconstrained 1T2R parallel mechanisms can have several advantages over their counterparts that lack overconstraints: higher stiffness, greater cost-effectiveness, and easier-to-obtain explicit inverse/forward kinematics.

Figure 2.

3D model of TriMule.

Fringe-projection-based measurement system

In the mobile robotic system, a fringe-projection-based measurement system is used to increase machining accuracy. The structure of a measurement system can be shown in Figure 3(a).The measurement principle is based on fringe projection profilometry.

Figure 3.

A fringe-projection-based measurement system: (a) the structure of the one projector-and-one camera, (b) fringe patterns and absolute phase maps, (c) the structure of the one projector-and-two cameras, and (d) the prototype of the three-dimensional measurement system.

First, four-step phase shifting method²⁷ and three-frequency unwrapped method²⁸ are applied to calculate the absolute phase. Twelve fringe patterns (four step phase shifting patterns on each of the three frequencies) are projected onto a standard plane and the camera captures these fringe patterns. The absolute phase map on the camera image plane can be calculated as the weighted average of the three wrapped phases.

Then, the one-to-one mapping relationship between the point on projector image plane and the corresponding point on camera image plane are established. The projector projects two sets of patterns (with horizontal and vertical fringes as shown in Figure 3(b)), the real one-to-one mapping relationship between the pixels on the projector and camera image plane can be established by searching the identical phases.

However, the optical structure of the projector in this measurement system can be regarded as a reverse of an imaging system (camera). The projector should be calibrated precisely to obtain three-dimensional point cloud accurately. Therefore, we use double cameras to avoid accurate calibration of the projector (as shown in Figure 3(c)). The phase information of the fringe image produced by the projector is only used as the basis of binocular matching, so the accuracy of the projector has little effect on the measurement system. Figure 3(d) shows the prototype of the three-dimensional measurement system.

3D measurement

Referencing

In order to determine the reference coordinate system efficiently and accurately, we propose a method based on binocular vision. The coarse position and orientation of the workpiece and robot with respect to a common reference frame is assumed to be known. When the end-effector of robot moves to the nominal coordinates, the measurement system can find the reference features. Here sticky tags are used as reference features. As shown in Figure 4(a), the reference features are circular reflective markers that provide a clear image when they are photographed by the camera. Three markers are used to define a reference coordinate system in the paper. It is noted that more markers can be used to mitigate measurement errors.

Figure 4.

Coordinate frame of the mobile robotic system: (a) circular reflective markers and (b) machining datum measurement and coordinate transformation among frames.

To determine the position and orientation of the machining data, the mobile robotic system approaches the workpiece at its nominal reference coordinate system ( $NRCS$ ), which is obtained from the CAD model. When the robot stops stabilizing, the cameras are used to capture the image with the reference markers. Then outline of the circular marker can be extracted and pose of the marker can be calculated in camera coordinate system ( $CCS$ ). As shown in Figure 4(b), when getting the pose of all the markers, the reference coordinate system ( $RCS$ ) is computed by following predefined rules. Then the transformation matrix of the robot base coordinate system ( $BCS$ ) system with respect to the $RCS$ can be derived as

{{}^{R}T}_{B} =^{R} T_{C} \cdot^{C} T_{E} \cdot^{E} T_{B}

(1)

where the homogeneous matrix of camera coordinate system to robot end-effector coordinate ( $ECF$ ) ${{}^{C}T}_{E}$ can be calibrated by using a hand-eye calibration method, and the relationship of $BCS$ to $ECS$ can be obtained from robot controller.

In our previous work, we proposed an algorithm for estimating the pose of circular marker.²⁹ The method uses the circular contour obtained by the monocular camera to estimate the pose information of the spatial circle whose radius is known. Firstly, the circular marker has been mapped to be an upright elliptic cone in the pinhole imaging model. Secondly, radius constraint has been applied to recover depth of the circular marker based on the upright elliptic cone. Finally, the pose of the circular marker has been estimated by single projection only.

Profile error estimation

For a large-scale structure part, the actual profile is likely to be different from that in the CAD model due to the tool error, cutting force, and deflection due to assembly stress. Hence, a method of online measurement and model matching is proposed to compensate error. The profile of the workpiece is measured online, and the actual profile is aligned with the ideal model. Then, the deviation of the actual workpiece profile and the CAD model is calculated.

The mobile robotic system can control the three-dimensional measurement system to measure the area to be processed. The point cloud $Ω^{C}$ sampled on a machining part in the camera coordinate system can be transformed to those in the robot base coordinate system as

Ω^{B} : {p | p =^{B} R_{C} p_{c} -^{B} t_{C}, p_{c} \in Ω^{C}}

(2)

where translation vector ${{}^{B}t}_{C}$ and rotation matrix ${{}^{B}R}_{C}$ can be obtained as

{{}^{C}T}_{B} =^{C} T_{E} \cdot^{E} T_{B}

(3)

Since both the robot and the measurement system have been calibrated, the point cloud $Ω^{B}$ and the CAD model are located to each other sufficiently close in the robot base coordinate system. In order to compare two models, some features such as points, lines, and faces are selected. These features are usually the edges of the machined area or areas of great stiffness. Then the features on the workpiece can be aligned to the corresponding features of its CAD model. The most successful methodology for date registration is the classical iterative closest points (ICP) algorithm.³⁰ The ICP algorithm tries to lookup the closet points and work out the optimal transformation between two point sets. Through the ICP algorithm, features can be aligned accurately. Supposing theoretical points on design surface constitute a point cloud $Ω^{Q}$ , the distance between a point $p_{i}$ in $Ω^{B}$ and $Ω^{Q}$ is

d (p_{i}, Ω^{Q}) = \min_{j \in {1, . . .}} d (p_{i}, q_{j})

(4)

where $q_{j} \in Ω^{Q}$ . Supposing the closet point $q_{j}$ in $Ω^{Q}$ marked ${q_{i}}^{'}$ satisfies

d (p_{i}, Ω^{Q}) = d (p_{i}, {q_{i}}^{'})

(5)

The mean square objective function to be minimized can be defined as

\min F (R, t) = \sum_{i = 1}^{n} | | {q_{i}}^{'} - R p_{i} - t | |^{2}

(6)

where $R$ denotes the rotation matrix, $t$ denotes the translation vector. By solving the equation, we can get the matching parameters between the two cloud points. After aligning the features, the deviation can be obtained by comparing the models.

Implementation of auto-normalization

Robotic drilling for large-scale parts demands high positioning accuracy of robots, which is usually achieved through error measurement and compensation online. In order to measure the normal direction of drilled surface efficiently and accurately, a method based on point cloud is introduced.

Normal direction measurement

At the beginning of the robotic drilling process, the robot moves to drilled point according to the NC program written by off-line programming system. As shown in Figure 5, the point cloud of the workpiece surface is acquired by the proposed the fringe-projection-based measurement system. It is well established in differential geometry that the local geometric shape of a point can be approximated by a quadric surface. So, in order to extract the normal direction at the target drilling position $P_{c}$ , the neighbor points are on the workpiece surface within 30 mm from point $P_{c}$ are selected and approximated by a quadric surface. The approximated quadric surface can be expressed as:

\begin{matrix} Φ (x, y, z) = A x^{2} + B y^{2} + C z^{2} + D + Exy \\ + Fxz + G yz + Hx + Jy + Kz \end{matrix}

(7)

where A∼K are the coefficients of quadratic surface. To solve the coefficients, an objective function of fitting quadric surface is established

ζ = \sum_{i = 1}^{N} {‖ A {x_{i}}^{2} + B {y_{i}}^{2} + C {z_{i}}^{2} + D + E x_{i} y_{i} + F x_{i} z_{i} + G y_{i} z_{i} + H x_{i} + J y_{i} + K z_{i} ‖}^{2}

(8)

where N is the number of neighborhood points, $(x_{i}, y_{i}, z_{i})$ is defined as the coordinate of the point. So, the coefficients can be calculated by the optimization method.

Then the normal vector ${\vec{n}}_{c}$ of a point on a quadratic surface is

{\vec{n}}_{c} = \nabla Φ (x, y, z) = \frac{{(\begin{matrix} \frac{\partial Φ}{\partial x} & \frac{\partial Φ}{\partial y} & \frac{\partial Φ}{\partial z} \end{matrix})}^{T}}{| {(\begin{matrix} \frac{\partial Φ}{\partial x} & \frac{\partial Φ}{\partial y} & \frac{\partial Φ}{\partial z} \end{matrix})}^{T} |}

(9)

The normal direction ${\vec{n}}_{c}$ and target drilling position $P_{c}$ in the camera coordinate can be transformed to tool coordinate as:

{\vec{n}}_{t} =^{Tool} R_{E} \cdot {({{}^{C}R}_{E})}^{- 1} \cdot {\vec{n}}_{c}

(10)

P_{t} =^{Tool} T_{E} \cdot {({{}^{C}T}_{E})}^{- 1} \cdot P_{c}

(11)

where ${{}^{tool}T}_{E}$ is the relationship between the robot end-effector coordinate system and the tool coordinate system.

Figure 5.

Measurement chain.

Alignment method

As shown in Figure 6, the unadjusted tool coordinate system is named $Tool$ , where the spindle axis direction is consistent with the negative direction of the $Z$ axis; $P_{t}$ is the target drilling position in the Tool coordinate. The coordinate system $Hole$ : $P_{t} X_{p} Y_{p} Z_{p}$ is established at point $P_{t}$ , and the coordinate axis direction of the $Hole$ is consistent with the $Tool$ . Assuming $P_{t} = {(x_{p}, y_{p}, z_{p})}^{T}$ , the homogeneous matrix of $Tool$ relative to $Hole$ is

{{}^{Hole}T}_{tool} = (\begin{matrix} E_{3 \times 3} & P_{t} \\ 0_{1 \times 3} & 1 \end{matrix})

(12)

where $E_{3 \times 3}$ is the identity matrix.

Figure 6.

Geometric model.

Supposing the outer normal vector to the drilling skin surface is shown as $\vec{P_{t} N}$ , the angle $θ$ is defined as the intersection angle between the spindle axis and the normal vector, so the problem to normalize the spindle axis direction can be simplified as minimizing the angle $θ$ .³¹

As shown in Figure 7, $\vec{P_{t} M}$ is the projection of $\vec{P_{t} N}$ on the $Z_{P} P_{t} Y_{P}$ coordinate plane; $\vec{P_{t} D}$ is the projection of $\vec{P_{t} N}$ on the $Z_{P} P_{t} X_{P}$ coordinate plane. Let $θ_{1}$ be the angle from $\vec{Z_{P}}$ to $\vec{P_{t} D}$ ; Let $θ_{2}$ be the angle from $\vec{Z_{P}}$ to $\vec{P_{t} M}$ ; Let $t$ be the angle from $\vec{P_{t} M}$ to $\vec{P_{t} N}$ . $θ_{1}$ , $θ_{2}$ and $t$ have both positive and negative sides, which are defined as follows: the rotation around $+ Y_{P}$ axis is the positive direction of $θ_{1}$ and $t$ ; the rotation around $+ X_{P}$ axis is the positive direction of $θ_{2}$ .

Figure 7.

Relationship between $θ_{1}$ , $θ_{2}$ and t.

Assuming $\vec{P_{t} N} = (x_{n}, y_{n}, z_{n})^{T}$ , the relationship between $θ_{1}$ , $θ_{2}$ and $t$ is presented as follows:

t = \arctan (\tan θ_{1} \times \cos θ_{2})

(13)

It is known that $\vec{Z_{P}}$ can be coincident with outer normal vector $\vec{P_{t} N}$ via pitch and yaw rotations . The adjusted hole coordinate system is named as $Hole'$ . The transformation can be described as

\begin{matrix} {{}^{HOLE'}T}_{HOLE} = Rot (x, θ_{2}) Rot (y, t) \end{matrix}

(14)

The homogeneous matrix of the robot base coordinate system with respect to $Hole$ is derived as

{{}^{HOLE}T}_{Base} =^{HOLE} T_{Tool} \cdot^{Tool} T_{E} \cdot^{E} T_{Base}

(15)

The homogeneous matrix of the robot base coordinate system with respect to $Hole'$ can be also derived as

{{}^{HOLE'}T}_{Base} =^{HOLE'} T_{HOLE} \cdot^{HOLE} T_{Base}

(16)

After the robot compensates for the error, the adjusted tool should be above the target drilled point, and the tool direction is consistent with the normal direction of drilled surface. So the relationship between the adjusted tool coordinate system $Tool'$ and $Hole'$ can be described as

{{}^{Hole'}T}_{Tool'} = (\begin{matrix} E_{3 \times 3} & P_{d} \\ 0_{1 \times 3} & 1 \end{matrix})

(17)

where $P_{d} = {(\begin{matrix} 0 & 0 & d \end{matrix})}^{T}$ is preset position value.

Thus, the target configuration of tool can be calculated as

{{}^{Tool'}T}_{Base} = {({{}^{HOLE'}T}_{Tool'})}^{- 1} \cdot^{HOLE'} T_{Base}

(18)

The procedure of the method is shown in Figure 8.

Figure 8.

Workflow of robotic drilling process.

Experiments and discussions

Measurement accuracy of the measurement system

Error of the measurement system will affect error compensation in robot machining, so the precision of measurement system is very important in this study. In the experiment, a standard part consisting of two standard balls has been used. As shown in Figure 9(a), the standard ball was bonded to a flat plate and the relative position of the standard ball is fixed.

Figure 9.

Accuracy verification of the measurement system: (a) standard part measurement and (b) point cloud of standard part.

The standard part was measured by fringe-projection-based measurement system. Then the point cloud data was used to fit two standard spheres A, B as shown in Figure 9(b). Supposing that the spherical coordinates of the fitted balls are $P_{A}$ , $P_{B}$ and the diameters are $D_{A}$ , $D_{B}$ , the distance $d$ between the two spheres can be obtained as $d = P_{A} - P_{B}$ .

In order to evaluate its measurement accuracy, the exact radius and distance were measured with a CMM (coordinate measuring machine). The maximal deviation of radius was 0.024 mm (see Table 1), and the maximum distance error was 0.004 mm (see Table 2), which demonstrates that the measuring accuracy of the measurement system is good enough.

Table 1.

Sphere fitting error.

Standard spheres	Measured radius (mm)	Exact radius (mm)	Error (mm)
A	20.010	19.989	0.021
	20.007	19.989	0.018
	20.010	19.989	0.021
B	20.011	19.988	0.023
	20.008	19.988	0.020
	20.012	19.988	0.024

Table 2.

Center distance error.

ID	Measured distance (mm)	Exact distance (mm)	Error (mm)
1	59.853	59.857	−0.004
2	59.856	59.857	−0.001
3	59.861	59.857	0.004

Measurement accuracy of the robotic system

In order to verify the accessible measurement accuracy in the course of processing, one measurement scheme was established as shown in Figure 10. The fringe-projection-based measurement system was mounted on TriMule’s end-effector and a test workpiece with reference features was fixed on the fixture.

Figure 10.

Experiment setup.

In the experiment, some benchmarks were placed near the reference features. The exact positions of reference features and benchmarks were measured with a CMM, transformed to reference coordinate system. In order to compute the exact normal directions of the benchmarks, the surface patches around the benchmarks were also measured with a CMM. Then the reference features were measured by the fringe-projection-based measurement system to establish the reference coordinate system, and the robot was programmed to approach each benchmark to measure the position and normal direction of each benchmark and transfer them to the reference coordinate system. The position of benchmarks was obtained via triangulation using the calibrated parameters of the two cameras, and the normal direction of the benchmarks was computed through the method described in Section 3.

Table 3 shows the position error $Δ P$ and normal error $Δ N$ between the actual and measured benchmarks. The average position error and the average normal error are 0.075 mm and 0.21°; The maximal position error and the maximal normal error are 0.102 mm and 0.27°. Compared to Table 1 and 2, measurement errors in Table 3 are comparatively large. This is because that errors in Table 3 contain the positioning error of the robot itself and the measurement error of the measurement system. The experimental results show that the measurement accuracy meets the technical requirements of large-scale manufacturing. Furthermore, the system demonstrates a great potential for improved measurement accuracy.

Table 3.

Position error and normal error.

ID	ΔP (mm)	ΔN (°)	ID	ΔP (mm)	ΔN (°)	ID	ΔP (mm)	ΔN (°)
1	0.049	0.09	4	0.074	0.27	7	0.054	0.22
2	0.088	0.24	5	0.056	0.11	8	0.087	0.11
3	0.083	0.25	6	0.102	0.24	9	0.080	0.27

Conclusion

In this paper a mobile robotic system for large-scale manufacturing has been introduced. The system is able to measure the workpiece profile online and obtain dense point cloud with high precision by the fringe-projection-based measurement system. The measurement system enables the robot to measure the pose of the reference feature and establish the reference coordinate system efficiently. 3D point cloud of actual surface profile is matched with CAD model to estimate the profile error, which can reduce the effect of workpiece deformation. Furthermore, a method of normal direction measurement based on 3D point cloud is proposed to reduce the normal error during drilling, and the process for aligning the robotic tool to the measured surface is theoretically analyzed. The proposed methods are verified by experiments. It can be concluded from the results that the integration of fringe-projection-based measurement system on a mobile robotic system offers a fully automated and high efficiently alternative solution for large-scale manufacturing. It is worth pointing out that one-station measurement and machining is studied in this paper. Multi-station measurement and machining will be analyzed soon and more experiments will be conducted to validate effectiveness of the mobile robotic system.

Due to large-scale of complex components, it brings great challenges to the measurement of component size and the evaluation of machining quality. Furthermore, it seriously affects the selection of machining allowance, trajectory planning and control parameters of the robot, so it is necessary to establish a flexible and precise measurement mechanism based on the multi-sensor information fusion, which can not only realize in-situ precise measurement and machining quality assessment oriented to multi object and multi process, but also provide powerful data support for autonomous decision-making and adjustment of robot machining process and meet the needs of robot autonomous machining.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Key Research and Development Program (Grant No. 2017YFE0111300), National Natural Science Foundation of China (Grant No. 51775376), and EU H2020 project (Grant No. 734272).

ORCID iDs

Yabin Ding

Xianping Liu

References

Mark

Robot capability test and development of industrial robot positioning system for the aerospace industry. SAE Trans 2005; 114(1): 1108–1118.

Savio

De Chiffre

Carmignato

, et al. Economic benefits of metrology in manufacturing. CIRP Ann 2016; 65(1): 495–498.

Peggs

Maropoulos

Hughes

, et al. Recent developments in large-scale dimensional metrology. Proc IMechE Part B J Eng Manuf 2009; 223(6): 571–595.

Olazagoitia

Wyatt

New PKM Tricept T9000 and its application to flexible manufacturing at aerospace industry. SAE Int 2007; 01: 3820.

KUKA. An omni-directional mobile robot system, https://www.kuka.com/en-gb/products/mobility/mobile-robots/kmr-quantec (accessed 22 May 2020).

Luo

Xie

Liu

, et al. Error modeling and sensitivity analysis of a novel 5-degree-of-freedom parallel kinematic machine tool. Proc IMechE Part B J Eng Manuf 2019; 233(6): 1637–1652.

Neumanm

KE.

Robot. Patent US 4732525, USA, 1988.

Neumanm

. Tricept application. In: Proceedings of the 3rd Chemnitz parallel kinematics seminar, Zwickau, Germany, 23–25 April 2002, pp.547–551. Zwickau: Verlag Wissenschaftliche Scripten.

Neumanm

KE.

Parallel-kinematical machine. Patent WO/2006/054935, 2006.

10.

Jin

Kinematic modeling of Exechon parallel kinematic machine. Robot Comput-Integr Manuf 2011; 27 (1): 186–193.

11.

Huang

Dong

Liu

, et al. Five-degree-of-freedom hybrid robot with rotational supports. Patent 9943967 B2, USA, 2018.

12.

Schmitt

Peterek

Morse

, et al. Advances in large-scale metrology – review and future trends. CIRP Ann Manuf Technol 2016; 65: 643–665.

13.

Prabhu

Muhandri

Song

, et al. Decision support system enabled by depth imaging sensor data for intelligent automation of moving assemblies. Proc IMechE Part B J Eng Manuf 2018; 232(1): 51–66.

14.

Muelaner

Cai

Maropoulos

PG.

Large-volume metrology instrument selection and measurability analysis. Proc IMechE Part B J Eng Manuf 2010; 224(6): 853–868.

15.

Maisano

Jamshidi

Franceschini

, et al. A comparison of two distributed large-volume measurement systems: the mobile spatial co-ordinate measuring system and the indoor global positioning system. Proc IMechE Part B J Eng Manuf 2009; 223(5): 511–521.

16.

Uekita

Takaya

On-machine dimensional measurement of large parts by compensating for volumetric errors of machine tools. Precis Eng 2016; 43: 200–210.

17.

Fan

, et al. A vision-based processing methodology for profile grinding of contour surfaces. Proc IMechE Part B J Eng Manuf 2020; 234(1–2): 27–39.

18.

Susemihl

Moeller

Kothe

, et al. High accuracy mobile robotic system for machining of large aircraft components. SAE Int J Aerospace 2016; 9: 231–238.

19.

Dong

Pose accuracy compensation technology in robot-aided aircraft assembly drilling process. Acta Astronaut 2011; 32(10): 1951–1960.

20.

Frommknecht

Kuehnle

Effenberger

, et al. Multi-sensor measurement system for robotic drilling. Robot Comput-Integr Manuf 2017; 47: 4–10.

21.

Rao

Wang

Yang

, et al. Normal direction measurement and optimization with a dense three-dimensional point cloud in robotic drilling. IEEE-ASME Trans Mech 2018; 23(3): 986–996.

22.

Zuo

Feng

Huang

, et al. Phase shifting algorithms for fringe projection profilometry: a review. Opt Laser Eng 2018; 109: 23–59.

23.

Sharan

Onwubolu

GC.

Client-server control architecture for a vision-based pick-and-place robot. Proc IMechE Part B J Eng Manuf 2012; 226(8): 1369–1378.

24.

Bordron

Mehai-Souzani

Bruneau

Inline measurement strategy for additive manufacturing. Proc IMechE Part B J Eng Manuf 2019; 233(5): 1402–1411.

25.

Zuo

Chen

, et al. High-speed three-dimensional shape measurement for dynamic scenes using bi-frequency tripolar pulse-width-modulation fringe projection. Opt Laser Eng 2013; 51(8): 953–960.

26.

Cheng

Liu

Yue

, et al. Stiffness modeling and analysis of a novel 5-DOF hybrid robot. Mech Mach Theory 2018; 125: 80–93.

27.

Surrel

Design of algorithms for phase measurements by the use of phase stepping. Appl Opt 1996; 35(1): 51–60.

28.

Towers

Jones

Absolute fringe order calculation using optimised multi-frequency selection in full-field profilometry. Opt Lasers Eng 2005; 43(7): 788–800.

29.

Mei

Zhang

Ding

YB.

Monocular vision for pose estimation in space based on cone projection. Opt Eng 2017; 56(10): 103108.

30.

Besl

McKay

A method for registration of 3-D shapes. IEEE Trans Pattern Anal 1992; 14(2): 239–256.

31.

Tian

Zhou

, et al. Auto-normalization algorithm for robotic precision drilling system in aircraft component assembly. Chin J Aeronaut 2013; 26(2): 495–500.

Development of a novel mobile robotic system for large-scale manufacturing

Abstract

Keywords

Introduction

Descriptions of the mobile robotic system

TriMule

Fringe-projection-based measurement system

3D measurement

Referencing

Profile error estimation

Implementation of auto-normalization

Normal direction measurement

Alignment method

Experiments and discussions

Measurement accuracy of the measurement system

Measurement accuracy of the robotic system

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iDs

References