Tractor-trailer-train braking time sequence detection based on monocular vision

Abstract

The tractor-trailer-train at the braking process prone to braking instability caused by asynchronous braking between the shafts. With respect to the lack of intelligent detection of Braking Time Sequence (BTS), a non-contact dynamic detection scheme of intelligent vehicle BTS is proposed. Based on the monocular vision principle, the edge markers of tractor-trailer train tires are identified, and the tire slip rate is solved. The noise reduction of the collected image is processed. The marker area is obtained by Blob analysis. This region at the image to be matched is identified by the template matching algorithm based on contour. The camera is calibrated by Zhang’s calibration method. In order to verify the effectiveness of the detection scheme, the real vehicle test was carried out. The test results show that the error of slip rate solution is below 4.2%.

Keywords

Tractor-trailer train braking time sequence slip rate template matching algorithm camera calibration real vehicle test

Introduction

Tractor-trailer-trains occupies an important position in the transportation industry because of its high fuel economy and convenient. Although tractor-trailer-trains have high transport efficiency and adapt to the current logistics development requirements, they also have many inevitable shortcomings. Since tractor-trailer-trains are composed of tractor and one or more trailers, the dynamic performance of the vehicle becomes complex and the handling stability becomes poor. Improving the handling stability of tractor-trailer-train has also become the focus of scholars’ research. In the braking process of tractor-trailer-train, if the front axle braking is prior to the rear axle, it is prone to folding accidents. If the front axle braking lags behind the rear axle, it is prone to dragging. Therefore, the inter-axle braking sequence affects the braking stability of the vehicle. At present, the test methods of vehicle braking performance mainly include bench test and road test. With the help of portable braking performance tester, the road test method can detect the braking distance, the fully emitted average deceleration and the braking deviation. The bench test method mainly relies on the drum reaction brake test bench and the flat plate brake test bench for testing, which can detect the braking force and braking imbalance and block force, and meet the detection of most vehicles. However, the above two test methods can not realize the detection of the BTS of the vehicle, which is difficult to reflect the braking performance of the vehicle. Therefore, it is particularly important to detect the BTS to determine whether it is necessary to adjust and maintain the braking system.

In order to improve the braking stability of the tractor-trailer-train, He and Jing¹ designed a new pneumatic braking system, and proposed a strategy to control the difference between the actual braking torque and the target braking torque of the semi-trailer. The simulation results show that the improved scheme reduces the braking response time and improves the braking stability of the vehicle. Elhemly and Fayed² established a Simulink model to study the BTS under different driving conditions. The model verified by experiments was used to predict the braking stability of tractor-trailer trains under different combinations of tractors and trailers. Castillo Aguilar et al.³ developed an extended Kalman filter slip rate estimation algorithm based on tire force and vehicle speed. The scheme obtains the real-time torque of the tire by six-component wheel force sensor, and obtains the best slip rate with GPS system. Although the detection results are accurate, the hardware installation is complex, and the method is not suitable for multi-axle tractor-trailer trains. Goodarzi et al.⁴ analyzed the braking process of tractor-trailer train, and proposed a scheme for braking force distribution by controlling the slip rate of tires on both sides of the coaxial line. This scheme improved the braking stability of tractor-trailer train. Wang et al.⁵ conducted experimental analysis on the braking hysteresis characteristics of the pneumatic braking system of the multi-axle tractor-trailer trains, but the test can only qualitatively explain the factors affecting the braking hysteresis, cannot be quickly and accurately measured for specific models. Xie et al.⁶ optimizes the braking force distribution coefficient of the tractor-trailer train and the hanging workshop by establishing the dynamic model of the tractor-trailer-train, so that the tractor-trailer-train brakes according to the best BTS and ensures the good braking performance of the vehicles.

For the detection of the BTS, Zhuang and Wu⁷ used infrared photoelectric sensors to scan a disc with 36 sector-shaped grooves, and obtained pulse signals by cutting infrared light from the disc. This method has low accuracy and too complicated equipment installation to meet the current requirements of automated vehicle detection. Ling⁸ detects the braking pressure of each brake by the accelerometer installed on the brake push rod of the vehicle, so as to obtain the BTS of each axle. However, this method belongs to static detection and cannot obtain the BTS under actual braking. Xiao⁹ developed a detection system for the flatbed brake test sets of tractor-trailer train, due to the limitation of the space layout of the detection system. This method is not suitable for different types of tractor-trailer-train.

As a non-contact detection method, visual detection has been widely used in vehicle detection field because of its high detection accuracy and easy arrangement of detection system. Furferi et al.¹⁰ designed a set of wheel angle detection system based on three-dimensional machine vision technology. The accuracy of the system in measuring the front beam angle and camber angle meets the requirements. Shan et al.¹¹ used stereo vision technology to realize the dynamic detection of vehicle axle distance, and analyzes the error caused by vehicle curve driving, and compensates the detection results. Liu et al.¹² based on machine vision technology, established a real-time detection system of vehicle running angle. By comparing and analyzing the system data and test results, it is proved that the method is reliable in monitoring the change of track vehicle posture. Creager et al.¹³ used DIC system to observe the force difference between wheel push-pull movement and traditional rolling in sandy soil. Through comparative tests, it was found that push-pull movement increased the driving force of the vehicle than the traditional rolling method, and also reduced the resistance that the wheel needed to overcome. Due to the high price of vehicle sideslip angle sensor device, it is only suitable for vehicle development and performance evaluation. Based on DIC system, Johnson et al.¹⁴ improved the existing algorithm, and used cheap cameras to accurately measure the vehicle sideslip angle on rugged cross-country terrain. The above results show that machine vision technology can be applied to the research of intelligent vehicle detection.

Based on the above research, it can be seen that the braking performance detection and braking stability analysis are the focus of the current research on the braking of tractor-trailer-train. The research on the BTS is still very shallow. The existing technologies and methods have poor detection accuracy, low efficiency and lack of intelligence. Therefore, a method using monocular vision technology to detect the BTS is proposed. This method can detect the BTS in real time under the state of vehicle motion, and achieve dynamic and high-precision detection. The structure of this paper is as follows: Section I is the introduction. Section II analyzes the movement of the tire in the braking slip process of the tractor-trailer train, and proposes a scheme to solve the tire slip rate by using the tire edge marker. In Section III, the camera is calibrated by Zhang’s calibration method, and the accuracy of spatial positioning of monocular camera is verified by external parameter matrix. The contour-based template matching algorithm is used to identify the tire edge markers, and the collected images are processed. Blob analysis is used to obtain the position of the template area in the image. The Section IV conducts the real vehicle test to verify the effectiveness of the detection scheme and analyze the test results.

Theoretical analysis and design of detection scheme

Rationality analysis of BTS detection based on slip ratio

Tire slip ratio represents the degree to which the tire rotational speed do not enable to calculate correctly the vehicle speed when the vehicle starts or brakes at a significant acceleration. Tire slip ratio directly affects vehicle braking performance. The variation trend of longitudinal and transverse adhesion coefficient with tire longitudinal slip rate is shown in Figure 1.

Figure 1.

Relationship between longitudinal slip ratio and adhesion coefficient.

When the tire slip rate is in the range of 0%–20%, the longitudinal adhesion coefficient increases rapidly, and the braking effect of the wheel in this region increases significantly, which can make the vehicle in a stable driving state. When the slip ratio is greater than 20%, the longitudinal and lateral adhesion coefficients show a downward trend, the braking effect and directional stability of the wheel become worse, and the vehicle is in an unstable state. Therefore, if the wheel can obtain high longitudinal and lateral adhesion coefficient at the same time, the wheel is easy to achieve good braking effect and has good directional stability. The BTS can be evaluated by the slip rate of the wheel: under the condition of a certain road adhesion coefficient, the time when the slip rate of each wheel reaches 20% is compared with the initial time when each wheel triggers the braking.

Solution of tire slip ratio and rotation angle

A circular marker is posted at the edge of the tire, and a monocular camera is used to detect the marker, and the trajectory of the marker is calculated to obtain the tire slip rate. In the rolling state, the cycloid formed by the movement of circular markers posted on the sidewall is a periodic function. The expression in the rectangular coordinate system is shown in. R and r denotes the radius of the tire and the distance between the tire center and the marker center respectively, α denotes the rotation angle of the tire, X and Y respectively denote the transverse and longitudinal coordinates of the marker center in the Cartesian coordinate system, X_a and Y_a denote respectively the transverse and longitudinal coordinates of the tire center in the Cartesian coordinate system.

{\begin{matrix} X = X_{a} - r \sin α = R α - r \sin α \\ Y = Y_{a} - r \cos α = R - r \cos α \end{matrix}

(1)

In the slip state, the tire increases a slip in the horizontal direction. The trajectory of the tire edge markers with and without slip is shown in Figure 2. The large circle in the figure represents the tire, and the small circle represents the circular marker. s₁ and s₂ denotes the cycloid trajectory of the tire edge marker without and with tire slip respectively, X_s denotes the horizontal displacement caused by tire slip. Under the slip state, the horizontal motion equation of the marker is shown in equation (2).

X = (X_{a} + X_{s}) - r \sin α = (R α + X_{s}) - r \sin α

(2)

Figure 2.

Slip trajectory diagram of tire edge marker.

The displacement D_a of the center of the adjacent two-point tire under slip state is shown in equation (3). Figure 3 shows the relationship between slip and horizontal displacement of unslipped markers.

D_{a} = X'_{a, i} - X'_{a, i - 1} = X_{a, i} + X_{s, i} - (X_{a, i - 1} + X_{s, i - 1})

(3)

Figure 3.

Relationship diagram of tire slip displacement.

In equation (3), X′_a,i denotes the lateral displacement of the wheel center in the slip state of the tire, and the time interval between the two points is T. If T is small enough, the instantaneous velocity of the tire at i time can be expressed by the average velocity, as shown in equation (4).

\begin{matrix} u_{i} = \frac{D_{a}}{T} = \frac{X_{a, i} + X_{s, i} - (X_{a, i - 1} + X_{s, i - 1})}{T} \\ = \frac{X_{i} + r \sin α_{i} - (X_{i - 1} + r \sin α_{i - 1})}{T} \end{matrix}

(4)

Let u_w be the translational velocity formed by the rotation of the tire at angular velocity ω. When T is small enough, the instantaneous velocity is expressed as the average velocity in time interval T, and the translational velocity u_w,i at i time is expressed as equation (5).

u_{w, i} = ω_{i} R = \frac{R (α_{i} - α_{i - 1})}{T}

(5)

According to formulas equations (4) and (5), the slip rate equation is shown in equation (6).

\begin{matrix} s_{i} = \frac{u_{i} - u_{w, i}}{u_{i}} \\ = \frac{X_{i} + r \sin α_{i} - X_{i - 1} - r \sin α_{i - 1} - R (α_{i} - α_{i - 1})}{X_{i} + r \sin α_{i} - X_{i - 1} - r \sin α_{i - 1}} \end{matrix}

(6)

In order to obtain more accurate angle values, the rotation angles of the two markers were solved by using the inverse tangent function. The trajectories of the two markers in a cycle are shown in Figure 4. The angle formula of tire marker is shown in equation (7)

\begin{matrix} α_{i} = \frac{π}{2} - \arctan (\frac{Y_{i, A} - Y_{i, B}}{X_{i, A} - X_{i, B}}) & X_{i, A} - X_{i, B} < 0 \\ α_{i} = \frac{3 π}{2} - \arctan (\frac{Y_{i, A} - Y_{i, B}}{X_{i, A} - X_{i, B}}) & X_{i, A} - X_{i, B} > 0 \\ \begin{matrix} α_{i} = π & X_{i - 1, A} - X_{i - 1, B} < 0 \\ α_{i} = 0 & X_{i - 1, A} - X_{i - 1, B} > 0 \end{matrix} & X_{i, A} - X_{i, B} = 0 \end{matrix}

(7)

Figure 4.

Trajectory diagram of dual-marker tire.

When vehicle is driving, tire subsidence will occur, affecting the marker coordinates. In order to reduce the tire subsidence, the vehicle should be no-load tested, and whether the tire pressure meets the standard tire pressure before testing. Affected by the internal force of the tire, the force of the tire near the rim is less than that of the tire edge, and the marker is posted near the rim.^15–17

Camera calibration and tire marker extraction

Monocular camera imaging principle

In the whole monocular vision imaging system, the coordinate systems involved are: world coordinate system (Xw, Yw, Zw), camera coordinate system (Xc, Yc, Zc), image plane coordinate system (X, Y), pixel coordinate system (u, v). Pixel coordinates are discrete integer coordinates. The imaging principle is shown in the Figure 5.

Figure 5.

Monocular vision system imaging diagram.

The coordinate of any point P in three-dimensional space is (Xc, Yc, Zc)^T, in the camera coordinate system, and the corresponding relationship with the coordinate (x, y) of the image point p in the physical imaging coordinate system is:

\frac{X}{X_{c}} = \frac{Y}{Y_{c}} = \frac{f}{Z_{c}}

(8)

Let the homogeneous coordinates of a space point P in the world coordinate system be (Xw, Yw, Zw)^T, and the homogeneous coordinates in the current camera coordinate system be (Xc, Yc, Zc)^T. The coordinate in the physical imaging coordinate system is (x, y), and the coordinate in the image plane coordinate system is (u,v). The following relationship can be obtained by establishing the above coordinate system:

(\begin{matrix} X_{C} \\ Y_{C} \\ Z_{C} \\ 1 \end{matrix}) = (\begin{matrix} R & t \\ 0 & 1 \end{matrix}) (\begin{matrix} X_{W} \\ Y_{W} \\ Z_{W} \\ 1 \end{matrix})

(9)

R denote rotation transformation matrix, and t denote translation vector. The transformation from camera coordinate system to physical imaging coordinate system is shown as follows:

Z_{C} (\begin{matrix} x \\ y \\ 1 \end{matrix}) = (\begin{matrix} f & 0 & 0 & 0 \\ 0 & f & 0 & 0 \\ 0 & 0 & 1 & 0 \end{matrix}) (\begin{matrix} X_{C} \\ Y_{C} \\ Z_{C} \\ 1 \end{matrix})

(10)

Conversion of physical imaging plane coordinate system to image plane coordinate system:

(\begin{matrix} u \\ v \\ 1 \end{matrix}) = (\begin{matrix} \frac{1}{d x} & 0 & u \\ 0 & \frac{1}{d y} & v \\ 0 & 0 & 1 \end{matrix}) (\begin{matrix} x \\ y \\ 1 \end{matrix})

(11)

It can be obtained equation (12) from equations (8)–(11):

\begin{array}{l} (\begin{matrix} u \\ v \\ 1 \end{matrix}) = (\begin{matrix} \frac{1}{d x} & 0 & u \\ 0 & \frac{1}{d y} & v \\ 0 & 0 & 1 \end{matrix}) \\ (\begin{matrix} f & 0 & 0 & 0 \\ 0 & f & 0 & 0 \\ 0 & 0 & 1 & 0 \end{matrix}) (\begin{matrix} R & t \\ 0 & 1 \end{matrix}) (\begin{matrix} X_{W} \\ Y_{W} \\ Z_{W} \\ 1 \end{matrix}) \end{array}

(12)

Camera calibration

The main purpose of camera calibration is to reduce the interference of camera imaging error on visual detection. The monocular camera is used to collect the marker images and obtain the marker center coordinates after image processing to obtain the tire slip rate, which requires the images collected by the camera to be consistent with the real scene as far as possible. In this paper, the monocular camera is calibrated based on Zhang’s¹⁸ calibration method. Considering the influence of lens distortion factors, LM algorithm¹⁹ is used to optimize the calibration parameters nonlinearly. LM algorithm combines the advantages of Newton’s downhill method and gradient descent method. By setting the adjustment factor according to the gradient change, it chooses which optimization method to use, so that the cost function is not easy to fall into local optimum. At the same time, LM algorithm can also provide accurate step size, local convergence faster. The optimization process of LM algorithm is as follows: Firstly, given the initial value x₀ and the trust region radius μ₀, the residual r_k at the initial point and the Jacobian matrix J_k are calculated, and the iterative step length λ_k is calculated. The Taylor approximation effect is calculated by the iterative calculation factor ρ_k, and the trust region radius is dynamically adjusted by the iterative calculation factor. The calculation formula of ρ_k is shown in equation (13).

ρ_{k} = \frac{f (x_{k} + Δ x_{k}) - f (x_{k})}{J (x_{k}) Δ x_{k}}

(13)

If ρ_k≤ 0.25, indicating that the step size is large, Taylor approximation effect is not good, reduce the trust region radius μ_k+1= 0.5 μ_k. If ρ_k≥ 0.75, it indicates that the step size is small, the Taylor approximation effect is accurate, and the radius of the expanded trust region μ_k+1= 2μ_k. If 0.25 ≤ ρ_k≤ 0.75, the step size is appropriate, μ_k+1= μ_k. When Δx_k is small enough to stop iteration, the value of x_k+1 is judged by ρ_k. When ρ_k≤ 0, equation (13) shows that the error increases, the optimization stops. When ρ_k≥ 0, the error can be further reduced, so x_k+1= x_k+ Δx_k. LM algorithm can dynamically adjust the radius of the trust region μ_k according to the error of the function value with the iterative calculation factor ρ_k, so as to dynamically select the optimization method according to the change of the function error value. It can speed up the optimization speed and reduce the possibility of obtaining local optimal solution.¹⁹

Matlab2019b is used to calibrate the monocular camera. The checkerboard is composed of black and white squares with 60 mm edge length. The array is 9 × 7 and the accuracy is ±0.001 mm. In order to better reflect the distortion degree of the image, the checkerboard target should traverse the entire field of view as much as possible. The chessboard and camera detail images are shown in Figure 6(a) and (b) respectively, and collected 16 calibrated target images are shown in Figure 7.

Figure 6.

(a) Chessboard detail image and (b) camera detail image.

Figure 7.

Chessboard image acquisition.

Through corner extraction, initial calculation of calibration parameters and LM algorithm optimization, the internal and external parameters of monocular camera are obtained, as shown in Table 1. The camera, lens type and parameters used in image acquisition are shown in Table 2.

Table 1.

Calibration parameters of monocular camera.

Parameter name	Parameter value
Intrinsic Matrix G	$[\begin{matrix} 1679.9171 & 0 & 964.2951 \\ 0 & 1680.3922 & 602.1930 \\ 0 & 0 & 1 \end{matrix}]$
Coefficient of radial distortion [k₁, k₂, k₃]	$[\begin{matrix} - 0.13373 & 0.11465 & - 0.00067 \end{matrix}]$
Tangential distortion coefficient [ p₁, p₂]	$[\begin{matrix} 0.000 & 0.0003 \end{matrix}]$

Table 2.

Camera and lens parameter table.

Camera parameters	Parameter value	Parameters of lens	Parameter value
Model	AI-230U150C	Model	HA0814-10M
Resolution	1920 × 1200	Focal length	8 mm
Pixel dimension	4.8 μm × 4.8 μm	Field angle (H × V × D)	84.8
Frame rate	30 fps		68
Sensor type	CMOS		98.4
Shutter mode	Global shutter	Optical interface	C-Mount
Buffer size	128 MB	Overall dimension	Φ66 × 77.8 (mm)

According to the camera parameters, the resolution of the camera is 1920 × 1200, and the main point coordinates of the camera should be in (960,600). It can be seen that the main point parameters of the camera obtained in Table 1 are basically near (960,600). At the same time, when the lens is 8 mm fixed focal lens and the pixel size is 4.8 μm × 4.8 μm, the theoretical value of f_x and f_y is about 1667. Table 1 shows that f_x, f_y are basically near 1667. This shows that the calibrated camera parameters conform to the actual camera parameters.

In order to further verify the calibration accuracy, the reprojection error is used to illustrate the calibration effect. The camera calibration reprojection error point cloud chart is shown in Figure 8. It can be seen from the reprojection error point cloud chart that the calibrated camera reprojection error is within ±1 pixel. It shows that the calibration results optimized by LM algorithm are ideal, and this calibration method can be applied to the BTS detection system.

Figure 8.

Camera reprojection error figure.

Image acquisition and processing

Image processing uses Halcon software. The Halcon software package contains a large number of image processing operators, and provides a custom operator package to meet the modification and design of operators by image processing personnel. It also supports C# and C++ conversion of image processing code, and is widely used in various image processing projects. In order to speed up the image process and reduce the search range of template matching, the collected images are segmented to find the Region of Interest (ROI). Since the markers are all located in the tire, the ROI area selects the strip area where the tire is located. The gen_rectangle operator in Halcon is used to select the ROI area, and the reduce_domain operator is used to segment the image. The selected of circular markers and ROI are shown in Figures 9 and 10. As the BTS detection is carried out outdoors, the camera is affected by many interference factors, and the images are often mixed with various noises, mainly Gaussian noise and impulse noise. In order to ensure the quality of the image, Gaussian noise and impulse noise are added to the image. The image is processed by Gaussian filtering and median filtering. The selection of the filter is determined by the processing effect. The noise addition and filtering effect are shown in Figure 11. It can be seen from the filtering effect that Gaussian filtering also blurs the image edge in the process of removing noise, which is not conducive to the feature extraction of markers. Compared with Gaussian filtering, median filtering achieves effective filtering of images on the premise of preserving edge information. Therefore, median filtering is selected for filtering.

Figure 9.

Tire concentric circular marker.

Figure 10.

Circular markers images captured by monocular camera.

Figure 11.

(a) Adding Gaussian noise, (b) adding pulse noise, (c) adding Gaussian noise three-dimensional graph, (d) adding impulse noise three-dimensional graph, (e) Gaussian filtering processing, and (f) median filtering processing.

Blob analysis segmentation template region

After image preprocessing, the Blob analysis algorithm is used to detect and segment the markers in the image to realize the identification of the marker area. The basic idea of Blob analysis is to filter the collected images to make the gray difference between the marker and the background image more obvious, and then extract the marker.²⁰ The operator of dyn_threshold in Halcon can perform dynamic threshold segmentation on the acquired image. In the operation, the “dark” mode is selected as the parameter, that is, the region where the gray value is lower than the offset gray value of the threshold image is selected for segmentation, and the expression is shown in equation (14). After processing the gray value images of the test area at different times from morning to night, it is determined that when the Offset value is 100, the segmentation effect is good.

f_{origImage} \leq f_{ThresholdImage} - Offset

(14)

The ring target is accurately segmented, but there are still some non-feature regions that interfere. Firstly, the segmented connected domain is opened, and the Halcon feature detection tool is used to combine with the ring feature of the marker. The processing results are shown in Figure 12(b).

Figure 12.

(a) Image connected domain and (b) ring feature selection.

Since there may be multiple ring features in the image after feature selection, select_obj operator is called to index the selected object, and the rightmost ring mark is extracted. The reduce_domain operator is called to cut out the corresponding region in the original color image,^21,22 so as to realize the segmentation process of identifying markers and regions.

Template matching algorithm

Template matching algorithm is to analyze the similarity of image gray and feature, and then find the method of similar regions in the image. Sobel operator has good detection effect, and has a smooth inhibitory effect on noise, so this paper selects Sobel operator to detect image edge. By obtaining the one-step degree of the image, the solution formulas of the edge images in the vertical and horizontal directions are shown in equation (15). Substituting the gradient value into equation (16), the gradient amplitude and direction of each pixel in the image can be obtained.

\begin{matrix} Gx = \sum \sum Sx (m, n) f (x - m, y - n) \\ Gy = \sum \sum Sy (m, n) f (x - m, y - n) \end{matrix}

(15)

\begin{matrix} G_{magnitude} = \sqrt{G x^{2} + G y^{2}} \\ G_{Direction} = inv \tan (Gy / Gx) \end{matrix}

(16)

The edge detection processing image is shown in Figure 13. Since there are many edge features after edge detection, it is necessary to carry out target detection on the image by NMS algorithm, so as to retain the maximum edge of the local area and remove unnecessary edge information. At this time, the image may have discontinuous edge features. The lag threshold is used to connect the disconnected edge lines to obtain accurate and concise edge features.

Figure 13.

Sobel operator edge detection chart.

After the template is created, the template is used to search the matching object in the source image. The shape-based template matching is used to characterize the similarity between the template and the matching area by similarity. The gradient of the template in the x and y directions is set as G_i^T(Gx_i^T, Gy_i^T), and the gradient of the matching image in the x and y directions is set as G_u,v^S(Gx_u,v^S, Gy_u,v^S), where u and v represent the number of rows and columns of the matching image, and the normalized sum of the product of the gradient points of the template and the matching image represents the similarity, as shown in equation (17).

\begin{matrix} S_{u, v} = \frac{1}{n} \sum_{i = 1}^{n} \\ \frac{({Gx}_{i}^{T} • {Gx}_{(u + Xi, v + Yi)}^{S}) + ({Gy}_{i}^{T} • {Gy}_{(u + Xi, v + Yi)}^{S})}{\sqrt{{Gx}_{i}^{T}^{2} + {Gy}_{i}^{T}^{2}} \sqrt{{Gx}_{(u + Xi, v + Yi)}^{T}^{2} + {Gy}_{(u + Xi, v + Yi)}^{T}^{2}}} \end{matrix}

(17)

When S_u,v = 1, there is a perfect matching between the template model and the search image. If S_u,v = 0, there is no similar object in the search image.

The position coordinates (Row, Column, Angle) obtained by template matching are transformed into rotation matrix R(θ) and translation matrix P. The sub-pixel contour of template image is transformed into the target area by affine transformation, and the matching effect is displayed. The affine transformation is used to transform the green sub-pixel contour of the concentric circle template to the matching area and the disp_cross operator is used to display the matching coordinate points. The effect image is shown in Figure 14.

Figure 14

Concentric circle marker recognition effect map..

Test and result analysis of BTS detection scheme

Verification test of slip rate solution scheme

In order to verify the actual effect of the slip rate solution scheme, in this section, the slip calibration device is used to verify the accuracy of slip rate obtained by monocular vision.²³ The device is shown in Figure 15. The slip rate verification device is composed of a disc rotation module and a platform translation module. The disc rotation module is driven by a stepper motor to rotate the disc around the rotating shaft. At the same time, the overall disc rotation module is placed on the mobile platform. The mobile platform is driven to do horizontal movement through the synchronous belt. The tire movement in the slip state can be simulated by adjusting the disc rotation speed and the platform movement speed.

Figure 15.

Structure diagram of slip rate verification device.

Due to the complexity of the simulation braking process, accelerated motion is used to simulate the reverse motion of the braking process, so that the disc and platform are accelerated from static to 10 km/h. The maximum moving distance of the mobile platform is 4 m. Combined with the velocity of the beginning and end of the movement, the acceleration of the mobile platform is 0.96 m/s² calculated by the displacement velocity formula of the uniform acceleration movement, and the slip rate is 0. When the acceleration of the disk rotation is less than 0.96 m/s², the product of the angular velocity of the disk and the radius of the disk is less than the horizontal velocity of the mobile platform, and the tire slip phenomenon is simulated. The acceleration of the mobile platform is set to 1 m/s², and the acceleration of the disk is set to 0.9 m/s². The slip rate value collected by the speed encoder and the slip rate value solved by the visual scheme are compared. The real-time slip rate is solved by visual scheme and slip rate verification device, and the curves of slip rate and error with time are obtained, as shown in Figure 16. The maximum error of the slip rate obtained by the test is 4.2%, which shows the rationality of the slip rate solution scheme. The scheme can be used for the brake timing detection of automobile trains.

Figure 16.

Acceleration to 10 km/h slip rate versus time curve.

The composition of BTS visual inspection system

The detection system hardware includes: Eight AI-230U150C high-speed industrial cameras, frame rate of 30 fps, 1920 × 1200 pixels, field of view angle of 84.8° × 68° × 98.4°, with 8 mm focal length lens. Two IPC-810 industrial control computers, EPE-1816 motherboard compatible with Intel 6th and 7th generation processors, using EPE-6111P6 motherboard, including ISA, PCI, and PCIe card slot. One pair of E3JK-5DM1-5L photoelectric sensor. The detection software system is designed based on Windows 7 operating system, and all functions are implemented in VC++. The camera calibration part is implemented by Matlab R2019b, and the image processing part is implemented by Halcon algorithm package.

Real vehicle test

Selected Jiefang brand J6P 550 6 × 4 three-axis heavy tank tractor-trailer-train as test vehicle. The tire specifications were 12R22.5. Test site road adhesion coefficient is 0.75, the test process is as follows: Camera positioning, leveling, debugging of BTS detection system, and posting of concentric circle markers. Detection site layout is shown in Figure 17. The train enters the detection area at a constant speed of 10 km/h. When the train passes through the opposite photoelectric switch to block the light path, the system identifies the train entering, starts the camera to collect the image, and distinguishes the tire markers of different axles. When the high frequency signal is the same as the number of registered axles of the database vehicle, the camera frame number i₀ is recorded, and the tire radius and the location radius of the marker are solved. The system determines the radius of each tire and the radius of the marker position, sends the braking command to the dot matrix screen, indicating the driver’s braking. Driver brakes and trigger pedal switch, computer record BTS detection initial time T_b. Calculation of BTS of the train by slip ratio.

Figure 17.

Hardware layout diagram of real vehicle test.

According to the above test process, T_i was recorded when the tire slip rate reached 20% on each axis of the test vehicle. The CFTOOL toolbox in Matlab2019b is used to obtain the slip rate-time curve by Gaussian fitting. As shown in Figure 18. The tire slip rate–time curves on the left and right sides of the vehicle are obtained from the marker coordinates collected by the camera, as shown in Figure 19.

Figure 18.

Fitting curve of slip ratio of right side of first axle of test vehicle.

Figure 19.

Fitting curve of left and right tire slip rate of test vehicle.

The time when the slip ratio of each tire reaches 20% can be read through the slip ratio curve. Five repeated tests are carried out on the test train. The time when the slip ratio of each tire reaches 20% under five tests is shown in Table 3.

Table 3.

Test data of vehicle BTS.

Axle 1		Axle 2		Axle 3		Axle 4		Axle 5		Axle 6
L	R	L	R	L	R	L	R	L	R	L	R
0.40	0.39	0.31	0.33	0.35	0.36	0.42	0.41	0.44	0.44	0.49	0.48
0.36	0.37	0.29	0.32	0.34	0.34	0.39	0.4	0.42	0.43	0.45	0.46
0.39	0.38	0.31	0.3	0.34	0.35	0.42	0.43	0.46	0.45	0.49	0.5
0.46	0.44	0.41	0.43	0.38	0.39	0.52	0.51	0.54	0.55	0.57	0.59
0.39	0.4	0.31	0.32	0.34	0.36	0.44	0.43	0.47	0.46	0.49	0.47

The average braking time of the axle was obtained by averaging the tire slip rates on both sides of the same axle, and the test results of the test train were analyzed. The BTS between test axles is 2-3-1-4-5-6. It can be seen that the braking time lag of the steering bridge is obvious. After analysis, it is found that the tractor-trailer-train often work in heavy load conditions, which makes the vertical force of the trailer ground larger. The tractor tires are often locked before the trailer’s, and the train loses the steering ability. The driver adjusts the braking force of the steering shaft to reduce this phenomenon.

Conclusion

In this paper, a solution scheme of tractor-trailer train slip rate based on cycloid formula is proposed. By collecting the lateral coordinates of tractor-trailer train tire markers and combining the tire rotation angle, the real-time tire slip rate is obtained to detect the BTS of the vehicle. The template matching algorithm was used to locate the markers, and the Blob analysis was used to obtain the location of the concentric circle markers, and the matching template was created. The tire markers were identified by the contour-based template matching algorithm. The slip rate verification test was established to verify the accuracy of slip rate solution. The real vehicle test of monocular vision tractor-trailer train BTS detection was completed. Compared with other results, this paper realizes the non-contact and dynamic intelligent vehicle BTS detection, and solves the defects of insufficient detection ability and low measurement accuracy of existing technologies.

Supplemental Material

sj-jpg-1-ade-10.1177_16878140211067045 – Supplemental material for Tractor-trailer-train braking time sequence detection based on monocular vision

Supplemental material, sj-jpg-1-ade-10.1177_16878140211067045 for Tractor-trailer-train braking time sequence detection based on monocular vision by Libin Zhang, Shiyuan Feng, Hongying Shan and Guanran Wang in Advances in Mechanical Engineering

Footnotes

Handling Editor: Chenhui Liang

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Shiyuan Feng

Supplemental material

Supplemental material for this article is available online.

References

Jing

Study on braking stability of commercial vehicles: an optimized air brake system. Adv Mech Eng 2019; 11: 10.

Elhemly

MAE

Fayed

MAG

. Simulation of tractor semitrailer manoeuvre at high speed using MATLAB/SIMULINK. Int J Heavy Veh Syst 2011; 18: 341–358.

Castillo Aguilar

Cabrera Carrillo

Guerra Fernández

, et al. Robust road condition detection system using in-vehicle standard sensors. Sensors 2015; 15: 32056–32078.

Goodarzi

Behmadi

Esmailzadeh

An optimised braking force distribution strategy for articulated vehicles. Veh Syst Dyn 2008; 46: 849–856.

Wang

Zhou

Yang

, et al. An experimental study on hysteresis characteristics of a pneumatic braking system for a multi-axle heavy vehicle in emergency braking situations. Appl Sci 2017; 7: 799.

Xie

Wei

FY.

Optimization of braking force distribution coefficient for car-trailer. 2nd international workshop on intelligent systems and applications. Wuhan: IEEE Computer Society; Harbin Institute of Technology; Huazhong University of Science and Technology; Hubei University of Technology; IEEE Harbin Section; Research Association of Modern Education and Computer Science, 2010.

Zhuang

QY.

Test of vehicle braking timing. Res Explor Lab 1991; 10: 98–99+106.

Ling

Research on detection method of braking sequence of truck’s air brake system. Changchun: Jilin University, 2015.

Xiao

ZM.

Detection system for flat brake table of automobile train. China Sci Technol Panor Mag 2019; 13: 55–56.

10.

Furferi

Governi

Volpe

, et al. Design and assessment of a machine vision system for automatic vehicle wheel alignment. Int J Adv Robot Syst 2013; 10: 242.

11.

Shan

Zhang

, et al. Dynamic detection method for non-contact vehicle wheelbase difference. J Central South Univ 2017; 48: 2959–2965.

12.

Liu

Cao

, et al. A real-time posture monitoring method for rail vehicle bodies based on machine vision. Veh Syst Dyn 2017; 55: 853–874.

13.

Creager

Johnson

Plant

, et al. Push–pull locomotion for vehicle extrication. J Terramechanics 2015; 57: 71–80.

14.

Johnson

Botha

Els

PS.

Real-time side-slip angle measurements using digital image correlation. J Terramechanics 2019; 81: 35–42.

15.

Performance and structure of inflatable tire. Guangzhou: South China University of Technology Press, 1998.

16.

All-steel radial tire finite element simulation analysis under the load condition. Qingdao: Qingdao University of Science and Technology, 2014.

17.

Wang

FY.

Advanced tire friction modeling and monitoring. In: Tham

Simpson

(eds.) Advanced motion control and sensing for intelligent vehicles. New York, NY: Springer, 2007, pp.153–162.

18.

Zhang

A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 2000; 22: 1330–1334.

19.

Madsen

Nielsen

Tingleff

Methods for non-linear least squares problems. Lyngby: Society for Industrial and Applied Mathematics, 2004.

20.

Gonzalez

Woods

Eddins

Digtal image processing using Matlab. Beijing: Tsinghua University Press, 2013.

21.

JY.

Research on voids detection technology in BGA solder balls based on X-ray image. Chengdu: University of Electronic Science and Technology of China, 2019.

22.

Liu

ZY.

Research on variety identification of rice seed using computer vision. Hangzhou: Zhejiang University, 2006.

23.

Zhang

Shan

, et al. Vehicle braking timing detection device and method based on stereo vision. CN105510052B. 2017.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.33 MB