A non-contact method for weighting of aggregate pile based on binocular vision

Abstract

Concrete is indispensable in contemporary industry, and the weighing of aggregates directly determines the quality of concrete preparation. In response to the difficulty of traditional methods in meeting the industrial requirements for accuracy, efficiency, and cost of measurement, this paper proposes a non-contact bone pile weight measurement method based on binocular vision. Firstly, the on-site image is obtained through a binocular camera, and the camera parameters are used to improve the image edge information and preprocess the image. Then, the improved semi global matching (SGM) algorithm is used to improve computational efficiency, obtaining three-dimensional information of the aggregate pile for volume calculation. Based on the watershed algorithm and T-S inference, the aggregate clearance rate is calculated, and the initial volume is corrected. Finally, a linear relationship is established between the density of the aggregate pile and the clearance rate, and the weight of the aggregate pile can be obtained by the density formula. The experimental results on different sizes of aggregate piles shows that the average error of the experiment is less than 4%. Efficiency of improved SGM matching algorithm has increased several times compared to the original algorithm, providing a low-cost and efficient measurement method for industrial aggregate weighing.

Keywords

Binocular vision semi global matching algorithm watershed algorithm T-S inference

Introduction

Concrete is one of the important materials for infrastructure construction, and the aggregate is one of the core components of concrete. During the preparation process of concrete, the accurate measurement of aggregate weight is essential for achieving the desired concrete properties and performance.¹ Improper weighing of aggregates can lead to variations in the mix proportions and result in inconsistencies in the concrete’s strength, workability, and durability, which directly impacts the concrete’s compressive strength, flexural strength, and the overall quality and properties. Therefore, precise weighing of aggregates is crucial in concrete mix design to produce a well-balanced and high-quality concrete mixture.

The traditional measurement methods are the contact measurements, such as direct measurement and sensor measurement.^2,3 At present, the common aggregate weighing instruments in the market have problems such as unstable drop values, large volume, high prices, and poor real-time performance.

The new measurement methods have emerged with the application of computer vision technology in image detection, such as laser diffraction method⁴ and image processing technology.^5,6 Qiu and Wang⁷ designed a system that uses non-contact optoelectronic methods to measure the volume of objects and uses laser to locate the center position. Pal et al.⁸ proposed an optical weight measurement system using Fiber Bragg Grating (FBG) sensors combined with machine learning, which greatly improves measurement speed and accuracy of the system. However, such methods have highly cost requirements, complex processes, and poor universality. Fauzi et al.⁹ conducted non-contact evaluation of the volume and weight of the target object using width and height based on captured 2D images. Some scholars^10,11 used tire image recognition markers to obtain tire deformation parameters and achieve dynamic weighing of vehicles. The non-contact methods to detect target objects without the need to install sensors for direct measurement, and it is easy to overcome many shortcomings of traditional methods.

In recent years, obtaining 3D information of target images based on binocular vision for 3D reconstruction and subsequent processing has become popular in the field of computer vision. Chen et al.¹² designed a new measurement method based on binocular vision, which achieved accurate 3D reconstruction and size measurement of rectangular workpieces. Liu et al.¹³ proposed a method to detect feed volume based on binocular stereo vision. Some scholars^14–16 combined binocular vision and Semi-global stereo matching (SGM) algorithm to estimate the weight of industrial coal and aggregate piles, which avoids excessive error caused by tedious human operations. In addition to its application in industrial products, it is also widely used in volume detection of agricultural products.^17–19 Table 1 provides a comparison of different algorithms of volume calculation.

Table 1.

Comparison of different algorithms of volume calculation.

Algorithms	Information	Findings	Cons
Cheng et al.²⁰	Proposes a method for volume calculation by an improved α-Shape algorithm.	High accuracy and good flexibility.	High computational complexity and high memory consumption.
Zheng et al.²¹	Provides a method based on improved point cloud slicing.	Diverse data sources and suitable for irregular shapes.	High requirement for data quality and Accuracy is severely affected by data density.
SGM	A widely used stereo matching algorithm in the field of computer vision.	It has high accuracy and robustness, and is suitable for environments with varying scene depths.	Poor robustness to complex situations such as noise and occlusion.

In response to the many shortcomings of traditional measurement methods and the current research on volume measurement based on binocular vision, the research on weight measurement is almost blank. Considering the huge advantages of machine vision in measurement, this article used binocular vision to study the weight measurement of aggregate piles, which intended to focus on improving efficiency and reducing costs. The contribution of this article is reflected in the following aspects:

(1) A volume measurement method based on SGM is proposed. The improved Census transform has greatly improved the computational speed of high-resolution images.

(2) Proposing a volume correction method. The initial volume is further corrected by the clearance rate, which is obtained by combining watershed algorithm and T-S inference.

(3) Introducing a simple and efficient method for measuring the weight of aggregate pile. The weight of aggregate pile can be calculated by a linear relationship between the porosity and aggregate density. It reduces the errors caused by complex transformations and makes the experimental results more convincing.

The remaining organizational arrangements of this article are as follows:

The second section mainly introduces the relevant work of this article. The third section specifically describes the new method proposed in this article and the specific steps of the method. The fourth section verifies the authenticity and effectiveness of the methods and assumptions proposed in this article through experiments. The research conclusion is presented in Section 5.

Binocular camera

Binocular camera model

The origin of binocular vision technology is related to the human visual system, which utilizes the disparity between left and right images and the principle of similar triangles to obtain three-dimensional information of the target object for analysis and utilization.²²

Figure 1 is a schematic diagram of binocular camera imaging. $f$ is the focal length, optical axis 1 is parallel to optical axis 2, B is the distance between the projection centers of two cameras, $z$ is the depth distance, $d$ is the disparity, pixel size is the size of the pixel.²³ Among them, points P₁ and P₂ are points in space, P represents the image point on the imaging surface of the left and right cameras, and $X_{1}$ and $X_{2}$ denote the distance from the left and right image points to the boundary of the camera imaging surface. Based on the geometric relationship, the triangle PP₁P₂ is similar to the triangle PO₁O₂, and the following relationship can be obtained:

\begin{matrix} X_{left} = f \frac{x_{c}}{z_{c}} \\ X_{right} = f \frac{(x_{c} - B)}{z_{c}} \\ Y = f \frac{y_{c}}{z_{c}} \end{matrix}

(1)

Figure 1.

Binocular camera imaging.

The disparity of point P on the left and right cameras is:

d = X_{left} - Y_{right}

(2)

Thus, the coordinates of the feature point $P$ in the three-dimensional coordinates can be calculated as follows.

\begin{matrix} x_{c} = \frac{B * X_{left}}{disparity} \\ y_{c} = \frac{B * Y}{disparity} \\ z_{c} = \frac{B \times f}{disparity} \end{matrix}

(3)

The distance $z$ from point $P$ to the camera is:

z = \frac{Bf}{d * pixelsize}

(4)

From the equation (4), it can be seen that depth distance is inversely proportional to parallax, and is proportional to focal distance. The closer the object is from the camera, the greater the parallax is, and vice versa. Therefore, the distance of the camera is proportional to the focal distance $f$ of the camera $B$ from the center of the camera.

Camera calibration

Calibration is crucial for 3D imaging, as its purpose is to determine the internal and external parameters of the camera. This article adopts the Zhang zhengyou’s calibration method,²⁴ and the object of reference with calibration is checkboard, with a single grid size of 26 mm × 26 mm, in total of 7 × 10 grids. The collected calibration board images are shown in Figure 2.

Figure 2.

Diagram of calibration board.

External parameters include rotation matrix $T$ and translation vector $R$ , internal parameters include focal length $f$ , radial lens distortion $k_{1}$ , $k_{2}$ , tangential distortion $P_{1}$ , $P_{2}$ , and principal coordinates (c_x, c_y).

Process and results of camera calibration:

(1) Using MATLAB to calibrate the camera. Selecting the tool of image calibration in the app’s column of MATLAB and adding 20 calibration board images with different angles as shown in Figure 3, and MATLAB will automatically remove some unqualified images.

(2) Using MATLAB to perform corner extraction on input images. The coordinate unit in the image is 0.1 mm, and the corner extraction process is shown in Figure 4.

(3) The input size of the block is 26 mm. As shown in Figure 5, after extracting the corner points, the reprojection error and the binocular camera model are calculated by MATLAB.

(4) Using MATLAB to calculate and display data of internal and external camera parameters. The parameters of the binocular camera can be obtained as shown in Tables 2 and 3 after repeated calculations and removing edge points.

Figure 3.

Add the image.

Figure 4.

Corner selection order and Corners display.

Figure 5.

The result of calculation: (a) reprojection error and (b) binocular camera model.

Table 2.

The parameters of binocular camera.

variable	Left camera	Right camera
f_x	1980.2	1977.5
f_y	1986.6	1976.3
c_x	1042.2	1151.6
c_y	845.1	793.9
k₁	−0.1648	0.2239
k₂	−0.0731	0.2273
P₁	−0.0048	−0.0014
P₂	−0.0006	0.0003

Table 3.

Rotating vector R and translation vector T.

Parameters	Values
R₁	−0.0028
R₂	−0.1273
R₃	−0.0753
T₁	368.2
T₂	18.0534
T₃	38.2013

All parameters of the two cameras are basically the same. The correction coefficients and error ranges of the four types of distortion, as well as the correction results, indicate the existence of a certain camera angle. The experimental results indicates that the distortion coefficient is small, and the visible and tangential distortions are not significant.

Proposed methods

The flow chart is as shown in Figure 6:

(1) Image preprocessing. The binocular camera is used to collect the image After the camera is calibrated with Zhang Zhengyou’s calibration method. In view of the noise in the image, Gaussian filter is used to increase the image clarity and Laplace operator is used to detect the edge.

(2) Initial volume calculation. The improved SGM matching algorithm is used to obtain the disparity map, and the three-dimensional information of the obtained image is used as input information, and the initial volume is calculated using the differential element method.

(3) Volume correction and weight calculation. Use the improved PSO-GA algorithm and watershed algorithm to calculate the porosity and correct the initial volume, and establish the relationship between the internal and external gaps and density to obtain the corresponding weight of the aggregate.

Figure 6.

Flow chart of aggregate pile weight measurement method.

Image processing

Most useful information is obtained from the initial image. However, the noise is inevitable during the process of image acquisition. In order to increasing the reliability of the experiment and minimizing the interference of noise on the image, this paper selects Gaussian filter to process it. The filtering effect diagram is shown in Figure 7.

Figure 7.

Gaussian filter: (a) original image, (b) add salt and pepper noise with a noise density of 0.1, and (c) result image.

As shown in the figure, adding the salt and pepper noise with density of 0.1 to the original image. Gaussian filter can effectively remove the noise, and then using the Laplace operator to detect the edge of the image. For image f(x, y), the Laplace operator can be defined as:

L (f (x, y)) = \frac{\partial^{2} f (x, y)}{\partial x^{2}} + \frac{\partial^{2} f (x, y)}{\partial y^{2}}

(5)

Firstly, extracting the RGB components from the target image and performing edge detection on each RGB component, and then synthesizing the above three scenarios. The effect of Laplace operator is shown in Figure 8.

Figure 8.

The flow chart of Laplace operator.

Initial volume calculation based on improved SGM

The traditional SMG algorithm has the following four steps: cost calculation, cost aggregation, disparity calculation, and disparity optimization.

(1) Matching Cost calculation. The cost calculation is based on the grayscale difference between the target pixel and its neighboring pixels, which is converted into a bit string through the census transform. The main idea is to calculate the matching cost by taking the Hamming distance between the bit strings as the input of the transform. Figure 9 illustrates the principle of census transform.

Figure 9.

Schematic diagram of census transform.

The expression for calculating the Hamming distance is as follows:

φ (I (p), I (q)) = {\begin{matrix} 1, I (p) > I (q) \\ 0, I (p) < I (q) \end{matrix}

(6)

Where I(p) represents the value of the window center point; I(q) represents the value of each point in the set except for the center point.

(2) Cost aggregation. The concept of the SGM algorithm is to transform the process of solving two-dimensional (2D) energy minimization into equal cost aggregation on multiple one-dimensional (1D) paths in multiple directions. The basic flow as shown in Figure 10:

Figure 10.

The schematic diagram of SGM algorithm.

The cost aggregation calculation formula is as follows:

\begin{matrix} L_{r}^{'} (p, d) = C (p, d) + \min {\begin{matrix} L_{r} (p - r, d) \\ L_{r}^{'} (p - r, d - 1) + P_{1} \\ L_{r}^{'} (p - r, d + 1) + P_{1} \\ \min_{i} L_{r}^{'} (p - r, i) + P_{2} \end{matrix}} \\ - \min L_{r} (p - r, i) \end{matrix}

(7)

where $L_{r}$ represents the aggregated cost value on path r, $p - r$ represents the last pixel of pixel p on path r, and $\min_{i} L_{r} (p - r, t)$ represents the minimum cost value of pixel $p - r$ under all disparity. The penalty coefficients $P_{1}$ and $P_{2}$ are determined by the color difference of the pixels.

(3) Disparity calculation. In the SGM algorithm, disparity calculation adopts the same winner takes all algorithm strategy as the local algorithm, and the final disparity is determined by the minimum aggregate value selected from all pixel values. The calculation formula is as follows:

d^{*} = argminS (p, d)

(8)

(4) Disparity optimization. This article adopts the left and right consistency method to eliminate incorrect matches. The obtained disparity map is shown in Figure 11:

Figure 11.

Disparity map: (a) left view, (b) disparity map, and (c) right view.

In order to quickly measure the volume of aggregate piles, this article proposes an improved SGM algorithm with hierarchical strategy which establishes P image layers with different resolutions for the original images and each layer has a resolution of one quarter of the upper layer, and a parallel optimization algorithm is adopted to address the efficiency issue of cost aggregation in multi-path aggregation process. By enhancing the existing SGM algorithm, the proposed algorithm enables real-time pixel-by-pixel stereo matching of images in a fraction of the time taken by traditional methods. The flowchart of the improved SGM is shown in Figure 12.

Figure 12.

The flow chart of the improved SGM algorithm.

The algorithm can be classified into three categories based on the level of optimization achieved. The first is the unoptimized version of the SGM algorithm, followed by the parallel optimization P-SGM algorithm. Finally, the PH-SGM optimization algorithm is developed, which includes both parallel optimization and hierarchical strategy to further improve the performance. The experimental objects are image pairs obtained from a binocular camera with a resolution of 1000 × 800, and the search range of parallax are set to 128, 256, and 512. Regarding the parameter settings for this process, specifically the SGM algorithm, there are 8 cost aggregation paths, 5 matching layers, and a window size of $5 \times 5$ for the Census transform. The experimental results as shown in Table 4.

Table 4.

Three algorithms consume time.

Algorithms	D1 = 256	D2 = 512	D3 = 1024
SGM	7.84 s	15.23 s	40.06 s
P-SGM	1.67 s	2.68 s	5.97s
PH-SGM	0.48 s	0.51 s	0.53 s

From Table 4, it can be seen that the SGM+P algorithm with parallel optimization has a 4–6 times improvement in efficiency compared to the unoptimized SGM algorithm, but it did not meet expectations. The SGM+P+H algorithm, which adopts parallel optimization and hierarchical matching strategies, has an efficiency improvement of 3–11 times compared to the SGM+P algorithm that only uses parallel optimization.

SGM features can be extracted from the left and right views of aggregate pile. Then, by comparing these features, point pairs that correspond to the same object in the left and right images can be identified, enabling the calculation of their three-dimensional coordinates.²⁵ The distance Z from a specific point to the camera plane can be computed using equation (4). By iterating through every pixel on the disparity map in a sequential order, the Z coordinates of each pixel in the global coordinate system can be determined, followed by the computation of complete three-dimensional coordinates for all pixels. As shown in Figure 13, importing coordinate data into MATLAB software can obtain a depth map of the aggregate pile.

Figure 13.

Depth map.

In this process, a binocular camera is used to capture depth information from the top of the aggregate pile by generating a depth map. As illustrated in Figure 13, the total volume of the pile is calculated by finding the cumulative sum of the volume of corresponding blocks per unit area. Furthermore, the average depth of each pixel in the pile corresponds to the overall average depth of the aggregate.

The three-dimensional coordinates of the two-dimensional point can be calculated by the disparity d corresponding to the coordinates (x, y) of a point in the disparity map.

Q = [\begin{matrix} 1 & 0 & 0 & - c_{x} \\ 0 & 1 & 0 & - c_{y} \\ 0 & 0 & 0 & f \\ 0 & 0 & - \frac{1}{T_{x}} & \frac{cx - c_{x}^{'}}{T_{x}} \end{matrix}]

(9)

Where, c_x and c_y are the abscissa of the principal point of the left image, x_c is the horizontal coordinate of the center point f of the right image, and T_x is a parameter in the translation vector T that represents the offset in the horizontal direction.

The method of 3D scene reconstruction is an approach to estimate initial volume. An irregular quadrilateral prism can be constructed by using 4 pixels in the image coordinates to obtain eight 3D points. The shape of the irregular quadrilateral is shown in Figure 14.

Figure 14.

Initial volume calculation.

The center point P can be obtained by calculating the center points at the top and bottom of an irregular hexahedron:

\begin{matrix} {(X, Y, Z)}_{p} = \frac{{(X, Y, Z)}_{i} + {(X, Y, Z)}_{r}}{2} {(X, Y, Z)}_{k} \\ = \frac{(X_{d 1}, Y_{d 1}, Z_{d 1}) + (X_{d 2}, Y_{d 2}, Z_{d 2})}{4} \\ + \frac{(X_{d 3}, Y_{d 3}, Z_{d 3}) + (X_{d 4}, Y_{d 4}, Z_{d 4})}{4} k \\ = top, bottom; d = i, r \end{matrix}

(10)

The volume of a single irregular quadrilateral can be calculated by dividing it into 12 irregular tetrahedrons: P-ABC, P-ADC, and P-ABE…, and then summing the volumes of each tetrahedron, and then the initial volume of the aggregate is calculated by summing the volumes of each irregular quadrilateral. Each irregular tetrahedron’s volume can be calculated using the Euler formula in the equation (11):

V = \frac{1}{6} | \begin{matrix} 1 & 1 & 1 & 1 \\ x_{1} & x_{2} & x_{3} & x_{4} \\ y_{1} & y_{2} & y_{3} & y_{4} \\ z_{1} & z_{2} & z_{3} & z_{4} \end{matrix} |

(11)

Volume correction

This section uses an improved genetic algorithm to segment the target image, and calculates the perimeter and area of the aggregate based on the watershed algorithm. Then, combined with T-S inference, the porosity of the aggregate pile is calculated to correct the volume.

Previous work^26–28 has demonstrated that genetic algorithm (GA) has excellent performance in the field of image segmentation. The workflow of learning frequent contour segments (FCS) using GA is shown in Figure 15.

Figure 15.

Genetic algorithm (GA) learns the workflow of FCS.

This article proposes a new hybrid PSO-GA algorithm that incorporates the steps of splitting and merging into the original iterative algorithm,^26,29 avoiding the ineffective evolution process of small genes while selecting suitable genes for effective contour recognition.

This section uses the MPEG-7 core experiment, the CE-Shape-1 dataset,^30,31 as the test data set. Figure 16 shows FCS obtained from the training images using proposed method.

Figure 16.

Examples of positive detection using FCS learned from training images.

In this study, the intersection-union ratio (IOU) is established as 0.4 in order to assess the coverage of the results. Compared to previous works,³² the coverage results are evaluated and compared accordingly.

The coverage is the percentage of ground-truth boundary points recovered by this method, and the accuracy is the percentage of output points located on the ground-truth boundary.¹⁶ As shown in Table 5, while the current method’s coverage results may be slightly lower than those of previous studies, its accuracy results demonstrate a significant improvement than previous reports. The findings indicate that the current method enables accurate identification of object boundaries in test images, while excluding the boundary box.

Table 5.

Coverage results.

	Aggregate pile	Debris	Mound	Big stone pile
Our method	82.4/90.5	71.9/82.5	73.8/84.2	70.2/79.4
	Mug	Swan	Bottle	apple
Ground-truth	83.2/75.7	78.4/77.0	82.7/84.2	91.3/93.6
Ferrari et al.³²	55.1/62.3	36.8/39.3	71.2/67.7	42.5/40.8

Figure 16 compares route optimality and convergence speed of the different algorithm.

The path optimality of all proposed algorithms is compared and analyzed in Figure 17. The results illustrate that the proposed hybrid PSO-GA algorithm significantly outperforms the other two algorithms in terms of routing success rate. This observation is supported by the clear difference in the success rates of the algorithms shown in the figure. Notably, the proposed hybrid algorithm achieves a routing success rate of almost 98% with 25 nodes, which is considerably higher than that of the other two algorithms. As the network size increases, the success rate slightly decreases, but it still outperforms the other two algorithms, with the hybrid PSO-GA algorithm exhibiting better performance than PSO and GA algorithms by 6% and 3%, respectively, in an ultra-dense network of 150 nodes. All experiments are conducted by keeping the mutation probability parameter at 0.05.

Figure 17.

Comparison of three algorithms: (a) comparison of path optimality for each algorithm and (b) results of convergence speed of each algorithm.

In the process of image segmentation, it is difficult to effectively distinguish local pixel values due to interconnectivity and illumination. Therefore, when using the improved PSO-GA algorithm for image segmentation, and this article proposes a watershed algorithm^33,34 to calculate the clearance rate of aggregates. Due to the difficulty in visually calculating the internal gap of the aggregate pile, and this article estimates the internal clearance rate of the aggregate by calculating the clearance rate of two-dimensional planar aggregates.

Based on the watershed algorithm, the area S₁ and perimeter l of each aggregate can be calculated from the segmented image to obtain a clearance rate $λ$ :

R_{i} : if s = A and l = B λ_{i} = a_{0}^{i} + a_{1}^{i} s + a_{2}^{i} l i = 1, 2, \dots, m

(12)

Where, M is the T-S theorem number for aggregate piles. $λ_{i}$ is the clearance rate. A is the set of S and B is the set of $l$ . a₀, a₁, a₂ are the unknown variables of the equation. The clearance rate $λ_{j}$ corresponding to the jth type of aggregate block is calculated by the weighted average method:

λ_{j} = \frac{\sum_{i = 1}^{m} μ_{Ai} μ_{Bi} λ_{i}}{\sum_{i = 1}^{m} μ_{Ai} μ_{Bi}} j = 1, 2, \dots, n,

(13)

Among them, $λ_{i}$ is determined by rule i. μ_Ai is the membership function of S. μ_Bi is the membership function of L.

The flowchart is shown in Figure 18, from which it can be seen that the improved watershed algorithm performs well in image segmentation of the aggregate pile surface.

Figure 18.

Aggregate pile surface segmentation treatment.

The initial calculation of volume ignores the existence of clearance rate in the aggregate pile, and the errors are inevitable. Therefore, the T-S inference³⁵ is used to correct the volume of the aggregate. The area and perimeter of the aggregate are used as input for inference to obtain the clearance rate, and is used as a correction factor.

\bar{V} = \bar{λ} V \bar{λ} = \frac{1}{n} \sum_{j}^{n} λ_{j}

(14)

The clearance rate of the aggregate can be obtained from equation (4), and then the volume of the aggregate can be corrected according to equation (14).

Weight calculation

This section establishes a database of the clearance rate corresponding to the density of different aggregate piles. Within a controllable error range, which confirms that there is a certain relationship between the clearance rate and density of the aggregate. The relationship is shown in Figure 19:

Figure 19.

The linear relationship between aggregate internal clearance and density.

As shown in Figure 19, there is a linear relationship between the internal clearance rate and density. If the density is set to y and the clearance rate is set to x, their relationship can be approximatively expressed by the function y = 9.091x-2.27, with 0.3589 < x < 0.4767.

The weight of the aggregate pile can be obtained by combining equation (14) and the relationship between density and clearance rate.

Experimental results

Verification of the feasibility of the clearance rate

This section confirms that the relationship between the internal and external clearance rate of the aggregate pile. Firstly, putting the aggregate pile into a bag and soaking the aggregate using the water intrusion method, and then measuring the volume of water and aggregate to obtain the internal porosity of the aggregate. However, due to the internal pressure of the actual aggregate pile is too high, so a large amount of aggregate should be loaded as much as possible during loading, and the heavy objects should be placed and compacted to simulate the high-pressure environment inside the aggregate pile. The weight of the heavy object should be 79.1 kg. The same method is used to calculate the external clearance rate, but the step of placing heavy objects is omitted. A total of 16 groups of aggregates with different densities were divided, and the corresponding clearance rate was measured 20 times for each group by replacing aggregates with different shapes, and the average value was taken as the final result.

The specific data is shown in Table 6.

Table 6.

Measurement parameters of aggregate heap.

Aggregate category	Internal clearance rate (%)	External clearance rate (%)	Error (%)	Density (g/m³)
Only large aggregates	47.67	49.24	1.57	1.99
Only small aggregates	35.89	41.12	5.22	2.97
Only medium aggregates	44.53	47.22	2.69	2.01
Large and small aggregates 1:1	43.61	44.11	0.50	2.11
Large and small aggregates 1:2	39.12	40.01	0.89	2.61
Large and small aggregates 2:1	39.28	39.98	0.70	2.58
Large and medium aggregates 1:2	46.24	48.41	2.17	2.16
Large and medium aggregates 1:1	46.26	47.10	0.84	2.04
Large and medium aggregates 2:1	46.80	48.58	1.78	2.10
Medium and small aggregates 1:1	41.65	41.88	0.23	2.55
Medium and small aggregates 2:1	40.12	41.36	1.24	2.64
Medium and small aggregates 1:2	41.54	41.74	0.20	2.57
Large, medium, and small aggregates 1:1:1	43.21	44.80	1.59	2.28
Large, medium, and small aggregates 2:1:1	42.18	43.12	0.92	2.18
Large, medium, and small aggregates 1:2:1	43.56	44.16	0.60	2.31
Large, medium, and small aggregates 1:1:2	44.78	45.22	0.42	2.06

As shown in Table 6, the aggregates are divided into different categories based on their size. The final data is obtained by measuring each group multiple times, then removing the extreme values and taking the average value. From the Table 6, it can be seen that the internal clearance rate is roughly the same value with external clearance rate. Among them, the pure small aggregates having the largest difference in clearance rate, with the error of 5.22%. This indicates that under high-pressure conditions, the clearance rate of small aggregates will become lower. In addition, the error in clearance rate of mixed aggregates is generally lower than 2.17%. Therefore, it is feasible to estimate the internal clearance rate of the aggregate pile based on the external clearance rate of the aggregate pile.

Experiment on weighing aggregates

This paper mainly uses a USB3.0 interface binocular camera as shown in Figure 20, with an output frame rate of 30 per second and a focal distance of 3.1 mm, as well as an AMD ^® The experimental platform consists of a R5 7520U laptop and a device that can be used to fix a binocular camera. The output frame rate of the camera is 30 frames per second, with a focal distance of 3.1 mm. The adjustable range of center distance between the two cameras is 3.5–16.5 cm. In this article, a center distance of 3.5 cm is used. Using software AMcap for image collection under Windows system, and using MATLAB R2017A for image processing and subsequent volume calculation. The basic camera parameters are shown in Table 7:

Figure 20.

Binocular camera.

Table 7.

The parameters of camera.

Resolution and frame rate of binocular cameras	2560*720@30fps
Single pixel size	$3 μ m \times 3 μ m$
Data interface	USB3.0
Camera center distance	3.5–16.5 cm
Photosensitive spectral range	380–1060 nm
Applicable platforms	Windows
Focal distance	3.1 mm aberration rate ≤0.8%

Using the method presented in this article to capture initial images using a binocular camera and calculate the weight of six different sized aggregate piles in Figure 21. The weight measurement results of the aggregate pile are shown in Table 8. The results of the further comparative experiments are shown in Table 9.

Figure 21.

Binocular view of aggregate piles of different sizes.

Table 8.

Weight measurement results and errors.

Order number	Actual weight/g	Measured weight/g	Error/%
(1)	2514.0	2613.0	3.94
(2)	2129.0	2075.2	2.53
(3)	1818.5	1740.1	4.31
(4)	1686.5	1643.6	2.54
(5)	1343.0	1374.0	2.31
(6)	1101.0	1135.2	3.11

Table 9.

Experimental results of large-sized aggregate piles.

Actual weight/kg	Results of pre-improved SGM	$E r r o r / % / T i m e / s$	Results of proposed method	$E r r o r / % / T i m e / s$
5.82	6.12	5.14/6.67	6.01	3.26/1.02
8.73	8.43	3.44/8.12	8.51	2.52/1.13
12.86	12.47	3.13/9.87	12.58	2.18/0.97
16.45	15.84	3.71/11.13	16.03	2.54/1.28

According to the weight data of the aggregate pile measured in Table 7, the average error can be calculated to be approximately 3.12%, and can be found that the maximum error is 4.31%. Although the SGM algorithm still has some flaws in matching accuracy, the main source of error for aggregate piles is that the surface colors of the pile are too similar, which makes the watershed algorithm difficult to segment and the accuracy of clearance rate measurement is low. There is a significant error in the relationship between aggregate density and clearance rate given. Table 8 demonstrates that when the measured weight of the aggregate pile reaches a large value, the method proposed in this paper is far superior to the pre-improved SGM algorithm in terms of both error control and efficiency, and the error value will not increase with the size of aggregate pile.

In industrial production, it has always been difficult to accurately measure the weight of aggregate piles through direct measurement. This article uses non-contact method to measure the weight of aggregate piles, and the results indicated that the proposed method can effectively measure the weight of aggregate piles, although there are certain errors, it also has certain reference value.

Conclusion

This article proposes a non-contact method for measuring the weight of aggregate piles based on binocular vision. Firstly, an improved SGM algorithm is used to calculate the initial volume of the aggregate pile, compared to the original SGM algorithm, it has huge advantages when facing high-resolution images. Furthermore, because of the special stacking space of the aggregate pile, watershed algorithm, improved PSO-GA method, and T-S inference are combined to calculate the clearance rate between the aggregates. The initial volume is corrected to reduce system errors, and a linear relationship between density and clearance rate is established to obtain the weight of the aggregate pile. The experimental results proved the feasibility of this scheme. The method in this article can quickly measure the weight of the target aggregate pile. The non-contact weighing method proposed in this article has advantages over previous contact weighing methods in terms of low cost, fast measurement, simple deployment, and applicability in narrow real-time scenarios.

There are still some remaining issues worth investigating in future work. This includes how to improve the accuracy of stereo matching, as well as combine image processing technology to enhance the applicability of the system for different environments and establish a more complete weight measurement system.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Independent Research Fund of the State Key Laboratory of Mining Response and Disaster Prevention and Control in Deep Coal Mines (No. SKLMRDPC20ZZ06) and the program in the Youth Elite Support Plan in Universities of Anhui Province (No. gxyq2020013) and Innovation fund of Anhui University of Science and Technology (2021CX2054).

ORCID iD

Ziqiang Li

Data availability statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

Jang

Kim

Sicakova

. Effect of aggregate size on recycled aggregate concrete under equivalent mortar volume mix design. Appl Sci 2021; 11: 11288.

Erdil

Ayrac

Akcan

, et al. Piezoresistive disposable weight sensor with increased sensitivity. Turk J Electr Eng Comput Sci 2020; 28: 984–998.

Oskoui

Taylor

Ansari

. Method and sensor for monitoring weight of trucks in motion based on bridge girder end rotations. Struct Infrastruct Eng 2020; 16: 481–494.

Gautam

Kumar

Thangaraj

, et al. Optical weight measurement system using FBG based D-IM edge filter detection. Opt Fiber Technol 2021; 61: 102386.

Wang

Pan

Rao

, et al. An image-assisted rod-platform weighing system for weight information sampling of broilers. Trans ASABE 2018; 61: 631–640.

Qiao

, et al. Automatic weight measurement of pigs based on 3D images and regression network. Comput Electron Agric 2021; 187: 106299.

Qiu

Z-W

Wang

. Non-contact measurement of complex surface object volume based on image processing of double image field. Chengdu, China: SPIE, 2019.

Pal

Kumar

Gautam

, et al. FBG based optical weight measurement system and its performance enhancement using machine learning. IEEE Sens J 2022; 22: 4113–4121.

Fauzi

AIM

Bin Ibrahim

Bin Saealal

. Estimation of volume and weight of apple by using 2D contactless computer vision measuring method. In: 2022 International conference on electrical, computer, communications and mechatronics engineering, ICECCME 2022, Male, Maldives, 16–18 November 2022. New York: Institute of Electrical and Electronics Engineers Inc.

10.

Feng

Leung

. Application of computer vision for estimation of moving vehicle weight. IEEE Sens J 2021; 21: 11588–11597.

11.

Kong

Zhang

Wang

, et al. Non-contact vehicle weighing method based on tire deformation using image recognition. Zhongguo Gonglu Xuebao/China J Highw Transp 2022; 35: 186–193.

12.

Chen

Zhong

Han

, et al. Binocular visual dimension measurement method for rectangular workpiece with a precise stereoscopic matching algorithm. Meas Sci Technol 2023; 34: 035010–035023.

13.

Liu

Dai

Zeng

, et al. Detection of bulk feed volume based on binocular stereo vision. Sci Rep 2022; 12: 9318–9334.

14.

Wang

Dai

Zhang

, et al. Coal weight measurement method of belt conveyor based on binocular stereo vision. In: 7th International conference on information, cybernetics, and computational social systems, ICCSS 2020, Guangzhou, China, 13–15 November 2020, pp.486–492. New York: Institute of Electrical and Electronics Engineers Inc.

15.

Dong

Song

. Dynamic coal quantity measurement method based on binocular vision. Meitan Kexue Jishu/Coal Sci Technol (Peking) 2022; 50: 196–203.

16.

Wang

, et al. Research on weighing of concrete aggregate pile based on binocular vision. Informatica 2022; 46: 1–12.

17.

Meyer

Eifert

Wang

, et al. Volume estimation of strawberries, mushrooms, and tomatoes with a machine vision system. Int J Food Prop 2018; 21: 1867–1874.

18.

Zhang

Guo

Huang

, et al. Computer Vision estimation of the volume and weight of apples by using 3D reconstruction and noncontact measuring methods. J Sens 2020; 2020: 1–12.

19.

Sun

Liu

. A novel tomato volume measurement method based on machine vision. Tehnicki Vjesn 2021; 28: 1674–1680.

20.

Cheng

Wang

Yang

, et al. Calculation method of 3D point cloud canopy volume based on improved α-shape algorithm. Nongye Jixie Xuebao/Trans Chin Soc Agric Mach 2021; 52: 175–183.

21.

Zheng

Yang

, et al. Research on volume calculation method based on improved point cloud slicing. In: 2nd International conference on sensors and information technology, ICSI 2022, 21–23 January 2022, Sanya, China. Academic Exchange Information Center (AEIC). SPIE.

22.

Zhang

Hao

Wang

, et al. Safety warning of mine conveyor belt based on binocular vision. Sustainability 2022; 14: 13276.

23.

Sun

Xia

, et al. Sample balancing of curves for lens distortion modeling and decoupled camera calibration. Opt Commun 2023; 537: 129221–129230.

24.

Zhang

. A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 2000; 22: 1330–1334.

25.

Liu

Zheng

. Accurate volume calculation driven by Delaunay triangulation for coal measurement. Sci Program 2021; 2021: 661326.

26.

Abdel-Gawad

Said

Radwan

. Optimized edge detection technique for brain tumor detection in MR images. IEEE Access 2020; 8: 136243–136259.

27.

Lei

Lin

. A Novel Dynamic fingerprint segmentation method based on fuzzy C-means and genetic algorithm. IEEE Access 2020; 8: 132694–132702.

28.

Shen

. Implementation of CT image segmentation based on an image segmentation algorithm. Appl Bionics Biomech 2022; 2022: 1–11.

29.

Wei

Tang

. A genetic-algorithm-based explicit description of object contour and its ability to facilitate recognition. IEEE Trans Cybern 2015; 45: 2558–2571.

30.

Latecki

Lakaemper

Eckhardt

. Shape descriptors for non-rigid shapes with a single closed contour. In: IEEE Conference on computer vision and pattern recognition, CVPR 2000, Hilton Head Island, SC, 13–15 June 2000, pp.424–429. New York: IEEE.

31.

El Rube

Alajlan

Kamel

, et al. Robust multiscale triangle-area representation for 2D shapes. In: IEEE International conference on image processing 2005, ICIP 2005, Genova, Italy, 11–14 September 2005, 2005, pp.545–548. New York: Institute of Electrical and Electronics Engineers Computer Society.

32.

Ferrari

Jurie

Schmid

. From images to shape models for object detection. Int J Comput Vis 2010; 87: 284–303.

33.

Haag

Schwartz

Shakibajahromi

, et al. A fast algorithm to delineate watershed boundaries for simple geometries. Environ Model Softw 2020; 134: 104842.

34.

Indriyani

Utoyo

Rulaningtyas

. A new watershed algorithm for pothole image segmentation. Stud Inform Control 2021; 30: 131–139.

35.

Huang

Dong

. An adaptive secure control scheme for T–S fuzzy systems against simultaneous stealthy sensor and actuator attacks. IEEE Trans Fuzzy Syst 2021; 29: 1978–1991.