Sage Journals: Discover world-class research

Abstract

Defect detection plays a crucial role in the manufacturing industry, ensuring the quality of industrial products. Despite advancements in this field, current defect detection methods face two primary challenges: (1) extracting visually similar features from the background poses difficult and (2) these methods struggle to identify tiny defects in the target objects. We present a feature enhancement module called the SE-CAR, which aims to handle the identified problems effectively. This module is designed to efficiently capture tiny defects in images, prioritize defect information features, and ultimately enhance the model’s predictive performance and accuracy in defect recognition tasks. In addition, Distance-IoU Non-Maximum Suppression is employed as a substitution for the original Non-Maximum Suppression. This enhances the recognition accuracy of bounding boxes, ensuring that the model maintains high detection accuracy even after complex scenarios. Moreover, the proposed methodology exhibits broad applicability across a wide spectrum of prevalent defect detection paradigms. In empirical experiments, we employ the YOLOv7 architecture as the foundation framework, integrating the proposed methodologies for the purpose of detecting defects in the membrane. The empirical evidence demonstrates a notable improvement in the performance of detecting defects in membrane products using the SE-CAR feature enhancement module and Distance-IoU Non-Maximum Suppression algorithm. In comparison to baseline networks, an increase in mAp by 6.29%, precision by 1.63%, and recall by 8.76%.

Keywords

Defect intelligent detection membrane products defect detection upsampling module improving YOLOv7

Introduction

The identification of surface defects in industrial products has been a long-standing challenge in industrial production.¹ Several objective factors, such as machine failures, material inhomogeneity, and environmental disturbances during production, frequently give rise to surface defects in products, resulting in significant economic losses for enterprises.² Automatic defect detection is indispensable in effectively controlling product quality. Traditionally, surface defect inspection has relied on manual visual inspection. Nevertheless, even the most proficient workers are capable of identifying only 70% of defects during the production process, and prolonged engagement in this labor-intensive task can deteriorate visual accuracy.^3,4 Therefore, it becomes imperative to propose an automatic defect detection method specifically for this purpose.

Within the contemporary domain of defect detection, target recognition strategies can be predominantly categorized into two primary methodologies: traditional approaches and deep learning-based methodologies. Traditional object detection methods primarily rely on feature extraction and machine learning approaches within computer vision. These methods typically employ manually designed features. Machine learning algorithms are subsequently employed for the purpose of classification. The primary advantages of traditional methods lie in their simplicity, speed, and suitability for real-time applications. Over time, deep learning-based object detection methodologies have emerged, facilitated by the advancements in deep neural network. These methods comprehensively learn object representations and classifiers directly from raw images through an end-to-end approach. Currently, there are two main kinds of deep learning-based object identification methodologies: one-stage and two-stage methods. One-stage methods, unify object detection and classification tasks within a single network.^5,6 Conversely, two-stage methods, initially employ a Region Proposal Network (RPN) to locate possible object areas first, and then categorize and localize them.^7,8 The primary advantages of deep learning methods lie in their automatic feature learning capabilities and robust expression, well-suited for addressing complex scenarios and various object classes.

Convolutional Neural Networks (CNNs) have emerged as the most critical and foundational deep learning methodology. CNNs have demonstrated unparalleled performance in feature extraction and spatial analysis, making them an indispensable tool for computer vision tasks. Despite the remarkable performance of the CNNs in object detection, the model faces shortcomings, including diminished accuracy and suboptimal detection of tiny objects in specific scenarios. To address these challenges, extensive efforts have been dedicated to enhancing and optimizing the CNNs model. An array of innovative approaches has been introduced, encompassing advancements in network architectures, data augmentation techniques, and refinements to loss functions. These strategies are all aimed at improving the model’s performance. As a result, the discipline of defect detection has seen notable advancements in terms of accuracy and resilience.⁹ Among these methods, Faster Region-based Convolutional Neural Network (Faster R-CNN) stands out as a swift algorithm for object detection, which harnesses the capabilities of region-based convolutional neural networks.¹⁰ Numerous researchers have conducted enhancements to Faster RCNN, with the aim of improving various aspects, such as detection accuracy and speed.^11–13 On the other hand, single shot multibox detector (SSD), which is based on convolutional neural networks, has become a well-known object detection technology.¹⁴ Numerous studies, research have made improvements to SSD in order to enhance its detection accuracy and speed.^15–17 Researchers have made significant progress in improving the original YOLO algorithm’s performance. Notable versions, such as YOLOv5^18,19 and YOLOv7,^20,21 have been developed to enhance both detection accuracy and speed. This work aims to enhance object detection algorithms’ efficiency and performance so they can be more accurate and versatile in a range of application scenarios.⁶ Nevertheless, there remain several challenges that necessitate further investigation within this domain. Firstly, the extraction of visually similar features from the background poses difficulty.²² Secondly, the existing methods tend to struggle in accurately identifying tiny defects in the target object.²³ This paper presents a detection framework with two parts to address these issues: the feature enhancement module and the Distance-IoU Non-Maximum Suppression (DIoU-NMS) technique.²⁴ The feature enhancement module integrates the Squeeze-and-Excitation (SE) attention module²⁵ and the Content-Aware ReAssembly of FEatures (CARAFE) up-sampling module²⁶ to effectively preserve detailed information, enhance feature representation, and improve overall performance by dynamically adjusting the importance of each channel. For the sake of brevity, the feature enhancement module is referred to as SE-CAR later. Additionally, in order to tackle the issue of overlapping bounding boxes in object recognition, DIoU-NMS is utilized. This method efficiently removes excessively overlapping bounding boxes and improves the precision and stability of object detection outcomes. To confirm that the suggested approach is effective, a specific dataset focused on membrane product defects was constructed, and the proposed framework was applied to YOLOv7²⁷ for experimentation. The empirical experiments conducted demonstrate substantial improvements in the proposed framework’s ability to detect membrane product defects. Impressively, the average precision (mAP) achieved an outstanding value of 97.87%, while precision and recall reached 99.5% and 97.12%, respectively. Compared to the baseline networks, our approach resulted in significant improvements, increasing the mAP, precision, and recall by 6.29%, 1.63%, and 8.76%, respectively. These experimental findings conclusively affirm the outstanding performance and applicability of the proposed framework. The following are this paper’s main contributions:

(1) We propose a feature enhancement module named SE-CAR, by integrating CARAFE into the SE attention mechanism. The SE-CAR model has the capability to enhance image features, thereby improving the overall performance of visual tasks.

(2) We introduce DIoU-NMS into the framework, which can be more effective in addressing challenges presented by overlapping or densely packed target boxes.

(3) We carried out comparative studies on the job of membrane product defect identification after integrating the suggested framework into YOLOv7. Comparing our experimental findings to the benchmark, we find substantial improvements of 6.29% in mAP, 1.63% in accuracy, and 8.76% in recall. The suggested technique achieves intelligent recognition when it is used in the industrial membrane product manufacturing line.

The rest of this paper is organized as follows: Section “Related work” reviews related work on traditional and deep learning-based defect detection methods. Section “Methodology” presents the proposed methodology, detailing the SE-CAR feature enhancement module, the DIoU-NMS algorithm and object detection techniques. Section “Empirical experiments” describes the empirical experiments, including data collection and preprocessing, evaluation metrics, and experimental results and analysis. Finally, Section “Conclusion and future work” concludes the paper and outlines future research directions.

Related work

Traditional detection methods

Traditional automated defect detection methods primarily focus on identifying and assessing defects or faults that could potentially impact the quality and performance of materials. These methods commonly involve a combination of manual visual inspection and machine vision techniques.^28,29 For instance, widely utilized approaches such as edge detection and threshold processing are employed to detect defect regions.³⁰ Several different methodologies have been suggested in prior scholarly works for identifying and categorizing defects. Ren et al.³¹ presented a novel method that makes use of color histograms to assess how comparable Electrical Resistance Tomography pictures are to one another. Lambert and Bock³² employed a multi-scale wavelet-based approach to detect texture defects, which not only allows for precise defect localization but also enables defect classification. Ma et al.³³ addressed the limitations of manually setting parameters and poor adaptability of the traditional Canny operator by proposing an adaptive Canny operator-based algorithm. Vladimir et al.³⁴ presented a digital image processing-based approach that achieves higher speed and accuracy. Ngan et al.³⁵ developed a direct threshold technique predicated on detailed sub-images of WT, which can effectively segment defect areas on patterned fabrics. Dou et al.³⁶ proposed a novel algorithm, named the Fast Railway Bolt Detection Algorithm, which overcomes the time-consuming task of manually collecting and training the training dataset, by utilizing rapid template matching techniques. These methods often rely on extensive manual feature extraction, making them time-consuming and constrained by resource limitations for manual inspection.

Deep-learning-based segmentation methods

The ascent of deep learning, notably through the utilization of CNNs, has heralded remarkable advancements in computer vision. CNNs serve as potent tools for feature learning, allowing the automatic acquisition of feature representations from unprocessed data, eliminating the requirement for manual design. The integration of this technology has empowered computer vision systems to gain a more profound comprehension of various image defects, thus bolstering the precision and resilience of defect detection. Numerous scholars have put forth diverse algorithms for detecting defects. For instance, Balzategui et al.³⁷ employed a fully convolutional network to accurately identify and localize defects in polycrystalline solar cells. Gao et al.³⁸ introduced a deep learning FCN-RCNN model to precisely and rapidly identify multiple tunnel defects, considering the intricate tunnel environments. Xia et al.³⁹ developed a algorithm that integrates the structural similarity index (SSIM) and MobileNet for accurate and efficient detection of surface defects in printed circuit boards. Their approach demonstrates a significant improvement in computational efficiency, achieving at least a 12-fold increase in speed compared to the Faster RCNN method while maintaining a high level of accuracy. Lei and Sui⁴⁰ presented a deep CNN adapted from the Faster R-CNN framework for intelligent fault detection in high-voltage power lines, specifically targeting the identification of damaged insulators and the presence of bird nests. Yu et al.⁴¹ addressed several critical challenges in the realm of X-ray flaw identification in castings, including the subtle differences between defect classes, the significant variations in defect sizes, and the inherent uncertainties in annotation. To effectively tackle these challenges, the authors proposed an innovative adaptive network that dynamically selects optimal deep and receptive fields for the semantic segmentation of defects. Using Generative Adversarial Networks (GANs) as the foundation, Liu et al.⁴² presented a framework for fabric fault detection. This framework addresses challenges associated with the intricate diversity of fabric textures and defects. Cui et al.⁴³ presented a swift and precise network for detecting surface defects, denoted as SDDNet. Their approach is particularly effective in handling issues like significant variations in texture and the detection of small-sized defects. Su and Yan⁴⁴ addressed various challenges in gear end face defect detection, including small defects, complex tooth surfaces, intricate processing environments, and the requirement for real-time detection. They proposed a method based on an improved YOLOv3 model. Li et al.⁴⁵ has proposed a lightweight and enhanced algorithm, leveraging YOLOv5, to tackle the challenges of low efficiency in detecting flat jujubes in the field, as well as the complexity of deploying target detection algorithms on low-cost devices.

Currently, small object detection, with a focus on identifying incredibly small objects within images, has recently emerged as a prominent research area. Due to the inherent characteristics of tiny objects, such as a reduced number of pixels and less pronounced object information, conventional object detection algorithms often grapple with the accurate detection and localization of such objects. To address this challenge, researchers have proposed a range of specialized detection algorithms specifically tailored for tiny objects. These methods leverage multi-scale feature fusion and attention mechanisms, yielding substantial enhancements in the detection performance of diminutive objects at the expense of increased computational requirements. Tiny object detection has emerged as highly active research topics within the field of object detection. Scholars consistently develop novel strategies to address these issues and improve object detection algorithms’ performance in progressively more challenging situations. To address the challenge of detecting tiny defects with complex multi-scale characteristics amidst intricate background interference prevalent in industrial environments, Zhang et al.⁴⁶ introduced a novel anchor-free network, DsPAN, intended for tiny object detection. In order to explicitly address the difficulties involved with recognizing microscopic objects, Deng et al.⁴⁷ presented an inventive Extended Feature Pyramid Network (EFPN) that smoothly incorporates pyramid levels with ultra-high resolutions. To improve the precision of identifying small objects, Lim et al.⁴⁸ presented a context-based object identification technique, which addresses the difficulty presented by poor resolution and sparse information related to these targets.

Methodology

Feature enhancement module

One of the most important tools used to improve the quality and recognition of features in picture data is the feature enhancement module. Its goal is to augment the valuable information present in the data, thus assisting algorithms or systems in processing the data more effectively.

We present a feature enhancement module called SE-CAR in this paper, which seamlessly integrates the CARAFE into the SE attention mechanism. The SE attention mechanism serves as a key component of the SE-CAR module, dynamically recalibrating the feature map weights to enhance representational power. This method begins by applying global average pooling to the feature maps, compressing the spatial information of each channel into a scalar, thereby creating a channel descriptor vector. Then, this vector is input into a small network composed of two fully connected layers, adaptively learning the importance weights for each channel. Finally, the learned weights are applied to the corresponding channels of the original feature maps, highlighting important features and suppressing less important ones.

Another significant component of the SE-CAR module is the CARAFE module, which leverages a learning-based approach to improve upsampling results through specific sampling and reconstruction processes. In the subsequent sections, we present a comprehensive explanation of the CARAFE module and the SE attention mechanism.

The SE attention module is a potent and flexible technique intended to improve deep neural network performance and representational capacity. Its purpose is to dynamically learn and adaptively recalibrate the importance weights of features across different channels. In convolutional neural networks, different channels have varying importance in representing distinct features. By incorporating the SE attention mechanism allows for the adjustment of channel weights to enhance the capture of crucial information. Figure 1 the architectural design and key components of the SE attention module. The SE attention module is composed of several key components, which include:

Figure 1.

The structure of SE attention module.

Squeeze

To get the global average value for each channel, the input defect feature map was first globally pooled in the spatial dimension. This provides, a scalar value that represents the global importance of each channel’s features. In Figure 1, the operation of squeeze is denoted by using $F_{sq} (\cdot)$ .

Excitation

Subsequently, a small multilayer perceptron is employed to learn the global average values, resulting in the acquisition of a new set of weights that represent the relative importance of each channel. In Figure 1, the operation of excitation is denoted by using $F_{ex} (\cdot, W)$ .

Scale

Finally, an element-wise multiplicative operation is used to modify the initial input feature map using the channel-wise significance weights that have been adaptively learnt via the attention process. The action of scale in Figure 1 is shown by $X$ .

In Figure 1 $F_{tr}$ is a simple convolution operation, represented by the $V = {v_{1}, v_{2}, . . ., v_{c}}$ . Squeeze excitation block: map the input $X \in R^{H' \times W' \times C'}$ to the feature map $U \in R^{H \times W \times C}$ , output $U = {u_{1}, u_{2}, . . ., u_{c}}$ . This is an appendage that may be built onto a transformation. $u_{c}$ may be calculated using the formula in equation (1).

u_{c} = v_{c} \otimes X = \sum_{s = 1}^{C'} v_{c}^{s} \otimes x^{s}

(1)

where ⊗ represents the convolution operation, $v_{c} = {v_{c}^{1}, v_{c}^{2}, . . ., v_{c}^{C'}}$ , $X = {x^{1}, x^{2}, . . ., x^{C'}}$ , and $u_{c} \in R^{H \times W}$ . Here, $v_{c}$ is the $c$ -th convolution kernel, and $x_{s}$ is a 2D spatial convolution.

The module CARAFE is designed for upsampling, as seen in Figure 2. To gradually raise the low-resolution feature map’s spatial resolution is the main goal of the upsampling process, ultimately aligning its dimensions with those of the original input image, thereby enabling pixel-wise correspondence and facilitating subsequent high-resolution spatial reasoning tasks. The CARAFE module is built upon the fundamental concept of leveraging both local pixel information and surrounding contextual information during the upsampling process. It extracts the context information from the feature map through reversible operations and combining it with the outcomes of conventional up-sampling. As a result, CARAFE enables more precise upsampling results and improved restoration of details from the original image.

Figure 2.

The structure of CARAFE up-sampling.

CARAFE network architecture comprises two modules: the Kernel Prediction Module and the Content-aware Reassembly Module. Within the Kernel Prediction Module, $1 \times 1$ convolution is initially applied to compress the input’s $H \times W \times C$ feature map channels to $H \times W \times C_{m}$ , thereby reducing computational load. Next, the number of channels is redistributed to encode the convolution, with $σ$ representing the up-sampling rate. Subsequently, pixel reassembly is conducted to enhance the up-sampling’s receptive field. Using normalization to cut down on the number of operation parameters is the last stage.

In the Content-aware Reassembly Module, the feature map patches extracted from the original high-resolution input undergo element-wise multiplication with their spatially corresponding adaptive kernels generated by the Kernel Prediction Module, yielding the final output.

To acquire defect feature maps of superior quality, the CARAFE upsampling method is employed. First, a convolutional layer is used to generate a reassembly kernel, which is responsible for capturing local contextual information. Then, the input feature maps are preliminarily upsampled, typically using the nearest neighbor interpolation method, to match the target output size. Next, the upsampled feature maps are weightedly reassembled using the generated reassembly kernel, a process that actually redistributes features while considering local context. Finally, the CARAFE module outputs the reassembled feature maps, which retain the information of the original features and integrate contextual information, thus obtaining a higher quality defect feature representation. This optimized feature map is then passed to subsequent layers of the network for further processing.

DIoU-NMS

The Intersection over Union (IoU) metric is used by NMS approaches to measure the extent of overlap between several detection bounding boxes. Based on a predefined threshold, this metric is used to determine whether to retain a specific detection box. Equation (2) illustrates the formula for calculating IoU.⁴⁹

IoU (A, B) = \frac{Area of Overlap}{Area of Union}

(2)

Traditional NMS methods have primarily relied on a single IoU metric. However, practical applications necessitate the consideration of various influential factors. It is important to take into account not just the common area between the grounded frame and the forecast frame, but also the aspect ratio and the separation from each centroid point. In cases where two targets are in close proximity and exhibit a substantial IoU value, traditional NMS may mistakenly identify them as a single object, which could result in unintended exclusion. On the other hand, DIoU-NMS offers advantages by not eliminating the other frame in instances where the centroids of the two frames are considerably distant, even when the IoU value is high. Consequently, the implementation of DIoU-NMS serves to diminish the incidence of false deletions. DIoU-NMS is better than NMS because it becomes possible to more effectively address challenges presented by overlapping or densely packed target boxes. This not only leads to better suppression of redundant detection results but also enhances the accuracy and stability of target detection. The formula for calculating DIoU⁵⁰ is demonstrated in equation (3). Figure 3 illustrates the concept. The anticipated box is represented by the black box, while the actual box is represented by the green box.

DIoU (A, B) = IoU (A, B) - \frac{d^{2}}{c^{2}}

(3)

where $c$ represents the diagonal length of the boxes, or the separation between the two farthest points, and $d$ represents the Euclidean distance between the center points of the two rectangular boxes.

Figure 3.

The schematic diagram of DIoU calculation.

In summary, Algorithm 1 illustrates the computational procedure of the SE-CAR and DIoU-NMS methods. The algorithm takes as input a feature vector $X \in R^{H' \times W' \times C'}$ , as well as the set of resulting detection boxes $D$ and their corresponding detection scores $S$ . Each score $s$ corresponds to a detection box $d$ in $D$ . The algorithm flow of SE-CAR is described in lines 1 to 3. The input $X$ is subjected to the SE attention mechanism in line 1 in order to produce a feature map $U \in R^{H \times W \times C}$ , which is made up of distinct feature maps $u_{c}$ . Each map $u_{c}$ has a transformation that is defined by equation (1). Line 2 takes the input feature map and performs a $1 \times 1$ convolution on it, compressing its channels to $H \times W \times C_{m}$ with a given upsample ratio, denoted as $σ$ (assuming $σ$ is an integer). Line 3 encompasses content encoding, up-sampling kernel prediction, and normalization using softmax to ensure the sum of convolutional kernel weights is 1. This step outputs the set of initial detection boxes $B = {b_{1}, . . ., b_{N}}$ and their corresponding detection scores $S = {s_{1}, . . ., s_{N}}$ , where each score $s_{i}$ corresponds to the detection box $b_{i}$ in $B$ . Lines 2 to 3 utilize the CARAFE upsample method. Line 4 represents $N_{t}$ as the threshold for NMS. Lines 5 to 16 outline the calculation process for DIoU-NMS, which is used to refine and post-process the detection results. Create an empty selected bounding box list $D$ . Iterate through each bounding box $b_{k}$ in $B$ . Add $b_{k}$ to $D$ . Compute the DIoU for $b_{k}$ with each bounding box in $B$ . If the DIoU value exceeds the predefined threshold, remove $b_{i}$ from $B$ and $s_{i}$ from $S$ . Return $S$ as the final selected bounding box list.

Algorithm 1 The algorithm of SE-CAR and DIoU.
Input: Feature $\in R^{H' \times W' \times C'}$ ; Output: The set of result detection boxes $D$ and the set of detection scores $S$ , where each $s$ corresponds to the detection box $d$ in $D$ .1: $U$ = SE_ATT( $X$ ) {SE_ATT(·) calculate by equation (1)} 2: $X'$ = CARAFE_UP( $U$ ) 3: { $B$ , $S$ } = VISUAL_MODEL( $X'$ ) 4: $N_{t} \leftarrow$ NMS threshold. 5: $D \leftarrow$ {} 6: while $\| B \| \neq$ 0 do 7: $s_{k} \leftarrow \max (S)$ 8: $D \leftarrow D \cup {b_{k}}$ 9: $B \leftarrow B ∖ {s_{k}}$ 10: for $b_{i} \in B$ do 11: if DIoU $(b_{k}, b_{i}) \geq N_{t}$ then {DIoU(·) calculate by equation (2).} 12: $B \leftarrow B ∖ {b_{i}}$ 13: $S \leftarrow S ∖ {s_{i}}$ 14: end if 15: end for 16: end while

Algorithm 1 The algorithm of SE-CAR and DIoU.

Input: Feature

\in R^{H' \times W' \times C'}

; Output: The set of result detection boxes

D

and the set of detection scores

S

, where each

s

corresponds to the detection box

d

D

.1:

U

= SE_ATT(

X

) {SE_ATT(·) calculate by equation (1)}
2:

X'

= CARAFE_UP(

U

) 3: {

B

S

} = VISUAL_MODEL(

X'

)
4:

N_{t} \leftarrow

NMS threshold.
5:

D \leftarrow

{}
6: while

| B | \neq

0 do
7:

s_{k} \leftarrow \max (S)

D \leftarrow D \cup {b_{k}}

B \leftarrow B ∖ {s_{k}}

10: for

b_{i} \in B

do
11: if DIoU

(b_{k}, b_{i}) \geq N_{t}

then {DIoU(·) calculate by equation (2).}
12:

B \leftarrow B ∖ {b_{i}}

13:

S \leftarrow S ∖ {s_{i}}

14: end if
15: end for
16: end while

Object detection techniques

Currently, there are several commonly used target detection algorithms in computer vision, namely SSD, Faster RCNN, and the YOLO series. The algorithms in question are engineered to ascertain and accurately demarcate objects spanning diverse classifications within both still imagery and video content. SSD, an anchor-based object detector, is capable of simultaneous prediction of multiple bounding boxes and corresponding classes within an image, enabling real-time detection. It accomplishes this by utilizing predetermined anchor boxes of various scales on the feature maps, enabling the detection of objects across a wide range of sizes. SSD, an anchor-based object detector, is capable of simultaneous prediction of multiple bounding boxes and corresponding classes within an image, enabling real-time detection. In contrast, Faster RCNN is a two-stage object detection algorithm that employs a RPN to generate initial candidate bounding boxes, which are then refined and classified. Although Faster RCNN achieves higher detection accuracy compared to SSD, it comes at the cost of slower processing speed. The complexity of the computation involved in the generation of candidate bounding boxes leads to an overall decrease in speed. On the other hand, YOLOv7 is among the most sophisticated single-level object detection systems out there. With YOLOv7, inference is remarkably rapid yet detection accuracy is still high. This is achieved via convolutional neural network prediction of both the bounding box and its associated class probabilities simultaneously. Due to its superior speed, YOLOv7 is highly suitable for real-time object detection applications, offering a notable advantage.

YOLOv7 model

YOLOv7 is an evolution of the YOLO series and is known for its high accuracy and real-time performance. The model consists of three main components: an input preprocessor, a backbone network, and a detection head.

The YOLOv7¹ model consists of three main components: an input terminal, a backbone network, and a detection head. Initially, the faulty image is first uniformly resized by the input component to a typical 640 × .640 three-channel RGB training image. Efficient Layer Aggregation Networks (ELAN) and Max-Pooling (MP) modules make up the Backbone module. The input feature layer’s dimensions are maintained by the stacked convolution blocks that make up the ELAN module. It enhances interaction between different feature layers by expanding, randomly combining, and concatenating them, thereby improving the model’s learning capability. The MP module ensures consistency in channel numbers between the input and output. The head structure comprises a Spatial Pyramid Pooling with Cross Stage Partial Convolution (SPPCSPC) layer, followed by a sequence of CBS layers, MPConv layers, CatConv layers, and Residual Pose Representation (RepConv) layers, which serve as the output layers of the network. Each layer is obtained by several distinct convolution layers performing various operations. The network performs feature extraction by iteratively fusing high-level and low-level feature maps, then uses these refined features to generate and output the final predictions.

Empirical experiments

The empirical trials that were conducted to confirm the suggested method’s effectiveness are described in this section. Every experiment was run on a GPU server with 64 GB of RAM. The server was equipped with an Intel^® Xeon^® Gold 5218 CPU operating at 2.30 GHz and an NVIDIA GeForce RTX 3090 GPU. The experimental code was implemented using Python 3.9. The process of software development can be divided into the following steps:

(1) Data acquisition: Acquire images using an industrial camera and preprocess the gathered data.

(2) Data preparation: Divide data into training, validation, and testing sets, and annotate the data.

(3) Model selection: Choose YOLOv7 as the fundamental model for training.

(4) Model improvement: Utilize an improved version of YOLOv7 to train the model for increased fault detection accuracy.

(5) Defect detection: Deploy the trained model to identify and precisely locate defects.

A visual representation of the software workflow can be observed in Figure 4.

Figure 4.

Software flowchart of membrane product detection algorithm.

Data collection and preprocess

The data for experiments was obtained from an operating membrane product factory located in Zhejiang Province. The dataset contains images of membrane products captured under various lighting conditions and manufacturing environments. Table 1 summarizes the various types of data collected and the environmental conditions under which the data was acquired.

Table 1.

Details of the membrane product defect dataset.

Data type	Lighting condition	Resolution
Images	Sunny, cloudy, night	2192 × 568

A high-resolution industrial camera mounted on the production line was used to capture real-time images, which were then transmitted to a GPU server for further processing. The process of data collection is depicted in Figure 5.

Figure 5.

Diagram illustrating data collection for membrane products.

The inspection process faces a significant challenge when dealing with membrane products, as their thin nature is a result of specific craftsmanship techniques. In order to address this challenge, we conducted multiple experiments and implemented several modifications to the existing production line. Ultimately, we decided to use a deep blue color with low saturation as the background. This choice aimed to reduce potential interference caused by complex backgrounds and improve the accuracy of defect detection. Figure 6 depicts a schematic diagram illustrating the process of data collection for membrane products under various conditions. The defect dataset in this paper focuses on a particular type of defect labeled as “hole”, which manifests as circular holes. To account for the influence of different environmental factors, images of the membrane products were uniformly collected under different light conditions, including sunny, cloudy, and nighttime scenarios, as depicted in Figure 7.

Figure 6.

(a) The original production environment, (b) the original data collection, (c) the production environment with added bottom plate, and (d) the data collection with added bottom plate.

Figure 7.

Schematic diagram of the data sets collected under different light environments.

To improve the quality and diversity of the dataset, several data preprocessing and augmentation techniques were applied:

(1) Normalization: The pixel values of each image were normalized to a range of 0 to 1.

(2) Resizing: Every image was resized to 568 × 568 pixels.

(3) Image augmentation: Conventional image augmentation techniques such as translation, rotation, flipping, and mirroring were applied during the training phase.

The initial dataset of 586 images was expanded to 2630 images after preprocessing and augmentation. The augmented dataset was then split into training (1841 images), validation (526 images), and test (263 images) sets with a 7:2:1 ratio. Figure 8 illustrates the schematic diagram of the image preprocessing procedure.

Figure 8.

Preprocessing and augmentation effects of membrane product images.

Determination of judgment index

In our experiment, the primary assessment statistic we employ to gauge the model’s effectiveness is the mAP. The mAP stands for the arithmetic mean of the Average Precision (AP) ratings for each class in the dataset, which offers a thorough assessment of the model’s overall accuracy. The steps involved in calculating mAP, recall, and accuracy are outlined below.

Precision

Out of all the samples that were expected to be positive, precision is the percentage of samples that the model properly identifies as positive. According to equation (4), precision can be computed mathematically by dividing the number of true positive samples (positive samples that were correctly predicted) by the total number of false positives (positive samples that were forecasted incorrectly).

Precision = \frac{TP}{TP + FP}

(4)

where True Positives (TP) are the samples the model correctly classified as positive, False Negatives (FN) are the samples the model incorrectly classified as negative, and False Positives (FP) are the number of negative samples the model mistakenly predicted to be positive.

Recall

Recall evaluates how well a model detects positive samples. It is also known as true positive rate or sensitivity at times. As shown in equation (5), it is computed by dividing the total number of TP by the sum of FN and TP.

Recall = \frac{TP}{TP + FN}

(5)

mAP

By calculating the arithmetic mean of the AP scores for each object category, mAP is a comprehensive performance statistic that provides an overview of an object detection system’s overall accuracy. The system’s comprehensive assessment of object detection and classification accuracy, as stated in equation (6), is provided.

mAP = \frac{\sum_{i = 1}^{n} A P_{i}}{n}

(6)

where $A P_{i}$ is the AP score for the i-th category and $n$ is the total number of object categories.

Higher precision and recall values generally result in an increase in the model’s accuracy and recall rate when it comes to target detection. Similarly, a higher mAP signifies improved detection performance in the model. This suggests that the model can detect targets more precisely and reduce the likelihood that they would be missed.

Experimental results and analysis

Comparison experiment

In order to determine the baseline network that can meet the industrial inspection needs, we employed the membrane defect dataset to train and evaluate several object detection models, including SSD, Faster RCNN, and various algorithms from the YOLO series (YOLOv5, YOLOv7, YOLOv8, YOLOv9). The detection results in terms of mAP are presented in Table 2. The mAP was employed as the primary metric to evaluate the models’ robustness and detection accuracy for identifying flaws in membrane products. The results indicate that YOLOv7 achieved the highest mAP of 91.58% among the compared models, outperforming SSD (86.53%), Faster RCNN (87.21%), YOLOv5 (88.98%), YOLOv8 (89.38%), and YOLOv9 (88.21%). The comparison reveals that the YOLOv7 algorithm significantly outperforms the other evaluated models in terms of mAP for the membrane defect detection task. Consequently, the YOLOv7 algorithm demonstrated superior performance in detecting defects on membrane products, rendering it the most suitable choice for practical production processes. Therefore, we selected YOLOv7 as the baseline architecture for further improvements and experiments.

Table 2.

Performance comparison of SSD, faster RCNN, and YOLOv7 with algorithm.

Experimental model	mAP (%)
SSD	86.53
Faster RCNN	87.21
YOLOv5	88.98
YOLOv7	91.58
YOLOv8	89.38
YOLOv9	88.21

Enhanced module ablation experiment

We employed YOLOv7 as the base network with the same configuration to validate the experimental findings. In an extensive series of ablation trials, we evaluated the effect of adding the SE-CAR module on detection accuracy. Table 3 presents the detection outcomes that were obtained as a result. Figure 9 shows the performance comparison of the models.

Table 3.

The results of ablation experimental results.

Model	mAP (%)	Precision (%)	Recall (%)
YOLOv7	91.58	97.87	88.36
YOLOv7-SE	96.75	99.10	95.39
YOLOv7-CARAFE	95.68	97.87	94.03
YOLOv7-SE-CAR	96.89	98.52	95.97

Figure 9.

Model performance comparison, YOLOv7-SE-CAR model has high performance in membrane product tiny defect detection tasks.

The analysis of the ablation study data reveals that the enhanced YOLOv7 architecture significantly outperforms the original YOLOv7 architecture in terms of defect detection performance for membrane products. The original YOLOv7 architecture achieved a mAP of 91.58%, a precision of 97.87%, and a recall of 88.36%, which served as the baseline performance for subsequent modifications. By introducing the selective attention mechanism, the mAP increased to 96.75%, the precision rose to 99.10%, and the recall improved to 95.39%. These findings imply that the model is better able to concentrate on important characteristics for detection tasks as a result of the SE attention mechanism’s integration, which raises detection accuracy and recall. Moreover, the incorporation of the CAFARE upsampling module further enhanced the model’s performance, with the mAP reaching 95.68%, the precision remaining at 97.87%, and the recall increasing to 94.03%. This indicates that the CAFARE upsampling module helps the model capture more detailed and edge information, resulting in improved detection accuracy. Furthermore, the SE-CAR feature enhancement module, which combines both the SE attention mechanism and the CAFARE upsampling module, achieved the highest performance, with a mAP of 96.89%, a precision of 98.52%, and a recall of 95.97%. This synergistic integration of the SE attention mechanism and CAFARE upsampling module further optimizes the model’s ability to adapt to task requirements and attain superior mAP and precision metrics for membrane product defect detection.

The experiment shows that using the SE-CAR feature improvement module can help identify defects in membrane products, compared to the original YOLOv7, mAP, precision, and recall increased by 5.31%, 0.65%, and 7.61% respectively. We believe that the SE-CAR module allows the model to concentrate on essential features, extract more sophisticated feature information, and enhance detection performance through the integration of the advantages of the SE attention mechanism and the CAFARE upsampling module.

Figure 10 illustrates the network structure proposed for improved the YOLOv7 model. Our aim is to establish an intelligent defect recognition model tailored specifically for membrane products. Initially, the SE-CAR feature enhancement module is incorporated to enhance the overall accuracy of recognition. The central portion depicts the overall architecture of the network, while the sections with a gray background represent the sub-modules within the architecture. The figure includes a highlighted dashed box, which represents our proposed SE-CAR feature enhancement module.

Figure 10.

The structure of the improved network architecture of YOLOv7.

DIoU-NMS ablation experiment

In the same experimental setting, the performance of DIoU-NMS was compared and evaluated under various conditions. The results are presented in Table 4. The initial YOLOv7 implementation employed the conventional NMS algorithm, which employs a threshold to determine whether to retain overlapping bounding boxes. A low threshold could lead to more duplicated bounding boxes, whereas a high threshold might miss detections. Moreover, the size of the boxes was not taken into account, potentially leading to the neglect or redundant detection of tiny objects. Consequently, the mAP achieved was 91.58%, with precision at 97.87% and recall at 88.36%. Figure 11 shows the performance comparison of the models.

Table 4.

The results of ablation experimental results.

Model	mAP (%)	Precision (%)	Recall (%)
YOLOv7	91.58	97.87	88.36
YOLOv7-DIoU	96.41	98.31	94.87
YOLOv7-CARAFE-DIoU	96.55	98.34	94.92
YOLOv7-SE-DIoU	97.41	99.20	96.32
YOLOv7-SE-CAR-DIoU	97.87	99.50	97.12

Figure 11.

Model performance comparison, YOLOv7-SE-CAR-DIOU model has high performance in membrane product defect detection tasks.

To assess the effectiveness of DIoU-NMS, experiments were conducted on various YOLOv7 models, including the original version, one incorporating CAFARE undersampling, another with an added SE attention mechanism, and a model featuring the SE-CAR feature enhancement module. Notably, replacing the traditional NMS with DIoU-NMS yielded a substantial improvement in mAP, precision, and recall. The most notable enhancement was observed in the YOLOv7 model with the SE-CAR feature enhancement module, yielding respective values of 97.81%, 99.50%, and 97.12% for mAP, precision, and recall. These metrics registered improvements of 6.29%, 1.63%, and 8.76%, respectively, compared to the original YOLOV7 model. The experimental results offered compelling evidence regarding the influence of the DIoU-NMS algorithm on the performance of object detection tasks. By incorporating a distance measurement factor, the DIoU-NMS algorithm effectively evaluates the overlap between bounding boxes utilizing distance information. This capability enables DIoU-NMS to deliver more accurate outcomes, particularly in scenarios with high levels of overlapping bounding boxes.

These findings confirm that our suggested model has a major influence on the identification of surface flaws in membrane products. As seen in Figure 12, we compare the detection results of our enhanced YOLOv7 model with those of the original YOLOv7 model. The comparison between the two images above is evident. When two defects are in close proximity, the original NMS algorithm of YOLOv7 disregards the detection boxes of smaller defects, consequently leading to missed detections. However, the subsequent pair of images demonstrates the successful detection of extremely tiny defects by our algorithm, effectively compensating for the deficiencies of YOLOv7. Furthermore, a clear observation from the images is that our algorithm has substantially enhanced the confidence level in detecting these defects.

Figure 12.

Comparison of test results under different models.

To evaluate the suggested algorithm’s performance in real-world industrial settings, we implemented this method in a membrane product factory production line. We deployed the improved YOLOv7 method in the actual environment for testing, enabling real-time intelligent defect detection. The results of these tests are illustrated in Figure 13.

Figure 13.

Defect detection results of membrane products.

Conclusion and future work

Identifying surface flaws in industrial items is one of the most crucial tasks in the production process. However, this particular area is replete with a myriad of challenges that necessitate further investigation and resolution. Firstly, the extraction of visually similar features from the background poses a difficulty, and secondly, the existing methods tend to struggle in accurately identifying tiny defects in the target object. This research proposes a comprehensive detection framework that encompasses two essential components: the SE-CAR feature enhancement module and the DIoU-NMS. The SE-CAR module is aimed at effectively preserving detailed information, enhancing feature representation, and improving overall performance by dynamically adjusting the importance of each channel. Moreover, the DIoU-NMS algorithm is employed to tackle the issue of overlapping bounding boxes in object detection tasks. This approach improves the object detection results in terms of accuracy and stability by eliminating bounding boxes that overlap excessively. An exclusive dataset designed specifically for membrane product defects is selected to confirm the effectiveness of the proposed method. The suggested framework is used to explore with YOLOv7. The effectiveness of these optimization techniques has been effectively shown by the experimental results. With a mAP of 97.87%, the enhanced method outperformed the original algorithm by a noteworthy 6.29%. The research findings clearly show how well the suggested method performs in identifying membrane faults, increasing its efficacy and broad application in actual production settings.

In future research, our objective is to precisely detect areas of defects and comprehensively classify these flaws utilizing the Rapid method proposed by Wang et al.⁵¹ Additionally, we aspire for our proposed method can be applied to other models such as the SSD, the Faster R-CNN, and the YOLO series for use in other target detection research areas.

Supplemental Material

sj-zip-1-mac-10.1177_00202940241268952 – Supplemental material for Defect intelligent recognition of membrane product based on deep learning

Supplemental material, sj-zip-1-mac-10.1177_00202940241268952 for Defect intelligent recognition of membrane product based on deep learning by Maonian Wu, Ling Li, Wei Peng, Tao Wu, Jinwei Yu, Bo Zheng and Shaojun Zhu in Measurement and Control

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: National Natural Science Foundation of China (61906066); 2024 Zhejiang University Students’ Science and Technology Innovation Activity Program (New Seedling Talent Program) Project (2024R430C028); A Project Supported by Scientific Research Fund of Zhejiang Provincial Education Department (Y202351126); Postgraduate Research and Innovation Project of Huzhou University (2023KYCX36); 2022 Huzhou Science and Technology Plan Project (2022GZ04).

ORCID iD

Ling Li

Data availability statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Notes

Supplemental material

Supplemental material for this article is available online.

References

Ioannou-Ttofa

Fattas

Eusebio

, et al. Treatment efficiency and economic feasibility of biological oxidation, membrane filtration and separation processes, and advanced oxidation for the purification and valorization of olive mill wastewater. Water Res 2017; 114: 1–13.

Singh

Desai

Automated surface defect detection framework using machine vision and convolutional neural networks. J Intell Manuf 2023; 34(4): 1995–2011.

Han

Chun

Lee

, et al. Stroboscopic data-driven, integrated, and intelligent machine learning-based algorithms for semiconductor wafer inspection. In: ASME 2023 18th international manufacturing science and engineering conference, New Brunswick, NJ, USA, June 12–16, 2023, vol. 87240. American Society of Mechanical Engineers.

Jia

Shi

Quan

, et al. Fabric defect detection based on transfer learning and improved faster R-CNN. J Eng Fiber Fabr 2022; 17: 1–18.

Redmon

Divvala

Girshick

, et al. You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2022, pp.779–788. New York: IEEE.

Yuan

Wang

, et al. Improved feature fusion in YOLOv5 for accurate detection and counting of Chinese flowering cabbage (Brassica campestris L. ssp. chinensis var. utilis Tsen et lee) buds. Agronomy 2023; 14(1): 42.

Girshick

. Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, 2015, pp.1440–1448. New York: IEEE.

Belova

Smutka

Rosochatecka

, et al. Agris on-line papers in economics and informatics. Czh Univ Life Sci Prag Faculty Econ Manag 2012; 4: 1–15.

Peng

Liu

, et al. Pattern recognition and computer vision: Third Chinese conference, PRCV 2020, Nanjing, China, Proceedings, Part III, 16–18 October 2020, vol. 12307. Heidelberg: Springer Nature, 2020.

10.

Cui

Pan

Zhang

, et al. Intelligence science and big data engineering. Visual data engineering: 9th international conference, IScIDE 2019, Proceedings, Part I, Nanjing, China, 17–20 October 2019, vol. 11935. Heidelberg: Springer Nature, 2019.

11.

Sun

Hoi

SC.

Face detection using deep learning: an improved faster RCNN approach. Neurocomputing 2018; 299: 42–50.

12.

Jiang

Tan

, et al. Semantic segmentation for multiscale target based on object recognition using the improved faster-RCNN model. Future Gener Comp Syst 2021; 123: 94–104.

13.

Chen

Wang

Huang

An improved faster RCNN-based weld ultrasonic atlas defect detection method. Meas Control 2023; 56(3–4): 832–843.

14.

Xiuming

Yeping

Shijuan

, et al. Scale adaptive small objects detection method in complex agricultural environment: taking bees as research object. Smart Agric 2022; 4(1): 140.

15.

Zhai

Shang

Wang

, et al. DF-SSD: an improved SSD object detection algorithm based on DenseNet and feature fusion. IEEE Access 2020; 8: 24344–24357.

16.

Yin

Yang

Ran

, et al. FD-SSD: an improved SSD object detection algorithm based on feature fusion and dilated convolution. Signal Process Image Commun 2021; 98: 116402.

17.

Chen

Guo

Yang

, et al. Fast vehicle detection algorithm in traffic scene based on improved SSD. Measurement 2022; 201: 111655.

18.

Dong

Wei

, et al. Defect signal intelligent recognition of weld radiographs based on YOLO v5-improvement. J Manuf Process 2023; 99: 373–381.

19.

Wang

, et al. SF-YOLOv5: improved YOLOv5 with swin transformer and fusion-concat method for multi-UAV detection. Meas Control 2023; 56(7–8): 1436–1445.

20.

Zhu

Jiang

Guo

, et al. Surface morphology inspection for directed energy deposition using small dataset with transfer learning. J Manuf Process 2023; 93: 101–115.

21.

Luo

Zhang

, et al. Mask wearing detection algorithm based on improved YOLOv7. Meas Control 2024; 57(6): 751–762.

22.

Song

Yan

Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics Lasers Eng 2020; 128: 106000.

23.

Tang

Wang

Zhang

, et al. A lightweight surface defect detection framework combined with dual-domain attention mechanism. Expert Syst Appl 2024; 238: 121726.

24.

Neubeck

Van Gool

. Efficient non-maximum suppression. In: Proceedings of the 18th international conference on pattern recognition (ICPR-06), Hong Kong, China, vol. 3, pp.850–855.

25.

Shen

Sun

. Squeeze-and-excitation networks. In: Proceedings of the 31th IEEE conference on computer vision and pattern recognition (CVPR-18), Salt Lake City, UT, USA, pp.7132–7141. New York: IEEE.

26.

Wang

Chen

, et al. Carafe: content-aware reassembly of features. In: Proceedings of the 17th IEEE/CVF international conference on computer vision (ICCV-19), Seoul, Korea (South), pp.3007–3016. New York: IEEE.

27.

Wang

Bochkovskiy

Liao

. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the 36th IEEE/CVF conference on computer vision and pattern recognition (CVPR-23), Vancouver, BC, Canada, pp.7464–7475. New York: IEEE.

28.

Schoonahd

Gould

Miller

Studies of visual inspection. Ergonomics 1973; 16(4): 365–379.

29.

Golnabi

Asadpour

Design and application of industrial machine vision systems. Robot Comput Integr Manuf 2007; 23(6): 630–637.

30.

Ziou

Tabbone

Edge detection techniques – an overview. Pattern Recognit Image Anal 1998; 8: 537–559.

31.

Ren

Tian

Hong

, et al. Visualized investigation of defect in cementitious materials with electrical resistance tomography. Constr Build Mater 2019; 196: 428–436.

32.

Lambert

Bock

. Wavelet methods for texture defect detection. In: Proceedings of the 4th international conference on image processing (ICIP-97), Santa Barbara, CA, USA, 1997, vol. 3, pp.201–204. New York: IEEE.

33.

Deng

Sun

, et al. An algorithm for fabric defect detection based on adaptive canny operator. In: Proceedings of the 1st international conference on intelligent computing, automation and systems (ICICAS-19), Chongqing, China, 2019, pp.475–481. New York: IEEE.

34.

Vladimir

Evgen

Aung

. Automatic detection and classification of weaving fabric defects based on digital image processing. In: Proceedings of the 6th IEEE conference of Russian young researchers in electrical and electronic engineering (EIConRus-19), Petersburg, Russia, 2019, pp.2218–2221. New York: IEEE.

35.

Ngan

Pang

Yung

, et al. Wavelet based methods on patterned fabric defect detection. Pattern Recognit 2005; 38(4): 559–576.

36.

Dou

Huang

, et al. A fast template matching-based algorithm for railway bolts detection. Int J Mach Learn Cybern 2014; 5: 835–844.

37.

Balzategui

Eciolaza

Arana-Arexolaleiba

. Defect detection on polycrystalline solar cells using electroluminescence and fully convolutional neural networks. In: Proceedings of the 12th IEEE/SICE international symposium on system integration (SII-20), Honolulu, HI, USA, 2020, pp.949–953. New York: IEEE.

38.

Gao

Jian

, et al. Faster multi-defect detection system in shield tunnel using combination of FCN and faster RCNN. Adv Struct Eng 2019; 22(13): 2907–2921.

39.

Xia

Cao

Wang

. SSIM-NET: real-time PCB defect detection based on SSIM and MobileNet-V3. In: Proceedings of the 2nd world conference on mechanical engineering and intelligent manufacturing (WCMEIM-19), Shanghai, China, 2019, pp.756–759. New York: IEEE.

40.

Lei

Sui

Intelligent fault detection of high voltage line based on the faster R-CNN. Measurement 2019; 138: 379–385.

41.

Song

, et al. Adaptive depth and receptive field selection network for defect semantic segmentation on castings x-rays. NDT E Int 2020; 116: 102345.

42.

Liu

Wang

, et al. Multistage GAN for fabric defect detection. IEEE Trans Image Process 2019; 29: 3388–3400.

43.

Cui

Jiang

, et al. SDDNET: a fast and accurate network for surface defect detection. IEEE Trans Instrum Meas 2021; 70: 1–13.

44.

Yan

. A defect detection method of gear end-face based on modified YOLO-v3. In: Proceedings of the 10th institute of electrical and electronics engineers international conference on cyber technology in automation, control, and intelligent systems (CYBER-20), Xi’an, China, 2020, pp.283–288. New York: IEEE.

45.

Zhang

Xue

, et al. Lightweight target detection for the field flat jujube based on improved YOLOv5. Comput Electron Agric 2022; 202: 107391.

46.

Zhang

Huang

, et al. DSP-YOLO: an anchor-free network with DSPAN for small object detection of multiscale defects. Expert Syst Appl 2024; 241: 122669.

47.

Deng

Wang

Liu

, et al. Extended feature pyramid network for small object detection. IEEE Trans Multimed 2021; 24: 1968–1979.

48.

Lim

Astrid

Yoon

, et al. Small object detection using context and attention. In: 2021 international conference on artificial intelligence in information and Communication (ICAIIC), 2021, pp.181–186. New York: IEEE.

49.

Jiang

Wang

, et al. Unitbox: an advanced object detection network. In: Proceedings of the 24th ACM international conference on multimedia (MM-16), Amsterdam, The Netherlands, 2016, pp.516–520. New York: ACM Press.

50.

Zheng

Wang

Liu

, et al. Distance-IOU loss: faster and better learning for bounding box regression. In: Proceedings of the 34th AAAI conference on artificial intelligence (AAAI-20), New York, NY, USA, pp.12993–13000. Washington, DC: Association for the Advancement of Artificial Intelligence.

51.

Wang

Xiang

, et al. Rapid image labeling via neuro-symbolic learning. In: Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining (KDD-23), Long Beach, CA, USA, pp.2467–2477. New York: ACM Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

196.51 MB

0.00 MB