Segmentation networks for detecting overlapping screws in 3D and color images for industrial quality control

Abstract

This study explores cost-effective, real-time strategies for bin picking in industrial quality control. An anomaly detection solution was developed for a screw production plant, utilizing machine vision and AI to identify overlapping screws as anomalies. Two improvements are proposed to a basic solution initially relying on a laser profiler for depth images. The first improvement applies a Convolutional Neural Network (CNN) to the laser profiler's output, and the second replaces the laser profiler with a camera that captures color images, applying a CNN to its output. The first improvement was tested with real laser profiler data using YOLOv8 and Mask R-CNN segmentation models. After achieving comparable results on the real dataset, the second improvement was tested on multiple synthetic datasets, simulating different scenarios, including setups with mixed screws. Results demonstrated that model performance on color images, represented in the RGB color space (red, green, and blue), was comparable to depth images, validating color cameras as an appropriate alternative. Since color cameras are cheaper and capture images faster, they are well-suited for high-speed quality control systems, offering significant cost and performance advantages. Code is available at: https://github.com/enmarchi/overlapping_screws_geneneration_code.