A highly efficient garbage pick-up embedded system based on improved SSD neural network using robotic arms

Abstract

With the social evolution, economic development, and continuously improved living standards, the dramatically increasing garbage produced by human beings has seriously affected our living environment. There are 3 main ways to dispose of garbage: sanitary landfill, incineration, or recycling. At present, a huge amount of labor resources is required for pre-sorting before garbage disposal, which greatly reduces efficiency, increases costs, and even leads to direct incineration without sorting. Hence, this study proposes a solution scenario of how to use object detection technology for garbage sorting. With the development of the deep learning theory, object detection technology has been widely used in all fields, thus, how to find target objects accurately and rapidly is one of the key technologies. This paper proposes a highly efficient garbage pick-up embedded system, where detection is optimized based on the Single Shot MultiBox Detector (SSD) neural network architecture and reduced model parameters. The experimental verification scenario was conducted in a dynamic environment integrating a robotic arm with a conveyor belt simulated by an electronic rotating turntable. The experimental results show that the modified model can accurately identify garbage types, with a significant speed of 27.8 FPS (Frames Per Second) on NVidia Jetson TX2, and an accuracy rate of approximately 87%.

Keywords

AI embedded system deep learning robotic arm control garbage picking up

Get full access to this article

View all access options for this article.

References

Anand ,

S.S.

Tripathy and

R.S.

Kumar , An improved edge detection using morphological Laplacian of Gaussian operator, in: Signal Processing and Integrated Networks (SPIN), 2015 2nd International Conference on, 2015, pp. 532–536. doi:10.1109/SPIN.2015.7095391.

Badrinarayanan ,

Kendall and

Cipolla , SegNet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (2017), 2481–2496. doi:10.1109/TPAMI.2016.2644615.

Bai ,

Lian and

Liu , Deep learning based robot for automatically picking up garbage on the grass, Proceedings of the IEEE Transactions on Consumer Electronics 64 (2018), 382–389. doi:10.1109/TCE.2018.2859629.

Bay ,

Tuytelaars and

L.V.

Gool , Surf: Speeded up robust features, in: European Conference on Computer Vision, 2006, pp. 404–417.

Carles ,

Stefano ,

Anthony ,

George and

Davy , Internet of things for enabling smart environments: A technology-centric perspective’, Journal of Ambient Intelligence and Smart Environments 11(1) (2019), 23–43. doi:10.3233/AIS-180509.

G.N.

Chaple ,

R.D.

Daruwala and

M.S.

Gofane , Comparisions of Robert, prewitt, Sobel operator based edge detection methods for real time uses on FPGA, in: Technologies for Sustainable Development (ICTSD), 2015 International Conference on, 2015.

Coccoli ,

V.D.

Francesco ,

Fusco and

Maresca , A cloud-based cognitive computing solution with interoperable applications to counteract illegal dumping in smart cities, Multimedia Tools and Applications (2021). doi:10.1007/s11042-021-11238-8.

G-code, Available online https://en.wikipedia.org/wiki/G-code (accessed on 21 September 2021).

Girshick , Fast r-cnn, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1440–1448.

10.

Girshick ,

Donahue ,

Darrell and

Malik , Rich feature hierarchies for accurate object detection and semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 580–587.

11.

Glorot and

Bengio , Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010, pp. 249–295.

12.

Goodfellow ,

Bengio and

Courville , Deep Learning (Adaption Computation and Machine Learning Series), MIT Press, 2016.

13.

He ,

Zhang ,

Ren and

Sun , Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 27–30.

14.

Jiang ,

Tang and

Chen , Tracking multiple targets based on min-cost network flows with detection in RGB-D data, International Journal of Computational Science and Engineering 15(3) (2017), 330–339. doi:10.1504/IJCSE.2017.087414.

15.

Kanopoulos ,

Vasanthavada and

R.L.

Baker , Design of an image edge detection filter using the Sobel operator, IEEE Journal of Solid-State Circuits 23 (1998), 358–367. doi:10.1109/4.996.

16.

Krizhevsky ,

Sutskever and

G.E.

Hinton , Imagenet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems (2012).

17.

Label-Annotation-VOC-Pascal, Available online https://github.com/manhcuogntin4/Label-Annotation-VOC-Pascal (accessed on 21 September 2021).

18.

S.H.

Lee ,

T.W.

Hou ,

C.H.

Yeh and

C.S.

Yang , A lightweight neural network based on AlexNet-SSD model for garbage detection, in: HPCCT 2019 – 3rd High Performance Computing and Cluster Technologies Conference and BDAI 2019 – 2nd International Conference on Big Data and Artificial Intelligence. Association for Computing Machinery, 2019, pp. 274–278.

19.

T.Y.

Lin ,

Goyal ,

Girshick ,

He and

Doll’ar , Focal loss for dense object detection, 2017, arXiv:1708.02002.

20.

Liu ,

Anguelov ,

Erhan ,

Szegedy and

Reed , SSD: Single Shot MultiBox Detector, 2015, arXiv:1512.02325.

21.

T.C.

Lu , CNN convolutional layer optimisation based on quantum evolutionary algorithm, Connection Science 15(3) (2020), 330–339.

22.

Matjaz ,

G.I.

Yu-Hua ,

Aki ,

Andres and

Vincent , Artificial intelligence and ambient intelligence, multimedia tools and applications, Journal of Ambient Intelligence and Smart Environments 11(1) (2019), 71–86. doi:10.3233/AIS-180508.

23.

D.O.

Melinte ,

A.-M.

Travediu and

D.N.

Dumitriu , Deep convolutional neural networks object detector for real-time waste identification, Applied Science 10 (2020), 7301. doi:10.3390/app10207301.

24.

Mikami ,

Chen ,

Nakazawa ,

Iida ,

Kishimoto and

Oya , DeepCounter: Using Deep Learning to Count Garbage Bags, 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Application (RTCSA), 2018.

25.

J.M.

Morel and

Yu , A SIFT: A new framework for fully affine invariant image comparison, SIAM Journal on Imaging Sciences 2 (2009), 438–469. doi:10.1137/080732730.

26.

Nie ,

Li ,

Wang ,

Guo ,

Deng ,

Deng and

Dou , Laius: An energy-efficient FPGA CNN accelerator with the support of a fixed-point training framework, International Journal of Computational Science and Engineering 21 (2020), 418–428. doi:10.1504/IJCSE.2020.106064.

27.

NVIDIA, Jetson TX2, available online https://www.nvidia.com/zh-tw/autonomous-machines/embedded-systems-dev-kits-modules/ (accessed on 21 September 2021).

28.

Redmon ,

Divvala ,

Girshick and

Farhadi , You only look once: Unified, real-time object detection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 779–788.

29.

Redmon and

Farhadi , Yolo9000: Better, faster, stronger, arXiv preprint, 2017, arXiv:1612.08242.

30.

Redmon and

Farhadi , Yolov3: An incremental improvement, 2018, arXiv:1804.02767.

31.

Ren ,

He ,

Girshick and

Sun , Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence 39(6) (2017). doi:10.1109/TPAMI.2016.2577031.

32.

Sadako Technologies, Available online http://sadako.es/ (accessed on 21 September 2021).

33.

V.G.

Santiago and

J.A.

Alvares , Deployment framework for the Internet of water meters using computer vision on ARM platform, Journal of Ambient Intelligence and Smart Environments 12(1) (2020), 35–60. doi:10.3233/AIS-200544.

34.

Simonyan and

Zisserman , Very Deep Convolutional Networks for Large-Scale Image Recognition, International Conference on Learning Representations, 2015.

35.

B.H.

Systems , Available online https://www.bulkhandlingsystems.com/ (accessed on 21 September 2021).

36.

C.H.

Tung ,

W.J.

Syu and

W.C.

Hunag , Hough transform-based cubic spline recognition for natural shapes, International Journal of Computational Science and Engineering 18(2) (2019), 164–175. doi:10.1504/IJCSE.2019.097943.

37.

UFACTOR, uArm Swift, available online https://www.ufactory.cc/pages/uarm (accessed on 21 September 2021).

38.

Wang ,

Xing and

Liu , Efficient deep convolutional model compression with an active stepwise pruning approach, International Journal of Computational Science and Engineering 22(4) (2020), 420–430. doi:10.1504/IJCSE.2020.109401.

39.

Yu ,

Rajesh ,

X.M.

Mercilin ,

Kritika ,

Kavinkumar and

S.B.

Shah , Machine learning-based ship detection and tracking using satellite images for maritime surveillance, Journal of Ambient Intelligence and Smart Environments 13(5) (2021), 361–371. doi:10.3233/AIS-210611.

40.

Zhihong ,

Hebin ,

Yan ,

Yanbo and

Binyan , Multi-task detection system for garbage sorting base on high-order fusion of convolutional feature hierarchical representation, in: 2018 37th Chinese Control Conference (CCC), 2018.