Lightweight robotic grasping detection network based on dual attention and inverted residual

Abstract

Grasping detection is one of the crucial capabilities for robot systems. Deep learning has achieved remarkable outcomes in robot grasping tasks; however, many deep neural networks were at the expense of high computation cost with memory requirements, which hindered their deployment on computing-constrained devices. To solve this problem, this paper proposes an end-to-end lightweight network with dual attention and inverted residual strategies (LiDAIR), which adopts a generative pixel-level prediction to achieve grasp detection. The LiDAIR is composed of the convolution modules (Conv), the inverted residual convolution module (IRCM), the convolutional block attention connection module (CBACM), and the transposed convolution modules (TConv). The Convs are utilized in downsampling processes to extract the input image features. Then, the IRCM is proposed as a bridge between the downsampling and upsampling phases. In the upsampling phase, the CBACM is designed to focus on the valuable regions from spatial and channel dimensions, where the skip connection is employed to attain multi-level feature fusion. Afterwards, the TConvs are used to restore image resolution. The LiDAIR is lightweight with 704K parameters and enjoys a good tradeoff among lightweight structure, accuracy, and speed. It was evaluated on both the Cornell data set and the Jacquard data set within 10 ms inference time, and the detection accuracy on both the data sets were 97.7% and 92.7%, respectively.

Keywords

Robotic grasping lightweight network attention strategy inverted residual pixel-level

Get full access to this article

View all access options for this article.

References

Asif

Tang

Harrer

(2018) GraspNet: An efficient convolutional neural network for real-time grasp detection for low-powered devices. In: Proceedings of the 27th international joint conference on artificial intelligence (IJCAI), Stockholm, 13–19 July, pp. 4875–4882. San Francisco: Margan Kaufmann.

Bohg

Morales

Asfour

, et al. (2014) Data-driven grasp synthesis—A survey. IEEE Transactions on Robotics 30: 289–309.

Chu

F-J

Vela

(2018) Real-world multiobject, multigrasp detection. IEEE Robotics and Automation Letters 3: 3355–3362.

Depierre

Dellandréa

Chen

(2018) Jacquard: A large scale dataset for robotic grasp detection. In: Proceedings of the 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), Madrid, 1–5 October, pp. 3511–3516. New York: IEEE.

Girshick

(2015) Fast R-CNN. In: Proceedings of the 2015 IEEE international conference on computer vision (ICCV), Santiago, 7–13 December, pp. 1440–1448. New York: IEEE.

Jiang

Moseson

Saxena

(2011) Efficient grasping from RGBD images: Learning using a new rectangle representation. In: Proceedings of the 2011 IEEE international conference on robotics and automation, Shanghai, China, 9–13 May, pp. 3304–3311. New York: IEEE.

Krizhevsky

Sutskever

Hinton

(2012) ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25. DOI:10.1145/3065386

Kumra

Kanan

(2017) Robotic grasp detection using deep convolutional neural networks. In: Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September, pp. 769–776. New York: IEEE.

Kumra

Joshi

Sahin

(2020) Antipodal robotic grasping using generative residual convolutional neural network. In: Proceedings of the 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS), Las Vegas, NV, 25–29 October, pp. 9626–9633. New York: IEEE.

10.

M-T

Lien

J-JJ

(2022) Lightweight robotic grasping model based on template matching and depth image. IEEE Embedded Systems Letters 14: 199–202.

11.

Lenz

Lee

Saxena

(2015) Deep learning for detecting robotic grasps. The International Journal of Robotics Research 34: 705–724.

12.

Liu

Tao

Yuan

, et al. (2022) Robotic objects detection and grasping in clutter based on cascaded deep convolutional neural network. IEEE Transactions on Instrumentation and Measurement 71: 5004210.

13.

Morrison

Corke

Leitner

(2019) Learning robust, real-time, reactive robotic grasping. The International Journal of Robotics Research 39: 183–201.

14.

Redmon

Angelova

(2015) Real-time grasp detection using convolutional neural networks. In: Proceedings of the 2015 IEEE international conference on robotics and automation (ICRA), Seattle, WA, 26–30 May, pp. 1316–1322. New York: IEEE.

15.

Sahbani

El-Khoury

Bidaud

(2012) An overview of 3D object grasp synthesis algorithms. Robotics and Autonomous Systems 60: 326–336.

16.

Sandler

Howard

Zhu

, et al. (2018) MobileNetV2: Inverted residuals and linear bottlenecks. In: Proceedings of the 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, 18–23 June, pp. 4510–4520. New York: IEEE.

17.

Teng

Gao

(2021) Generative robotic grasping using depthwise separable convolution. Computers & Electrical Engineering 94: 107318.

18.

Tian

Song

, et al. (2022) Lightweight pixel-wise generative robot grasping detection based on RGB-D dense fusion. IEEE Transactions on Instrumentation and Measurement 71: 5017912.

19.

Wang

Jiang

Zhao

, et al. (2019) Efficient fully convolution neural network for generating pixel wise robotic grasps with high resolution images. In: Proceedings of the 2019 IEEE international conference on robotics and biomimetics (ROBIO), Dali, China, 6–8 December, pp. 474–480. New York: IEEE.

20.

Wang

, et al. (2016) Robot grasp detection using multimodal deep convolutional neural networks. Advances in Mechanical Engineering. Epub ahead of print 23 September. DOI: 10.1177/1687814016668077.

21.

Woo

Park

Lee

J-Y

, et al. (2018) CBAM: Convolutional block attention module. In: Proceedings of the 15th European conference on computer vision (ECCV), Munich, 8–14 September, pp. 3–19. New York: ACM.

22.

Zhang

Lan

Bai

, et al. (2019a) A multi-task convolutional neural network for autonomous robotic grasping in object stacking scenes. In: Proceedings of the 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS), Macau, China, 3–8 November, pp. 6435–6442. New York: IEEE.

23.

Zhang

Lan

Bai

, et al. (2019b) ROI-based robotic grasp detection for object overlapping scenes. In: Proceedings of the 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS), Macau, China, 3–8 November, pp. 4768–4775. New York: IEEE.

24.

Zhang

, et al. (2017) Robust robot grasp detection in multimodal fusion. MATEC Web of Conferences 139: 00060.

25.

Zhou

Lan

Zhang

, et al. (2018) Fully convolutional grasp detection network with oriented anchor box. In: Proceedings of the 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), Madrid, 1–5 October, pp. 7223–7230. New York: IEEE.