In this paper, a hybrid method based on deep learning is proposed to visually classify terrains encountered by mobile robots. Considering the limited computing resource on mobile robots and the requirement for high classification accuracy, the proposed hybrid method combines a convolutional neural network with a support vector machine to keep a high classification accuracy while improve work efficiency. The key idea is that the convolutional neural network is used to finish a multi-class classification and simultaneously the support vector machine is used to make a two-class classification. The two-class classification performed by the support vector machine is aimed at one kind of terrain that users are mostly concerned with. Results of the two classifications will be consolidated to get the final classification result. The convolutional neural network used in this method is modified for the on-board usage of mobile robots. In order to enhance efficiency, the convolutional neural network has a simple architecture. The convolutional neural network and the support vector machine are trained and tested by using RGB images of six kinds of common terrains. Experimental results demonstrate that this method can help robots classify terrains accurately and efficiently. Therefore, the proposed method has a significant potential for being applied to the on-board usage of mobile robots.
AngelovaAMatthiesLHelmickD, et al. (2007) Learning and prediction of slip from visual information. Journal of Field Robotics24(3): 205–231.
2.
BelterDWietrzykowskiJSkrzypczynskiP (2019) Employing natural terrain semantics in motion planning for a multi-legged robot. Journal of Intelligent & Robotic Systems93(3): 723–743.
3.
BermudezFJulianRHaldaneD, et al. (2012) Performance analysis and terrain classification for a legged robot over rough terrain. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal, 7–12 October 2012, pp. 513–519IEEE.
4.
BorensteinJKorenY (1991) The vector field histogram-fast obstacle avoidance for mobile robots. IEEE Transactions on Automation Science and Engineering7(3): 278–288.
5.
BrandaoMShiguematsuYMHashimotoK, et al. (2016) Material recognition CNNs and hierarchical planning for biped robot locomotion on slippery terrain. In: 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), Cancun, Mexico, 15–17 November 2016, pp. 81–88. IEEE.
6.
BrooksCIagnemmaK (2009) Visual detection of novel terrain via two-class classification. In: 2009 ACM Symposium on Applied Computing, Honolulu, HI, USA, 8–12 March 2009, pp. 1145–1150. ACM.
7.
BrooksCAIagnemmaK (2012) Self-supervised terrain classification for planetary surface exploration rovers. Journal of Field Robotics29(3): 445–468.
8.
CaoFZhuangYZhangH, et al. (2018) Robust place recognition and loop closing in laser-based SLAM for UGVs in urban environments. IEEE Sensors Journal18(10): 4242–4252.
9.
CaoFZhuangYYanF, et al. (2020) Long-term autonomous environment adaptation of mobile robots: State-of-the-art methods and prospects. Acta Automatica Sinica46(2): 205–221.
10.
CastelnoviMArkinRCollinsT (2005) Reactive speed control system based on terrain roughness detection. In: 2005 IEEE International Conference on Robotics and Automation, Barcelona, Spain, 18–22 April 2005, pp. 891–896. IEEE.
11.
ChagantiS YNandaIPandiKR, et al. (2020) Image classification using SVM and CNN. In: 2020 International Conference on Computer Science, Engineering and Applications (ICCSEA), July 25–26, 2020, London, United Kingdom, pp. 1–5. CCSEA.
12.
EbadiFNorouziM (2017) Road terrain detection and classification algorithm based on the color feature extraction. Artificial Intelligence and Robotics (IRANOPEN). Qazvin: IEEE.
13.
FilitchkinPBylK (2012) Feature-based terrain classification for littledog. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal, 7–12 October 2012, pp. 1387–1392. IEEE.
14.
GonzalezRApostolopoulosDIagnemmaK (2018) Slippage and immobilization detection for planetary exploration rovers via machine learning and proprioceptive sensing. Journal of Field Robotics35(2): 231–247.
15.
HangSYingbaiH, et al. (2020) Improved recurrent neural network-based manipulator control with remote center of motion constraints: Experimental result. Neural Networks131(1): 291–299.
16.
KhanYNKommaPZellA (2011) High resolution visual terrain classification for outdoor robots. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, 6–13 November 2011, pp.1014–1021. IEEE.
17.
KozlowskiPWalasK (2018) Deep neural networks for terrain recognition task. In: 2018 Baltic URSI Symposium (URSI), Poznan, Poland, 14–17 May, 2018, pp. 283–286. IEEE.
18.
KrizhevskyASutskeverIHintonG (2012) Imagenet classification with deep convolutional neural networks. In: Advances Neural Information Processing Systems 25, Lake Tahoe, NV, USA, 36 December 2012; pp. 1097–1105. Springer.
19.
LeCunYKavukcuogluKFarabetC (2010) Convolutional networks and applications in vision. In: 2010 IEEE International Symposium on Circuits and Systems, Paris, France, May 30–June 2 2010, pp. 253–256.
20.
LeCunYBoserBDenkerJS, et al. (1989) Backpropagation applied to handwritten zip code recognition. Neural Computation1(4): 541–551.
21.
LiangMHuX (2015) Recurrent convolutional neural network for object recognition. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015, pp. 3367–3375.
22.
LinJDWuXYChaiY, et al. (2020) Structure optimization of convolutional neural networks: A survey. Acta Automatica Sinica46(1): 24–37.
23.
IshigamiGNagataniKYoshidaK (2011) Path planning and evaluation for planetary rovers based on dynamic mobility index. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA, 25–30 September 2011, pp. 601–606. IEEE.
24.
NadavIKatzE (2006) Off-road path and obstacle detection using monocular camera. In: 2016 IEEE International Conference on the Science of Electrical Engineering (ICSEE), Eilat, Israel, 16–18 November 2016, pp. 1–5. IEEE.
25.
PapadakisP (2013) Terrain traversability analysis methods for unmanned ground vehicles: A survey. Engineering Applications of Artificial Intelligence26(4): 1373–1385.
26.
RothrockBKennedyRCunninghamC, et al. (2016) Spoc: Deep learning-based terrain classification for Mars rover missions. In: AIAA SPACE 2016, Long Beach, CA, USA, 14 September 2016, p. 5539. AIAA.
27.
SimonyanKZissermanA (2015) Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations, San Diego, CA, USA, 7–9 May 2015Conference Track Proceedings.
28.
SrivastavaNHintonGKrizhevskyA, et al. (2014) Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research15(1): 1929–1958.
29.
TanakaYJiYYamashitaA, et al. (2015) Fuzzy based traversability analysis for a mobile robot on rough terrain. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 26–30 May 2015, pp. 3965–3970. IEEE.
30.
VernazaPTaskarBLeeD (2008) Online, self-supervised terrain classification via discriminatively trained submodular Markov random fields. In: 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, 19–23 May 2008, pp. 2750–2757.
31.
ZhangHDaiXSunF, et al. (2016) Terrain classification in field environment. based on Random Forest for the mobile robot. In: 35th Chinese Control Conference (CCC), Chengdu, 27–29 July China, 2016, pp. 6074–6079. IEEE.
32.
ZhilenkovANyrkovAChernyiSSokolovS (2017) Simulation of in-sensor processes in the sensor – Object system type when scanning the elements of underwater communication lines with a probe beam. International Review on Modelling and Simulations10(5): 363–370.
33.
WangZJTurkoRShaikhO, et al. (2020) CNN EXPLAINER: Learning convolutional neural networks with interactive visualization. IEEE Transactions on Visualization and Computer Graphics: 1–1.
