This paper proposes a framework for trajectory tracking of wheeled robots in indoor environment. Being robust against uncertainty and scalability to large environments are essential factors for this task. Here, it is supposed that the robot is only equipped with a vision system e.g. a Kinect camera. Generally, some challenges in this problem are: determining suitable control architecture, adjusting the parameters of this architecture according to the given purposes, extracting proper features from high-dimensional input images. In this paper, using deep learning methods the proper features are extracted. The controller is designed based on weighted sum of these features. A new method to combine supervised learning and reinforcement learning is introduced to adjust the proposed controller parameters. The mobile robot and the experimental environments are established on the WEBOTS and MATLAB co-simulation platform. The simulation experimental results indicate that the designed control is robust and effective for tracking trajectories in the indoor environment.
BengioY., CourvilleA. and VincentP., Representation learning: A review and new perspectives, IEEE Transactions on Pattern Analysis and Machine Intelligence35 (2013), 1798–1828.
2.
CorreaD.S.O., SciottiD.F., PradoM.G.., Mobile robots navigation in indoor environments using kinect sensor, in: Second Brazilian Conference on Critical Embedded Systems (CBSEC), 2012, pp. 36–41.
3.
DerhamiV., Similarity of learned helplessness in human being and fuzzy reinforcement learning algorithms, Journal of Intelligent & Fuzzy Systems24 (2013), 347–354.
4.
DerhamiV., MajdV.J. and AhmadabadiM.N., Fuzzy Sarsa learning and the proof of existence of its stationary points, Asian Journal of Control10 (2008), 535–549.
5.
DeSouzaG.N. and KakA.C., Vision for mobile robot navigation: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence24 (2002), 237–267.
6.
FathinezhadF., DerhamiV. and RezaeianM., Supervised fuzzy reinforcement learning for robot navigation, Applied Soft Computing40 (2016), 33–41.
7.
GuS., LillicrapT., GhahramaniZ., TurnerR.E. and LevineS., Q-Prop: Sample-efficient policy gradient with an off-policy critic, International Conference on Learning Representations (ICLR)2017.
8.
GüntherJ., PilarskiP.M., HelfrichG., ShenH. and DiepoldK., Intelligent laser welding through representation, prediction, and control learning: An architecture with deep neural networks and reinforcement learning, Mechatronics34 (2016), 1–11.
9.
HadsellR., SermanetP., BenJ.., Online learning for offroad robots: Using spatial label propagation to learn long-range traversability, Proc of Robotics: Science and Systems (RSS) (2007), 32.
10.
HanafiD., AbueejelaY.M. and ZakariaM.F., Wall follower autonomous robot development applying fuzzy incremental controller, Intelligent Control and Automation4 (2013), 18.
11.
HintonG.E. and SalakhutdinovR.R., Reducing the dimensionality of data with neural networks, Science313 (2006), 504–507.
12.
HoffmannG.M., TomlinC.J., MontemerloM. and ThrunS., Autonomous automobile trajectory tracking for offroad driving: Controller design, experimental validation and racing, in: American Control Conference, 2007, pp. 2296–2301.
13.
IglesiasR., RegueiroC.V., CorreaJ. and BarroS., Supervised reinforcement learning: Application to a wall following behaviour in a mobile robot, in: International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, 1998, pp. 300–309.
14.
JafarF.A., ZakariaN.A. and YokotaK., Visual features based motion controller for mobile robot navigation, International Journal of Simulation Systems, Science & Technology15 (2014), 7–14.
15.
JangJ., SunC., MizutaniE.Neuro-Fuzzy and Soft Computing, Prentice-Hall, upper Sanddle River, 1997.
16.
KoberJ., BagnellJ.A. and PetersJ., Reinforcement learning in robotics: A survey, The International Journal of Robotics Research32 (2013), 1238–1274.
17.
KonoligeK., AgrawalM., BollesR.C.., Outdoor mapping and navigation using stereo vision, in: Experimental Robotics, 2008, pp. 179–190.
18.
LampleG. and ChaplotD.S., Playing FPS Games with Deep Reinforcement Learning, in: AAAI Conference on Artificial Intelligence, 2017, pp. 2140–2146.
19.
LangeS. and RiedmillerM., Deep auto-encoder neural networks in reinforcement learning, in: International Joint Conference on Neural Networks (IJCNN), 2010, pp. 1–8.
20.
LillicrapT.P., HuntJ.J., PritzelA.., Continuous control with deep reinforcement learning, Google Patents2016–in.
21.
LinY.-P. and LiX.-Y., Reinforcement learning based on local state feature learning and policy adjustment, Information Sciences154 (2003), 59–70.
22.
MiljkovićZ., MitićM., LazarevićM. and BabićB., Neural network reinforcement learning for visual control of robot manipulators, Expert Systems with Applications40 (2013), 1721–1736.
23.
MnihV., BadiaA.P., MirzaM.., Asynchronous methods for deep reinforcement learning, in: International Conference on Machine Learning, 2016, pp. 1928–1937.
24.
MnihV., KavukcuogluK., SilverD.., Human-level control through deep reinforcement learning, Nature518 (2015), 529–533.
25.
OliverA., KangS., WünscheB.C. and MacDonaldB., Using the Kinect as a navigation sensor for mobile robotics, in: Proceedings of the 27th Conference on Image and Vision Computing New Zealand, 2012, pp. 509–514.
26.
PerkinsT.J. and PrecupD., A convergent form of approximate policy iteration, in: Advances in Neural Information Processing Systems, 2002, pp. 1595–1602.
27.
PilarskiP.M., DawsonM.R., DegrisT.., Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning, in: IEEE International Conference on Rehabilitation Robotics (ICORR), 2011, pp. 1–7.
28.
QiuJ., WeiY. and WuL., A novel approach to reliable control of piecewise affine systems with actuator faults, IEEE Transactions on Circuits and Systems II: Express Briefs64 (2017), 957–961.
29.
QuintiaP., DomenechJ.E., RegueiroC.V., GamalloC. and IglesiasR., Learning a wall following behaviour in mobile robotics using stereo and mono vision, in: IX Workshop en Agentes Fisicos, Vigo, Espana, 2008.
30.
RiedmillerM., Neural fitted Q iteration–first experiences with a data efficient neural reinforcement learning method, in: European Conference on Machine Learning, 2005, pp. 317–328.
31.
SaeediP., LawrenceP.D. and LoweD.G., Vision-based 3-D trajectory tracking for unknown environments, IEEE Transactions on Robotics22 (2006), 119–136.
32.
SchulmanJ., MoritzP., LevineS., JordanM. and AbbeelP., High-dimensional continuous control using generalized advantage estimation, International Conference on Learning Representations (ICLR)2016.
33.
SuttonR.S. and BartoA.G., Reinforcement learning: An introduction, MIT Press, Cambridge1998.
34.
ThrunS., MontemerloM., DahlkampH.., Stanley: The robot that won the DARPA Grand Challenge, Journal of Field Robotics23 (2006), 661–692.
35.
WeiY., QiuJ. and KarimiH.R., Reliable output feedback control of discrete-time fuzzy affine systems with actuator faults, IEEE Transactions on Circuits and Systems I: Regular Papers64 (2017), 170–181.
36.
WieringM., Convergence and divergence in standard and averaging reinforcement learning, Machine Learning: ECML2004 (2004), 477–488.
37.
YangY., FuM., ZhuH., XiongG. and ChangshengS., Control methods of mobile robot rough-terrain trajectory tracking, in: 8th IEEE International Conference on Control and Automation (ICCA), 2010, pp. 731–738.
38.
YeC., YungN.H. and WangD., A fuzzy controller with supervised learning assisted reinforcement learning algorithm for obstacle avoidance, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)33 (2003), 17–27.
39.
YungN.H. and YeC., An intelligent mobile vehicle navigator based on fuzzy logic and reinforcement learning, Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)29 (1999), 314–321.
40.
ZhouZ., ChenT., WuD. and YuC., Corridor navigation and obstacle distance estimation for monocular vision mobile robots, JDCTA: International Journal of Digital Content Technology and its Applications5 (2011), 192–202.
41.
ZhuY., MottaghiR., KolveE.., Target-driven visual navigation in indoor scenes using deep reinforcement learning, in: IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 3357–3364.
